Pipeline
How an IDP template's pipeline is built in the builder: the order of the stages, what passes from one to the next, each layer's card and auto-save.
Pipeline
The Builder shows the path every document takes, top to bottom: from the file received to the JSON returned. Each stage has its own settings, and the stage map sums up the state of each one — how many filters, whether identification is on, how many post-processing steps and how many validations.
Stage order
| # | Stage | Receives | Hands over |
|---|---|---|---|
| 1 | Document | The file sent (PDF or image). | The pages that go through the pipeline, according to the analysis scope. |
| 2 | Pre-processing | The selected pages. | The images with the filters applied, top to bottom. |
| 3 | Identification | The first page of the incoming document, before the filters. | Accepts the document or, when it rejects it, stops the pipeline. |
| 4 | Extraction | The pre-processed images. | The extracted text and, depending on the method, tables, forms and text blocks. |
| 5 | Post-processing | The extraction result. | The transformed result, step by step: each step receives the previous one's output. |
| 6 | Validations | The final result: the last post-processing step's output or, without any, the extraction result. | A pass or fail for each check. |
| 7 | Result | The final result. | The JSON returned to whoever called the API and sent to the webhooks. |
Stages with nothing configured don't block the path: with no filters, the document goes to extraction as it was sent; with identification off, no document is rejected; with no post-processing, the extraction result is returned as is; with no validations, every result is accepted.
Extraction is the only stage that always has a configuration: a new template starts with Roberty Cloud OCR and table extraction turned on.
Layer cards
Pre-processing, post-processing and validations are lists of layers — filters, steps or checks — run top to bottom. Each layer is a card with the same controls:
| Control | What it does |
|---|---|
| Add | Creates a layer of the chosen type. New layers go to the end of the list. |
| Move up / Move down | Changes the layer's position and, with it, the order of execution. |
| Turn on / Turn off switch | Turns the layer on or off. A layer that is off keeps its settings but is skipped in runs. |
| Show settings / Hide settings | Opens or closes the layer's fields. |
| Remove | Deletes the layer from the list. |
Turning a layer off is handy for comparing results in the Playground with and without it, without losing what was configured.
Auto-save
Every change in the builder is saved automatically to the template's draft right after it is made — there is no save button. If a save fails, the header shows Save failed with a Retry save option.
The draft is what the Playground, Simulate API and the evaluation sets run. Production and the API keep using the active version until you publish a new one. See Versions.
Only workspace developers and administrators can edit templates.
Result
The builder's last stage is the Result: the final JSON, returned to whoever called the API and sent to the webhooks. From there you open the Playground, to run the draft on a sample document and see each stage's result before publishing, or Simulate API, to build the call the way an external system would. See Playground and Simulate API.