Schema Validation

AutoFlow validates all generated workflows against two JSON schemas plus custom structural rules implemented in src/autoflow/schemas/validator.py.

Workflow Schema

schemas/asksage_workflow/schema.json defines the structure of Agent Builder workflows:

Node Types

7 supported node types:

Type	Purpose
`llm`	Call an LLM. Files attach via `file_variables` referencing `input.<varname>` — there is no separate file-reader node.
`loop`	Iterate over a list, exposing `{{loop.item}}` and collecting `{{loop.results}}`
`if_else`	Binary conditional routing (`if`/`else` canonical, `true`/`false` legacy alias). Conditions reference variables WITHOUT mustache braces.
`variable_assignment`	Save a single `source` to a `variable_name` (UI: "Save Variable")
`flat_response`	Return a templated text or JSON response (UI: "Return Response") — always terminal
`decision_tree`	Multi-way routing with positional handles `cat_0`, `cat_1`, ...
`variable_transform`	Apply an array of operations under a `transformations` field (UI: "Transform Variables")

Edge Rules

sourceHandle values: "out", "if"/"else" (or legacy "true"/"false"), "iteration"/"complete" (or legacy "loop"/"aggregated"), "pass"/"fail", or positional "cat_0", "cat_1", ... from decision_tree
targetHandle values: "in"
Every edge must reference existing node IDs

Variable Patterns

Workflows use double-brace variable references:

Pattern	Meaning
`{{input.X}}`	An `agent_config` input variable
`{{node_id.response}}`	Output from an LLM node (uses raw node id)
`{{loop.item}}`	Current item in a loop iteration (inside the loop body)
`{{loop.results}}`	Collected results after the loop completes
`{{node_id.category}}`	Category output from a decision_tree
`{{node_id.confidence}}`	Confidence score from a decision_tree
`{{variable_name}}`	Output of `variable_assignment` or any `variable_transform` operation — bare name

Templates on disk are real Ask Sage v1 exports. Nodes use kebab-case or snake_case string IDs, and edges reference nodes by these string IDs. Workflows have nodes, edges, and agent_config at the top level (Agent Builder import requires agent_config). AWC adds an optional metadata block (catalog discovery only — Agent Builder ignores it) and an optional slug field on each node (human-readable convenience — Agent Builder ignores it).

{
  "id": "intake",
  "slug": "intake",
  "type": "llm",
  "label": "Intake",
  "position": { "x": 0, "y": 0 },
  "config": { ... }
}

Variable references use the raw node id: {{intake.response}}.

v2 (UUID + slug) — AWC-Internal Only

AWC has an internal v2 format (UUID id + slug + key) used during pipeline execution and validation. It is never written to disk for templates. The validator can convert in-memory between formats via migration.py and auto-detects via detect_workflow_version().

Custom Validation Rules

Structural Rules

Unique node IDs — Every id must be unique
Valid edge references — All source and target in edges must reference existing node IDs
No orphan nodes — Every node must be connected by at least one edge (except the entry node)
Position bounds — Node positions must be within -10000 to 10000 on both axes
Variable resolution — All {{variable}} references must resolve to a node that appears earlier in the graph

Semantic Rules

Beyond structural checks, workflows are validated for execution safety:

Terminal node required — At least one flat_response node or an llm node with is_terminal: true in its config
Dangling handle detection — if_else must have both branches wired (if/else canonical, or legacy true/false); decision_tree must have an edge for every category (positional handles cat_0, cat_1, ...); loop must have both iteration and complete wired (or legacy loop/aggregated)
Mustache restriction — if_else condition fields must not contain {{...}} variable syntax
Loop bounds — loop nodes must have max_iterations within 1-1000
Registry config validation — Each node's config is validated against the node registry's parameter definitions (required fields, types, ranges, enums)
agent_config required for Agent Builder import — workflows missing agent_config cannot be imported into Agent Builder

Node Registry

The node registry (node_registry/asksage_nodes.yaml) is the single source of truth for all 7 node types. It defines each type's parameters, handles, and constraints. The validator uses the registry for config validation, and per-node JSON Schemas are auto-generated from it into schemas/nodes/.

Output Envelope Schema

schemas/universal_output_envelop/schema.json validates the wrapper around every pipeline output:

{
  "objective": "string",
  "deliverable_artifact": "object",
  "quality_scores": {
    "correctness": "integer (1-5)",
    "completeness": "integer (1-5)",
    "format_validity": "integer (1-5)",
    "efficiency": "integer (1-5)",
    "reusability": "integer (1-5)"
  },
  "confidence": "number (0-1)",
  "assumptions": ["string"],
  "constraints": ["string"],
  "known_gaps": ["string"],
  "estimated_human_edit_minutes": "integer",
  "recommended_next_actions": ["string"],
  "metadata": "object"
}

Quality Rubric

The 5-point scoring scale used by the EvaluatorAgent:

Dimension	What It Measures
Correctness	Does the workflow solve the stated objective?
Completeness	Are all necessary steps and edge cases covered?
Format Validity	Is the output valid Ask Sage Agent Builder JSON?
Efficiency	Minimal nodes, smart routing, no redundancy?
Reusability	Can the workflow be generalized for similar tasks?