Guides

• 1: DataCanon input
• 2: Schema projection
• 3: Schema composition
• 4: Child schemas
• 5: Application construction
• 6: SchemaCommandLine
• 7: Attribute output
• 8: Output and application reports
• 9: Migrating from numeric coercion

1 - DataCanon input

A DataCanon Instance (DCI) is a value in DataCanon’s data model. SchemaProjector accepts a Mapping DCI whose keys are String DCIs. The values already have DataCanon Types (DCTs); the schema can optionally constrain them. Parsing text and converting conventional objects happen before projection.

From conventional objects

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

projector = SchemaProjector({"variables": {"matrix": {}, "iterations": {"type": "Integer"}}})
input_dci = as_dc_instance({"matrix": [[2, -1], [-1, 2]], "iterations": 20})
projection = projector.get_projection(input_dci)

as_dc_instance() applies DataCanon’s conversion rules. The resulting DCI is the input to DataSchemer (DS). Passing the conventional dictionary directly to get_projection() is an error.

The String-key restriction applies to the Mapping that supplies schema fields and to Mappings supplying child-schema data. A Mapping used as a field value can retain arbitrary DCI keys:

input_dci = as_dc_instance({"samples": {("sensor", 0): [1.0, 2.0]}})
projection = SchemaProjector({"variables": {"samples": {}}}).get_projection(input_dci)
assert projection.data["samples"][("sensor", 0)] == [1.0, 2.0]

From Shorthand

from data_canon import parse_shorthand

input_dci = parse_shorthand('{"matrix": [[2, -1], [-1, 2]], "iterations": 20}')
projection = projector.get_projection(input_dci)

Shorthand preserves structured numerical DCIs. For example, sqrt(3) describes a Power DCI, and 1/2 * sqrt(3) describes a Product containing a Fraction and a Power. DataCanon (DC) does not perform mathematical simplification during parsing.

Optional input preprocessing

Preprocessing makes manually written input easier to maintain. Enable it explicitly when calling the DC parser:

source = """
diagonal = 2
off_diagonal = -1 |
{
  "matrix": <diagonal off_diagonal; off_diagonal diagonal>,
  "iterations": 20
}
"""
input_dci = parse_shorthand(source, preprocess=True)
projection = projector.get_projection(input_dci)
assert projection.data["matrix"] == [[2, -1], [-1, 2]]

Substitutions insert DC structure. Compact List notation establishes nested Lists. An explicit eval(...) requests scalar mathematical evaluation during preprocessing:

input_dci = parse_shorthand(
    'side=2.5 | {"area": eval(side * side)}',
    preprocess=True,
)
projection = SchemaProjector({"variables": {"area": {}}}).get_projection(input_dci)
assert projection.data["area"] == 6.25

Preprocessing belongs to DC’s input facilities. SchemaProjector receives the resulting DCI and does not evaluate text inside String DCIs.

Application-defined command-line presets are available through SCL’s per-field substitutions property. See Application-defined substitutions for an identity-matrix example using 2 * I. Presets affect command-line option values; SchemaProjector continues to receive DCIs without interpreting strings.

From files

For a standalone projector, load the DCI before projection:

from data_canon import read

input_dci = read("calculation.dcb")
projection = projector.get_projection(input_dci)

DC’s read() handles canonical binary .dcb, strict Shorthand .dcs, and DC HDF5 files. For a standalone preprocessed Shorthand file, read its text and call parse_shorthand(..., preprocess=True) explicitly. If external resources are present, use an appropriate source when converting conventional output; DC’s persistence readers retain source context automatically.

SchemaCommandLine (SCL) provides an additional input boundary. It accepts .dcs with preprocessing, .dcb, DC .h5/.hdf5, and DC YAML .yaml/.yml. Its source loading retains context for deferred external-resource access. See SCL input files.

YAML interchange

Use DC’s YAML interchange when importing YAML directly:

from data_canon import parse_yaml

input_dci = parse_yaml("""
iterations: 20
samples:
  $ ("sensor", 0): [1.0, 2.0]
phase: $ complex(1/2, 1/2 * sqrt(3))
""")
projection = SchemaProjector({"variables": {
    "iterations": {"type": "Integer"}, "samples": {}, "phase": {},
}}).get_projection(input_dci)

The $ marker embeds strict DC Shorthand in YAML scalars, including Mapping keys. YAML sequences become Lists. YAML airlocks do not run Shorthand preprocessing.

For DC syntax, representations, and custom conversion registries, consult the DataCanon documentation.

2 - Schema projection

SchemaProjector compiles a schema for repeated use. A DataCanon Instance (DCI) is a value in DataCanon’s data model. Calling get_projection(input_dci) validates one Mapping DCI and returns a SchemaProjection containing the original input and processed output.

Compile once, project multiple inputs

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

schema = {"variables": {
    "iterations": {"type": "Integer"},
    "method": {"optional": True, "default": "direct", "choices": ["direct", "iterative"]},
}}
projector = SchemaProjector(schema)
first = projector.get_projection(as_dc_instance({"iterations": 10}))
second = projector.get_projection(as_dc_instance({"iterations": 20, "method": "iterative"}))

assert first.data == {"iterations": 10, "method": "direct"}
assert second.data == {"iterations": 20, "method": "iterative"}

Schema compilation resolves field definitions and validates their metadata. Projection validates each input separately. A projector can also receive an initial input through the keyword-only input_data argument; its projection, data_dci, and data properties then refer to that initial projection.

Optional type constraints

type names a DataCanon Type (DCT) using its canonical name, such as Integer, Float, List, or Array. The names are case-sensitive. Omitting type accepts any DCI.

schema = {"variables": {
    "iterations": {"type": "Integer"},
    "matrix": {},
}}

The processor rejects a Float DCI for iterations, even if its value is 20.0. An Integer DCI is likewise rejected for a field constrained to Float. Boolean and Integer remain distinct DCTs. DataSchemer (DS) performs no implicit numerical coercion.

The matrix field can accept a nested List, an Array, or another DCI. The application constructor is responsible for the domain-specific interpretation.

Constrain an Array’s element type

An Array DCI has a portable Array element type in addition to its DCT. Declare array_type when the application requires a particular one:

import numpy as np

array_projector = SchemaProjector({"variables": {
    "matrix": {"type": "Array", "array_type": "float64"},
}})
node = as_dc_instance({"matrix": np.eye(2, dtype=np.float64)})
assert array_projector.get_projection(node).data_dci == node

This accepts only an Array DCI with portable type float64. A float32 Array fails, as does a nested List. Validation preserves the supplied shape and elements. Omitting array_type allows every portable Array element type.

Supported names are bool, int8, int16, int32, int64, uint8, uint16, uint32, uint64, float16, float32, float64, complex64, and complex128. The names are case-sensitive DC names; NumPy aliases such as f8 are rejected.

array_type requires an explicit type: "Array"; it does not infer a DCT. The constraint applies to defaults and supplied values, including through SCL. It follows normal field-property inheritance, copying, overrides, and removal. It is checked after final schema composition, so delete_properties can remove an inherited constraint. ExternalArray is a separate DCT and is not covered by this Array constraint.

Required fields and defaults

Fields are required by default. Mark a field optional: True to permit absence. Only optional fields may declare a default.

schema = {"variables": {
    "iterations": {"type": "Integer"},
    "tolerance": {"optional": True, "default": 1e-8},
    "label": {"optional": True},
}}

An absent tolerance receives the default; an absent label stays absent. Supplying a default does not infer a DCT constraint. Add "type": "Float" if tolerance should accept only Float DCIs.

Defaults are converted to DCIs during compilation. Explicit defaults can already be DCIs. A supplied value always takes precedence over the default, including Null or false when allowed by the field’s constraints.

The original input is preserved. Defaults appear only in the processed result.

Choices, dependencies, and conflicts

choices compares values using DCI equality. Choice values may be conventional objects or explicit DCIs; schema compilation converts them to DCIs. An Integer choice does not admit the corresponding Float DCI.

requires lists companion fields that must be present when the field is present. Schema-level conflicts lists groups in which at most one field may be present.

schema = {
    "variables": {
        "method": {"optional": True, "default": "direct", "choices": ["direct", "iterative"]},
        "restart": {"optional": True, "requires": ["checkpoint"]},
        "checkpoint": {"optional": True},
        "fresh_start": {"type": "Boolean", "optional": True},
    },
    "conflicts": [["restart", "fresh_start"]],
}

Defaults are present for requires and conflicts. A false Boolean or Null is also present; presence is determined by the field’s existence, not its truth value. Unknown fields are rejected at every projected schema level.

Original input, local output, and complete output

Without child schemas, the local and complete processed views have the same contents. Supplied field order is preserved; added defaults follow in schema order. Child Mapping positions are preserved in the complete output.

Conventional views are computed on access and are not cached. Mutating a returned dictionary does not mutate the processed Mapping DCI. Conventional conversion follows DataCanon (DC)’s rules and may simplify values according to their destination types.

Deferred external resources and custom Records

Projection checks external-reference metadata without fetching the payloads. It also leaves custom Record conversion handlers uncalled. Accessing data, data_tree, or selected constructor arguments can retrieve external resources or invoke explicitly registered handlers.

Pass registry and source to the projector when needed:

# registry is a configured DataCanon ConversionRegistry.
# source is the namespace containing any referenced external resources.
projector = SchemaProjector(schema, registry=registry, source=source)

Those arguments are forwarded to DC conversion. Conversion failures are separate from successful schema validation.

Handle user errors

from data_schemer.errors import DSUserError

try:
    projection = projector.get_projection(input_dci)
except DSUserError as error:
    print(error.user_message())

See Errors for DCT diagnostics and exception classes. Continue with schema composition, child schemas, or application construction.

3 - Schema composition

Schema inheritance reuses definitions. A derived schema can adjust field properties, copy field definitions, or remove constraints. Inheritance does not create nested input data; that is the role of child schemas.

Inherit field properties

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

definitions = {
    "base": {"variables": {
        "iterations": {
            "type": "Integer", "optional": True, "default": 20,
            "help": "Maximum iteration count",
        },
    }},
    "production": {
        "inherit": ["base"],
        "variables": {"iterations": {"default": 100}},
    },
}

projector = SchemaProjector(definitions, schema_name="production")
projection = projector.get_projection(as_dc_instance({}))
assert projection.data == {"iterations": 100}
assert projector.schema_variables["iterations"]["type"] == "Integer"

The local default replaces the inherited default while type, optional, and help remain inherited. Overrides merge field properties. Each property’s value is replaced as a whole: a new choices list replaces the inherited list, and a new Mapping default replaces the inherited Mapping default.

For inherit: ["first", "second"], the first base wins when both explicitly define the same property. The second base fills properties absent from the first. Local variables properties take precedence over every base. Implicit defaults such as optional: False are added after composition.

Copy a field definition

definitions["production"]["copy_variables"] = {"report_limit": "base@iterations"}
definitions["production"]["variables"]["report_limit"] = {
    "default": 5,
    "help": "Maximum number of reported entries",
}
projector = SchemaProjector(definitions, schema_name="production")
assert projector.get_projection(as_dc_instance({})).data == {
    "iterations": 100, "report_limit": 5,
}

copy_variables maps a destination name to "schema@field". The source field uses its resolved definition, including inheritance. A copy replaces an inherited destination definition, then local properties for the destination override the copied properties.

Same-schema copies can refer to later copy entries. Cyclic copies and cyclic schema inheritance are rejected. Copies run before removals, so a same-schema source may be copied and then removed from the final schema.

Remove properties or fields

definitions["flexible"] = {
    "inherit": ["base"],
    "delete_properties": {"iterations": ["type", "default"]},
}
projector = SchemaProjector(definitions, schema_name="flexible")
assert projector.get_projection(as_dc_instance({})).data == {}
assert projector.get_projection(as_dc_instance({"iterations": "automatic"})).data == {
    "iterations": "automatic",
}

The field remains optional, but its type constraint and default are removed. Use delete_variables: ["iterations"] to remove the whole field definition. Removing an absent field or property is an error, catching misspelled names.

Setting default to None supplies a Null DataCanon Instance (DCI) default. It does not remove a default. Use delete_properties for removal.

Resolution order

For each schema, DataSchemer (DS) composes inherited field properties, applies local overrides and copies, then removes properties and fields. The copied destination always receives its local overrides after the copy. Implicit property defaults and final validation follow composition and removal.

The schema’s original definition dictionaries are not modified by field resolution. A schema_variables property returns a copy of the compiled metadata.

update_variables has been removed. Put property overrides directly under variables. An occurrence of update_variables, including an empty Mapping, raises a migration error.

Other inherited metadata

child_schemas accumulates in base order, followed by locally declared names, with duplicates removed. conflicts includes inherited groups.

print_attribute has its own merge rules for output configuration, including extending lists of choices and exclusions. Those rules are separate from ordinary field-property replacement; see the behavioral reference.

4 - Child schemas

A schema can declare child schemas to validate nested input Mappings. Each child schema is a free-standing definition with its own fields and constraints.

Child schemas describe nested data. Schema inheritance reuses definitions. A child schema need not inherit from its parent, and a derived schema need not be used as a child schema.

Declare and supply child data

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

definitions = {
    "run": {
        "variables": {"name": {"type": "String"}},
        "child_schemas": ["solver", "report"],
    },
    "solver": {"variables": {
        "iterations": {"type": "Integer", "optional": True, "default": 20},
    }},
    "report": {"variables": {
        "filename": {"type": "String"},
    }},
}

projector = SchemaProjector(definitions, schema_name="run")
projection = projector.get_projection(as_dc_instance({
    "name": "example", "solver": {},
}))

assert projection.data == {"name": "example"}
assert projection.child_projections["solver"].data == {"iterations": 20}
assert projection.data_tree == {"name": "example", "solver": {"iterations": 20}}
assert "report" not in projection.child_projections

The supplied empty Mapping for solver receives defaults and validation. The absent report entry remains absent; its required filename is not checked until report data is supplied.

Presence is explicit

A supplied Mapping can still fail because a required field has no value. Supplying a parent child Mapping does not activate absent entries for that schema’s children. The presence rule applies independently at every level.

child_schema_mode has no effect on projection. It controls only SchemaCommandLine (SCL) command navigation. Selecting a child command supplies its Mapping even without options, which requests the same default processing as an explicitly empty input Mapping.

Nest child schemas

definitions["solver"]["child_schemas"] = ["advanced"]
definitions["advanced"] = {"variables": {
    "threshold": {"type": "Float", "optional": True, "default": 0.01},
}}

projection = SchemaProjector(definitions, schema_name="run").get_projection(
    as_dc_instance({"name": "example", "solver": {"advanced": {}}})
)
advanced = projection.child_projections["solver"].child_projections["advanced"]
assert advanced.data == {"threshold": 0.01}

Validation stays local to each schema. An error for a nested DataCanon Type (DCT) mismatch includes its full data path, such as solver.advanced.threshold.

Inspect definitions and results separately

projection.child_projections contains per-input results for supplied children. Use a child projection’s data_dci, data_tree_dci, or get_instance() to work with that particular input.

projector.child_projectors contains reusable compiled child projectors. Compilation can happen during projection or through compile_child(name). Compiling a child does not supply input or activate it in later projections.

projector = SchemaProjector(definitions, schema_name="run")
projector.compile_child("solver").compile_child("advanced")
matches = projector.find_projectors_with_paths("advanced")
assert [path for path, compiled in matches] == [("solver", "advanced")]

The search covers already compiled projectors. It preserves distinct paths when a schema definition is reused in different branches, and does not fetch external content or convert DCIs to conventional objects.

SCL places an additional restriction on command definitions: a child schema cannot be directly declared by multiple parents. Standalone projection can reuse definitions across branches; SCL rejects ambiguous command ownership.

Structural restrictions

Every declared child name must identify an existing schema. A field name cannot also be a child-schema name at the same level. Child-schema cycles are rejected. A schema used as a child cannot declare non-None required_file metadata; required positional files belong to the SCL root.

The complete processed tree preserves the order of supplied child entries. For constructing objects from children, see Application construction.

5 - Application construction

DataSchemer validates input DataCanon Instances (DCIs) and applies schema defaults. Application constructors receive instances of conventional types, such as int, list, and NumPy arrays. A constructor can then perform the transformations its application requires.

Keep validation and application transformations separate

import numpy as np

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector


class Calculation:
    def __init__(self, matrix, iterations, tolerance=1e-8):
        self.matrix = np.asarray(matrix, dtype=np.float64)
        self.iterations = iterations
        self.tolerance = tolerance


schema = {
    "variables": {
        "coefficients": {"code_alias": "matrix"},
        "iterations": {"type": "Integer"},
        "tolerance": {"type": "Float", "optional": True},
    },
}

input_dci = as_dc_instance({
    "coefficients": [[1, 0], [0, 1]],
    "iterations": 20,
})
projection = SchemaProjector(schema).get_projection(input_dci)
calculation = projection.get_instance(Calculation)

assert calculation.matrix.shape == (2, 2)
assert calculation.matrix.dtype == np.dtype("float64")
assert calculation.tolerance == 1e-8
assert projection.data_dci == input_dci

The schema requires an Integer DCI for iterations. The untyped coefficients field can accept a nested List DCI or an Array DCI. Conventional conversion produces the corresponding objects; Calculation chooses its own array dtype. That transformation does not alter the processed DCI. The code_alias maps the external field name coefficients to constructor argument matrix.

Schema defaults and constructor defaults

A schema default becomes part of the processed DCI and is passed to a matching constructor argument. An absent optional field without a schema default remains absent, allowing the constructor’s own default to apply. Explicitly supplied values, including Null, take precedence over defaults when allowed by the schema.

In the example, tolerance remains absent from projection.data_dci, while the constructed object uses Calculation’s default of 1e-8. Adding "default": 1e-6 to the field definition would put that value in the processed DCI and pass it to the constructor instead.

A default does not infer a type constraint:

variables = {"value": {"optional": True, "default": 1}}

The default is an Integer DCI, but an explicitly supplied value may have any DataCanon Type (DCT). Add "type": "Integer" to constrain both supplied values and the default. DataSchemer (DS) never uses constructor annotations or constructor defaults to infer DCT constraints. A constructor default cannot satisfy a required schema field; schema validation happens before application construction.

Constructor argument selection

get_instance() accepts a class or callable factory with an inspectable signature. It selects matching local fields after applying code_alias, and passes arguments by keyword. Keyword-only parameters are supported. An omitted parameter retains its callable default.

Other local fields are ignored unless the callable accepts **kwargs, in which case additional fields are passed using their aliases. An unused external reference is not materialized merely to construct an object. Similarly, a registered Record conversion handler runs only if its value is selected for conversion.

Signature binding happens before conventional conversion. Missing constructor arguments raise DSUserError without first retrieving external payloads or calling Record conversion handlers. A required positional-only argument needs a wrapper that accepts keyword arguments:

obj = projection.get_instance(lambda matrix, iterations: factory(matrix, iterations))

A callable without an inspectable signature also needs an inspectable wrapper. Exceptions raised by the application constructor or by conventional conversion propagate to the caller; DS does not disguise them as schema validation errors.

Child projections

Root construction uses local fields only, even if the root constructor accepts **kwargs. Child data is not automatically passed to a root constructor, nor does DS choose classes for child schemas.

Construct a child object explicitly from its projection:

solver_projection = projection.child_projections["solver"]
solver = solver_projection.get_instance(Solver)

The application decides how to assemble independently constructed objects. data_tree_dci and data_tree remain available when the application wants the complete nested data instead. A child projection exists only when its Mapping was supplied; an empty supplied Mapping receives defaults and validation.

SchemaCommandLine

Passing target_class=Calculation to SchemaCommandLine uses the same construction contract and makes the result available through target_obj. Standalone projection and SchemaCommandLine (SCL) share one implementation of argument selection, alias handling, deferred conversion, and signature checking.

6 - SchemaCommandLine

SchemaCommandLine (SCL) turns schema definitions into a command-line interface. It parses option values and files into DataCanon Instances (DCIs), merges the sources, and uses SchemaProjector for validation and defaults. SCL can then construct an application object.

A complete program

Save the following as calculate.py:

import numpy as np
from data_schemer.schema_command_line import SchemaCommandLine

class Calculation:
    def __init__(self, matrix, iterations, tolerance):
        self.matrix = np.asarray(matrix, dtype=np.float64)
        self.iterations = iterations
        self.tolerance = tolerance

    @property
    def trace(self):
        return float(np.trace(self.matrix))

schema = {
    "variables": {
        "matrix": {"help": "Numerical matrix"},
        "iterations": {"type": "Integer", "help": "Maximum iteration count"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
        "input_files": {"type": "List", "optional": True},
    },
    "print_attribute": {"choices": ["trace"]},
}

if __name__ == "__main__":
    raise SystemExit(SchemaCommandLine.main(schema, target_class=Calculation))

Run it with a Shorthand matrix and request an application property:

python calculate.py --matrix '[[2, -1], [-1, 2]]' --iterations 20 --print-attribute trace

trace: 4.0

SCL validates the input, adds the tolerance default, and constructs Calculation. The class performs its own NumPy conversion. Without --print-attribute, this example constructs the object without printing it.

SchemaCommandLine.main() returns zero on success. It catches DSUserError, prints the message to stderr, and returns its exit code, normally 2. Passing debug=True to main() reraises DataSchemer (DS) user errors for debugging. Exceptions from application code propagate. Help exits normally before loading input files or constructing the application.

Option names and values

A schema field iteration_limit becomes --iteration-limit. An alias list can add short or alternate options:

field = {"type": "Integer", "alias": ["-n", "num_iterations"]}

That field would offer its normal option, -n, and --num-iterations. An option alias changes the command-line spelling. A code_alias changes the constructor argument name; the two mechanisms have separate purposes.

SCL parses ordinary option values as DataCanon (DC) Shorthand with preprocessing enabled:

python calculate.py --matrix '<2 -1; -1 2>' --iterations 20

See DataCanon input for substitutions, compact Lists, and explicit eval(...). A Sum or Product expression remains structured DC data unless preprocessing explicitly requests evaluation.

A field declared String accepts ordinary text, including numeric-looking text such as 1.0. A value beginning with a double quote is parsed as a DC String literal. Other fields accept simple bare identifiers and paths as a string convenience when Shorthand parsing fails. Malformed structured input is rejected; it is not silently turned into a String. For an untyped string containing spaces, include DC quotes inside the shell argument: '"sample label"'.

Declared types are exact DataCanon Type (DCT) constraints. --iterations 20.0 is rejected for Integer; --tolerance 1 is rejected for Float. Use 1.0 when a Float is required.

Application-defined substitutions

An application can define named input presets using a field’s substitutions property. The following field definition makes I available when entering a matrix on the command line:

schema = {
    "variables": {
        "matrix": {
            "help": "Matrix expression; I denotes the identity matrix",
            "substitutions": {"I": "array(<1 0, 0 1>)"},
        },
        "iterations": {"type": "Integer"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
    },
    "print_attribute": {"choices": ["trace"]},
}

Use this definition in calculate.py from the complete example above:

python calculate.py --matrix '2 * I' --iterations 20 --print-attribute trace

trace: 4.0

SCL passes the preset to DC preprocessing. Parsing 2 * I constructs an OrderedProduct DCI containing an Integer and an Array. Conversion to its conventional counterpart, followed by Calculation’s explicit np.asarray(..., dtype=np.float64), produces the numerical matrix. DC identity remains structural; application construction requests numerical evaluation.

The matrix field deliberately has no type constraint: both an Array DCI and an OrderedProduct DCI can supply a usable matrix to this application. Declaring type: "Array" would reject 2 * I, because an OrderedProduct is a different DCT. DS does not turn expressions into Arrays to satisfy a constraint.

Each preset value must be self-contained Shorthand text. Compact Lists and explicit eval(...) are prepared when the schema is compiled. Presets do not refer to one another. User-supplied local definitions can refer to presets:

python calculate.py --matrix 'A=I | 3 * A' --iterations 20 --print-attribute trace

A local definition cannot redefine an application preset with the same name. Names follow DC’s substitution rules, and quoted strings are protected. Replacement is textual, following DC preprocessing rules; include explicit constructors when an expression’s grouping must be retained.

Substitution scope is one field. For nargs, the same presets apply to every supplied item. Declared String fields retain their literal-text input policy. Defaults, file contents, and programmatically supplied DCIs receive no application substitutions. A file can still declare its own local substitutions when its format supports preprocessing.

Inheritance and copying preserve substitutions like other field properties. Overriding the property replaces the whole preset mapping; DS does not merge individual preset names. Invalid names, nontext values, and malformed preset text are reported with the field name during schema compilation.

Boolean flags and multiple option values

A Boolean field normally becomes a flag that supplies true when present. Omission leaves the field absent for schema processing, where a default may apply. Set store_false: True for a false flag, or store_true: False to require an explicit Boolean value such as --enabled false.

nargs controls option arity independently of its DCT constraint. For example:

field = {"type": "List", "nargs": "+", "optional": True}

This option consumes one or more shell arguments and builds a List DCI. Each item is parsed as Shorthand. A List field without nargs can instead accept one whole Shorthand List expression. nargs: "?" permits an option with no value; it then supplies the configured const, or Null when no const is configured. That explicitly supplied Null is distinct from an omitted option.

Input files and source precedence

Declare an optional input_files field to enable --input-files. A List DCT constraint is suitable; SCL supplies a default nargs: "+" for this control. An alternative control name can be passed as the constructor’s input_files_tag.

python calculate.py --input-files baseline.yaml overrides.dcs --iterations 40

Each file must represent a Mapping DCI with String field keys. Supported formats:

Source precedence, from lowest to highest, is:

1. Files selected by --input-files, in their supplied order.
2. A required positional file configured with apply_as_schema_data: True.
3. Command-line options and selected child commands.

Later values replace earlier values. Declared child-schema Mappings merge recursively, allowing a command-line option to override one child field while retaining its siblings from a file. An ordinary Mapping-valued field is replaced as a whole. Existing field positions are retained; new fields append.

Schema defaults fill remaining absent fields after sources are merged. SCL retains source context on external references without fetching their payloads. The input_files control itself remains in the input and processed local data.

For example, baseline.yaml for the program above can contain:

matrix:
  - [2, -1]
  - [-1, 2]
iterations: 20

Standard input and required positional files

Use - in the input-file list to read structured input from stdin. The default stdin format is DC YAML; pass stdin_format="shorthand" to the SCL constructor when the application wants Shorthand instead. Empty structured text input denotes an empty Mapping. Stdin can be consumed only once per invocation.

A root schema can require a positional file:

required_file = {
    "enabled": True,
    "metavar": "FILE",
    "help": "Primary input file",
    "read_mode": "path",
    "stdin_ok": True,
    "apply_as_schema_data": True,
}

Place that dictionary under the schema-level key required_file. With apply_as_schema_data: True, SCL loads structured data by suffix, merges it at the precedence above, and leaves required_file_content as None.

Without structured loading, read_mode selects application-specific handling: path retains the path without reading, text reads UTF-8 text, and binary reads bytes. Access the path and content through required_file_path and required_file_content. Child schemas cannot declare required_file metadata.

Child commands

Declared child schemas become bare command tokens. Schema name post_process appears as command post-process. Options after a child token belong to that child; place root options before descending into a child.

definitions = {
    "run": {"child_schemas": ["solver", "report"], "child_schema_mode": "inclusive"},
    "solver": {"variables": {
        "iterations": {"type": "Integer", "optional": True, "default": 20},
    }},
    "report": {"variables": {"filename": {"type": "String"}}},
}

scl = SchemaCommandLine(
    definitions, schema_name="run",
    argv=["solver", "report", "--filename", "results.txt"],
)
assert scl.data_tree == {"solver": {"iterations": 20}, "report": {"filename": "results.txt"}}

The solver token supplies an empty child Mapping, which receives its default. No command or file entry for a child means that child remains absent.

child_schema_mode controls navigation only:

• exclusive (default) permits descent into child commands and their descendants.
• inclusive additionally permits a one-level sibling jump between that schema’s children.

SCL does not search across multiple ancestors for a sibling. Both navigation modes use the same explicit child-data presence rules.

Required option values are consumed before command recognition. A String value matching a child command stays a value. For variable-arity options, a command can begin after the option’s minimum arity is satisfied. Use --option=value to disambiguate a first value, or supply a whole List expression as one argument.

Help lists child commands and their schema descriptions. The schema-level child_schema_label can change that help group’s heading.

Application construction and operations

With target_class, SCL constructs an object using selected local fields and exposes it as target_obj. Child objects are constructed explicitly from child_projections. See Application construction.

Subclasses can override _operations(). SCL calls it after successful projection and optional construction, then handles requested attribute output. The hook can inspect data_dci, convert selected data, use target_obj, or write results.

SCL also exposes input_dci, data, data_tree_dci, data_tree, child_data, and child_projections. Conventional views are computed on access.

Print application attributes

The complete program enables --print-attribute trace. Default output uses DC YAML interchange, preserving supported DCI distinctions with $ airlocks. Attributes are read after _operations() runs.

For automatic property discovery, external names, and exclusions, see Attribute output.

Tab completion

Install argcomplete and register your application’s installed executable:

eval "$(register-python-argcomplete my-program)"

Replace my-program with the executable that invokes SCL. DS does not install a universal command named my-program. Completion offers options, declared choices, and child commands in the current navigation context. It runs before input files are loaded or the target application is constructed.

7 - Attribute output

SchemaCommandLine (SCL) can print selected attributes of the constructed application object. The schema-level print_attribute setting enables --print-attribute. A target_class is required, and print_attribute is reserved as an injected SCL control field.

Discover public properties

from data_schemer.schema_command_line import SchemaCommandLine

class Summary:
    def __init__(self, values):
        self.values = values

    @property
    def mean(self):
        return sum(self.values) / len(self.values)

    @property
    def maximum(self):
        return max(self.values)

schema = {
    "variables": {"values": {}},
    "print_attribute": True,
}

scl = SchemaCommandLine(
    schema, target_class=Summary,
    argv=["--values", "[1, 2, 3]", "--print-attribute", "mean"],
)

mean: 2.0

Automatic discovery exposes public @property names, including inherited properties, including functools.cached_property descriptors. It does not discover every instance attribute. The requested attribute is read after application construction and _operations().

Select external names

Use a configuration dictionary for explicit choices or aliases:

schema["print_attribute"] = {
    "choices": ["average", "maximum"],
    "code_alias": {"average": "mean"},
}
scl = SchemaCommandLine(
    schema, target_class=Summary,
    argv=["--values", "[1, 2, 3]", "--print-attribute", "average"],
)

average: 2.0

The command-line spelling is external; mean is the internal attribute name. Schema field code_alias mappings also participate in attribute-name mapping. The explicit output mapping must not overlap their external names or internal targets. Attribute aliases must be one-to-one.

When choices is a list, it controls validation and completion. Manual choices can name attributes that are not @property descriptors, provided they exist on the constructed object.

Automatic exclusions

schema["print_attribute"] = {"exclude": ["maximum"]}

Exclusions apply during automatic discovery only when the configuration has no choices key. An excluded internal name or its external alias removes the property. An explicit choices: None selects automatic discovery but suppresses exclusions; key presence is significant.

Output representation

Default attribute output uses DataCanon YAML interchange. A supported value that YAML cannot represent faithfully uses a $ Shorthand airlock. Each requested attribute is printed separately under its external name.

Set omit_key: True to print just the value, still using DataCanon (DC) YAML interchange. For example:

schema["print_attribute"] = {"choices": ["mean"], "omit_key": True}

This output path converts the application attribute using DC’s conventional conversion rules. The original input DataCanon Instance (DCI) is not changed.

Application reports

SCL always uses DC YAML interchange for attribute output. An application that wants rounded tables, labels, or a specialized report should format that output in its own code, for example in _operations(). The former precision_from and array_style_from settings are rejected with migration guidance.

The behavioral reference describes output-configuration inheritance.

8 - Output and application reports

SchemaCommandLine (SCL) uses DataCanon (DC) YAML interchange for attribute output. Use DC’s renderers and persistence writers when working with DataCanon Instances (DCIs). Application code is responsible for rounded tables, labels, or other specialized reports.

Render or store a DCI

from data_canon import as_dc_instance, render_shorthand, render_yaml

node = as_dc_instance({"samples": {("sensor", 0): [1.0, 2.0]}})
print(render_shorthand(node))
print(render_yaml(node))

Use write(node, filename="result.dcb") for canonical binary persistence. Shorthand and YAML interchange provide readable representations of the DCI. SCL’s --print-attribute uses that same YAML interchange, including when omit_key requests the value without its external-name wrapper.

Format application reports

An application may choose to round numbers, arrange rows into tables, or add labels. Perform those operations explicitly in application code. An SCL subclass can write such a report in its _operations() hook after construction.

A numerical report is a presentation chosen by the application. If the original DCI must also be retained, store it separately using DC persistence.

Removed formatting interfaces

The former DataSchemer (DS) numpy_print.py and render_data.py modules have been removed, along with their array, Mapping, and labeled-table formatters. SCL no longer supports the precision constructor argument, its precision property or set_precision() method. The precision_from and array_style_from output settings are rejected with migration guidance.

This page remains at its previous URL so existing documentation links continue to resolve. See Attribute output for the supported SCL output configuration.

9 - Migrating from numeric coercion

The DataSchemer (DS) processor accepts DataCanon Instances (DCIs) and replaces the former numeric coercion engine. data_schemer.coerce_numbers and the temporary data_schemer.dci_projector module have been removed. The public processor is data_schemer.schema_projector.SchemaProjector.

Move parsing to the input boundary

from data_canon import parse_shorthand
from data_schemer.schema_projector import SchemaProjector

input_dci = parse_shorthand('{"matrix": <1 0; 0 1>}', preprocess=True)
projector = SchemaProjector({"variables": {"matrix": {}}})
projection = projector.get_projection(input_dci)
assert projection.data == {"matrix": [[1, 0], [0, 1]]}

A standalone projector receives a DCI. SchemaCommandLine (SCL) handles text parsing at its own boundary and then invokes that same processor.

Replace the former contracts

Field-property inheritance is now explicit: the first listed base wins explicit property conflicts, and local properties override inherited ones. A supplied child Mapping is projected; an absent child stays absent. SCL’s inclusive navigation does not activate children.

The old numpy_print.py and render_data.py formatting modules and SCL’s numerical report controls have also been removed. Use DC rendering for interchange and application code for specialized reports.

The input guide and projection guide cover the new workflow. The behavioral reference records the resolved rules.