DataSchemer

• 1: Getting Started
• 1.1: Installation
• 2: Guides
• 2.1: DataCanon input
• 2.2: Schema projection
• 2.3: Schema composition
• 2.4: Child schemas
• 2.5: Application construction
• 2.6: SchemaCommandLine
• 2.7: Attribute output
• 2.8: Output and application reports
• 2.9: Migrating from numeric coercion
• 3: Reference
• 3.1: Terminology
• 3.2: Behavioral rules
• 3.3: Python API
• 3.4: Errors

schema = {
    "variables": {
        "matrix": {},
        "iterations": {"type": "Integer"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
    },
}

1 - Getting Started

DataSchemer (DS) validates DataCanon Instances (DCIs) against an application’s schema. A DCI is a value in the DataCanon data model. The same schema can serve a Python interface, configuration files, and SchemaCommandLine (SCL).

Follow the installation instructions first. This example uses DataCanon and NumPy alongside DS.

Define the expected input

Suppose an application needs a matrix, an iteration count, and a tolerance.

schema = {
    "variables": {
        "matrix": {},
        "iterations": {"type": "Integer"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
    },
}

Fields are required unless marked optional. A type constraint is optional: matrix can be any DCI, while iterations must be an Integer DCI. The application will choose how to turn the matrix data into its preferred numerical container.

Supply a DCI and project it

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

input_dci = as_dc_instance({
    "matrix": [[2, -1], [-1, 2]],
    "iterations": 20,
})

projector = SchemaProjector(schema)
projection = projector.get_projection(input_dci)

assert projection.input_dci is input_dci
assert projection.data_dci == as_dc_instance({
    "matrix": [[2, -1], [-1, 2]],
    "iterations": 20,
    "tolerance": 1e-8,
})

The original input remains available through input_dci. The processed data_dci includes the default tolerance. Unknown fields, missing required fields, and incorrect declared DataCanon Types (DCTs) raise user-facing errors.

Access conventional objects

data = projection.data
print(data)

{'matrix': [[2, -1], [-1, 2]], 'iterations': 20, 'tolerance': 1e-08}

data converts the processed fields to instances of conventional Python types. Here those types are dict, list, int, and float. The authoritative processed DCI remains available separately through data_dci.

Construct the application object

import numpy as np

class Calculation:
    def __init__(self, matrix, iterations, tolerance):
        self.matrix = np.asarray(matrix, dtype=np.float64)
        self.iterations = iterations
        self.tolerance = tolerance

calculation = projection.get_instance(Calculation)

assert calculation.matrix.shape == (2, 2)
assert calculation.matrix.dtype == np.dtype("float64")
assert calculation.iterations == 20
assert calculation.tolerance == 1e-8

DS selects the constructor arguments and converts their DCIs. Calculation chooses its own array representation. A nested List DCI or an Array DCI can therefore serve as the matrix input without a DS-specific matrix type.

Use the same schema with SCL

from data_schemer.schema_command_line import SchemaCommandLine

scl = SchemaCommandLine(
    schema,
    argv=["--matrix", "[[2, -1], [-1, 2]]", "--iterations", "20"],
    target_class=Calculation,
)

assert scl.target_obj.iterations == 20
assert scl.data_dci == projection.data_dci

SCL parses the option values as DataCanon Shorthand before calling the same schema processor. A program normally omits argv to use its actual command line.

Continue

• DataCanon input covers Shorthand, preprocessing, and input files.
• Schema projection covers constraints and output views.
• Application construction explains aliases, defaults, and deferred conversion.
• SchemaCommandLine builds a complete command-line program.

1.1 - Installation

Install the current DataCanon and DataSchemer checkouts into the same environment. Both projects use Meson through meson-python. Python 3.10 or newer is required.

From a directory containing the two code repositories:

python3 -m venv DS
source DS/bin/activate
python -m pip install --upgrade pip
python -m pip install meson meson-python ninja pytest
python -m pip install -e ./data_canon --no-build-isolation
python -m pip install -e ./data_schemer --no-build-isolation

Adjust the checkout paths to match your filesystem. The code repositories are DataCanon and DataSchemer.

Editable installation makes changes in the checkouts available during development. Run the DataSchemer (DS) suite from its code repository:

cd data_schemer
python -m pytest

For shell completion in an application built with SchemaCommandLine, install argcomplete into the same environment:

python -m pip install argcomplete

Register the application’s executable with your shell as described in the SchemaCommandLine (SCL) guide.

2 - Guides

Start with Getting Started, then choose the guide for the boundary you are working on.

• DataCanon input: create DataCanon Instances (DCIs) from conventional objects, Shorthand, or files.
• Schema projection: optional DataCanon Type (DCT) constraints, defaults, choices, and processed output.
• Schema composition: inheritance, field-property overrides, copying, and removal.
• Child schemas: validate nested data while keeping inheritance separate.
• Application construction: conventional conversion and constructor arguments.
• SchemaCommandLine: options, files, nested commands, completion, and attribute output.
• Attribute output: property discovery, external names, and output controls.
• Output and application reports: DataCanon (DC) output and application-owned presentation.
• Migrating from numeric coercion: replace the former DataSchemer (DS) input model.

The Reference collects the behavioral rules, Python interfaces, and error details.

2.1 - DataCanon input

A DataCanon Instance (DCI) is a value in DataCanon’s data model. SchemaProjector accepts a Mapping DCI whose keys are String DCIs. The values already have DataCanon Types (DCTs); the schema can optionally constrain them. Parsing text and converting conventional objects happen before projection.

From conventional objects

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

projector = SchemaProjector({"variables": {"matrix": {}, "iterations": {"type": "Integer"}}})
input_dci = as_dc_instance({"matrix": [[2, -1], [-1, 2]], "iterations": 20})
projection = projector.get_projection(input_dci)

as_dc_instance() applies DataCanon’s conversion rules. The resulting DCI is the input to DataSchemer (DS). Passing the conventional dictionary directly to get_projection() is an error.

The String-key restriction applies to the Mapping that supplies schema fields and to Mappings supplying child-schema data. A Mapping used as a field value can retain arbitrary DCI keys:

input_dci = as_dc_instance({"samples": {("sensor", 0): [1.0, 2.0]}})
projection = SchemaProjector({"variables": {"samples": {}}}).get_projection(input_dci)
assert projection.data["samples"][("sensor", 0)] == [1.0, 2.0]

From Shorthand

from data_canon import parse_shorthand

input_dci = parse_shorthand('{"matrix": [[2, -1], [-1, 2]], "iterations": 20}')
projection = projector.get_projection(input_dci)

Shorthand preserves structured numerical DCIs. For example, sqrt(3) describes a Power DCI, and 1/2 * sqrt(3) describes a Product containing a Fraction and a Power. DataCanon (DC) does not perform mathematical simplification during parsing.

Optional input preprocessing

Preprocessing makes manually written input easier to maintain. Enable it explicitly when calling the DC parser:

source = """
diagonal = 2
off_diagonal = -1 |
{
  "matrix": <diagonal off_diagonal; off_diagonal diagonal>,
  "iterations": 20
}
"""
input_dci = parse_shorthand(source, preprocess=True)
projection = projector.get_projection(input_dci)
assert projection.data["matrix"] == [[2, -1], [-1, 2]]

Substitutions insert DC structure. Compact List notation establishes nested Lists. An explicit eval(...) requests scalar mathematical evaluation during preprocessing:

input_dci = parse_shorthand(
    'side=2.5 | {"area": eval(side * side)}',
    preprocess=True,
)
projection = SchemaProjector({"variables": {"area": {}}}).get_projection(input_dci)
assert projection.data["area"] == 6.25

Preprocessing belongs to DC’s input facilities. SchemaProjector receives the resulting DCI and does not evaluate text inside String DCIs.

Application-defined command-line presets are available through SCL’s per-field substitutions property. See Application-defined substitutions for an identity-matrix example using 2 * I. Presets affect command-line option values; SchemaProjector continues to receive DCIs without interpreting strings.

From files

For a standalone projector, load the DCI before projection:

from data_canon import read

input_dci = read("calculation.dcb")
projection = projector.get_projection(input_dci)

DC’s read() handles canonical binary .dcb, strict Shorthand .dcs, and DC HDF5 files. For a standalone preprocessed Shorthand file, read its text and call parse_shorthand(..., preprocess=True) explicitly. If external resources are present, use an appropriate source when converting conventional output; DC’s persistence readers retain source context automatically.

SchemaCommandLine (SCL) provides an additional input boundary. It accepts .dcs with preprocessing, .dcb, DC .h5/.hdf5, and DC YAML .yaml/.yml. Its source loading retains context for deferred external-resource access. See SCL input files.

YAML interchange

Use DC’s YAML interchange when importing YAML directly:

from data_canon import parse_yaml

input_dci = parse_yaml("""
iterations: 20
samples:
  $ ("sensor", 0): [1.0, 2.0]
phase: $ complex(1/2, 1/2 * sqrt(3))
""")
projection = SchemaProjector({"variables": {
    "iterations": {"type": "Integer"}, "samples": {}, "phase": {},
}}).get_projection(input_dci)

The $ marker embeds strict DC Shorthand in YAML scalars, including Mapping keys. YAML sequences become Lists. YAML airlocks do not run Shorthand preprocessing.

For DC syntax, representations, and custom conversion registries, consult the DataCanon documentation.

2.2 - Schema projection

SchemaProjector compiles a schema for repeated use. A DataCanon Instance (DCI) is a value in DataCanon’s data model. Calling get_projection(input_dci) validates one Mapping DCI and returns a SchemaProjection containing the original input and processed output.

Compile once, project multiple inputs

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

schema = {"variables": {
    "iterations": {"type": "Integer"},
    "method": {"optional": True, "default": "direct", "choices": ["direct", "iterative"]},
}}
projector = SchemaProjector(schema)
first = projector.get_projection(as_dc_instance({"iterations": 10}))
second = projector.get_projection(as_dc_instance({"iterations": 20, "method": "iterative"}))

assert first.data == {"iterations": 10, "method": "direct"}
assert second.data == {"iterations": 20, "method": "iterative"}

Schema compilation resolves field definitions and validates their metadata. Projection validates each input separately. A projector can also receive an initial input through the keyword-only input_data argument; its projection, data_dci, and data properties then refer to that initial projection.

Optional type constraints

type names a DataCanon Type (DCT) using its canonical name, such as Integer, Float, List, or Array. The names are case-sensitive. Omitting type accepts any DCI.

schema = {"variables": {
    "iterations": {"type": "Integer"},
    "matrix": {},
}}

The processor rejects a Float DCI for iterations, even if its value is 20.0. An Integer DCI is likewise rejected for a field constrained to Float. Boolean and Integer remain distinct DCTs. DataSchemer (DS) performs no implicit numerical coercion.

The matrix field can accept a nested List, an Array, or another DCI. The application constructor is responsible for the domain-specific interpretation.

Constrain an Array’s element type

An Array DCI has a portable Array element type in addition to its DCT. Declare array_type when the application requires a particular one:

import numpy as np

array_projector = SchemaProjector({"variables": {
    "matrix": {"type": "Array", "array_type": "float64"},
}})
node = as_dc_instance({"matrix": np.eye(2, dtype=np.float64)})
assert array_projector.get_projection(node).data_dci == node

This accepts only an Array DCI with portable type float64. A float32 Array fails, as does a nested List. Validation preserves the supplied shape and elements. Omitting array_type allows every portable Array element type.

Supported names are bool, int8, int16, int32, int64, uint8, uint16, uint32, uint64, float16, float32, float64, complex64, and complex128. The names are case-sensitive DC names; NumPy aliases such as f8 are rejected.

array_type requires an explicit type: "Array"; it does not infer a DCT. The constraint applies to defaults and supplied values, including through SCL. It follows normal field-property inheritance, copying, overrides, and removal. It is checked after final schema composition, so delete_properties can remove an inherited constraint. ExternalArray is a separate DCT and is not covered by this Array constraint.

Required fields and defaults

Fields are required by default. Mark a field optional: True to permit absence. Only optional fields may declare a default.

schema = {"variables": {
    "iterations": {"type": "Integer"},
    "tolerance": {"optional": True, "default": 1e-8},
    "label": {"optional": True},
}}

An absent tolerance receives the default; an absent label stays absent. Supplying a default does not infer a DCT constraint. Add "type": "Float" if tolerance should accept only Float DCIs.

Defaults are converted to DCIs during compilation. Explicit defaults can already be DCIs. A supplied value always takes precedence over the default, including Null or false when allowed by the field’s constraints.

The original input is preserved. Defaults appear only in the processed result.

Choices, dependencies, and conflicts

choices compares values using DCI equality. Choice values may be conventional objects or explicit DCIs; schema compilation converts them to DCIs. An Integer choice does not admit the corresponding Float DCI.

requires lists companion fields that must be present when the field is present. Schema-level conflicts lists groups in which at most one field may be present.

schema = {
    "variables": {
        "method": {"optional": True, "default": "direct", "choices": ["direct", "iterative"]},
        "restart": {"optional": True, "requires": ["checkpoint"]},
        "checkpoint": {"optional": True},
        "fresh_start": {"type": "Boolean", "optional": True},
    },
    "conflicts": [["restart", "fresh_start"]],
}

Defaults are present for requires and conflicts. A false Boolean or Null is also present; presence is determined by the field’s existence, not its truth value. Unknown fields are rejected at every projected schema level.

Original input, local output, and complete output

Without child schemas, the local and complete processed views have the same contents. Supplied field order is preserved; added defaults follow in schema order. Child Mapping positions are preserved in the complete output.

Conventional views are computed on access and are not cached. Mutating a returned dictionary does not mutate the processed Mapping DCI. Conventional conversion follows DataCanon (DC)’s rules and may simplify values according to their destination types.

Deferred external resources and custom Records

Projection checks external-reference metadata without fetching the payloads. It also leaves custom Record conversion handlers uncalled. Accessing data, data_tree, or selected constructor arguments can retrieve external resources or invoke explicitly registered handlers.

Pass registry and source to the projector when needed:

# registry is a configured DataCanon ConversionRegistry.
# source is the namespace containing any referenced external resources.
projector = SchemaProjector(schema, registry=registry, source=source)

Those arguments are forwarded to DC conversion. Conversion failures are separate from successful schema validation.

Handle user errors

from data_schemer.errors import DSUserError

try:
    projection = projector.get_projection(input_dci)
except DSUserError as error:
    print(error.user_message())

See Errors for DCT diagnostics and exception classes. Continue with schema composition, child schemas, or application construction.

2.3 - Schema composition

Schema inheritance reuses definitions. A derived schema can adjust field properties, copy field definitions, or remove constraints. Inheritance does not create nested input data; that is the role of child schemas.

Inherit field properties

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

definitions = {
    "base": {"variables": {
        "iterations": {
            "type": "Integer", "optional": True, "default": 20,
            "help": "Maximum iteration count",
        },
    }},
    "production": {
        "inherit": ["base"],
        "variables": {"iterations": {"default": 100}},
    },
}

projector = SchemaProjector(definitions, schema_name="production")
projection = projector.get_projection(as_dc_instance({}))
assert projection.data == {"iterations": 100}
assert projector.schema_variables["iterations"]["type"] == "Integer"

The local default replaces the inherited default while type, optional, and help remain inherited. Overrides merge field properties. Each property’s value is replaced as a whole: a new choices list replaces the inherited list, and a new Mapping default replaces the inherited Mapping default.

For inherit: ["first", "second"], the first base wins when both explicitly define the same property. The second base fills properties absent from the first. Local variables properties take precedence over every base. Implicit defaults such as optional: False are added after composition.

Copy a field definition

definitions["production"]["copy_variables"] = {"report_limit": "base@iterations"}
definitions["production"]["variables"]["report_limit"] = {
    "default": 5,
    "help": "Maximum number of reported entries",
}
projector = SchemaProjector(definitions, schema_name="production")
assert projector.get_projection(as_dc_instance({})).data == {
    "iterations": 100, "report_limit": 5,
}

copy_variables maps a destination name to "schema@field". The source field uses its resolved definition, including inheritance. A copy replaces an inherited destination definition, then local properties for the destination override the copied properties.

Same-schema copies can refer to later copy entries. Cyclic copies and cyclic schema inheritance are rejected. Copies run before removals, so a same-schema source may be copied and then removed from the final schema.

Remove properties or fields

definitions["flexible"] = {
    "inherit": ["base"],
    "delete_properties": {"iterations": ["type", "default"]},
}
projector = SchemaProjector(definitions, schema_name="flexible")
assert projector.get_projection(as_dc_instance({})).data == {}
assert projector.get_projection(as_dc_instance({"iterations": "automatic"})).data == {
    "iterations": "automatic",
}

The field remains optional, but its type constraint and default are removed. Use delete_variables: ["iterations"] to remove the whole field definition. Removing an absent field or property is an error, catching misspelled names.

Setting default to None supplies a Null DataCanon Instance (DCI) default. It does not remove a default. Use delete_properties for removal.

Resolution order

For each schema, DataSchemer (DS) composes inherited field properties, applies local overrides and copies, then removes properties and fields. The copied destination always receives its local overrides after the copy. Implicit property defaults and final validation follow composition and removal.

The schema’s original definition dictionaries are not modified by field resolution. A schema_variables property returns a copy of the compiled metadata.

update_variables has been removed. Put property overrides directly under variables. An occurrence of update_variables, including an empty Mapping, raises a migration error.

Other inherited metadata

child_schemas accumulates in base order, followed by locally declared names, with duplicates removed. conflicts includes inherited groups.

print_attribute has its own merge rules for output configuration, including extending lists of choices and exclusions. Those rules are separate from ordinary field-property replacement; see the behavioral reference.

2.4 - Child schemas

A schema can declare child schemas to validate nested input Mappings. Each child schema is a free-standing definition with its own fields and constraints.

Child schemas describe nested data. Schema inheritance reuses definitions. A child schema need not inherit from its parent, and a derived schema need not be used as a child schema.

Declare and supply child data

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector

definitions = {
    "run": {
        "variables": {"name": {"type": "String"}},
        "child_schemas": ["solver", "report"],
    },
    "solver": {"variables": {
        "iterations": {"type": "Integer", "optional": True, "default": 20},
    }},
    "report": {"variables": {
        "filename": {"type": "String"},
    }},
}

projector = SchemaProjector(definitions, schema_name="run")
projection = projector.get_projection(as_dc_instance({
    "name": "example", "solver": {},
}))

assert projection.data == {"name": "example"}
assert projection.child_projections["solver"].data == {"iterations": 20}
assert projection.data_tree == {"name": "example", "solver": {"iterations": 20}}
assert "report" not in projection.child_projections

The supplied empty Mapping for solver receives defaults and validation. The absent report entry remains absent; its required filename is not checked until report data is supplied.

Presence is explicit

A supplied Mapping can still fail because a required field has no value. Supplying a parent child Mapping does not activate absent entries for that schema’s children. The presence rule applies independently at every level.

child_schema_mode has no effect on projection. It controls only SchemaCommandLine (SCL) command navigation. Selecting a child command supplies its Mapping even without options, which requests the same default processing as an explicitly empty input Mapping.

Nest child schemas

definitions["solver"]["child_schemas"] = ["advanced"]
definitions["advanced"] = {"variables": {
    "threshold": {"type": "Float", "optional": True, "default": 0.01},
}}

projection = SchemaProjector(definitions, schema_name="run").get_projection(
    as_dc_instance({"name": "example", "solver": {"advanced": {}}})
)
advanced = projection.child_projections["solver"].child_projections["advanced"]
assert advanced.data == {"threshold": 0.01}

Validation stays local to each schema. An error for a nested DataCanon Type (DCT) mismatch includes its full data path, such as solver.advanced.threshold.

Inspect definitions and results separately

projection.child_projections contains per-input results for supplied children. Use a child projection’s data_dci, data_tree_dci, or get_instance() to work with that particular input.

projector.child_projectors contains reusable compiled child projectors. Compilation can happen during projection or through compile_child(name). Compiling a child does not supply input or activate it in later projections.

projector = SchemaProjector(definitions, schema_name="run")
projector.compile_child("solver").compile_child("advanced")
matches = projector.find_projectors_with_paths("advanced")
assert [path for path, compiled in matches] == [("solver", "advanced")]

The search covers already compiled projectors. It preserves distinct paths when a schema definition is reused in different branches, and does not fetch external content or convert DCIs to conventional objects.

SCL places an additional restriction on command definitions: a child schema cannot be directly declared by multiple parents. Standalone projection can reuse definitions across branches; SCL rejects ambiguous command ownership.

Structural restrictions

Every declared child name must identify an existing schema. A field name cannot also be a child-schema name at the same level. Child-schema cycles are rejected. A schema used as a child cannot declare non-None required_file metadata; required positional files belong to the SCL root.

The complete processed tree preserves the order of supplied child entries. For constructing objects from children, see Application construction.

2.5 - Application construction

DataSchemer validates input DataCanon Instances (DCIs) and applies schema defaults. Application constructors receive instances of conventional types, such as int, list, and NumPy arrays. A constructor can then perform the transformations its application requires.

Keep validation and application transformations separate

import numpy as np

from data_canon import as_dc_instance
from data_schemer.schema_projector import SchemaProjector


class Calculation:
    def __init__(self, matrix, iterations, tolerance=1e-8):
        self.matrix = np.asarray(matrix, dtype=np.float64)
        self.iterations = iterations
        self.tolerance = tolerance


schema = {
    "variables": {
        "coefficients": {"code_alias": "matrix"},
        "iterations": {"type": "Integer"},
        "tolerance": {"type": "Float", "optional": True},
    },
}

input_dci = as_dc_instance({
    "coefficients": [[1, 0], [0, 1]],
    "iterations": 20,
})
projection = SchemaProjector(schema).get_projection(input_dci)
calculation = projection.get_instance(Calculation)

assert calculation.matrix.shape == (2, 2)
assert calculation.matrix.dtype == np.dtype("float64")
assert calculation.tolerance == 1e-8
assert projection.data_dci == input_dci

The schema requires an Integer DCI for iterations. The untyped coefficients field can accept a nested List DCI or an Array DCI. Conventional conversion produces the corresponding objects; Calculation chooses its own array dtype. That transformation does not alter the processed DCI. The code_alias maps the external field name coefficients to constructor argument matrix.

Schema defaults and constructor defaults

A schema default becomes part of the processed DCI and is passed to a matching constructor argument. An absent optional field without a schema default remains absent, allowing the constructor’s own default to apply. Explicitly supplied values, including Null, take precedence over defaults when allowed by the schema.

In the example, tolerance remains absent from projection.data_dci, while the constructed object uses Calculation’s default of 1e-8. Adding "default": 1e-6 to the field definition would put that value in the processed DCI and pass it to the constructor instead.

A default does not infer a type constraint:

variables = {"value": {"optional": True, "default": 1}}

The default is an Integer DCI, but an explicitly supplied value may have any DataCanon Type (DCT). Add "type": "Integer" to constrain both supplied values and the default. DataSchemer (DS) never uses constructor annotations or constructor defaults to infer DCT constraints. A constructor default cannot satisfy a required schema field; schema validation happens before application construction.

Constructor argument selection

get_instance() accepts a class or callable factory with an inspectable signature. It selects matching local fields after applying code_alias, and passes arguments by keyword. Keyword-only parameters are supported. An omitted parameter retains its callable default.

Other local fields are ignored unless the callable accepts **kwargs, in which case additional fields are passed using their aliases. An unused external reference is not materialized merely to construct an object. Similarly, a registered Record conversion handler runs only if its value is selected for conversion.

Signature binding happens before conventional conversion. Missing constructor arguments raise DSUserError without first retrieving external payloads or calling Record conversion handlers. A required positional-only argument needs a wrapper that accepts keyword arguments:

obj = projection.get_instance(lambda matrix, iterations: factory(matrix, iterations))

A callable without an inspectable signature also needs an inspectable wrapper. Exceptions raised by the application constructor or by conventional conversion propagate to the caller; DS does not disguise them as schema validation errors.

Child projections

Root construction uses local fields only, even if the root constructor accepts **kwargs. Child data is not automatically passed to a root constructor, nor does DS choose classes for child schemas.

Construct a child object explicitly from its projection:

solver_projection = projection.child_projections["solver"]
solver = solver_projection.get_instance(Solver)

The application decides how to assemble independently constructed objects. data_tree_dci and data_tree remain available when the application wants the complete nested data instead. A child projection exists only when its Mapping was supplied; an empty supplied Mapping receives defaults and validation.

SchemaCommandLine

Passing target_class=Calculation to SchemaCommandLine uses the same construction contract and makes the result available through target_obj. Standalone projection and SchemaCommandLine (SCL) share one implementation of argument selection, alias handling, deferred conversion, and signature checking.

2.6 - SchemaCommandLine

SchemaCommandLine (SCL) turns schema definitions into a command-line interface. It parses option values and files into DataCanon Instances (DCIs), merges the sources, and uses SchemaProjector for validation and defaults. SCL can then construct an application object.

A complete program

Save the following as calculate.py:

import numpy as np
from data_schemer.schema_command_line import SchemaCommandLine

class Calculation:
    def __init__(self, matrix, iterations, tolerance):
        self.matrix = np.asarray(matrix, dtype=np.float64)
        self.iterations = iterations
        self.tolerance = tolerance

    @property
    def trace(self):
        return float(np.trace(self.matrix))

schema = {
    "variables": {
        "matrix": {"help": "Numerical matrix"},
        "iterations": {"type": "Integer", "help": "Maximum iteration count"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
        "input_files": {"type": "List", "optional": True},
    },
    "print_attribute": {"choices": ["trace"]},
}

if __name__ == "__main__":
    raise SystemExit(SchemaCommandLine.main(schema, target_class=Calculation))

Run it with a Shorthand matrix and request an application property:

python calculate.py --matrix '[[2, -1], [-1, 2]]' --iterations 20 --print-attribute trace

trace: 4.0

SCL validates the input, adds the tolerance default, and constructs Calculation. The class performs its own NumPy conversion. Without --print-attribute, this example constructs the object without printing it.

SchemaCommandLine.main() returns zero on success. It catches DSUserError, prints the message to stderr, and returns its exit code, normally 2. Passing debug=True to main() reraises DataSchemer (DS) user errors for debugging. Exceptions from application code propagate. Help exits normally before loading input files or constructing the application.

Option names and values

A schema field iteration_limit becomes --iteration-limit. An alias list can add short or alternate options:

field = {"type": "Integer", "alias": ["-n", "num_iterations"]}

That field would offer its normal option, -n, and --num-iterations. An option alias changes the command-line spelling. A code_alias changes the constructor argument name; the two mechanisms have separate purposes.

SCL parses ordinary option values as DataCanon (DC) Shorthand with preprocessing enabled:

python calculate.py --matrix '<2 -1; -1 2>' --iterations 20

See DataCanon input for substitutions, compact Lists, and explicit eval(...). A Sum or Product expression remains structured DC data unless preprocessing explicitly requests evaluation.

A field declared String accepts ordinary text, including numeric-looking text such as 1.0. A value beginning with a double quote is parsed as a DC String literal. Other fields accept simple bare identifiers and paths as a string convenience when Shorthand parsing fails. Malformed structured input is rejected; it is not silently turned into a String. For an untyped string containing spaces, include DC quotes inside the shell argument: '"sample label"'.

Declared types are exact DataCanon Type (DCT) constraints. --iterations 20.0 is rejected for Integer; --tolerance 1 is rejected for Float. Use 1.0 when a Float is required.

Application-defined substitutions

An application can define named input presets using a field’s substitutions property. The following field definition makes I available when entering a matrix on the command line:

schema = {
    "variables": {
        "matrix": {
            "help": "Matrix expression; I denotes the identity matrix",
            "substitutions": {"I": "array(<1 0, 0 1>)"},
        },
        "iterations": {"type": "Integer"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
    },
    "print_attribute": {"choices": ["trace"]},
}

Use this definition in calculate.py from the complete example above:

python calculate.py --matrix '2 * I' --iterations 20 --print-attribute trace

trace: 4.0

SCL passes the preset to DC preprocessing. Parsing 2 * I constructs an OrderedProduct DCI containing an Integer and an Array. Conversion to its conventional counterpart, followed by Calculation’s explicit np.asarray(..., dtype=np.float64), produces the numerical matrix. DC identity remains structural; application construction requests numerical evaluation.

The matrix field deliberately has no type constraint: both an Array DCI and an OrderedProduct DCI can supply a usable matrix to this application. Declaring type: "Array" would reject 2 * I, because an OrderedProduct is a different DCT. DS does not turn expressions into Arrays to satisfy a constraint.

Each preset value must be self-contained Shorthand text. Compact Lists and explicit eval(...) are prepared when the schema is compiled. Presets do not refer to one another. User-supplied local definitions can refer to presets:

python calculate.py --matrix 'A=I | 3 * A' --iterations 20 --print-attribute trace

A local definition cannot redefine an application preset with the same name. Names follow DC’s substitution rules, and quoted strings are protected. Replacement is textual, following DC preprocessing rules; include explicit constructors when an expression’s grouping must be retained.

Substitution scope is one field. For nargs, the same presets apply to every supplied item. Declared String fields retain their literal-text input policy. Defaults, file contents, and programmatically supplied DCIs receive no application substitutions. A file can still declare its own local substitutions when its format supports preprocessing.

Inheritance and copying preserve substitutions like other field properties. Overriding the property replaces the whole preset mapping; DS does not merge individual preset names. Invalid names, nontext values, and malformed preset text are reported with the field name during schema compilation.

Boolean flags and multiple option values

A Boolean field normally becomes a flag that supplies true when present. Omission leaves the field absent for schema processing, where a default may apply. Set store_false: True for a false flag, or store_true: False to require an explicit Boolean value such as --enabled false.

nargs controls option arity independently of its DCT constraint. For example:

field = {"type": "List", "nargs": "+", "optional": True}

This option consumes one or more shell arguments and builds a List DCI. Each item is parsed as Shorthand. A List field without nargs can instead accept one whole Shorthand List expression. nargs: "?" permits an option with no value; it then supplies the configured const, or Null when no const is configured. That explicitly supplied Null is distinct from an omitted option.

Input files and source precedence

Declare an optional input_files field to enable --input-files. A List DCT constraint is suitable; SCL supplies a default nargs: "+" for this control. An alternative control name can be passed as the constructor’s input_files_tag.

python calculate.py --input-files baseline.yaml overrides.dcs --iterations 40

Each file must represent a Mapping DCI with String field keys. Supported formats:

Source precedence, from lowest to highest, is:

1. Files selected by --input-files, in their supplied order.
2. A required positional file configured with apply_as_schema_data: True.
3. Command-line options and selected child commands.

Later values replace earlier values. Declared child-schema Mappings merge recursively, allowing a command-line option to override one child field while retaining its siblings from a file. An ordinary Mapping-valued field is replaced as a whole. Existing field positions are retained; new fields append.

Schema defaults fill remaining absent fields after sources are merged. SCL retains source context on external references without fetching their payloads. The input_files control itself remains in the input and processed local data.

For example, baseline.yaml for the program above can contain:

matrix:
  - [2, -1]
  - [-1, 2]
iterations: 20

Standard input and required positional files

Use - in the input-file list to read structured input from stdin. The default stdin format is DC YAML; pass stdin_format="shorthand" to the SCL constructor when the application wants Shorthand instead. Empty structured text input denotes an empty Mapping. Stdin can be consumed only once per invocation.

A root schema can require a positional file:

required_file = {
    "enabled": True,
    "metavar": "FILE",
    "help": "Primary input file",
    "read_mode": "path",
    "stdin_ok": True,
    "apply_as_schema_data": True,
}

Place that dictionary under the schema-level key required_file. With apply_as_schema_data: True, SCL loads structured data by suffix, merges it at the precedence above, and leaves required_file_content as None.

Without structured loading, read_mode selects application-specific handling: path retains the path without reading, text reads UTF-8 text, and binary reads bytes. Access the path and content through required_file_path and required_file_content. Child schemas cannot declare required_file metadata.

Child commands

Declared child schemas become bare command tokens. Schema name post_process appears as command post-process. Options after a child token belong to that child; place root options before descending into a child.

definitions = {
    "run": {"child_schemas": ["solver", "report"], "child_schema_mode": "inclusive"},
    "solver": {"variables": {
        "iterations": {"type": "Integer", "optional": True, "default": 20},
    }},
    "report": {"variables": {"filename": {"type": "String"}}},
}

scl = SchemaCommandLine(
    definitions, schema_name="run",
    argv=["solver", "report", "--filename", "results.txt"],
)
assert scl.data_tree == {"solver": {"iterations": 20}, "report": {"filename": "results.txt"}}

The solver token supplies an empty child Mapping, which receives its default. No command or file entry for a child means that child remains absent.

child_schema_mode controls navigation only:

• exclusive (default) permits descent into child commands and their descendants.
• inclusive additionally permits a one-level sibling jump between that schema’s children.

SCL does not search across multiple ancestors for a sibling. Both navigation modes use the same explicit child-data presence rules.

Required option values are consumed before command recognition. A String value matching a child command stays a value. For variable-arity options, a command can begin after the option’s minimum arity is satisfied. Use --option=value to disambiguate a first value, or supply a whole List expression as one argument.

Help lists child commands and their schema descriptions. The schema-level child_schema_label can change that help group’s heading.

Application construction and operations

With target_class, SCL constructs an object using selected local fields and exposes it as target_obj. Child objects are constructed explicitly from child_projections. See Application construction.

Subclasses can override _operations(). SCL calls it after successful projection and optional construction, then handles requested attribute output. The hook can inspect data_dci, convert selected data, use target_obj, or write results.

SCL also exposes input_dci, data, data_tree_dci, data_tree, child_data, and child_projections. Conventional views are computed on access.

Print application attributes

The complete program enables --print-attribute trace. Default output uses DC YAML interchange, preserving supported DCI distinctions with $ airlocks. Attributes are read after _operations() runs.

For automatic property discovery, external names, and exclusions, see Attribute output.

Tab completion

Install argcomplete and register your application’s installed executable:

eval "$(register-python-argcomplete my-program)"

Replace my-program with the executable that invokes SCL. DS does not install a universal command named my-program. Completion offers options, declared choices, and child commands in the current navigation context. It runs before input files are loaded or the target application is constructed.

2.7 - Attribute output

SchemaCommandLine (SCL) can print selected attributes of the constructed application object. The schema-level print_attribute setting enables --print-attribute. A target_class is required, and print_attribute is reserved as an injected SCL control field.

Discover public properties

from data_schemer.schema_command_line import SchemaCommandLine

class Summary:
    def __init__(self, values):
        self.values = values

    @property
    def mean(self):
        return sum(self.values) / len(self.values)

    @property
    def maximum(self):
        return max(self.values)

schema = {
    "variables": {"values": {}},
    "print_attribute": True,
}

scl = SchemaCommandLine(
    schema, target_class=Summary,
    argv=["--values", "[1, 2, 3]", "--print-attribute", "mean"],
)

mean: 2.0

Automatic discovery exposes public @property names, including inherited properties, including functools.cached_property descriptors. It does not discover every instance attribute. The requested attribute is read after application construction and _operations().

Select external names

Use a configuration dictionary for explicit choices or aliases:

schema["print_attribute"] = {
    "choices": ["average", "maximum"],
    "code_alias": {"average": "mean"},
}
scl = SchemaCommandLine(
    schema, target_class=Summary,
    argv=["--values", "[1, 2, 3]", "--print-attribute", "average"],
)

average: 2.0

The command-line spelling is external; mean is the internal attribute name. Schema field code_alias mappings also participate in attribute-name mapping. The explicit output mapping must not overlap their external names or internal targets. Attribute aliases must be one-to-one.

When choices is a list, it controls validation and completion. Manual choices can name attributes that are not @property descriptors, provided they exist on the constructed object.

Automatic exclusions

schema["print_attribute"] = {"exclude": ["maximum"]}

Exclusions apply during automatic discovery only when the configuration has no choices key. An excluded internal name or its external alias removes the property. An explicit choices: None selects automatic discovery but suppresses exclusions; key presence is significant.

Output representation

Default attribute output uses DataCanon YAML interchange. A supported value that YAML cannot represent faithfully uses a $ Shorthand airlock. Each requested attribute is printed separately under its external name.

Set omit_key: True to print just the value, still using DataCanon (DC) YAML interchange. For example:

schema["print_attribute"] = {"choices": ["mean"], "omit_key": True}

This output path converts the application attribute using DC’s conventional conversion rules. The original input DataCanon Instance (DCI) is not changed.

Application reports

SCL always uses DC YAML interchange for attribute output. An application that wants rounded tables, labels, or a specialized report should format that output in its own code, for example in _operations(). The former precision_from and array_style_from settings are rejected with migration guidance.

The behavioral reference describes output-configuration inheritance.

2.8 - Output and application reports

SchemaCommandLine (SCL) uses DataCanon (DC) YAML interchange for attribute output. Use DC’s renderers and persistence writers when working with DataCanon Instances (DCIs). Application code is responsible for rounded tables, labels, or other specialized reports.

Render or store a DCI

from data_canon import as_dc_instance, render_shorthand, render_yaml

node = as_dc_instance({"samples": {("sensor", 0): [1.0, 2.0]}})
print(render_shorthand(node))
print(render_yaml(node))

Use write(node, filename="result.dcb") for canonical binary persistence. Shorthand and YAML interchange provide readable representations of the DCI. SCL’s --print-attribute uses that same YAML interchange, including when omit_key requests the value without its external-name wrapper.

Format application reports

An application may choose to round numbers, arrange rows into tables, or add labels. Perform those operations explicitly in application code. An SCL subclass can write such a report in its _operations() hook after construction.

A numerical report is a presentation chosen by the application. If the original DCI must also be retained, store it separately using DC persistence.

Removed formatting interfaces

The former DataSchemer (DS) numpy_print.py and render_data.py modules have been removed, along with their array, Mapping, and labeled-table formatters. SCL no longer supports the precision constructor argument, its precision property or set_precision() method. The precision_from and array_style_from output settings are rejected with migration guidance.

This page remains at its previous URL so existing documentation links continue to resolve. See Attribute output for the supported SCL output configuration.

2.9 - Migrating from numeric coercion

The DataSchemer (DS) processor accepts DataCanon Instances (DCIs) and replaces the former numeric coercion engine. data_schemer.coerce_numbers and the temporary data_schemer.dci_projector module have been removed. The public processor is data_schemer.schema_projector.SchemaProjector.

Move parsing to the input boundary

from data_canon import parse_shorthand
from data_schemer.schema_projector import SchemaProjector

input_dci = parse_shorthand('{"matrix": <1 0; 0 1>}', preprocess=True)
projector = SchemaProjector({"variables": {"matrix": {}}})
projection = projector.get_projection(input_dci)
assert projection.data == {"matrix": [[1, 0], [0, 1]]}

A standalone projector receives a DCI. SchemaCommandLine (SCL) handles text parsing at its own boundary and then invokes that same processor.

Replace the former contracts

Field-property inheritance is now explicit: the first listed base wins explicit property conflicts, and local properties override inherited ones. A supplied child Mapping is projected; an absent child stays absent. SCL’s inclusive navigation does not activate children.

The old numpy_print.py and render_data.py formatting modules and SCL’s numerical report controls have also been removed. Use DC rendering for interchange and application code for specialized reports.

The input guide and projection guide cover the new workflow. The behavioral reference records the resolved rules.

3 - Reference

This reference describes DataSchemer (DS), whose schema processor accepts DataCanon Instances (DCIs).

• Terminology: types, instances, conventional objects, schemas, and projections.
• Behavioral rules: validation, schema composition, child presence, source merging, and construction.
• Python API: supported entry points and output properties.
• Errors: user-facing diagnostics and application exception boundaries.

DataCanon (DC)’s specification defines DataCanon Types (DCTs), DCI equality, canonical encoding, fingerprints, and representation formats. DS uses those definitions and adds schema behavior. The DS reference does not introduce another DCI identity model or a portable schema encoding.

3.1 - Terminology

DataCanon (DC) defines a data model, equality, canonical bytes, and fingerprinting. DataSchemer (DS) applies application schemas to values in that model. SchemaCommandLine (SCL) is DS’s command-line interface builder.

Data types and values

A DataCanon Type (DCT) defines a kind of data, the information constituting its values, its validity rules, and equality. Integer, List, and Array are DCTs.

A DataCanon Instance (DCI) is a value of exactly one DCT. For example, IntegerDC(3) is a DCI of the Integer DCT. A DCI has canonical bytes and can be represented in Shorthand, YAML interchange, or supported persistence formats. The representation is separate from the value’s identity.

Conventional types are the usual types used in a programming language: builtins, library types, and user-defined classes. Python’s list, int, fractions.Fraction, and NumPy arrays are examples. A ListDC instance and a list are both Python objects but have different roles. DS preserves the DCI and converts it to conventional objects when requested.

Schemas and data

A schema definition is metadata describing permitted fields, optional DCT constraints, defaults, and other input rules. In the Python implementation, definitions are conventional dictionaries. The data supplied to the schema processor is a Mapping DCI with String field keys.

A field is a named entry governed by a schema. The Python schema API uses variables as the key containing field definitions.

A base schema supplies definitions through inheritance. A derived schema inherits and may override those definitions.

A child schema validates a nested Mapping entry in input data. Child schemas and inheritance are separate relationships. A supplied empty child Mapping is empty data for a defined schema; it is not an empty schema definition.

A projection is the validated result for one input DCI, after schema defaults have been applied. A compiled SchemaProjector can produce multiple independent SchemaProjection results. Each result retains the original input separately from processed output.

3.2 - Behavioral rules

This reference records the schema-processing contract for DataCanon Instances (DCIs), the values defined by DataCanon’s data model. A schema definition is conventional Python metadata. A projection is the validated result for one input DCI. A child schema describes nested data; a derived schema inherits definitions from bases.

Schema definitions

The single-schema form contains a top-level variables Mapping and has default name "default". Otherwise definitions map schema names to schema dictionaries, and schema_name is required. A schema with no local variables can use an empty variables dictionary in the single-schema form or a named definition in the multiple-schema form.

substitutions, help, metavar, alias, nargs, const, store_true, and store_false provide SchemaCommandLine (SCL) interface metadata. They do not change DCI equality or turn DCT constraints into conversion instructions.

Field names must be nonempty strings. Constructor aliases must be nonempty strings and unique at the schema level. A local field name cannot also be a declared child-schema name.

Schema definitions and named schema entries must be dictionaries. Schema names must be nonempty strings, and inheritance must list nonempty schema names. Malformed field choices, dependency names, and conflict groups are rejected with DS user-facing errors. Composition and removal still precede final field validation.

DCT constraints

The allowed type names are:

Null Boolean Integer Float String Bytes Constant Fraction
Power Quotient Complex
List Tuple Set Multiset Sum Product OrderedProduct Mapping UnorderedMapping Record
Array ExternalFile ExternalArray

Constraints use exact DCT identity. Integer does not satisfy Float, Float does not satisfy Integer, and Boolean does not satisfy Integer. DataSchemer (DS) performs no implicit numerical coercion. An omitted type accepts any DCI that satisfies the field’s other constraints.

A default has its own DCT but does not imply a field constraint. Defaults and choices may be conventional objects or DCIs. They are converted to DCIs at compilation using the supplied conversion registry, if any. Declared type and choice constraints apply to defaults as well as supplied values.

Array element-type constraint

An Array field may declare array_type as one of bool, int8, int16, int32, int64, uint8, uint16, uint32, uint64, float16, float32, float64, complex64, or complex128. Names are the exact portable DC Array type names.

array_type requires an explicit type: "Array". It validates the existing element type of supplied values and defaults; Array type, shape, and contents are preserved. It does not accept a List or an ExternalArray. Without array_type, the Array DCT constraint accepts every portable Array element type.

Field composition resolves this property like any other property. Validation of the constraint follows overrides and removals. A mismatch raises DSInvalidArrayTypeError with the field’s data path and required/received portable Array type names.

Field composition

Field definitions are resolved with these precedence rules:

1. Resolve listed bases recursively. For each field property, the first base explicitly defining it wins; later bases fill absent properties.
2. Apply local variables property overrides.
3. Resolve copy_variables destinations from their source field definitions. A copied definition replaces an inherited destination definition. Local destination properties override the copy.
4. Apply delete_properties, then delete_variables.
5. Fill implicit property defaults and validate the final field definitions.

A property value is replaced as a whole, including a choices list or Mapping default. Property values are not recursively merged. Implicit defaults are not inserted into base definitions before composition.

Copy references use "schema@field". Cross-schema sources include their resolved inheritance and removals. Same-schema copies can reference another copy declared later, and source fields before current-schema removals. Cyclic inheritance or copy references are errors. Missing copy sources or removal targets are errors.

delete_properties maps field names to lists of property names. delete_variables lists fields to remove. Final validation follows removals, allowing an invalid inherited definition to be removed before validation. update_variables is rejected, including an empty occurrence; local property overrides belong under variables.

Field order follows first appearance through the bases, then new local fields; new copy destinations follow their insertion order. Composition does not mutate the caller’s field definitions.

Effective data and validation

Each projected schema level receives a Mapping DCI with String field keys. Values of ordinary fields may contain arbitrary nested DCIs and Mapping keys.

Projection retains the original complete input DCI and computes processed data:

1. Separate declared child entries from local fields and reject unknown local fields.
2. Fill absent local fields from schema defaults.
3. Check required fields, DCT constraints, choices, requires, and conflicts.
4. Project supplied child Mappings recursively.
5. Assemble the complete processed Mapping.

Required fields cannot declare defaults. Optional fields without defaults remain absent when not supplied. Explicit Null and false are present values. Defaults satisfy requires and participate in conflicts. A conflict group permits at most one present member, regardless of the members’ truth values.

input_dci is unchanged. data_dci contains local processed fields, while data_tree_dci also includes processed child entries. Supplied Mapping order is preserved; added defaults append in schema order.

Conventional output is converted on access. Processing alone does not retrieve external payloads or invoke Record-to-conventional conversion handlers. Those operations can occur when a conventional view or selected constructor argument is requested, using DataCanon (DC)’s conversion and integrity rules.

Child schemas

child_schemas may name one child or list child names. Inheritance accumulates base declarations first, then local declarations, removing duplicates while preserving first occurrence. Every declared child must exist. Child-schema cycles and field/child-name collisions are rejected. A schema used as a child cannot declare non-None required_file metadata.

A child is projected only if its entry is present in the data. An absent child stays absent. An explicitly empty child Mapping receives defaults and validation. Null and other non-Mapping values are rejected. A supplied parent Mapping does not implicitly activate its children.

child_schema_mode belongs to SCL navigation and is not interpreted or exposed by SchemaProjector. Compiling or caching a child projector does not activate child data in any projection. Projector searches cover already compiled definitions and preserve distinct paths for reused schema names.

SCL input boundaries

SCL uses the same schema processor after parsing its input into DCIs.

• Option names use kebab-case; aliases add command-line spellings.
• Ordinary option values use Shorthand with preprocessing enabled.
• Declared String fields accept ordinary text unless the argument begins with a double quote, in which case it is parsed as a DC literal.
• After failed Shorthand parsing, simple bare identifiers and paths can become String DCIs. Malformed structured expressions are rejected.
• Boolean flags and nargs determine interface arity; supplied values still undergo schema validation. Omitted options remain absent until defaults apply.
• Required option values and the required positional file are consumed before command-token recognition. Variable arity can end at a command once its minimum number of values has been consumed.
• Structured files use .dcs with preprocessing, canonical .dcb, DC HDF5, or DC YAML according to suffix. Every file root must be a String-keyed Mapping DCI.
• Stdin uses DC YAML unless the constructor selects Shorthand. Blank structured text denotes an empty Mapping. Stdin cannot be consumed twice.
• Loaded external references retain nonsemantic source context for deferred access.

The input_files control is enabled only when declared in the schema, with default nargs: "+". Its paths are literal strings. A constructor argument can select another control name. SCL control fields remain in its processed local data and are subject to normal constructor argument selection.

SCL source merging

Files selected through the input-file control merge in order. A structured required positional file overlays them, then command-line values overlay both. Schema defaults fill remaining absent fields after merging.

Only declared child-schema blocks merge recursively. Ordinary Mapping-valued fields replace earlier values as a whole. Existing field positions remain; new fields append. The merged DCI becomes input_dci, before defaults. It is not a collection of separate snapshots of the original source files.

SCL navigation

Selecting a child command supplies its Mapping, even without options. Selecting a nested command supplies Mappings along that path. File-supplied child entries can also be projected without selecting their commands.

child_schema_mode: "exclusive" is the default: navigation descends into child commands. "inclusive" also permits a one-level sibling jump between that schema’s children. Navigation does not search upward across multiple ancestors, and neither mode activates absent data. Invalid navigation settings raise DSUserError in SCL.

SCL rejects child schemas directly declared under multiple parents. The standalone projector can reuse a child definition across branches.

required_file is root SCL metadata and is not inherited. With apply_as_schema_data: True, structured loading selects the format by suffix and required_file_content stays None. Otherwise read_mode selects path, text, or binary access.

Required-file switches are Booleans. Invalid modes, malformed metadata, and unknown required-file keys are rejected before access. Application-specific text mode decodes UTF-8; read failures are DS user errors. Path mode retains the path without opening its contents.

Application construction

get_instance(callable) selects local processed fields by constructor argument name after code_alias mapping. It passes keyword arguments, supports keyword-only parameters, and supplies additional local fields when the callable accepts **kwargs. Child projections are constructed separately.

A schema default is processed data and takes precedence over a callable default. An absent optional field leaves a callable default intact. Callable defaults and annotations do not define DCT constraints or satisfy required schema fields.

Signature binding happens before conventional conversion. Only selected argument values are converted. A callable without an inspectable signature, or one needing required positional-only arguments, can be adapted with a Python wrapper. Application constructor exceptions and conventional conversion failures propagate.

SCL’s target_class uses this same construction contract. SCL calls _operations() after construction, then prints requested attributes.

Print attribute configuration

print_attribute is disabled when absent or false; true enables automatic discovery of public properties. A configuration dictionary can control:

Attribute aliases must be one-to-one and disjoint from schema aliases in both external names and internal targets. print_attribute is a reserved injected SCL field; applications must not also declare it in variables.

Output configuration has separate inheritance semantics from field properties. Bases are applied in listed order, then local configuration. A false value disables output; true preserves an inherited configuration dictionary. Dictionaries merge alias maps with later values overriding earlier ones, extend and deduplicate choices and exclude lists, and replace other properties. Explicit choices: None clears manual choices; exclude: None clears exclusions. Later-base dictionary settings can therefore override earlier-base settings in this output configuration.

Attribute output always uses DC YAML interchange, including with omit_key. Numerical report formatting belongs to application code. The removed precision_from and array_style_from settings raise DSUserError, including when inherited from a base schema.

Errors and scope

Input constraint failures use DS user-facing errors, including structured DSInvalidTypeError for DCT mismatches. SCL’s main() catches DSUserError and normally returns status 2. Application and conversion errors remain separate.

The error reference gives exact type diagnostics. DCT identity, canonical bytes, resource integrity, and the syntax of DC representations remain defined by DataCanon.

Application-defined CLI substitutions

A field’s optional substitutions property is a dictionary mapping DC substitution identifiers to self-contained Shorthand text. Schema compilation validates names and text, prepares optional preprocessing in each value, and retains the prepared replacements in the compiled field metadata. Original schema definitions are not modified.

SCL supplies the replacements to DC preprocessing for that field’s command-line values, including individual nargs items. String fields retain their existing literal-input behavior. Local input definitions may use preset names but cannot redefine them. Field constraints apply to the resulting DCI.

Substitutions do not apply to defaults, input files, or programmatically supplied DCIs. Inheritance, copying, overriding, and deletion use ordinary field-property composition; overriding substitutions replaces the entire mapping.

3.3 - Python API

Import the public classes from their defining modules:

from data_schemer.schema_projector import SchemaProjector, SchemaProjection
from data_schemer.schema_command_line import SchemaCommandLine

SchemaProjector

SchemaProjector(schema_definitions, *, schema_name=None, input_data=None,
                registry=None, source=None)

schema_definitions is a conventional Python dictionary in single-schema or named-schema form. schema_name selects a definition in named-schema form. registry supplies a DataCanon (DC) conversion registry. source supplies external-resource context for conventional conversion. An optional input_data DataCanon Instance (DCI) creates an initial projection.

Accessing initial-projection members without an initial input raises DSUserError. Reusable projectors and per-input projections have distinct roles; passing a new input to get_projection() does not replace the initial projection.

For a field declared type: "Array", optional array_type constrains its portable element type, for example float64. It requires an exact match and preserves the supplied Array. This metadata is resolved during schema composition and validated before projection; defaults are checked as well.

SchemaProjection

Obtain a projection through projector.get_projection(input_dci).

Conventional views are computed on request. get_instance() selects fields before converting, so unrelated external references need not be materialized. See Application construction.

SchemaCommandLine

SchemaCommandLine(schema_definitions, target_class=None, schema_name=None,
                  argv=None, input_files_tag="input_files",
                  prog=None, registry=None, stdin_format="yaml")

Construction parses sources, validates the merged DCI, optionally constructs the target, calls _operations(), and handles requested attribute output. Omit argv to use sys.argv[1:]. prog selects the displayed executable name. stdin_format accepts "yaml" or "shorthand" for structured stdin.

SchemaCommandLine (SCL) exposes input_dci, data_dci, data, data_tree_dci, data_tree, child_projections, and child_data from its projection. Additional members:

main() forwards arguments to the constructor. debug=True reraises caught DS user errors. Argparse help can exit through normal SystemExit(0) behavior.

Schema metadata helpers

The following helpers are available from data_schemer.schema_projector:

get_schema_variables(schema_name, schema_definitions, allowed_types=None, defaults=None)
get_schema_child_schemas(schema_name, schema_definitions)
get_schema_conflicts(schema_name, schema_definitions)
get_schema_print_attribute(schema_name, schema_definitions)

get_schema_variables() resolves metadata. SchemaProjector performs additional compilation checks and converts defaults and choices to DCIs; use the projector when processing input.

For SCL tooling, compute_print_attribute_choices() is available from data_schemer.schema_command_line:

compute_print_attribute_choices(print_attribute, *, target_class, schema_vars)

It returns (choices, external_to_internal) for attribute validation and completion. Callers should reuse it when they need SCL’s naming rules.

Output formatting

SCL attribute output uses DC YAML interchange. Applications handle their own numerical reports. render_data.py, numpy_print.py, and SCL’s numerical report controls have been removed; see Output and application reports.

3.4 - Errors

DataSchemer (DS) validates DataCanon Instances (DCIs), the values in DataCanon’s data model. A DataCanon Type (DCT) constraint requires a particular data type. Catch DSUserError at the application’s user-input boundary:

from data_schemer.errors import DSUserError

try:
    projection = projector.get_projection(input_dci)
except DSUserError as error:
    print(error.user_message())

The base class provides user_message() and a default exit_code of 2. SchemaCommandLine.main() prints caught DS user errors to stderr and returns their exit code. Calling the constructor directly allows the caller to handle exceptions itself.

DCT mismatch

DSInvalidTypeError records a field whose DCT differs from its declared constraint. For a nested Float supplied where Integer is required:

Invalid type for field "solver.iterations".
Expected: Integer
Received: Float
Value: 1.0

The exception exposes:

Paths describe the input data, independently of schema names and constructor aliases. A literal field name containing a dot uses bracket notation:

Invalid type for field ["solver.iterations"].

That path is ("solver.iterations",), distinct from the nested path above. Diagnostic summaries describe numerical arrays by dtype and shape, bound long values, and do not retrieve external payloads or invoke conventional conversion.

Array element-type mismatch

DSInvalidArrayTypeError reports an Array DCI that does not satisfy its field’s array_type constraint:

Invalid Array type for field "solver.matrix".
Expected: float64
Received: float32
Value: Array(type=float32, shape=(2, 2))

It is a subclass of DSInvalidTypeError and DSUserError, retaining path, expected, received, and value_summary. For this exception, expected and received name portable Array element types. Nested invalid defaults preserve the same exception class and data path. A non-Array value fails the DCT check first and raises the ordinary DSInvalidTypeError.

Other input errors

The unused legacy DSCoercionError and DSInvalidChoiceError classes have been removed. Failed field choices are reported through DSUserError.

Invalid configuration and required files

Malformed schema dictionaries, inheritance lists, field choices or dependencies, conflict groups, and attribute-output settings produce DSUserError diagnostics. SCL also reports invalid option metadata and conflicting aliases as DS user errors rather than exposing argparse construction errors.

required_file metadata is checked before file access. Its switches must be Booleans, read_mode must be path, text, or binary, and unknown metadata keys are rejected. Missing or unreadable files and invalid UTF-8 text produce user-facing messages. Path-only mode does not open or check a file’s contents.

Constructor and conversion failures

Missing constructor arguments, non-callables, and uninspectable callable signatures produce DSUserError during get_instance(). Signature binding runs before conventional conversion.

Once selected arguments are converted and application construction starts, conversion failures and application exceptions propagate. For example, a constructor’s own ValueError remains a ValueError. A successful DCI projection does not guarantee that an external resource can later be retrieved or that an application constructor will accept its conventional arguments.

Validation of schema and SCL metadata is separate from application execution. DS does not broadly wrap arbitrary implementation failures or exceptions from registered conventional conversion handlers into input errors.