SchemaCommandLine

SchemaCommandLine (SCL) turns schema definitions into a command-line interface. It parses option values and files into DataCanon Instances (DCIs), merges the sources, and uses SchemaProjector for validation and defaults. SCL can then construct an application object.

A complete program

Save the following as calculate.py:

import numpy as np
from data_schemer.schema_command_line import SchemaCommandLine

class Calculation:
    def __init__(self, matrix, iterations, tolerance):
        self.matrix = np.asarray(matrix, dtype=np.float64)
        self.iterations = iterations
        self.tolerance = tolerance

    @property
    def trace(self):
        return float(np.trace(self.matrix))

schema = {
    "variables": {
        "matrix": {"help": "Numerical matrix"},
        "iterations": {"type": "Integer", "help": "Maximum iteration count"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
        "input_files": {"type": "List", "optional": True},
    },
    "print_attribute": {"choices": ["trace"]},
}

if __name__ == "__main__":
    raise SystemExit(SchemaCommandLine.main(schema, target_class=Calculation))

Run it with a Shorthand matrix and request an application property:

python calculate.py --matrix '[[2, -1], [-1, 2]]' --iterations 20 --print-attribute trace

trace: 4.0

SCL validates the input, adds the tolerance default, and constructs Calculation. The class performs its own NumPy conversion. Without --print-attribute, this example constructs the object without printing it.

SchemaCommandLine.main() returns zero on success. It catches DSUserError, prints the message to stderr, and returns its exit code, normally 2. Passing debug=True to main() reraises DataSchemer (DS) user errors for debugging. Exceptions from application code propagate. Help exits normally before loading input files or constructing the application.

Option names and values

A schema field iteration_limit becomes --iteration-limit. An alias list can add short or alternate options:

field = {"type": "Integer", "alias": ["-n", "num_iterations"]}

That field would offer its normal option, -n, and --num-iterations. An option alias changes the command-line spelling. A code_alias changes the constructor argument name; the two mechanisms have separate purposes.

SCL parses ordinary option values as DataCanon (DC) Shorthand with preprocessing enabled:

python calculate.py --matrix '<2 -1; -1 2>' --iterations 20

See DataCanon input for substitutions, compact Lists, and explicit eval(...). A Sum or Product expression remains structured DC data unless preprocessing explicitly requests evaluation.

A field declared String accepts ordinary text, including numeric-looking text such as 1.0. A value beginning with a double quote is parsed as a DC String literal. Other fields accept simple bare identifiers and paths as a string convenience when Shorthand parsing fails. Malformed structured input is rejected; it is not silently turned into a String. For an untyped string containing spaces, include DC quotes inside the shell argument: '"sample label"'.

Declared types are exact DataCanon Type (DCT) constraints. --iterations 20.0 is rejected for Integer; --tolerance 1 is rejected for Float. Use 1.0 when a Float is required.

Application-defined substitutions

An application can define named input presets using a field’s substitutions property. The following field definition makes I available when entering a matrix on the command line:

schema = {
    "variables": {
        "matrix": {
            "help": "Matrix expression; I denotes the identity matrix",
            "substitutions": {"I": "array(<1 0, 0 1>)"},
        },
        "iterations": {"type": "Integer"},
        "tolerance": {"type": "Float", "optional": True, "default": 1e-8},
    },
    "print_attribute": {"choices": ["trace"]},
}

Use this definition in calculate.py from the complete example above:

python calculate.py --matrix '2 * I' --iterations 20 --print-attribute trace

trace: 4.0

SCL passes the preset to DC preprocessing. Parsing 2 * I constructs an OrderedProduct DCI containing an Integer and an Array. Conversion to its conventional counterpart, followed by Calculation’s explicit np.asarray(..., dtype=np.float64), produces the numerical matrix. DC identity remains structural; application construction requests numerical evaluation.

The matrix field deliberately has no type constraint: both an Array DCI and an OrderedProduct DCI can supply a usable matrix to this application. Declaring type: "Array" would reject 2 * I, because an OrderedProduct is a different DCT. DS does not turn expressions into Arrays to satisfy a constraint.

Each preset value must be self-contained Shorthand text. Compact Lists and explicit eval(...) are prepared when the schema is compiled. Presets do not refer to one another. User-supplied local definitions can refer to presets:

python calculate.py --matrix 'A=I | 3 * A' --iterations 20 --print-attribute trace

A local definition cannot redefine an application preset with the same name. Names follow DC’s substitution rules, and quoted strings are protected. Replacement is textual, following DC preprocessing rules; include explicit constructors when an expression’s grouping must be retained.

Substitution scope is one field. For nargs, the same presets apply to every supplied item. Declared String fields retain their literal-text input policy. Defaults, file contents, and programmatically supplied DCIs receive no application substitutions. A file can still declare its own local substitutions when its format supports preprocessing.

Inheritance and copying preserve substitutions like other field properties. Overriding the property replaces the whole preset mapping; DS does not merge individual preset names. Invalid names, nontext values, and malformed preset text are reported with the field name during schema compilation.

Boolean flags and multiple option values

A Boolean field normally becomes a flag that supplies true when present. Omission leaves the field absent for schema processing, where a default may apply. Set store_false: True for a false flag, or store_true: False to require an explicit Boolean value such as --enabled false.

nargs controls option arity independently of its DCT constraint. For example:

field = {"type": "List", "nargs": "+", "optional": True}

This option consumes one or more shell arguments and builds a List DCI. Each item is parsed as Shorthand. A List field without nargs can instead accept one whole Shorthand List expression. nargs: "?" permits an option with no value; it then supplies the configured const, or Null when no const is configured. That explicitly supplied Null is distinct from an omitted option.

Input files and source precedence

Declare an optional input_files field to enable --input-files. A List DCT constraint is suitable; SCL supplies a default nargs: "+" for this control. An alternative control name can be passed as the constructor’s input_files_tag.

python calculate.py --input-files baseline.yaml overrides.dcs --iterations 40

Each file must represent a Mapping DCI with String field keys. Supported formats:

Source precedence, from lowest to highest, is:

1. Files selected by --input-files, in their supplied order.
2. A required positional file configured with apply_as_schema_data: True.
3. Command-line options and selected child commands.

Later values replace earlier values. Declared child-schema Mappings merge recursively, allowing a command-line option to override one child field while retaining its siblings from a file. An ordinary Mapping-valued field is replaced as a whole. Existing field positions are retained; new fields append.

Schema defaults fill remaining absent fields after sources are merged. SCL retains source context on external references without fetching their payloads. The input_files control itself remains in the input and processed local data.

For example, baseline.yaml for the program above can contain:

matrix:
  - [2, -1]
  - [-1, 2]
iterations: 20

Standard input and required positional files

Use - in the input-file list to read structured input from stdin. The default stdin format is DC YAML; pass stdin_format="shorthand" to the SCL constructor when the application wants Shorthand instead. Empty structured text input denotes an empty Mapping. Stdin can be consumed only once per invocation.

A root schema can require a positional file:

required_file = {
    "enabled": True,
    "metavar": "FILE",
    "help": "Primary input file",
    "read_mode": "path",
    "stdin_ok": True,
    "apply_as_schema_data": True,
}

Place that dictionary under the schema-level key required_file. With apply_as_schema_data: True, SCL loads structured data by suffix, merges it at the precedence above, and leaves required_file_content as None.

Without structured loading, read_mode selects application-specific handling: path retains the path without reading, text reads UTF-8 text, and binary reads bytes. Access the path and content through required_file_path and required_file_content. Child schemas cannot declare required_file metadata.

Child commands

Declared child schemas become bare command tokens. Schema name post_process appears as command post-process. Options after a child token belong to that child; place root options before descending into a child.

definitions = {
    "run": {"child_schemas": ["solver", "report"], "child_schema_mode": "inclusive"},
    "solver": {"variables": {
        "iterations": {"type": "Integer", "optional": True, "default": 20},
    }},
    "report": {"variables": {"filename": {"type": "String"}}},
}

scl = SchemaCommandLine(
    definitions, schema_name="run",
    argv=["solver", "report", "--filename", "results.txt"],
)
assert scl.data_tree == {"solver": {"iterations": 20}, "report": {"filename": "results.txt"}}

The solver token supplies an empty child Mapping, which receives its default. No command or file entry for a child means that child remains absent.

child_schema_mode controls navigation only:

• exclusive (default) permits descent into child commands and their descendants.
• inclusive additionally permits a one-level sibling jump between that schema’s children.

SCL does not search across multiple ancestors for a sibling. Both navigation modes use the same explicit child-data presence rules.

Required option values are consumed before command recognition. A String value matching a child command stays a value. For variable-arity options, a command can begin after the option’s minimum arity is satisfied. Use --option=value to disambiguate a first value, or supply a whole List expression as one argument.

Help lists child commands and their schema descriptions. The schema-level child_schema_label can change that help group’s heading.

Application construction and operations

With target_class, SCL constructs an object using selected local fields and exposes it as target_obj. Child objects are constructed explicitly from child_projections. See Application construction.

Subclasses can override _operations(). SCL calls it after successful projection and optional construction, then handles requested attribute output. The hook can inspect data_dci, convert selected data, use target_obj, or write results.

SCL also exposes input_dci, data, data_tree_dci, data_tree, child_data, and child_projections. Conventional views are computed on access.

Print application attributes

The complete program enables --print-attribute trace. Default output uses DC YAML interchange, preserving supported DCI distinctions with $ airlocks. Attributes are read after _operations() runs.

For automatic property discovery, external names, and exclusions, see Attribute output.

Tab completion

Install argcomplete and register your application’s installed executable:

eval "$(register-python-argcomplete my-program)"

Replace my-program with the executable that invokes SCL. DS does not install a universal command named my-program. Completion offers options, declared choices, and child commands in the current navigation context. It runs before input files are loaded or the target application is constructed.