Behavioral rules

This reference records the schema-processing contract for DataCanon Instances (DCIs), the values defined by DataCanon’s data model. A schema definition is conventional Python metadata. A projection is the validated result for one input DCI. A child schema describes nested data; a derived schema inherits definitions from bases.

Schema definitions

The single-schema form contains a top-level variables Mapping and has default name "default". Otherwise definitions map schema names to schema dictionaries, and schema_name is required. A schema with no local variables can use an empty variables dictionary in the single-schema form or a named definition in the multiple-schema form.

substitutions, help, metavar, alias, nargs, const, store_true, and store_false provide SchemaCommandLine (SCL) interface metadata. They do not change DCI equality or turn DCT constraints into conversion instructions.

Field names must be nonempty strings. Constructor aliases must be nonempty strings and unique at the schema level. A local field name cannot also be a declared child-schema name.

Schema definitions and named schema entries must be dictionaries. Schema names must be nonempty strings, and inheritance must list nonempty schema names. Malformed field choices, dependency names, and conflict groups are rejected with DS user-facing errors. Composition and removal still precede final field validation.

DCT constraints

The allowed type names are:

Null Boolean Integer Float String Bytes Constant Fraction
Power Quotient Complex
List Tuple Set Multiset Sum Product OrderedProduct Mapping UnorderedMapping Record
Array ExternalFile ExternalArray

Constraints use exact DCT identity. Integer does not satisfy Float, Float does not satisfy Integer, and Boolean does not satisfy Integer. DataSchemer (DS) performs no implicit numerical coercion. An omitted type accepts any DCI that satisfies the field’s other constraints.

A default has its own DCT but does not imply a field constraint. Defaults and choices may be conventional objects or DCIs. They are converted to DCIs at compilation using the supplied conversion registry, if any. Declared type and choice constraints apply to defaults as well as supplied values.

Array element-type constraint

An Array field may declare array_type as one of bool, int8, int16, int32, int64, uint8, uint16, uint32, uint64, float16, float32, float64, complex64, or complex128. Names are the exact portable DC Array type names.

array_type requires an explicit type: "Array". It validates the existing element type of supplied values and defaults; Array type, shape, and contents are preserved. It does not accept a List or an ExternalArray. Without array_type, the Array DCT constraint accepts every portable Array element type.

Field composition resolves this property like any other property. Validation of the constraint follows overrides and removals. A mismatch raises DSInvalidArrayTypeError with the field’s data path and required/received portable Array type names.

Field composition

Field definitions are resolved with these precedence rules:

1. Resolve listed bases recursively. For each field property, the first base explicitly defining it wins; later bases fill absent properties.
2. Apply local variables property overrides.
3. Resolve copy_variables destinations from their source field definitions. A copied definition replaces an inherited destination definition. Local destination properties override the copy.
4. Apply delete_properties, then delete_variables.
5. Fill implicit property defaults and validate the final field definitions.

A property value is replaced as a whole, including a choices list or Mapping default. Property values are not recursively merged. Implicit defaults are not inserted into base definitions before composition.

Copy references use "schema@field". Cross-schema sources include their resolved inheritance and removals. Same-schema copies can reference another copy declared later, and source fields before current-schema removals. Cyclic inheritance or copy references are errors. Missing copy sources or removal targets are errors.

delete_properties maps field names to lists of property names. delete_variables lists fields to remove. Final validation follows removals, allowing an invalid inherited definition to be removed before validation. update_variables is rejected, including an empty occurrence; local property overrides belong under variables.

Field order follows first appearance through the bases, then new local fields; new copy destinations follow their insertion order. Composition does not mutate the caller’s field definitions.

Effective data and validation

Each projected schema level receives a Mapping DCI with String field keys. Values of ordinary fields may contain arbitrary nested DCIs and Mapping keys.

Projection retains the original complete input DCI and computes processed data:

1. Separate declared child entries from local fields and reject unknown local fields.
2. Fill absent local fields from schema defaults.
3. Check required fields, DCT constraints, choices, requires, and conflicts.
4. Project supplied child Mappings recursively.
5. Assemble the complete processed Mapping.

Required fields cannot declare defaults. Optional fields without defaults remain absent when not supplied. Explicit Null and false are present values. Defaults satisfy requires and participate in conflicts. A conflict group permits at most one present member, regardless of the members’ truth values.

input_dci is unchanged. data_dci contains local processed fields, while data_tree_dci also includes processed child entries. Supplied Mapping order is preserved; added defaults append in schema order.

Conventional output is converted on access. Processing alone does not retrieve external payloads or invoke Record-to-conventional conversion handlers. Those operations can occur when a conventional view or selected constructor argument is requested, using DataCanon (DC)’s conversion and integrity rules.

Child schemas

child_schemas may name one child or list child names. Inheritance accumulates base declarations first, then local declarations, removing duplicates while preserving first occurrence. Every declared child must exist. Child-schema cycles and field/child-name collisions are rejected. A schema used as a child cannot declare non-None required_file metadata.

A child is projected only if its entry is present in the data. An absent child stays absent. An explicitly empty child Mapping receives defaults and validation. Null and other non-Mapping values are rejected. A supplied parent Mapping does not implicitly activate its children.

child_schema_mode belongs to SCL navigation and is not interpreted or exposed by SchemaProjector. Compiling or caching a child projector does not activate child data in any projection. Projector searches cover already compiled definitions and preserve distinct paths for reused schema names.

SCL input boundaries

SCL uses the same schema processor after parsing its input into DCIs.

• Option names use kebab-case; aliases add command-line spellings.
• Ordinary option values use Shorthand with preprocessing enabled.
• Declared String fields accept ordinary text unless the argument begins with a double quote, in which case it is parsed as a DC literal.
• After failed Shorthand parsing, simple bare identifiers and paths can become String DCIs. Malformed structured expressions are rejected.
• Boolean flags and nargs determine interface arity; supplied values still undergo schema validation. Omitted options remain absent until defaults apply.
• Required option values and the required positional file are consumed before command-token recognition. Variable arity can end at a command once its minimum number of values has been consumed.
• Structured files use .dcs with preprocessing, canonical .dcb, DC HDF5, or DC YAML according to suffix. Every file root must be a String-keyed Mapping DCI.
• Stdin uses DC YAML unless the constructor selects Shorthand. Blank structured text denotes an empty Mapping. Stdin cannot be consumed twice.
• Loaded external references retain nonsemantic source context for deferred access.

The input_files control is enabled only when declared in the schema, with default nargs: "+". Its paths are literal strings. A constructor argument can select another control name. SCL control fields remain in its processed local data and are subject to normal constructor argument selection.

SCL source merging

Files selected through the input-file control merge in order. A structured required positional file overlays them, then command-line values overlay both. Schema defaults fill remaining absent fields after merging.

Only declared child-schema blocks merge recursively. Ordinary Mapping-valued fields replace earlier values as a whole. Existing field positions remain; new fields append. The merged DCI becomes input_dci, before defaults. It is not a collection of separate snapshots of the original source files.

SCL navigation

Selecting a child command supplies its Mapping, even without options. Selecting a nested command supplies Mappings along that path. File-supplied child entries can also be projected without selecting their commands.

child_schema_mode: "exclusive" is the default: navigation descends into child commands. "inclusive" also permits a one-level sibling jump between that schema’s children. Navigation does not search upward across multiple ancestors, and neither mode activates absent data. Invalid navigation settings raise DSUserError in SCL.

SCL rejects child schemas directly declared under multiple parents. The standalone projector can reuse a child definition across branches.

required_file is root SCL metadata and is not inherited. With apply_as_schema_data: True, structured loading selects the format by suffix and required_file_content stays None. Otherwise read_mode selects path, text, or binary access.

Required-file switches are Booleans. Invalid modes, malformed metadata, and unknown required-file keys are rejected before access. Application-specific text mode decodes UTF-8; read failures are DS user errors. Path mode retains the path without opening its contents.

Application construction

get_instance(callable) selects local processed fields by constructor argument name after code_alias mapping. It passes keyword arguments, supports keyword-only parameters, and supplies additional local fields when the callable accepts **kwargs. Child projections are constructed separately.

A schema default is processed data and takes precedence over a callable default. An absent optional field leaves a callable default intact. Callable defaults and annotations do not define DCT constraints or satisfy required schema fields.

Signature binding happens before conventional conversion. Only selected argument values are converted. A callable without an inspectable signature, or one needing required positional-only arguments, can be adapted with a Python wrapper. Application constructor exceptions and conventional conversion failures propagate.

SCL’s target_class uses this same construction contract. SCL calls _operations() after construction, then prints requested attributes.

Print attribute configuration

print_attribute is disabled when absent or false; true enables automatic discovery of public properties. A configuration dictionary can control:

Attribute aliases must be one-to-one and disjoint from schema aliases in both external names and internal targets. print_attribute is a reserved injected SCL field; applications must not also declare it in variables.

Output configuration has separate inheritance semantics from field properties. Bases are applied in listed order, then local configuration. A false value disables output; true preserves an inherited configuration dictionary. Dictionaries merge alias maps with later values overriding earlier ones, extend and deduplicate choices and exclude lists, and replace other properties. Explicit choices: None clears manual choices; exclude: None clears exclusions. Later-base dictionary settings can therefore override earlier-base settings in this output configuration.

Attribute output always uses DC YAML interchange, including with omit_key. Numerical report formatting belongs to application code. The removed precision_from and array_style_from settings raise DSUserError, including when inherited from a base schema.

Errors and scope

Input constraint failures use DS user-facing errors, including structured DSInvalidTypeError for DCT mismatches. SCL’s main() catches DSUserError and normally returns status 2. Application and conversion errors remain separate.

The error reference gives exact type diagnostics. DCT identity, canonical bytes, resource integrity, and the syntax of DC representations remain defined by DataCanon.

Application-defined CLI substitutions

A field’s optional substitutions property is a dictionary mapping DC substitution identifiers to self-contained Shorthand text. Schema compilation validates names and text, prepares optional preprocessing in each value, and retains the prepared replacements in the compiled field metadata. Original schema definitions are not modified.

SCL supplies the replacements to DC preprocessing for that field’s command-line values, including individual nargs items. String fields retain their existing literal-input behavior. Local input definitions may use preset names but cannot redefine them. Field constraints apply to the resulting DCI.

Substitutions do not apply to defaults, input files, or programmatically supplied DCIs. Inheritance, copying, overriding, and deletion use ordinary field-property composition; overriding substitutions replaces the entire mapping.