Specifications on Data Canon

DataCanon Core

Mon, 01 Jan 0001 00:00:00 +0000

Version 1 Draft

Introduction

DataCanon (DC) is a data model for structured scientific data with a canonical identity encoding.

DataCanon defines a set of data types and their equality relations, together with a unique canonical byte encoding for every instance.

The type and encoding definitions provide a common, representation-independent basis for communicating, identifying, comparing, and preserving structured scientific data across software implementations and persistence mechanisms.

The identity of a DataCanon Instance is determined solely by its type and its constituent information. It does not depend on any particular programming language, textual notation, parser, renderer, or persistence format.

Shorthand

Mon, 01 Jan 0001 00:00:00 +0000

DataCanon Shorthand — Version 1 Draft

Status: Draft for independent implementation review
Depends on: DataCanon Core Version 1
Scope: Strict Shorthand parsing and identity-preserving rendering
Out of scope: Optional Shorthand preprocessing, YAML interchange, HDF5, and persistence conventions

Introduction

DataCanon Shorthand is a human-readable textual representation of DataCanon Instances (DCIs). It provides notation for all 24 DataCanon Types (DCTs) defined by DataCanon Core Version 1.

Shorthand does not define DCI identity. The core specification defines DCTs, equality, canonical binary encoding, and fingerprinting. A Shorthand document represents exactly one DCI; its whitespace and textual layout do not participate in that DCI’s identity.

Shorthand Preprocessing

Mon, 01 Jan 0001 00:00:00 +0000

DataCanon Shorthand Preprocessing — Version 1 Draft

Status: Draft for independent implementation review
Depends on: DataCanon Shorthand Version 1; DataCanon Core Version 1
Scope: Optional textual transformation of augmented Shorthand into strict Shorthand
Out of scope: DCI construction, canonical encoding, persistence, application-specific mathematical aliases

Introduction

DataCanon Shorthand preprocessing is an optional text-to-text transformation intended for human-authored Shorthand. It supports three conveniences:

compact List notation;
named textual substitutions; and
explicitly requested scalar arithmetic with eval(...).

The preprocessor does not construct, inspect, or validate DataCanon Instances (DCIs). Its output is intended to be parsed by a strict Shorthand parser, which remains responsible for all syntax and DCT validity checks. Preprocessing does not change DataCanon Core identity or canonical encoding.

YAML Interchange

Mon, 01 Jan 0001 00:00:00 +0000

DataCanon YAML Interchange — Version 1 Draft

Status: Draft for independent implementation review
Depends on: DataCanon Core Version 1; DataCanon Shorthand Version 1
Scope: Interchange between YAML documents and DataCanon Instances (DCIs)
Out of scope: Canonical binary encoding rules, HDF5 persistence, Shorthand preprocessing, and application-specific YAML tags

Introduction

DataCanon YAML Interchange permits ordinary YAML values to represent DataCanon Instances (DCIs), while preserving DataCanon-specific structures through an embedded Shorthand convention called the $ airlock.

HDF5

Mon, 01 Jan 0001 00:00:00 +0000

DataCanon HDF5 — Version 1 Draft

Status: Draft for independent implementation review
Depends on: DataCanon Core Version 1
Scope: Identity-preserving HDF5 representation of DataCanon Instances (DCIs), including embedded external resources
Out of scope: Canonical byte encoding, Shorthand and YAML syntax, filesystem packaging conventions, and application-specific source-resolution APIs

Introduction

DataCanon HDF5 defines a portable representation of DataCanon Instances (DCIs) in HDF5 files. Its purpose is to permit independent implementations, including Python and Julia, to exchange structured scientific data while preserving DataCanon identity.

Persistence

Mon, 01 Jan 0001 00:00:00 +0000

DataCanon Persistence — Version 1 Draft

Status: Draft for independent implementation review
Depends on: DataCanon Core Version 1; DataCanon Shorthand Version 1; DataCanon HDF5 Version 1
Related: DataCanon YAML Interchange Version 1; DataCanon Shorthand Preprocessing Version 1
Scope: Storage-format selection, reading and writing, external-resource namespaces, deferred retrieval, integrity verification, and identity-preserving transfer
Out of scope: DCT definitions, canonical byte encoding, Shorthand grammar, HDF5 object layout, YAML scalar semantics, and implementation-language APIs

Conventional Types

Mon, 01 Jan 0001 00:00:00 +0000

DataCanon Conventional Types — Version 1 Draft

Status: Draft for independent implementation review
Depends on: DataCanon Core Version 1
Related: DataCanon Persistence Version 1
Scope: Optional conventional numerical and collection types, their operations, and conversion relationships with DCIs
Out of scope: DCT definitions, canonical encoding, textual syntax, and complete symbolic algebra

Purpose and boundaries

DataCanon Core defines data and identity. Core does not perform arithmetic. An implementation can also provide conventional types used directly for data and computation in its programming language. Conventional types include built-in, library-provided, and user-defined types.