A schema language and compiler. Describe a data model once and compile it to Protobuf, Postgres DDL, XML Schema, and JSON Schema, and the services that use it to OpenAPI 3.1 and gRPC. Every lossy decision gets reported.
Open source · Apache-2.0v2.11.0
Compiler pipeline. A .schemata model is parsed into a typed IR and checked by analysis passes, then lowered to each target. Anything a target can't carry becomes a diagnostic. schemata diff checks each new version against every target's compatibility rules.
01Context
Real systems describe the same data model in several dialects: Protobuf on the wire, SQL at rest, XML Schema at the edges, JSON Schema in APIs. Hand-maintaining each copy drifts, and every format silently drops things the others can express. schemata makes one .schemata file the source of truth and compiles it to all of them, or imports an existing XML Schema to get started. Version 2 adds the services that use those models, compiled to OpenAPI 3.1 and gRPC.
Schemata is a purpose-built language with a parser, a typed IR, and analysis passes. Lossy mappings are never silent: every construct a target cannot carry (a regex pattern in Protobuf, a default in SQL) becomes a diagnostic, and --strict promotes them to errors.
02Key decisions
Four decisions shaped Schemata. Each card gives the problem, the options weighed, what was chosen, and what it cost.
Split each target into lower and render
Problem
Testing a code generator by diffing its output text is what makes codegen projects "miserable to maintain". Lossy mappings also needed one place to live.
Options
Generate in one pass and test the output text against golden files, or split each target into two stages.
Chose
lower is a pure transform of the schema into a target model. It makes every representational decision and reports a diagnostic for each lossy one. render only prints a model that is already legal. Lowering is tested against the model, and golden text comparison is kept for the render stage. A review of the first two targets, Protobuf and SQL, found that neither renderer imports a Diagnostic and that the lossy diagnostic was raised at exactly one call site.
Cost
The boundary did not enforce itself. Target.compile() still rendered after a lowering error until commit 42c5a35c made it skip the render.
Mapping nested data to Postgres
Problem
Tree-shaped models have to become relational tables. The roadmap calls this the place where Schemata "either earns its keep or produces schemas a DBA would reject".
Options
A nested record could be flattened into columns, moved to a side table with a foreign key, or stored as JSONB. A repeated scalar could use a junction table, an array column, or JSONB. A sum type could be one table with a discriminator, or a table per variant.
Chose
A record gets a table only if it has a key. Each field type has a default strategy: a keyless record embeds as columns, a keyed record becomes a foreign key, a list of records becomes a child table, and a union becomes a discriminator column. embed, table, and json override the default. Names are hash-truncated at Postgres's 63-byte limit.
Cost
Some shapes have no relational mapping. A list of unions, lists, or maps is an error unless the field is set to json. Postgres does not type jsonb contents or enforce refinements on array elements, so both are reported as lossy. A list's count bounds are lossy on every strategy.
Measure before abstracting
Problem
Each target repeated similar lowering logic. Was a generic capability-profile engine warranted?
Options
A generic capability-profile engine, or a handful of small shared utilities.
Chose
The engine was deferred at two targets, because it would have been fitted to one example. At four targets the repeated code was measured: about 10.5% of 3,222 lowering lines had the same shape in three or more targets, and 2.3% were identical. The targets differ in decisions, not in yes/no abilities, so the result was six small shared utilities and no engine.
Cost
Adopting the utilities exposed nine inconsistencies between targets. Eight were fixed as behaviour changes in v0.5.1. The ninth, @deprecated dropped silently on XSD and SQL, is documented rather than diagnosed and never became a behaviour change. The change added more lines than it removed, so the gain was the fixes.
Native binaries only after a spike
Problem
A command-line tool on the JVM pays for JVM startup time.
Options
A fat jar only, or a GraalVM native-image build "if it's worth the build complexity".
Chose
A native build, after a spike. For cold check (v0.1.0, Linux x64), the jar took 0.47 s and 141 MB RSS. The native binary took effectively 0.00 s and 20.8 MB, and is 22.2 MB itself. A parity test runs the jar and the binary on the same inputs and compares exit code, output, and written files byte for byte.
Cost
Getting CI green took six dry runs. Native jobs run with the Gradle configuration cache off. The macOS minimum is forced to 12 with a linker flag, and the Linux binary needs glibc 2.34. The spike needed no reachability metadata, but the language server's lsp4j dependency does, so a drift test guards it.
03Outcome
A single command that turns one schema into consistent Protobuf, Postgres DDL, XSD, and JSON Schema, plus OpenAPI 3.1 and gRPC service definitions, with evolution checks and editor support around it. It installs with brew install msbolton/schemata/schemata or a downloaded release binary, with no JDK required.
schemata diff judges changes between two schema versions against each target's compatibility rules, so breaking changes are caught before they ship. Importers read existing XML Schema, Protobuf, Postgres DDL, and OpenAPI 3.0/3.1. schemata migrate writes the Postgres DDL between two schema versions and refuses data-losing steps unless told otherwise.
A formatter and a language server come with VS Code and Zed extensions. The build has nineteen Gradle modules, with dependency direction checked by the build. GraalVM native binaries cover Linux, macOS (x64 + ARM), and Windows, alongside a JDK 21 jar, released through CI and Homebrew.