diff --git a/.gitignore b/.gitignore index 5bafa3b..8540cca 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,6 @@ __pycache__ build dist dtschema.egg-info + +.venv +rust/target diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..5d7cc12 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,264 @@ +# AGENTS.md + +Guidance for AI agents (and humans) working in the **dt-schema** repository. + +## What this repo is + +This is **devicetree-org/dt-schema** — the `dtschema` Python package: tools and schema +data for validating Devicetree files and Devicetree *binding* documents using the +[json-schema](https://json-schema.org) vocabulary. Schema files are written in a +JSON-compatible subset of YAML so they are both human- and machine-readable. + +There are **two kinds of data files**: + +- **Schemas** (`dtschema/schemas/`) — constrain actual Devicetree *data*. This repo + holds only the *core* schemas: properties from the DT Specification plus common + bindings (GPIO, clock, PHY, interrupts, PCI, …). **Device-specific bindings do NOT + live here** — they are maintained in the Linux kernel tree alongside the `.dts` files. +- **Meta-schemas** (`dtschema/meta-schemas/`) — constrain the *schema files themselves*. + Plain json-schema silently ignores unknown keywords; the meta-schemas restrict what a + binding may contain and catch common authoring mistakes. + +License: BSD-2-Clause. Author: Rob Herring . + +## Repo layout + +| Path | Purpose | +|------|---------| +| `dtschema/` | The Python package: library modules, CLI `main()`s, and bundled `schemas/` + `meta-schemas/` data | +| `dtschema/schemas/` | Core/common Devicetree binding schemas (constrain DT data) | +| `dtschema/meta-schemas/` | Meta-schemas (constrain the binding schema files) | +| `rust/` | Additive Rust reimplementation (Cargo workspace): the `dtschema` core lib + Linux-integrated CLIs under `cli/`, command-line-compatible with the corresponding Python tools. Python stays authoritative; the Rust tree ports its behaviour and differential-tests against it | +| `test/` | Test suite `test-dt-validate.py`, `.dts` fixtures, example schemas under `test/schemas/` | +| `tools/` | Standalone helper scripts: `dt-prop-populate`, `yaml-format`, `yaml2json` | +| `.github/workflows/` | `ci.yml` (lint + test matrix) and `publish.yml` (PyPI on tags) | +| `pyproject.toml` | Package metadata, deps, and `[project.scripts]` entry points | +| `example-schema.yaml` | Annotated reference template for authoring a new binding | +| `.yamllint` | YAML lint config (enforced in CI) | +| `README.md` | User-facing docs (see note below) | + +> **Note:** `README.md` refers to `tools/dt-validate`, `tools/dt-mk-schema`, etc. That +> wording is dated — those are installed as **console entry points** (see below), not +> scripts in `tools/`. The `tools/` directory only contains the three helper scripts. +> There is no `Makefile`, `setup.py`, `setup.cfg`, `tox.ini`, or `CONTRIBUTING`. + +## CLI tools + +Console entry points are declared in `pyproject.toml` under `[project.scripts]`; each +maps to a `main()` in a `dtschema/*.py` module. All argparse tools accept `@file` to +read arguments from a file (`fromfile_prefix_chars='@'`). + +| Command | Module | Purpose | +|---------|--------|---------| +| `dt-validate` | `dtb_validate.py` | Validate DTB(s) (or a dir of `**/*.dtb`) against schemas | +| `dt-doc-validate` | `doc_validate.py` | Validate binding YAML file(s) against the meta-schema | +| `dt-mk-schema` | `mk_schema.py` | Preprocess schemas into a single processed schema file | +| `dt-check-compatible` | `check_compatible.py` | Test whether compatible string(s) are documented | +| `dt-extract-example` | `extract_example.py` | Emit the DTS example(s) from a binding (pipe to `dtc`) | +| `dt-extract-props` | `extract_props.py` | Dump the property→type(s) map derived from schemas | +| `dt-cmp-schema` | `cmp_schema.py` | Compare two schema sets for possible ABI regressions | +| `dtb2py` | `dtb2py.py` | Decode a DTB into a Python dict dump | + +Also present but **not** console entry points: `dtschema/extract_compatibles.py` (a +runnable module that prints `enum` compatibles from a single binding) and the `tools/` +helpers (`yaml2json`, `yaml-format`, `dt-prop-populate`). + +## Core library & how validation works + +Module map (all under `dtschema/`): + +- **`lib.py`** — low-level helpers: `sized_int` (an `int` carrying a bit `.size`), + `_is_int_schema`/`_is_string_schema`, `extract_compatibles`, `_get_array_range`, and + `format_error` (the central human-readable error formatter). +- **`schema.py`** — `DTSchema` represents **one** binding file, loads its YAML, + meta-validates it (`iter_errors`/`is_valid`) against the meta-schema named by the + binding's `$schema`, applies `fixup()`, and runs `check_schema_refs()` (verifies `$id` + matches the file path, refs resolve, and node schemas carry an + additional/unevaluatedProperties constraint). +- **`fixups.py`** — `fixup_schema` expands the compact DT authoring syntax into strict + json-schema before validation: string→array, `items` list→fixed-size array (adds + `minItems`/`maxItems`), unit-suffix typing (`-hz`, `-microvolt`, `-ohms`, …), + `interrupts`/`interrupts-extended` handling, and implicit node props (`phandle`, + `status`, `pinctrl-*`, `bootph-*`, …). +- **`validator.py`** — `DTValidator` loads/preprocesses **all** schemas, builds a + `compat_map` (compatible → schema `$id`) and `always_schemas` (schemas with a + `select`), exposes `iter_errors`, the custom `typeSize` keyword, the property-type + cache, and the synthetic `generated-compatibles` schema. +- **`dtb.py`** — decodes a flattened DTB via `pylibfdt` into a typed nested Python tree, + using the property-type cache to turn raw bytes into ints / strings / matrices / + phandle tuples. + +Two validation flows: + +- **Flow A — binding vs meta-schema** (`dt-doc-validate`): `DTSchema.iter_errors()` + validates the binding against the meta-schema referenced by its `$schema`, then + `check_schema_refs()` checks `$id`/references. +- **Flow B — DTB vs schemas** (`dt-validate`): `decode_dtb` unflattens the DTB, then + each node is validated by (1) the schema matched from its first known `compatible` in + `compat_map`, and (2) every `select`-bearing `always_schemas` entry applied as + `{if: select, then: schema}`. Disabled nodes suppress `required`/`unevaluatedProperties` + errors. Nodes matching no schema are surfaced only with `-m/--show-unmatched`. + +Preprocessing & caching: + +- **`dt-mk-schema`** serializes the fully processed `.schemas` dict (including + `generated-types`, `generated-pattern-types`, `generated-compatibles`, and a `version` + stamp). `DTValidator` can reload that file directly to skip all fixup/type work; + a `version` mismatch raises *"Processed schema out of date, delete and retry"*. +- **`dt-validate --cache-dir`** is a *separate* per-DTB diagnostics cache: one + `.json` per DTB, keyed on cache/dtschema versions + DTB hash + schema hash + + options, with file paths normalized to the `$dtb` sentinel so entries are + path-independent. + +`dt-validate` also supports structured/CI-friendly output: `--json-output` (writes +diagnostics as JSON), relative in-tree DTB paths (`_display_path`), and the `--cache-dir` +diagnostics cache above — all implemented in `dtschema/dtb_validate.py`. + +## Dev setup & build + +Editable install against this tree (pulls all deps from `pyproject.toml`): + +``` +pip3 install -e . +``` + +> **Modern Linux note:** most current distros mark the system Python as +> *externally managed* (PEP 668), so a bare `pip3 install` fails unless you're inside a +> virtualenv. Either work in a venv (`python3 -m venv .venv && . .venv/bin/activate`, +> then `pip install -e .`), or use **pipx** to install the CLIs into an isolated +> environment: +> ``` +> pipx install dtschema # end users +> pipx install --editable . # this tree, for development +> pipx install git+https://github.com/devicetree-org/dt-schema.git@main +> ``` +> pipx puts the `dt-*` executables on `PATH` (`~/.local/bin`) while keeping their deps +> off the system Python. For editable dev work where you `import dtschema` in your own +> scripts/tests, prefer a venv. + +Runtime deps: `ruamel.yaml>0.15.69`, `jsonschema>=4.18`, `rfc3987`, `pylibfdt` +(building pylibfdt needs `swig`). The `dtc` device-tree-compiler is needed to produce +DTBs for tests. Version is dynamic via `setuptools_scm` (writes `dtschema/version.py`). + +> **Sanity check your install:** if `dt-*` binaries are already on `PATH` from a distro +> package or an old `pip`/`pipx` install, they can shadow this tree — confirm with +> `dt-validate --version` and `python3 -c "import dtschema; print(dtschema.__file__)"`. +> A base `python3` that can't `import dtschema` (e.g. missing `referencing`, a +> `jsonschema` dep) just means the deps aren't on that interpreter; install into a +> venv/pipx as above. + +## Testing & CI + +Tests are a single self-executing `unittest` script (no pytest/tox config). It needs +`dtc` on `PATH`. + +``` +test/test-dt-validate.py # or: python -m unittest test/test-dt-validate.py +``` + +It covers: all meta-schemas are valid Draft2019-09; the good/bad example schemas +pass/fail as expected; **every** bundled `dtschema/schemas/**/*.yaml` validates against +the meta-schema, has a unique `$id`, and resolves its refs; and DTB fixtures — files +named `*-fail.dts` must raise `ValidationError`, others must pass. + +CI (`.github/workflows/ci.yml`) runs on push/PR across Python 3.9–3.14: + +``` +flake8 . --select=E9,F63,F7,F82 --show-source --statistics # fatal +flake8 . --exit-zero --max-complexity=10 --max-line-length=127 --statistics # advisory +yamllint --strict $(git ls-files '*.yaml') +test/test-dt-validate.py +``` + +`publish.yml` builds with `python -m build` and publishes to PyPI on `v*` tags +(excluding `v*-pre`). + +## Conventions for editing schemas + +- Use **`example-schema.yaml`** (repo root) as the authoring template — it is heavily + annotated with the meaning of each keyword. For a minimal binding that is *guaranteed + valid* (the test suite asserts it passes strict meta-schema validation), copy + `test/schemas/good-example.yaml`; `test/schemas/bad-example.yaml` is the deliberately + invalid counterpart. Reformat with `tools/yaml-format`. +- YAML style is enforced by `.yamllint`: 2-space indentation (sequences indented), + single quotes only-when-needed, files must start with `---` (`document-start`), + no empty values, line-length warning at 110. Property names starting with `#` (e.g. + `'#interrupt-cells'`) must be quoted. +- Every binding needs `$id` (a `http://devicetree.org/schemas/...` URL matching its + path), `$schema` (usually `.../meta-schemas/core.yaml#`), `title`, `maintainers`, and + an `additionalProperties`/`unevaluatedProperties` constraint on node schemas. +- After editing a binding, run `dt-doc-validate ` and the full test suite. + +## Quick command reference + +``` +# editable dev install (deps from pyproject.toml) +# on PEP 668 distros, do this in a venv, or use: pipx install --editable . +pip3 install -e . + +# run the test suite (needs dtc) +test/test-dt-validate.py + +# lint exactly as CI does +flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics +yamllint --strict $(git ls-files '*.yaml') + +# validate a binding against the meta-schema +dt-doc-validate test/schemas/good-example.yaml + +# build a processed schema, then validate a DTB against it +dt-mk-schema -j test/schemas/ > processed-schema.json +dtc -O dtb -o device.dtb test/device.dts +dt-validate -s processed-schema.json device.dtb + +# check whether a compatible is documented +dt-check-compatible -s processed-schema.json vendor,a-compatible +``` + +## Linux kernel integration + +dt-schema is the engine behind the kernel's `make dt_binding_check` and +`make dtbs_check` targets — but those Makefile targets live in the Linux source tree +(`Documentation/devicetree/bindings/Makefile`), **not** here. The integration contract +is the installed CLIs (`dt-mk-schema`, `dt-doc-validate`, `dt-validate`, +`dt-extract-example`), which the kernel runs against its own in-tree bindings. + +The core schemas bundled here are **always** merged in during processing: +`process_schemas()` unconditionally appends this package's `schemas/` directory +(`core_schema=True`), so the kernel's bindings are validated alongside them. Note +`dt-mk-schema`'s `-u/--useronly` flag is currently a **no-op** (declared in argparse but +never read), and the `-u` on `dt-validate`/`dt-doc-validate` is the unrelated, +deprecated `--url-path` — don't confuse the two. + +## AI-assisted contribution rules + +dt-schema feeds directly into the Linux kernel workflow, so contributions here follow +the kernel's expectations for AI-assisted work — see +. In short: + +- **A human is accountable.** The human contributor must review all AI-generated code, + ensure it is correct and licensing-compliant, and take full responsibility for it. An + AI assistant is a tool, not a submitter. +- **AI agents MUST NOT add `Signed-off-by` tags.** Only a human can certify the + [Developer Certificate of Origin](https://developercertificate.org/); the human + submitter adds their own `Signed-off-by`. +- **Disclose assistance with an `Assisted-by` trailer** in the commit message, using the + format `Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]` — e.g. + `Assisted-by: Claude:claude-opus-4 coccinelle sparse`. List specialized analysis tools + (coccinelle, sparse, smatch, …) but **not** basic tooling (git, make, editors). +- **Licensing:** keep the `# SPDX-License-Identifier: BSD-2-Clause` header on new + Python/YAML files and match the existing copyright style. (Note: dt-schema itself is + BSD-2-Clause; the kernel tree it serves is GPL-2.0-only.) +- Otherwise follow the normal process in this file: pass the test suite, `flake8` + (fatal set) and `yamllint --strict`, and keep changes minimal and in-style. + +## Keeping this file current + +Treat `AGENTS.md` as living documentation. When a change would make anything here +inaccurate, update it in the **same** commit — for example: adding/renaming/removing a +CLI entry point in `pyproject.toml`, changing a tool's flags or behavior, adding or +reorganizing library modules or `schemas/`/`meta-schemas/` directories, bumping the +supported Python range or dependencies, or altering the CI/lint/test commands. If you +notice this file has already drifted from the code, fix it as part of your change rather +than leaving it stale. `CLAUDE.md` is a symlink to this file, so there is only one place +to edit. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 0000000..47dc3e3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/rust/Cargo.lock b/rust/Cargo.lock new file mode 100644 index 0000000..1419b16 --- /dev/null +++ b/rust/Cargo.lock @@ -0,0 +1,1385 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if", + "getrandom", + "once_cell", + "serde", + "version_check", + "zerocopy", +] + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "allocator-api2" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923" + +[[package]] +name = "anstream" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + +[[package]] +name = "anstyle-parse" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys", +] + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "argfile" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0a1cc0ba69de57db40674c66f7cf2caee3981ddef084388482c95c0e2133e5e8" +dependencies = [ + "fs-err", + "os_str_bytes", +] + +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + +[[package]] +name = "bit-set" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3" +dependencies = [ + "bit-vec", +] + +[[package]] +name = "bit-vec" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "borrow-or-share" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc0b364ead1874514c8c2855ab558056ebfeb775653e7ae45ff72f28f8f3166c" + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "bytecount" +version = "0.6.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "175812e0be2bccb6abe50bb8d566126198344f707e304f45c648fd8f2cc0365e" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "clap" +version = "4.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d91e0c145792ef73a6ad36d27c75ac09f1832222a3c209689d90f534685ee5b7" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f09628afdcc538b57f3c6341e9c8e9970f18e4a481690a64974d7023bd33548b" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_derive" +version = "4.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d012d2b9d65aca7f18f4d9878a045bc17899bba951561ba5ec3c2ba1eed9a061" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "clap_lex" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" + +[[package]] +name = "colorchoice" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "crossbeam-deque" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb" +dependencies = [ + "crossbeam-epoch", + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-epoch" +version = "0.9.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" + +[[package]] +name = "crypto-common" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "data-encoding" +version = "2.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4ae5f15dda3c708c0ade84bfee31ccab44a3da4f88015ed22f63732abe300c8" + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer", + "crypto-common", +] + +[[package]] +name = "displaydoc" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ac70aa55017e108007fbaf5aa0f54b021c98f92ff8af59d42eda9da96e3dd4f" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "dt-check-compatible" +version = "0.1.0" +dependencies = [ + "clap", + "dtschema", +] + +[[package]] +name = "dt-doc-validate" +version = "0.1.0" +dependencies = [ + "clap", + "dtschema", +] + +[[package]] +name = "dt-extract-example" +version = "0.1.0" +dependencies = [ + "clap", + "dtschema", + "regex", +] + +[[package]] +name = "dt-mk-schema" +version = "0.1.0" +dependencies = [ + "anyhow", + "argfile", + "clap", + "dtschema", + "serde_json", + "serde_yaml", +] + +[[package]] +name = "dt-validate" +version = "0.1.0" +dependencies = [ + "anyhow", + "argfile", + "clap", + "dtschema", + "rayon", + "serde_json", +] + +[[package]] +name = "dtschema" +version = "0.1.0" +dependencies = [ + "anyhow", + "fancy-regex 0.14.0", + "fdt", + "jsonschema", + "rayon", + "regex", + "serde", + "serde_json", + "serde_yaml", + "sha2", + "thiserror", +] + +[[package]] +name = "either" +version = "1.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e5e8f6c15a24b9a3ee5efec809ccd006d3b30e8b3bb63c39af737c7f87daa1d" + +[[package]] +name = "email_address" +version = "0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e079f19b08ca6239f47f8ba8509c11cf3ea30095831f7fed61441475edd8c449" +dependencies = [ + "serde", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "fancy-regex" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e24cb5a94bcae1e5408b0effca5cd7172ea3c5755049c5f3af4cd283a165298" +dependencies = [ + "bit-set", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "fancy-regex" +version = "0.18.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e1e1dacd0d2082dfcf1351c4bdd566bbe89a2b263235a2b50058f1e130a47277" +dependencies = [ + "bit-set", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "fdt" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "784a4df722dc6267a04af36895398f59d21d07dce47232adf31ec0ff2fa45e67" + +[[package]] +name = "fluent-uri" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc74ac4d8359ae70623506d512209619e5cf8f347124910440dbc221714b328e" +dependencies = [ + "borrow-or-share", + "ref-cast", + "serde", +] + +[[package]] +name = "foldhash" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" + +[[package]] +name = "fraction" +version = "0.15.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e076045bb43dac435333ed5f04caf35c7463631d0dae2deb2638d94dd0a5b872" +dependencies = [ + "lazy_static", + "num", +] + +[[package]] +name = "fs-err" +version = "2.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "88a41f105fe1d5b6b34b2055e3dc59bb79b46b48b2040b9e6c7b4b5de097aa41" +dependencies = [ + "autocfg", +] + +[[package]] +name = "generic-array" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "getrandom" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" +dependencies = [ + "cfg-if", + "js-sys", + "libc", + "r-efi", + "wasip2", + "wasm-bindgen", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" +dependencies = [ + "allocator-api2", + "equivalent", + "foldhash", +] + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "icu_collections" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2984d1cd16c883d7935b9e07e44071dca8d917fd52ecc02c04d5fa0b5a3f191c" +dependencies = [ + "displaydoc", + "potential_utf", + "utf8_iter", + "yoke", + "zerofrom", + "zerovec", +] + +[[package]] +name = "icu_locale_core" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92219b62b3e2b4d88ac5119f8904c10f8f61bf7e95b640d25ba3075e6cac2c29" +dependencies = [ + "displaydoc", + "litemap", + "tinystr", + "writeable", + "zerovec", +] + +[[package]] +name = "icu_normalizer" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c56e5ee99d6e3d33bd91c5d85458b6005a22140021cc324cea84dd0e72cff3b4" +dependencies = [ + "icu_collections", + "icu_normalizer_data", + "icu_properties", + "icu_provider", + "smallvec", + "zerovec", +] + +[[package]] +name = "icu_normalizer_data" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da3be0ae77ea334f4da67c12f149704f19f81d1adf7c51cf482943e84a2bad38" + +[[package]] +name = "icu_properties" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bee3b67d0ea5c2cca5003417989af8996f8604e34fb9ddf96208a033901e70de" +dependencies = [ + "icu_collections", + "icu_locale_core", + "icu_properties_data", + "icu_provider", + "zerotrie", + "zerovec", +] + +[[package]] +name = "icu_properties_data" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e2bbb201e0c04f7b4b3e14382af113e17ba4f63e2c9d2ee626b720cbce54a14" + +[[package]] +name = "icu_provider" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "139c4cf31c8b5f33d7e199446eff9c1e02decfc2f0eec2c8d71f65befa45b421" +dependencies = [ + "displaydoc", + "icu_locale_core", + "writeable", + "yoke", + "zerofrom", + "zerotrie", + "zerovec", +] + +[[package]] +name = "idna" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de" +dependencies = [ + "idna_adapter", + "smallvec", + "utf8_iter", +] + +[[package]] +name = "idna_adapter" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714" +dependencies = [ + "icu_normalizer", + "icu_properties", +] + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown", +] + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53b44bfcdb3f8d5837a46dae1ca9660a837176eee74a28b229bc626816589102" +dependencies = [ + "cfg-if", + "wasm-bindgen", +] + +[[package]] +name = "jsonschema" +version = "0.49.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8a77951c56b6a0af03c22af6953d6ffcfba9403c765578921f3f1089b999db6" +dependencies = [ + "ahash", + "bytecount", + "data-encoding", + "email_address", + "fancy-regex 0.18.0", + "fraction", + "getrandom", + "idna", + "itoa", + "jsonschema-regex", + "jsonschema-value", + "num-cmp", + "num-traits", + "percent-encoding", + "referencing", + "regex", + "serde", + "serde_json", + "strum", + "unicode-general-category", + "uuid-simd", +] + +[[package]] +name = "jsonschema-regex" +version = "0.49.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4c2e64f341a1d6a15d2daa3c967e48921cf15b9c7f23a9899e8b63c7f7c4199" +dependencies = [ + "regex-syntax", +] + +[[package]] +name = "jsonschema-value" +version = "0.49.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94e097778f3e9c6a33862077dbf07aca0360d0b18d2a7abc323da132df49ee83" +dependencies = [ + "ahash", + "bytecount", + "fraction", + "num-cmp", + "num-traits", + "serde_json", +] + +[[package]] +name = "lazy_static" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" + +[[package]] +name = "libc" +version = "0.2.189" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" + +[[package]] +name = "litemap" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0" + +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "micromap" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a86d3146ed3995b5913c414f6664344b9617457320782e64f0bb44afd49d74" + +[[package]] +name = "num" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23" +dependencies = [ + "num-bigint", + "num-complex", + "num-integer", + "num-iter", + "num-rational", + "num-traits", +] + +[[package]] +name = "num-bigint" +version = "0.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367" +dependencies = [ + "num-integer", + "num-traits", +] + +[[package]] +name = "num-cmp" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63335b2e2c34fae2fb0aa2cecfd9f0832a1e24b3b32ecec612c3426d46dc8aaa" + +[[package]] +name = "num-complex" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-integer" +version = "0.1.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7969661fd2958a5cb096e56c8e1ad0444ac2bbcd0061bd28660485a44879858f" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-iter" +version = "0.1.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c92800bd69a1eac91786bcfe9da64a897eb72911b8dc3095decbd07429e8048b" +dependencies = [ + "num-integer", + "num-traits", +] + +[[package]] +name = "num-rational" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824" +dependencies = [ + "num-bigint", + "num-integer", + "num-traits", +] + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "os_str_bytes" +version = "7.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73b489f051c9d2f4299ddf8d55e33c65c86a7b76f49359af15f79ebe541d1595" +dependencies = [ + "memchr", +] + +[[package]] +name = "outref" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1a80800c0488c3a21695ea981a54918fbb37abf04f4d0720c453632255e2ff0e" + +[[package]] +name = "parking_lot" +version = "0.12.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-link", +] + +[[package]] +name = "percent-encoding" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" + +[[package]] +name = "potential_utf" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0103b1cef7ec0cf76490e969665504990193874ea05c85ff9bab8b911d0a0564" +dependencies = [ + "zerovec", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "5.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" + +[[package]] +name = "rayon" +version = "1.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d" +dependencies = [ + "either", + "rayon-core", +] + +[[package]] +name = "rayon-core" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91" +dependencies = [ + "crossbeam-deque", + "crossbeam-utils", +] + +[[package]] +name = "redox_syscall" +version = "0.5.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" +dependencies = [ + "bitflags", +] + +[[package]] +name = "ref-cast" +version = "1.0.26" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "216e8f773d7923bcba9ceb86a86c93cabb3903a11872fc3f138c49630e50b96d" +dependencies = [ + "ref-cast-impl", +] + +[[package]] +name = "ref-cast-impl" +version = "1.0.26" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2c9283685feec7d69af75fb0e858d5e7378f33fe4fc699383b2916ab9273e03c" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "referencing" +version = "0.49.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b772e96f8eb6badd4eb10fbc08b8a98244c09b4af37e72e8fa612c854604b870" +dependencies = [ + "ahash", + "fluent-uri", + "getrandom", + "hashbrown", + "itoa", + "micromap", + "parking_lot", + "percent-encoding", + "serde_json", +] + +[[package]] +name = "regex" +version = "1.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fcfdb36bda0c880c5931cdc7a2bcdc8ba4556847b9d912bca70bc94708711ad" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "ryu" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "serde_json" +version = "1.0.151" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14" +dependencies = [ + "indexmap", + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_yaml" +version = "0.9.34+deprecated" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a8b1a1a2ebf674015cc02edccce75287f1a0130d394307b36743c2f5d504b47" +dependencies = [ + "indexmap", + "itoa", + "ryu", + "serde", + "unsafe-libyaml", +] + +[[package]] +name = "sha2" +version = "0.10.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "smallvec" +version = "1.15.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" + +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "strum" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9628de9b8791db39ceda2b119bbe13134770b56c138ec1d3af810d045c04f9bd" +dependencies = [ + "strum_macros", +] + +[[package]] +name = "strum_macros" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ab85eea0270ee17587ed4156089e10b9e6880ee688791d45a905f5b1ca36f664" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "synstructure" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "thiserror" +version = "2.0.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09a43598840e33d5b0331f38c5e30d13bb11c11210a4b58f0d9b18a5a5eefcd9" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43cbfe0cf76104d42a574802844187e84a305e531ed54455f11fbde0f10541cd" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "tinystr" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8323304221c2a851516f22236c5722a72eaa19749016521d6dff0824447d96d" +dependencies = [ + "displaydoc", + "zerovec", +] + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "unicode-general-category" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b993bddc193ae5bd0d623b49ec06ac3e9312875fdae725a975c51db1cc1677f" + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "unsafe-libyaml" +version = "0.2.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "673aac59facbab8a9007c7f6108d11f63b603f7cabff99fabf650fea5c32b861" + +[[package]] +name = "utf8_iter" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "uuid-simd" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23b082222b4f6619906941c17eb2297fff4c2fb96cb60164170522942a200bd8" +dependencies = [ + "outref", + "vsimd", +] + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "vsimd" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c3082ca00d5a5ef149bb8b555a72ae84c9c59f7250f013ac822ac2e49b19c64" + +[[package]] +name = "wasip2" +version = "1.0.4+wasi-0.2.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b67efb37e106e55ce722a510d6b5f9c17f083e5fc79afc2badeb12cc313d9487" +dependencies = [ + "wit-bindgen", +] + +[[package]] +name = "wasm-bindgen" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b067c0c11094aef6b7a801c1e34a26affafdf3d051dba08456b868789aaf9a4" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "167ce5e579f6bcf889c4f7175a8a5a585de84e8ff93976ce393efa5f2837aab1" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3997c7839262f4ef12cf90b818d6340c18e80f263f1a94bf157d0ec4420380e" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 2.0.119", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc1b4cb0cc549fcf58d7dfc081778139b3d283a081644e833e84682ad71cea24" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "wit-bindgen" +version = "0.57.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e" + +[[package]] +name = "writeable" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" + +[[package]] +name = "yoke" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "709fe23a0424b6a435d82152b1bd3fdfb0833487d5fa90d05d42762a9891fef5" +dependencies = [ + "stable_deref_trait", + "yoke-derive", + "zerofrom", +] + +[[package]] +name = "yoke-derive" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + +[[package]] +name = "zerocopy" +version = "0.8.55" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b5a105cd7b140f6eeec8acff2ea38135d3cab283ada58540f629fe51e46696eb" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.55" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fe976fb70c78cd64cccfe3a6fc142244e8a77b70959b30faf9d0ac37ee228eb" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "zerofrom" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ec05a11813ea801ff6d75110ad09cd0824ddba17dfe17128ea0d5f68e6c5272" +dependencies = [ + "zerofrom-derive", +] + +[[package]] +name = "zerofrom-derive" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + +[[package]] +name = "zerotrie" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0f9152d31db0792fa83f70fb2f83148effb5c1f5b8c7686c3459e361d9bc20bf" +dependencies = [ + "displaydoc", + "yoke", + "zerofrom", +] + +[[package]] +name = "zerovec" +version = "0.11.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "90f911cbc359ab6af17377d242225f4d75119aec87ea711a880987b18cd7b239" +dependencies = [ + "yoke", + "zerofrom", + "zerovec-derive", +] + +[[package]] +name = "zerovec-derive" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" diff --git a/rust/Cargo.toml b/rust/Cargo.toml new file mode 100644 index 0000000..6cdd737 --- /dev/null +++ b/rust/Cargo.toml @@ -0,0 +1,37 @@ +# SPDX-License-Identifier: BSD-2-Clause +# Copyright 2026 dt-schema contributors +[workspace] +resolver = "2" +members = [ + "dtschema", + "cli/dt-validate", + "cli/dt-doc-validate", + "cli/dt-mk-schema", + "cli/dt-check-compatible", + "cli/dt-extract-example", +] + +[workspace.package] +version = "0.1.0" +edition = "2024" +license = "BSD-2-Clause" +repository = "https://github.com/devicetree-org/dt-schema" + +[workspace.dependencies] +dtschema = { path = "dtschema" } +serde = { version = "1", features = ["derive"] } +serde_json = { version = "1", features = ["preserve_order"] } +serde_yaml = "0.9" +jsonschema = { version = "0.49", default-features = false } +fdt = "0.1.5" +clap = { version = "4", features = ["derive"] } +argfile = "0.2" +regex = "1" +fancy-regex = "0.14" +sha2 = "0.10" +rayon = "1" +anyhow = "1" +thiserror = "2" + +[profile.release] +lto = "thin" diff --git a/rust/cli/dt-check-compatible/Cargo.toml b/rust/cli/dt-check-compatible/Cargo.toml new file mode 100644 index 0000000..08841e1 --- /dev/null +++ b/rust/cli/dt-check-compatible/Cargo.toml @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-2-Clause +# Copyright 2026 dt-schema contributors +[package] +name = "dt-check-compatible" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true + +[[bin]] +name = "dt-check-compatible" +path = "src/main.rs" + +[dependencies] +dtschema.workspace = true +clap.workspace = true diff --git a/rust/cli/dt-check-compatible/src/main.rs b/rust/cli/dt-check-compatible/src/main.rs new file mode 100644 index 0000000..d827837 --- /dev/null +++ b/rust/cli/dt-check-compatible/src/main.rs @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! `dt-check-compatible`: check whether compatible strings are documented by +//! the schema set. +//! +//! Accepts positional `compatible_str...`, `-q/--quiet`, `-v/--invert-match`, +//! `-s/--schema` (required), and `-V/--version`. + +use std::path::{Path, PathBuf}; +use std::process::ExitCode; + +use clap::Parser; +use dtschema::validator::DTValidator; + +#[derive(Parser)] +#[command(disable_version_flag = true)] +struct Args { + /// 1 or more compatible strings to check for a match. + #[arg(required = true)] + compatible_str: Vec, + /// Suppress printing matches. + #[arg(short = 'q', long = "quiet")] + quiet: bool, + /// Invert sense of matching, printing compatibles which don't match. + #[arg(short = 'v', long = "invert-match")] + invert_match: bool, + /// Path to processed schema file or schema directory. + #[arg(short = 's', long = "schema")] + schema: String, + /// Print version number. + #[arg(short = 'V', long = "version")] + version: bool, +} + +fn main() -> ExitCode { + let args = Args::parse(); + + if args.version { + println!("{}", dtschema::version()); + return ExitCode::SUCCESS; + } + + if !args.schema.is_empty() && !Path::new(&args.schema).exists() { + return failure(); + } + + let validator = match DTValidator::new(&[PathBuf::from(&args.schema)]) { + Ok(v) => v, + Err(e) => { + eprintln!("{e}"); + return failure(); + } + }; + + let undoc = validator.get_undocumented_compatibles(&args.compatible_str); + + if args.invert_match { + if !undoc.is_empty() { + if !args.quiet { + println!("{}", undoc.join("\n")); + } + return ExitCode::SUCCESS; + } + } else { + // Matches = inputs that ARE documented. Preserve input order and drop + // duplicates. + let mut seen = std::collections::HashSet::new(); + let matches: Vec<&String> = args + .compatible_str + .iter() + .filter(|c| !undoc.contains(c) && seen.insert((*c).clone())) + .collect(); + if !matches.is_empty() { + if !args.quiet { + for m in matches { + println!("{m}"); + } + } + return ExitCode::SUCCESS; + } + } + + failure() +} + +/// Legacy failure status. +fn failure() -> ExitCode { + ExitCode::from(255) +} diff --git a/rust/cli/dt-check-compatible/tests/cli.rs b/rust/cli/dt-check-compatible/tests/cli.rs new file mode 100644 index 0000000..9593046 --- /dev/null +++ b/rust/cli/dt-check-compatible/tests/cli.rs @@ -0,0 +1,90 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! CLI integration tests for `dt-check-compatible`. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +fn dt_check_compatible_bin() -> PathBuf { + PathBuf::from(env!("CARGO_BIN_EXE_dt-check-compatible")) +} + +fn run(args: &[&str]) -> (i32, String, String) { + let out = Command::new(dt_check_compatible_bin()) + .args(args) + .output() + .expect("run dt-check-compatible"); + ( + out.status.code().unwrap_or(-1), + String::from_utf8_lossy(&out.stdout).into_owned(), + String::from_utf8_lossy(&out.stderr).into_owned(), + ) +} + +fn assert_documented(schema: &Path) { + let (rc, stdout, _stderr) = run(&["-s", schema.to_str().unwrap(), "vendor,soc1-ip"]); + assert_eq!(rc, 0); + assert_eq!(stdout, "vendor,soc1-ip\n"); +} + +fn assert_undocumented_invert(schema: &Path) { + let (rc, stdout, _stderr) = run(&["-s", schema.to_str().unwrap(), "-v", "vendor,missing"]); + assert_eq!(rc, 0); + assert_eq!(stdout, "vendor,missing\n"); +} + +fn write_processed_schema(path: &Path) { + let version = dtschema::version(); + std::fs::write( + path, + format!( + r#"{{ + "generated-compatibles": {{ + "$id": "generated-compatibles", + "$filename": "Generated schema of documented compatible strings", + "select": true, + "properties": {{ + "compatible": {{ + "items": {{ + "anyOf": [ + {{ "enum": ["vendor,soc1-ip"] }} + ] + }} + }} + }} + }}, + "version": "{version}" +}} +"# + ), + ) + .unwrap(); +} + +#[test] +fn schema_directory_and_processed_file() { + let repo = repo_root(); + let schemas = repo.join("test/schemas"); + assert_documented(&schemas); + assert_undocumented_invert(&schemas); + + let tmp = std::env::temp_dir().join(format!("dt-check-compatible-cli-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&tmp); + std::fs::create_dir_all(&tmp).unwrap(); + let processed = tmp.join("schema.json"); + write_processed_schema(&processed); + + assert_documented(&processed); + assert_undocumented_invert(&processed); +} diff --git a/rust/cli/dt-doc-validate/Cargo.toml b/rust/cli/dt-doc-validate/Cargo.toml new file mode 100644 index 0000000..17502e1 --- /dev/null +++ b/rust/cli/dt-doc-validate/Cargo.toml @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-2-Clause +# Copyright 2026 dt-schema contributors +[package] +name = "dt-doc-validate" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true + +[[bin]] +name = "dt-doc-validate" +path = "src/main.rs" + +[dependencies] +dtschema.workspace = true +clap.workspace = true diff --git a/rust/cli/dt-doc-validate/src/main.rs b/rust/cli/dt-doc-validate/src/main.rs new file mode 100644 index 0000000..52122e0 --- /dev/null +++ b/rust/cli/dt-doc-validate/src/main.rs @@ -0,0 +1,102 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! `dt-doc-validate`: meta-validate binding schema YAML files. +//! +//! Accepts positional `yamldt...`, `-v/--verbose`, `-n/--line-number` +//! (obsolete here; DTBs/loads carry no positions), `-u/--url-path`, and +//! `-V/--version`. Exits non-zero if any file has errors. + +use std::path::{Path, PathBuf}; +use std::process::ExitCode; + +use clap::Parser; +use dtschema::schema::DTSchema; + +#[derive(Parser)] +#[command(disable_version_flag = true)] +struct Args { + /// Directory or filename of YAML encoded devicetree schema file(s). + yamldt: Vec, + /// Verbose mode. + #[arg(short = 'v', long = "verbose")] + verbose: bool, + /// Print line and column numbers (obsolete, accepted for compatibility). + #[arg(short = 'n', long = "line-number")] + line_number: bool, + /// Additional search path for references. + #[arg(short = 'u', long = "url-path")] + url_path: Option, + /// Print version number. + #[arg(short = 'V', long = "version")] + version: bool, +} + +fn main() -> ExitCode { + let args = Args::parse(); + + if args.version { + println!("{}", dtschema::version()); + return ExitCode::SUCCESS; + } + let _ = (args.line_number, &args.url_path, args.verbose); + + let mut ret = 0u8; + for f in &args.yamldt { + if f.is_dir() { + let mut files = Vec::new(); + collect_yaml(f, &mut files); + for filename in files { + ret |= check_doc(&filename); + } + } else { + ret |= check_doc(f); + } + } + + ExitCode::from(ret) +} + +/// Meta-validate one file, print each error, then run the reference/constraint +/// check. Returns 1 if the file had validation errors. +fn check_doc(filename: &Path) -> u8 { + let dtsch = match DTSchema::load(filename) { + Ok(s) => s, + Err(e) => { + eprintln!("{}: {e}", filename.display()); + return 1; + } + }; + + let mut ret = 0; + match dtsch.format_errors() { + Ok(errors) => { + for e in errors { + eprintln!("{e}"); + ret = 1; + } + } + Err(e) => { + eprintln!("{}: error checking schema file: {e}", filename.display()); + return 1; + } + } + + dtsch.check_schema_refs(); + ret +} + +/// Recursively collect `*.yaml` files under `dir`, sorted for stable output. +fn collect_yaml(dir: &Path, out: &mut Vec) { + let Ok(rd) = std::fs::read_dir(dir) else { + return; + }; + let mut entries: Vec = rd.flatten().map(|e| e.path()).collect(); + entries.sort(); + for p in entries { + if p.is_dir() { + collect_yaml(&p, out); + } else if p.extension().and_then(|s| s.to_str()) == Some("yaml") { + out.push(p); + } + } +} diff --git a/rust/cli/dt-extract-example/Cargo.toml b/rust/cli/dt-extract-example/Cargo.toml new file mode 100644 index 0000000..1c8e57b --- /dev/null +++ b/rust/cli/dt-extract-example/Cargo.toml @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-2-Clause +# Copyright 2026 dt-schema contributors +[package] +name = "dt-extract-example" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true + +[[bin]] +name = "dt-extract-example" +path = "src/main.rs" + +[dependencies] +dtschema.workspace = true +clap.workspace = true +regex.workspace = true diff --git a/rust/cli/dt-extract-example/src/main.rs b/rust/cli/dt-extract-example/src/main.rs new file mode 100644 index 0000000..7972f79 --- /dev/null +++ b/rust/cli/dt-extract-example/src/main.rs @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Emits the DTS example(s) from a binding YAML file, wrapped in a minimal +//! device tree so the output can be piped to `dtc`. This tool is fully +//! self-contained templating; it does not use the dtschema validation library +//! beyond the shared YAML loader. + +use std::path::PathBuf; +use std::process::ExitCode; + +use clap::Parser; +use regex::Regex; + +// Template strings preserve the legacy output format. Literal braces are +// written directly and `{...}` placeholders are substituted by hand. + +// interrupt_template, with `{index}` and `{int_cells}` substituted. +fn interrupt_template(index: usize, int_cells: usize) -> String { + format!( + "\n interrupt-parent = <&fake_intc{index}>;\n fake_intc{index}: fake-interrupt-controller {{\n interrupt-controller;\n #interrupt-cells = < {int_cells} >;\n }};\n" + ) +} + +// example_template, with `{example_num}`, `{interrupt}`, `{example}` substituted. +fn example_template(example_num: usize, interrupt: &str, example: &str) -> String { + format!( + "\n example-{example_num} {{\n #address-cells = <1>;\n #size-cells = <1>;\n\n {interrupt}\n\n {example}\n }};\n}};\n" + ) +} + +const EXAMPLE_HEADER: &str = "\n/dts-v1/;\n/plugin/; // silence any missing phandle references\n"; + +const EXAMPLE_START: &str = "\n/{\n compatible = \"foo\";\n model = \"foo\";\n #address-cells = <1>;\n #size-cells = <1>;\n\n"; + +#[derive(Parser)] +#[command(version, about = None, long_about = None)] +struct Args { + /// Filename of YAML encoded schema input file + yamlfile: PathBuf, +} + +/// Split lines while preserving the line terminators that occur in these YAML +/// documents (`\n`, `\r\n`, `\r`). +fn splitlines_keepends(s: &str) -> Vec<&str> { + let bytes = s.as_bytes(); + let mut result = Vec::new(); + let mut start = 0; + let mut i = 0; + while i < bytes.len() { + match bytes[i] { + b'\n' => { + result.push(&s[start..=i]); + i += 1; + start = i; + } + b'\r' => { + if i + 1 < bytes.len() && bytes[i + 1] == b'\n' { + result.push(&s[start..=i + 1]); + i += 2; + } else { + result.push(&s[start..=i]); + i += 1; + } + start = i; + } + _ => i += 1, + } + } + if start < bytes.len() { + result.push(&s[start..]); + } + result +} + +/// Compute `int_cells` for one example. +fn interrupt_cells(ex: &str, int_re: &Regex, paren_re: &Regex) -> usize { + let Some(caps) = int_re.captures(ex) else { + return 0; + }; + let Some(int_val) = caps.get(1) else { + return 0; + }; + let int_val = paren_re.replace_all(int_val.as_str(), "0"); + // `split_whitespace` already skips leading/trailing whitespace. + int_val.split_whitespace().count() +} + +fn run(args: &Args) -> ExitCode { + let value = match dtschema::yaml::from_file(&args.yamlfile) { + Ok(v) => v, + Err(e) => { + // Best-effort: ruamel exposes a problem_mark (line:col) which + // serde_yml does not surface in the same shape, so the message + // text differs, but the exit status matches. + eprintln!("{e}"); + return ExitCode::from(1); + } + }; + + // Non-object YAML documents do not contain binding examples. + if !value.is_object() { + return ExitCode::SUCCESS; + } + + let root_re = Regex::new(r"/\s*\{").unwrap(); + let int_re = Regex::new(r"\sinterrupts\s*=\s*<([0-9a-zA-Z |()_]+)>").unwrap(); + let paren_re = Regex::new(r"\(.+|\)").unwrap(); + + let mut example_dts = String::from(EXAMPLE_HEADER); + + if let Some(examples) = value.get("examples") { + // Real bindings always store examples as a list of strings. + for (idx, item) in examples.as_array().into_iter().flatten().enumerate() { + let ex = item.as_str().unwrap_or(""); + + if root_re.is_match(ex) { + example_dts.push_str(ex); + } else { + let int_cells = interrupt_cells(ex, &int_re, &paren_re); + example_dts.push_str(EXAMPLE_START); + let ex_joined = splitlines_keepends(ex).join(" "); + let int_props = if int_cells > 0 { + interrupt_template(idx, int_cells) + } else { + String::new() + }; + example_dts.push_str(&example_template(idx, &int_props, &ex_joined)); + } + } + } else { + example_dts.push_str(EXAMPLE_START); + example_dts.push_str("\n};"); + } + + // Preserve the legacy trailing newline. + println!("{example_dts}"); + ExitCode::SUCCESS +} + +fn main() -> ExitCode { + let args = Args::parse(); + run(&args) +} diff --git a/rust/cli/dt-extract-example/tests/fixtures/no_examples.expected.dts b/rust/cli/dt-extract-example/tests/fixtures/no_examples.expected.dts new file mode 100644 index 0000000..83cb5a5 --- /dev/null +++ b/rust/cli/dt-extract-example/tests/fixtures/no_examples.expected.dts @@ -0,0 +1,12 @@ + +/dts-v1/; +/plugin/; // silence any missing phandle references + +/{ + compatible = "foo"; + model = "foo"; + #address-cells = <1>; + #size-cells = <1>; + + +}; diff --git a/rust/cli/dt-extract-example/tests/fixtures/no_examples.yaml b/rust/cli/dt-extract-example/tests/fixtures/no_examples.yaml new file mode 100644 index 0000000..7f85358 --- /dev/null +++ b/rust/cli/dt-extract-example/tests/fixtures/no_examples.yaml @@ -0,0 +1,6 @@ +--- +$id: http://devicetree.org/schemas/test2.yaml# +title: Test2 +properties: + reg: + maxItems: 1 diff --git a/rust/cli/dt-extract-example/tests/fixtures/with_examples.expected.dts b/rust/cli/dt-extract-example/tests/fixtures/with_examples.expected.dts new file mode 100644 index 0000000..bf6d9cf --- /dev/null +++ b/rust/cli/dt-extract-example/tests/fixtures/with_examples.expected.dts @@ -0,0 +1,52 @@ + +/dts-v1/; +/plugin/; // silence any missing phandle references + +/{ + compatible = "foo"; + model = "foo"; + #address-cells = <1>; + #size-cells = <1>; + + + example-0 { + #address-cells = <1>; + #size-cells = <1>; + + + interrupt-parent = <&fake_intc0>; + fake_intc0: fake-interrupt-controller { + interrupt-controller; + #interrupt-cells = < 3 >; + }; + + + device@0 { + compatible = "vendor,dev"; + reg = <0x0 0x100>; + interrupts = <0 1 2>; + }; + + }; +}; + +/{ + compatible = "foo"; + model = "foo"; + #address-cells = <1>; + #size-cells = <1>; + + + example-1 { + #address-cells = <1>; + #size-cells = <1>; + + + + other { + compatible = "vendor,other"; + }; + + }; +}; + diff --git a/rust/cli/dt-extract-example/tests/fixtures/with_examples.yaml b/rust/cli/dt-extract-example/tests/fixtures/with_examples.yaml new file mode 100644 index 0000000..e352bf3 --- /dev/null +++ b/rust/cli/dt-extract-example/tests/fixtures/with_examples.yaml @@ -0,0 +1,14 @@ +--- +$id: http://devicetree.org/schemas/test.yaml# +title: Test +examples: + - | + device@0 { + compatible = "vendor,dev"; + reg = <0x0 0x100>; + interrupts = <0 1 2>; + }; + - | + other { + compatible = "vendor,other"; + }; diff --git a/rust/cli/dt-extract-example/tests/parity.rs b/rust/cli/dt-extract-example/tests/parity.rs new file mode 100644 index 0000000..08e2a11 --- /dev/null +++ b/rust/cli/dt-extract-example/tests/parity.rs @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Parity tests: `dt-extract-example` must reproduce the expected output +//! byte-for-byte for the same input. +//! +//! Expected outputs are checked-in fixtures, so this test is hermetic. + +use std::process::Command; + +fn run(fixture: &str) -> String { + let dir = env!("CARGO_MANIFEST_DIR"); + let input = format!("{dir}/tests/fixtures/{fixture}.yaml"); + let output = Command::new(env!("CARGO_BIN_EXE_dt-extract-example")) + .arg(&input) + .output() + .expect("failed to run dt-extract-example"); + assert!( + output.status.success(), + "non-zero exit for {fixture}: {}", + String::from_utf8_lossy(&output.stderr) + ); + String::from_utf8(output.stdout).expect("stdout not utf-8") +} + +#[test] +fn with_examples_and_interrupts() { + let expected = include_str!("fixtures/with_examples.expected.dts"); + assert_eq!(run("with_examples"), expected); +} + +#[test] +fn no_examples_key() { + let expected = include_str!("fixtures/no_examples.expected.dts"); + assert_eq!(run("no_examples"), expected); +} diff --git a/rust/cli/dt-mk-schema/Cargo.toml b/rust/cli/dt-mk-schema/Cargo.toml new file mode 100644 index 0000000..266dc77 --- /dev/null +++ b/rust/cli/dt-mk-schema/Cargo.toml @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: BSD-2-Clause +# Copyright 2026 dt-schema contributors +[package] +name = "dt-mk-schema" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true + +[[bin]] +name = "dt-mk-schema" +path = "src/main.rs" + +[dependencies] +dtschema.workspace = true +clap.workspace = true +argfile.workspace = true +serde_json.workspace = true +serde_yaml.workspace = true +anyhow.workspace = true diff --git a/rust/cli/dt-mk-schema/src/main.rs b/rust/cli/dt-mk-schema/src/main.rs new file mode 100644 index 0000000..8c0ef2f --- /dev/null +++ b/rust/cli/dt-mk-schema/src/main.rs @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! `dt-mk-schema`: build a processed schema from raw binding YAML directories. +//! +//! Reads directories or YAML files, meta-validates and fixes up each, attaches +//! the generated type / compatible caches, and emits the result as JSON (`-j`) +//! or YAML. + +use std::io::Write; +use std::path::PathBuf; + +use clap::Parser; +use dtschema::process::ProcessedSchemas; + +#[derive(Parser)] +#[command(name = "dt-mk-schema", about = "Build a processed devicetree schema")] +struct Args { + /// Filename of the processed schema (default: stdout). + #[arg(short = 'o', long = "outfile")] + outfile: Option, + + /// Encode the processed schema in JSON. + #[arg(short = 'j', long = "json")] + json: bool, + + /// Only process user schemas (skip the bundled core schemas). + #[arg(short = 'u', long = "useronly")] + useronly: bool, + + /// Names of directories, or YAML encoded schema files. + schemas: Vec, + + /// Print version number. + #[arg(short = 'V', long = "version")] + version: bool, +} + +fn main() -> anyhow::Result<()> { + let args = Args::parse_from(argfile::expand_args( + argfile::parse_fromfile, + argfile::PREFIX, + )?); + + if args.version { + println!("{}", dtschema::version()); + return Ok(()); + } + + let ps = ProcessedSchemas::build(&args.schemas, !args.useronly, &dtschema::version()); + if ps.schemas.len() <= 1 { + // Only the `version` marker → nothing processed. + std::process::exit(255); + } + + let mut out: Box = match &args.outfile { + Some(p) => Box::new(std::fs::File::create(p)?), + None => Box::new(std::io::stdout()), + }; + + if args.json { + let text = serde_json::to_string_pretty(&ps.schemas)?; + writeln!(out, "{text}")?; + } else { + let value = serde_json::to_value(&ps.schemas)?; + let s = serde_yaml::to_string(&value)?; + write!(out, "{s}")?; + } + + Ok(()) +} diff --git a/rust/cli/dt-mk-schema/tests/cli.rs b/rust/cli/dt-mk-schema/tests/cli.rs new file mode 100644 index 0000000..e93de14 --- /dev/null +++ b/rust/cli/dt-mk-schema/tests/cli.rs @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors + +use std::path::{Path, PathBuf}; +use std::process::Command; + +use serde_json::Value; + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +fn find_python(repo: &Path) -> Option { + let venv = repo.join(".venv/bin/python3"); + for c in [venv, PathBuf::from("python3")] { + let ok = Command::new(&c) + .args(["-c", "import dtschema"]) + .status() + .map(|s| s.success()) + .unwrap_or(false); + if ok { + return Some(c); + } + } + None +} + +fn canonical_string(v: &Value) -> String { + fn canon(v: &Value) -> Value { + match v { + Value::Object(m) => { + let mut keys: Vec<&String> = m.keys().collect(); + keys.sort(); + let mut o = serde_json::Map::new(); + for k in keys { + o.insert(k.clone(), canon(&m[k])); + } + Value::Object(o) + } + Value::Array(a) => Value::Array(a.iter().map(canon).collect()), + other => other.clone(), + } + } + serde_json::to_string(&canon(v)).unwrap() +} + +fn normalize_generated_order(value: &mut Value) { + let any_of = &mut value["generated-compatibles"]["properties"]["compatible"]["items"]["anyOf"]; + if let Some(items) = any_of.as_array_mut() + && items.len() > 1 + { + items[1..].sort_by(|a, b| a["pattern"].as_str().cmp(&b["pattern"].as_str())); + } + + for genkey in ["generated-types", "generated-pattern-types"] { + if let Some(props) = value[genkey]["properties"].as_object_mut() { + for entries in props.values_mut() { + if let Some(entries) = entries.as_array_mut() { + entries.sort_by_key(canonical_string); + } + } + } + } +} + +#[test] +fn json_output_matches_python_by_value() { + let repo = repo_root(); + let Some(python) = find_python(&repo) else { + eprintln!("SKIP: no python with `dtschema` importable"); + return; + }; + let schemas = repo.join("test/schemas"); + let args_file = std::env::temp_dir().join("dt-mk-schema-args.txt"); + std::fs::write(&args_file, format!("{}\n", schemas.display())).unwrap(); + let response_arg = format!("@{}", args_file.display()); + + let rust = Command::new(env!("CARGO_BIN_EXE_dt-mk-schema")) + .args(["-j", &response_arg]) + .current_dir(&repo) + .output() + .expect("run rust dt-mk-schema"); + assert!( + rust.status.success(), + "rust dt-mk-schema failed: {}", + String::from_utf8_lossy(&rust.stderr) + ); + + let py = Command::new(&python) + .args([ + "-c", + "import sys; from dtschema.mk_schema import main; sys.exit(main())", + "-j", + schemas.to_str().unwrap(), + ]) + .current_dir(&repo) + .output() + .expect("run python dt-mk-schema"); + assert!( + py.status.success(), + "python dt-mk-schema failed: {}", + String::from_utf8_lossy(&py.stderr) + ); + + let mut got: Value = serde_json::from_slice(&rust.stdout).expect("parse rust json"); + let mut want: Value = serde_json::from_slice(&py.stdout).expect("parse python json"); + normalize_generated_order(&mut got); + normalize_generated_order(&mut want); + + assert_eq!(got, want); +} diff --git a/rust/cli/dt-validate/Cargo.toml b/rust/cli/dt-validate/Cargo.toml new file mode 100644 index 0000000..0d219e1 --- /dev/null +++ b/rust/cli/dt-validate/Cargo.toml @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: BSD-2-Clause +# Copyright 2026 dt-schema contributors +[package] +name = "dt-validate" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true + +[[bin]] +name = "dt-validate" +path = "src/main.rs" + +[dependencies] +dtschema.workspace = true +clap.workspace = true +argfile.workspace = true +serde_json.workspace = true +anyhow.workspace = true +rayon.workspace = true diff --git a/rust/cli/dt-validate/src/main.rs b/rust/cli/dt-validate/src/main.rs new file mode 100644 index 0000000..453febc --- /dev/null +++ b/rust/cli/dt-validate/src/main.rs @@ -0,0 +1,558 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! `dt-validate`: validate devicetree DTBs against the schema set. +//! +//! Accepts positional `dtbs`, `-s/--schema`, `-p/--preparse`, `-l/--limit`, +//! `-c/--compatible-match`, `-m/--show-unmatched`, `-n/--line-number` +//! (obsolete), `-v/--verbose`, `--json-output`, `--cache-dir`, +//! `-u/--url-path`, and `-V/--version`. + +use std::path::{Path, PathBuf}; + +use anyhow::{Context, Result}; +use clap::Parser; +use dtschema::cache::{CacheOptions, ValidationCache}; +use dtschema::diagnostic::{ + Diagnostic, decode_diagnostic, diagnostic_text, error_diagnostic, format_error_display, + unmatched_diagnostic, +}; +use dtschema::dtb::DtValue; +use dtschema::process::ProcessedSchemas; +use dtschema::validator::{DTValidator, DtError}; +use rayon::prelude::*; +use serde_json::Value; + +#[derive(Parser)] +#[command(disable_version_flag = true)] +struct Args { + /// Filename or directory of devicetree DTB input file(s). + dtbs: Vec, + /// Preparsed schema file or path to schema files. + #[arg(short = 's', long = "schema")] + schema: Option, + /// Preparsed schema file (deprecated, use '-s'). + #[arg(short = 'p', long = "preparse")] + preparse: Option, + /// Limit validation to schemas with $id matching LIMIT substring(s), + /// separated by ':'. + #[arg(short = 'l', long = "limit")] + limit: Option, + /// Limit validation to schema matching nodes' most specific compatible. + #[arg(short = 'c', long = "compatible-match")] + compatible_match: bool, + /// Print out node 'compatible' strings which don't match any schema. + #[arg(short = 'm', long = "show-unmatched")] + show_unmatched: bool, + /// Obsolete. + #[arg(short = 'n', long = "line-number")] + line_number: bool, + /// Verbose mode. + #[arg(short = 'v', long = "verbose")] + verbose: bool, + /// Write diagnostics in JSON format to the specified file. + #[arg(long = "json-output")] + json_output: Option, + /// Cache validation diagnostics in CACHE_DIR. + #[arg(long = "cache-dir")] + cache_dir: Option, + /// Additional search path for references (deprecated). + #[arg(short = 'u', long = "url-path")] + url_path: Option, + /// Print version number. + #[arg(short = 'V', long = "version")] + version: bool, +} + +/// Runtime options threaded through the node walk. +struct RunOpts { + verbose: bool, + show_unmatched: bool, + match_schema_file: Option>, + compatible_match: bool, + collect_diagnostics: bool, +} + +/// Per-file result, buffered so parallel workers can flush output in the +/// deterministic input order rather than racing on stderr. +#[derive(Default)] +struct FileOutput { + /// Lines destined for stderr, in emission order. + stderr: Vec, + /// Lines destined for stdout (verbose `Check:` notices). + stdout: Vec, + /// JSON diagnostics (only populated when collecting). + diagnostics: Vec, +} + +impl FileOutput { + fn flush(&self) { + use std::io::Write; + // Batch each stream behind a single lock to keep a file's lines + // contiguous even if another thread's flush interleaves at the syscall + // level (the ordering across files is still serialized by the caller). + if !self.stdout.is_empty() { + let out = std::io::stdout(); + let mut lock = out.lock(); + for l in &self.stdout { + let _ = writeln!(lock, "{l}"); + } + } + if !self.stderr.is_empty() { + let err = std::io::stderr(); + let mut lock = err.lock(); + for l in &self.stderr { + let _ = writeln!(lock, "{l}"); + } + } + } +} + +fn main() -> Result<()> { + let args = Args::parse_from(argfile::expand_args( + argfile::parse_fromfile, + argfile::PREFIX, + )?); + + if args.version { + println!("{}", dtschema::version()); + return Ok(()); + } + let _ = args.line_number; // obsolete, accepted for compatibility. + + // Compute the limit list, applying the deprecated url-path stripping. + let mut match_schema_file = args + .limit + .as_ref() + .map(|l| l.split(':').map(str::to_string).collect::>()); + if let (Some(url_path), Some(list)) = (&args.url_path, match_schema_file.as_mut()) { + for m in list.iter_mut() { + for d in url_path.split(std::path::MAIN_SEPARATOR) { + if !d.is_empty() && m.starts_with(d) { + *m = m[(d.len() + 1)..].to_string(); + } + } + } + } + + // Resolve the schema file: -p wins over -s for compatibility. + let schema_file: Option = args.preparse.clone().or_else(|| args.schema.clone()); + + // Cache setup. + let mut cache: Option = None; + if let Some(cache_dir) = &args.cache_dir { + let ok_schema = schema_file.as_ref().is_some_and(|p| p.is_file()); + if !ok_schema { + eprintln!("--cache-dir requires a schema file"); + std::process::exit(-1i32 as u8 as i32); + } + let opts = CacheOptions { + compatible_match: args.compatible_match, + limit: match_schema_file.clone(), + show_unmatched: args.show_unmatched, + verbose: args.verbose, + }; + cache = Some( + ValidationCache::new( + cache_dir.clone(), + schema_file.as_deref(), + dtschema::version(), + &opts, + ) + .context("initializing cache")?, + ); + } + + let collect_diagnostics = args.json_output.is_some() || cache.is_some(); + + let run = RunOpts { + verbose: args.verbose, + show_unmatched: args.show_unmatched, + match_schema_file, + compatible_match: args.compatible_match, + collect_diagnostics, + }; + + // Build the validator (once). A missing schema file is fatal. + if let Some(sf) = &schema_file + && !sf.exists() + { + std::process::exit(-1i32 as u8 as i32); + } + let validator = build_validator(schema_file.as_deref())?; + + // Validate every DTB. Files are independent, so run them across a rayon + // pool (the single built `validator` is `Sync` and its compiled-schema + // cache is shared), then flush each file's buffered output in the original + // input order so stderr/stdout and the JSON list stay deterministic. + let filenames = dtb_filenames(&args.dtbs); + let outputs: Vec = filenames + .par_iter() + .map(|filename| { + let mut out = FileOutput::default(); + if run.verbose { + out.stdout.push(format!("Check: {}", filename.display())); + } + + if let Some(cache) = &cache + && let Some(cached) = cache.load(filename) + { + for d in &cached { + out.stderr.push(diagnostic_text(d)); + } + out.diagnostics = cached; + return out; + } + + let diags = check_dtb(&validator, filename, &run, &mut out); + if let Some(cache) = &cache { + cache.store(filename, &diags); + } + out.diagnostics = diags; + out + }) + .collect(); + + let mut all_diagnostics: Vec = Vec::new(); + for out in outputs { + out.flush(); + all_diagnostics.extend(out.diagnostics); + } + + if let Some(json_path) = &args.json_output { + let mut text = serde_json::to_string_pretty(&all_diagnostics)?; + text.push('\n'); + std::fs::write(json_path, text) + .with_context(|| format!("writing {}", json_path.display()))?; + } + + Ok(()) +} + +/// Build a [`DTValidator`] from the schema file/dir (or bundled-only when none). +fn build_validator(schema_file: Option<&Path>) -> Result { + let version = dtschema::version(); + match schema_file { + Some(p) if p.is_file() => { + // A processed schema JSON file. + let processed = load_processed_schema(p, &version)?; + DTValidator::from_processed(processed) + } + Some(p) => DTValidator::new(&[p.to_path_buf()]), + None => DTValidator::new(&[]), + } +} + +/// Load a processed-schema JSON file into [`ProcessedSchemas`] using the fast +/// path for version checks and `generated-*` reuse. +fn load_processed_schema(path: &Path, version: &str) -> Result { + let text = + std::fs::read_to_string(path).with_context(|| format!("reading {}", path.display()))?; + let value: Value = serde_json::from_str(&text) + .with_context(|| format!("{}: not valid JSON", path.display()))?; + let obj = value + .as_object() + .with_context(|| format!("{}: processed schema is not an object", path.display()))?; + + if obj.contains_key("$id") { + anyhow::bail!( + "{}: looks like a single schema, not a processed schema set", + path.display() + ); + } + if let Some(v) = obj.get("version").and_then(Value::as_str) + && v != version + { + anyhow::bail!( + "Processed schema out of date, delete and retry: {}", + path.display() + ); + } + + ProcessedSchemas::from_value(&value, version) +} + +/// Decode a DTB and walk the resulting tree. Buffered diagnostics are returned; +/// human-readable lines are appended to `out`. +fn check_dtb( + validator: &DTValidator, + filename: &Path, + run: &RunOpts, + out: &mut FileOutput, +) -> Vec { + let mut diagnostics: Vec = Vec::new(); + + let data = match std::fs::read(filename) { + Ok(d) => d, + Err(e) => { + out.stderr.push(format!("{}: {e}", filename.display())); + return diagnostics; + } + }; + + let mut decode_errors: Vec = Vec::new(); + let mut tree = match validator.decode_dtb(&data, &mut decode_errors) { + Ok(t) => t, + Err(e) => { + out.stderr.push(format!("{}: {e}", filename.display())); + return diagnostics; + } + }; + let fname = filename.to_string_lossy(); + for msg in &decode_errors { + // Decode errors always print; they become diagnostics only when + // collecting (matching the previous serial behaviour). + out.stderr.push(msg.clone()); + if run.collect_diagnostics { + diagnostics.push(decode_diagnostic(&fname, msg).to_value()); + } + } + + // The decoded tree is a single root node. + check_subtree( + validator, + &mut tree, + false, + "/", + "/", + &fname, + run, + &mut diagnostics, + out, + ); + diagnostics +} + +/// Recurse through the tree, tracking the disabled state via `status`. +#[allow(clippy::too_many_arguments)] +fn check_subtree( + validator: &DTValidator, + subtree: &mut DtValue, + mut disabled: bool, + nodename: &str, + fullname: &str, + filename: &str, + run: &RunOpts, + diagnostics: &mut Vec, + out: &mut FileOutput, +) { + if nodename.starts_with("__") { + return; + } + { + let DtValue::Node(map) = subtree else { + return; + }; + map.insert( + "$nodename".to_string(), + DtValue::List(vec![DtValue::Str(nodename.to_string())]), + ); + if let Some(status) = map.get("status") { + disabled = status_disabled(status); + } + } + + check_node( + validator, + subtree, + disabled, + nodename, + fullname, + filename, + run, + diagnostics, + out, + ); + + let base = if fullname == "/" { + String::from("/") + } else { + format!("{fullname}/") + }; + let child_names: Vec = match subtree { + DtValue::Node(map) => map + .iter() + .filter_map(|(name, value)| { + if matches!(value, DtValue::Node(_)) { + Some(name.clone()) + } else { + None + } + }) + .collect(), + _ => Vec::new(), + }; + for name in child_names { + if let DtValue::Node(map) = subtree + && let Some(value) = map.get_mut(&name) + { + let child_full = format!("{base}{name}"); + check_subtree( + validator, + value, + disabled, + &name, + &child_full, + filename, + run, + diagnostics, + out, + ); + } + } +} + +/// Run the validator against one node, applying disabled-node suppression and +/// unmatched-compatible handling. +#[allow(clippy::too_many_arguments)] +fn check_node( + validator: &DTValidator, + node: &DtValue, + disabled: bool, + nodename: &str, + fullname: &str, + filename: &str, + run: &RunOpts, + diagnostics: &mut Vec, + out: &mut FileOutput, +) { + let DtValue::Node(map) = node else { + return; + }; + + // Skip example nodes; their contents have already been checked elsewhere. + if map.contains_key("example-0") || nodename.contains("example-") { + return; + } + + let errors = validator.iter_errors( + node, + run.match_schema_file.as_deref(), + run.compatible_match, + run.show_unmatched, + ); + + let compat = first_compatible(map); + + for error in &errors { + // Disabled-node suppression: drop missing-property style errors. + if (disabled || node_status_disabled(&error_instance_disabled(node, error))) + && error.has_suppressible_disabled_context + { + continue; + } + + if error.schema_file == dtschema::GENERATED_COMPATIBLES_SCHEMA { + let compat_list = compatible_list(map); + let diag = unmatched_diagnostic(filename, fullname, &compat_list); + let text = diag.text(); + emit_diagnostic(diagnostics, run, diag, Some(&text), out); + continue; + } + + let text = format_error_display(filename, error, Some(nodename), compat.as_deref()); + let diag = error_diagnostic( + filename, + error, + Some(nodename), + Some(fullname), + compat.as_deref(), + Some(&text), + ); + emit_diagnostic(diagnostics, run, diag, Some(&text), out); + } +} + +/// Emit a diagnostic to the buffered stderr and (when collecting) to the +/// diagnostics list. +fn emit_diagnostic( + diagnostics: &mut Vec, + run: &RunOpts, + mut diag: Diagnostic, + text: Option<&str>, + out: &mut FileOutput, +) { + if let Some(t) = text { + diag.set_formatted_if_missing(t); + } + let line = text.map(str::to_string).unwrap_or_else(|| diag.text()); + out.stderr.push(line); + if run.collect_diagnostics { + diagnostics.push(diag.to_value()); + } +} + +/// Whether a `status` value means "disabled". +fn status_disabled(status: &DtValue) -> bool { + match status { + DtValue::Str(s) => s.contains("disabled"), + DtValue::List(l) => l.iter().any(status_disabled), + _ => false, + } +} + +/// The failing node's own `status`-disabled flag, already carried by +/// [`DtError`]. +fn error_instance_disabled(_node: &DtValue, error: &DtError) -> bool { + error.instance_is_disabled_node +} + +fn node_status_disabled(flag: &bool) -> bool { + *flag +} + +/// The node's first compatible string, if any. +fn first_compatible(map: &std::collections::BTreeMap) -> Option { + match map.get("compatible") { + Some(DtValue::List(l)) => l.first().and_then(|v| match v { + DtValue::Str(s) => Some(s.clone()), + _ => None, + }), + _ => None, + } +} + +/// The node's full compatible string list. +fn compatible_list(map: &std::collections::BTreeMap) -> Vec { + match map.get("compatible") { + Some(DtValue::List(l)) => l + .iter() + .filter_map(|v| match v { + DtValue::Str(s) => Some(s.clone()), + _ => None, + }) + .collect(), + _ => Vec::new(), + } +} + +/// Expand directory arguments to `**/*.dtb`, then append plain-file arguments +/// in stable yield order. +fn dtb_filenames(dtbs: &[PathBuf]) -> Vec { + let mut out = Vec::new(); + for d in dtbs { + if d.is_dir() { + collect_dtbs(d, &mut out); + } + } + for f in dtbs { + if f.is_file() { + out.push(f.clone()); + } + } + out +} + +fn collect_dtbs(dir: &Path, out: &mut Vec) { + let Ok(rd) = std::fs::read_dir(dir) else { + return; + }; + let mut entries: Vec = rd.flatten().map(|e| e.path()).collect(); + entries.sort(); + for p in entries { + if p.is_dir() { + collect_dtbs(&p, out); + } else if p.extension().and_then(|s| s.to_str()) == Some("dtb") { + out.push(p); + } + } +} diff --git a/rust/cli/dt-validate/tests/cli.rs b/rust/cli/dt-validate/tests/cli.rs new file mode 100644 index 0000000..5ab2519 --- /dev/null +++ b/rust/cli/dt-validate/tests/cli.rs @@ -0,0 +1,261 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! CLI integration tests for `dt-validate`. +//! +//! These drive the built `dt-validate` binary against the repo's `test/*.dts` +//! fixtures (compiled on the fly with `dtc`) and the `test/schemas/` bindings. +//! Skipped (not failed) when `dtc` is not on `PATH`. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +use dtschema::process::ProcessedSchemas; +use serde_json::Value; + +/// repo root: `rust/cli/dt-validate/` → up three. +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +fn dt_validate_bin() -> PathBuf { + PathBuf::from(env!("CARGO_BIN_EXE_dt-validate")) +} + +fn have_dtc() -> bool { + Command::new("dtc") + .arg("--version") + .output() + .map(|o| o.status.success()) + .unwrap_or(false) +} + +fn write_processed_schema(repo: &Path, path: &Path) { + let schemas = repo.join("test/schemas"); + let processed = ProcessedSchemas::build(&[schemas], true, &dtschema::version()); + let mut text = serde_json::to_string_pretty(&processed.schemas).unwrap(); + text.push('\n'); + std::fs::write(path, text).unwrap(); +} + +/// Compile a `.dts` fixture to a `.dtb` under `out_dir`, returning its path. +fn compile_dtb(repo: &Path, dts_rel: &str, out: &Path) -> PathBuf { + let dts = repo.join(dts_rel); + let dtb = out.join(format!( + "{}.dtb", + dts.file_stem().unwrap().to_string_lossy() + )); + let status = Command::new("dtc") + .args(["-Odtb", "-o"]) + .arg(&dtb) + .arg(&dts) + .output() + .expect("run dtc"); + assert!( + status.status.success(), + "dtc failed for {dts_rel}:\n{}", + String::from_utf8_lossy(&status.stderr) + ); + dtb +} + +/// Run `dt-validate -s `, returning (stdout, stderr). +fn run_validate(repo: &Path, extra: &[&str]) -> (String, String) { + let schemas = repo.join("test/schemas"); + let mut cmd = Command::new(dt_validate_bin()); + cmd.arg("-s").arg(&schemas); + for a in extra { + cmd.arg(a); + } + let out = cmd.output().expect("run dt-validate"); + ( + String::from_utf8_lossy(&out.stdout).into_owned(), + String::from_utf8_lossy(&out.stderr).into_owned(), + ) +} + +/// Number of real diagnostic lines on stderr (ignoring the harmless +/// `bad-example.yaml: ignoring, error in schema` meta-validation notice that the +/// `test/schemas` set always emits). +fn diag_line_count(stderr: &str) -> usize { + stderr + .lines() + .filter(|l| !l.is_empty() && !l.contains("ignoring, error in schema")) + .count() +} + +#[test] +fn test_dtb_validation() { + if !have_dtc() { + eprintln!("SKIP: dtc not available"); + return; + } + let repo = repo_root(); + let tmp = std::env::temp_dir().join("dt-validate-cli-dtbs"); + std::fs::create_dir_all(&tmp).unwrap(); + + let mut entries: Vec = std::fs::read_dir(repo.join("test")) + .unwrap() + .flatten() + .map(|e| e.path()) + .filter(|p| p.extension().and_then(|s| s.to_str()) == Some("dts")) + .collect(); + entries.sort(); + assert!(!entries.is_empty(), "no .dts fixtures found"); + + for dts in entries { + let rel = format!("test/{}", dts.file_name().unwrap().to_string_lossy()); + let name = dts.file_stem().unwrap().to_string_lossy().into_owned(); + let expect_fail = name.contains("-fail"); + let dtb = compile_dtb(&repo, &rel, &tmp); + let (stdout, stderr) = run_validate(&repo, &[dtb.to_str().unwrap()]); + assert_eq!(stdout, "", "{name}: stdout should be empty"); + let diags = diag_line_count(&stderr); + if expect_fail { + assert!( + diags > 0, + "{name}: expected validation errors, got none.\nstderr:\n{stderr}" + ); + } else { + assert_eq!( + diags, 0, + "{name}: expected clean validation, got diagnostics:\n{stderr}" + ); + } + } +} + +#[test] +fn test_json_cli_output_file() { + if !have_dtc() { + eprintln!("SKIP: dtc not available"); + return; + } + let repo = repo_root(); + let tmp = std::env::temp_dir().join("dt-validate-cli-json"); + std::fs::create_dir_all(&tmp).unwrap(); + let dtb = compile_dtb(&repo, "test/device-fail.dts", &tmp); + let json_out = tmp.join("out.json"); + + let (stdout, stderr) = run_validate( + &repo, + &[ + "--json-output", + json_out.to_str().unwrap(), + dtb.to_str().unwrap(), + ], + ); + + assert_eq!(stdout, ""); + assert!( + stderr.contains("from schema $id:"), + "stderr missing schema note:\n{stderr}" + ); + + let text = std::fs::read_to_string(&json_out).unwrap(); + let diagnostics: Vec = serde_json::from_str(&text).unwrap(); + assert!(!diagnostics.is_empty()); + + let validation = diagnostics + .iter() + .find(|d| d["type"] == "validation") + .expect("a validation diagnostic"); + assert_eq!(validation["level"], "error"); + assert!(validation.get("message").is_some()); + assert!(validation.get("formatted").is_some()); + assert!(validation.get("schema").is_some()); +} + +#[test] +fn test_cli_cache_output() { + if !have_dtc() { + eprintln!("SKIP: dtc not available"); + return; + } + let repo = repo_root(); + let tmp = std::env::temp_dir().join("dt-validate-cli-cache"); + let _ = std::fs::remove_dir_all(&tmp); + std::fs::create_dir_all(&tmp).unwrap(); + + let dtb = compile_dtb(&repo, "test/device-fail.dts", &tmp); + let dtb2 = tmp.join("copy.dtb"); + std::fs::copy(&dtb, &dtb2).unwrap(); + + let schema = tmp.join("schema.json"); + write_processed_schema(&repo, &schema); + + let cache_dir = tmp.join("cache"); + std::fs::create_dir_all(&cache_dir).unwrap(); + let json_out = tmp.join("out.json"); + + let run = |target: &Path| -> (String, String) { + let out = Command::new(dt_validate_bin()) + .args(["--json-output"]) + .arg(&json_out) + .arg("--cache-dir") + .arg(&cache_dir) + .arg("-s") + .arg(&schema) + .arg(target) + .output() + .expect("run dt-validate"); + ( + String::from_utf8_lossy(&out.stdout).into_owned(), + String::from_utf8_lossy(&out.stderr).into_owned(), + ) + }; + + // First run: populates the cache. + let (stdout, stderr) = run(&dtb); + assert_eq!(stdout, ""); + assert!(stderr.contains("from schema $id:")); + assert!( + stderr.contains("vendor,bool-prop: size (5) error for type flag"), + "missing decode error:\n{stderr}" + ); + let first: Vec = + serde_json::from_str(&std::fs::read_to_string(&json_out).unwrap()).unwrap(); + let decode = first + .iter() + .find(|d| { + d["type"] == "decode" + && d["message"] == "vendor,bool-prop: size (5) error for type flag" + }) + .expect("decode diagnostic present"); + assert_eq!(decode["file"], dtb.to_string_lossy().into_owned()); + + // Second run of the same file: served from cache, identical output. + let (stdout, stderr) = run(&dtb); + assert_eq!(stdout, ""); + assert!(stderr.contains("from schema $id:")); + assert!(stderr.contains("vendor,bool-prop: size (5) error for type flag")); + let second: Vec = + serde_json::from_str(&std::fs::read_to_string(&json_out).unwrap()).unwrap(); + assert_eq!(second, first); + assert_eq!(std::fs::read_dir(&cache_dir).unwrap().count(), 1); + + // A byte-identical copy reuses the cache (via the `$dtb` sentinel) but + // reports its own path. + let (stdout, stderr) = run(&dtb2); + assert_eq!(stdout, ""); + assert!(stderr.contains("from schema $id:")); + let third: Vec = + serde_json::from_str(&std::fs::read_to_string(&json_out).unwrap()).unwrap(); + let validation = third + .iter() + .find(|d| d["type"] == "validation") + .expect("a validation diagnostic"); + assert_eq!(validation["file"], dtb2.to_string_lossy().into_owned()); + let formatted = validation["formatted"].as_str().unwrap(); + assert!( + formatted.starts_with(&format!("{}:", dtb2.to_string_lossy())), + "formatted should start with the copy's path: {formatted}" + ); + assert_eq!(std::fs::read_dir(&cache_dir).unwrap().count(), 1); +} diff --git a/rust/dtschema/Cargo.toml b/rust/dtschema/Cargo.toml new file mode 100644 index 0000000..6588cb5 --- /dev/null +++ b/rust/dtschema/Cargo.toml @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: BSD-2-Clause +# Copyright 2026 dt-schema contributors +[package] +name = "dtschema" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +description = "Devicetree schema validation library (Rust port)" + +[dependencies] +serde.workspace = true +serde_json.workspace = true +serde_yaml.workspace = true +jsonschema.workspace = true +fdt.workspace = true +regex.workspace = true +fancy-regex.workspace = true +sha2.workspace = true +rayon.workspace = true +anyhow.workspace = true +thiserror.workspace = true + +[build-dependencies] +# Bundled schema/meta-schema data path is resolved at runtime relative to the +# repository; see lib.rs `bundled_dir`. diff --git a/rust/dtschema/examples/fixupdiff.rs b/rust/dtschema/examples/fixupdiff.rs new file mode 100644 index 0000000..ba5f804 --- /dev/null +++ b/rust/dtschema/examples/fixupdiff.rs @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Differential harness: fixup every bundled/test schema in Rust and diff the +//! canonical (sorted-key) JSON against the Python goldens in `$GOLDEN_DIR` +//! (default `/tmp/golden/fixup`). Prints a per-file PASS/FAIL summary. +//! +//! Run: `cargo run -p dtschema --example fixupdiff` + +use dtschema::schema::DTSchema; +use serde_json::Value; +use std::path::Path; + +/// Recursively sort object keys so serialization is canonical. +fn canonicalize(v: &Value) -> Value { + match v { + Value::Object(m) => { + let mut keys: Vec<&String> = m.keys().collect(); + keys.sort(); + let mut out = serde_json::Map::new(); + for k in keys { + out.insert(k.clone(), canonicalize(&m[k])); + } + Value::Object(out) + } + Value::Array(a) => Value::Array(a.iter().map(canonicalize).collect()), + other => other.clone(), + } +} + +fn collect(glob_dir: &str, pattern_ext: &str, out: &mut Vec) { + fn walk(dir: &Path, out: &mut Vec) { + if let Ok(rd) = std::fs::read_dir(dir) { + for e in rd.flatten() { + let p = e.path(); + if p.is_dir() { + walk(&p, out); + } else if p.extension().and_then(|s| s.to_str()) == Some("yaml") { + out.push(p); + } + } + } + } + let _ = pattern_ext; + walk(Path::new(glob_dir), out); +} + +fn main() { + let repo = Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf(); + let golden_dir = + std::env::var("GOLDEN_DIR").unwrap_or_else(|_| "/tmp/golden/fixup".to_string()); + + let mut files = Vec::new(); + collect( + repo.join("dtschema/schemas").to_str().unwrap(), + "yaml", + &mut files, + ); + collect( + repo.join("test/schemas").to_str().unwrap(), + "yaml", + &mut files, + ); + files.sort(); + + let mut pass = 0; + let mut fail = 0; + let mut skip = 0; + let mut fails: Vec = Vec::new(); + + for f in &files { + let rel = f.strip_prefix(&repo).unwrap().to_str().unwrap(); + let golden_name = rel.replace('/', "__") + ".json"; + let golden_path = Path::new(&golden_dir).join(&golden_name); + if !golden_path.is_file() { + skip += 1; + continue; + } + let golden: Value = + serde_json::from_str(&std::fs::read_to_string(&golden_path).unwrap()).unwrap(); + + let sch = match DTSchema::load(f) { + Ok(s) => s, + Err(e) => { + fail += 1; + fails.push(format!("{rel}: load error: {e}")); + continue; + } + }; + let got = canonicalize(&sch.fixup()); + let want = canonicalize(&golden); + if got == want { + pass += 1; + } else { + fail += 1; + fails.push(rel.to_string()); + } + } + + println!("PASS={pass} FAIL={fail} SKIP={skip} TOTAL={}", files.len()); + if !fails.is_empty() { + println!("--- failures ---"); + for f in fails.iter().take(60) { + println!(" {f}"); + } + } +} diff --git a/rust/dtschema/examples/loadcheck.rs b/rust/dtschema/examples/loadcheck.rs new file mode 100644 index 0000000..559bb77 --- /dev/null +++ b/rust/dtschema/examples/loadcheck.rs @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: BSD-2-Clause +use dtschema::schema::DTSchema; +fn main() { + for f in [ + "../test/schemas/good-example.yaml", + "../test/schemas/bad-example.yaml", + ] { + let s = DTSchema::load(std::path::Path::new(f)).unwrap(); + println!( + "{f}: id={:?} schema={:?}", + s.id(), + s.value.get("$schema").and_then(|v| v.as_str()) + ); + } +} diff --git a/rust/dtschema/examples/metacheck.rs b/rust/dtschema/examples/metacheck.rs new file mode 100644 index 0000000..7e30f75 --- /dev/null +++ b/rust/dtschema/examples/metacheck.rs @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: BSD-2-Clause +use dtschema::schema::DTSchema; +fn main() { + let mut args: Vec = std::env::args().skip(1).collect(); + let summary = args.first().map(|s| s == "--summary").unwrap_or(false); + if summary { + args.remove(0); + } + let mut invalid = 0; + let mut total = 0; + for f in &args { + total += 1; + let s = match DTSchema::load(std::path::Path::new(f)) { + Ok(s) => s, + Err(e) => { + println!("{f}: LOAD-ERR {e}"); + invalid += 1; + continue; + } + }; + match s.meta_validate() { + Ok(errs) if errs.is_empty() => { + if !summary { + println!("{f}: VALID"); + } + } + Ok(errs) => { + invalid += 1; + println!("{f}: {} errors", errs.len()); + if !summary { + for e in errs.iter().take(4) { + println!(" {e}"); + } + } + } + Err(e) => { + invalid += 1; + println!("{f}: ERR {e}"); + } + } + } + if summary { + println!("TOTAL={total} INVALID={invalid}"); + } +} diff --git a/rust/dtschema/examples/mkschemadiff.rs b/rust/dtschema/examples/mkschemadiff.rs new file mode 100644 index 0000000..560bf97 --- /dev/null +++ b/rust/dtschema/examples/mkschemadiff.rs @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Differential harness for the processed-schema pipeline: build the Rust +//! processed schema set for `test/schemas` and diff it structurally against a +//! Python `dt-mk-schema -j` golden (`$MK_GOLDEN`, default +//! `/tmp/golden/mkschema-test.json`). +//! +//! Type-lists and the `generated-compatibles` enum/pattern lists are compared as +//! unordered sets (their order is Python glob/set-iteration dependent); the +//! `version` key is ignored. + +use dtschema::process::ProcessedSchemas; +use serde_json::Value; +use std::collections::BTreeSet; +use std::path::{Path, PathBuf}; + +fn canon(v: &Value) -> Value { + match v { + Value::Object(m) => { + let mut keys: Vec<&String> = m.keys().collect(); + keys.sort(); + let mut out = serde_json::Map::new(); + for k in keys { + out.insert(k.clone(), canon(&m[k])); + } + Value::Object(out) + } + Value::Array(a) => Value::Array(a.iter().map(canon).collect()), + other => other.clone(), + } +} + +/// Multiset of canonical-JSON strings for an array. +fn as_set(v: &Value) -> BTreeSet { + v.as_array() + .map(|a| a.iter().map(|x| canon(x).to_string()).collect()) + .unwrap_or_default() +} + +fn main() { + let repo = Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf(); + let golden_path = + std::env::var("MK_GOLDEN").unwrap_or_else(|_| "/tmp/golden/mkschema-test.json".to_string()); + let golden: Value = + serde_json::from_str(&std::fs::read_to_string(&golden_path).unwrap()).unwrap(); + let version = golden.get("version").and_then(Value::as_str).unwrap_or("0"); + + let input = std::env::var("MK_INPUT").unwrap_or_else(|_| "test/schemas".to_string()); + let ps = ProcessedSchemas::build(&[repo.join(&input)], true, version); + let got = &ps.schemas; + let want = golden.as_object().unwrap(); + + let mut problems = 0usize; + + // Key set parity. + let gk: BTreeSet<&String> = got.keys().collect(); + let wk: BTreeSet<&String> = want.keys().collect(); + for k in wk.difference(&gk) { + println!("MISSING key: {k}"); + problems += 1; + } + for k in gk.difference(&wk) { + println!("EXTRA key: {k}"); + problems += 1; + } + + // generated-types / generated-pattern-types: compare each prop's type-list as a set. + for genkey in ["generated-types", "generated-pattern-types"] { + let (Some(g), Some(w)) = (got.get(genkey), want.get(genkey)) else { + continue; + }; + let gp = g["properties"].as_object().unwrap(); + let wp = w["properties"].as_object().unwrap(); + let gpk: BTreeSet<&String> = gp.keys().collect(); + let wpk: BTreeSet<&String> = wp.keys().collect(); + for k in wpk.symmetric_difference(&gpk) { + println!("{genkey}: prop key mismatch: {k}"); + problems += 1; + } + for k in gpk.intersection(&wpk) { + if as_set(&gp[*k]) != as_set(&wp[*k]) { + println!( + "{genkey}: {k}:\n got {}\n want {}", + canon(&gp[*k]), + canon(&wp[*k]) + ); + problems += 1; + } + } + } + + // generated-compatibles: compare enum + patterns as sets. + if let (Some(g), Some(w)) = ( + got.get("generated-compatibles"), + want.get("generated-compatibles"), + ) { + let ga = &g["properties"]["compatible"]["items"]["anyOf"]; + let wa = &w["properties"]["compatible"]["items"]["anyOf"]; + let genum = as_set(&ga[0]["enum"]); + let wenum = as_set(&wa[0]["enum"]); + if genum != wenum { + let only_w: Vec<_> = wenum.difference(&genum).collect(); + let only_g: Vec<_> = genum.difference(&wenum).collect(); + println!("compatibles enum differs: only_want={only_w:?} only_got={only_g:?}"); + problems += 1; + } + let gpat: BTreeSet = ga.as_array().unwrap()[1..] + .iter() + .map(|e| e["pattern"].as_str().unwrap().to_string()) + .collect(); + let wpat: BTreeSet = wa.as_array().unwrap()[1..] + .iter() + .map(|e| e["pattern"].as_str().unwrap().to_string()) + .collect(); + if gpat != wpat { + println!("compatibles patterns differ:\n got {gpat:?}\n want {wpat:?}"); + problems += 1; + } + } + + // Per-schema-entry structural parity (ignoring $filename absolute paths). + for k in gk.intersection(&wk) { + if k.starts_with("generated-") || *k == "version" { + continue; + } + let mut g = got[*k].clone(); + let mut w = want[*k].clone(); + for v in [&mut g, &mut w] { + if let Some(o) = v.as_object_mut() { + o.remove("$filename"); + } + } + if canon(&g) != canon(&w) { + println!("entry differs: {k}"); + problems += 1; + } + } + + let _ = PathBuf::new(); + println!("\nPROBLEMS={problems}"); +} diff --git a/rust/dtschema/src/cache.rs b/rust/dtschema/src/cache.rs new file mode 100644 index 0000000..7e08ee8 --- /dev/null +++ b/rust/dtschema/src/cache.rs @@ -0,0 +1,239 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Per-DTB diagnostics cache for `dt-validate`. +//! +//! A content-addressed JSON file per DTB, keyed by a SHA-256 of +//! `{cache_version, dtschema_version, dtb_hash, schema_hash, options}`. Stored +//! diagnostics use the `$dtb` filename sentinel so a cache entry is reusable +//! when only the DTB path (not its content) changes. + +use std::fs; +use std::io::Read; +use std::path::{Path, PathBuf}; + +use serde_json::{Value, json}; +use sha2::{Digest, Sha256}; + +use crate::diagnostic::display_path; + +/// Cache format version shared with the installed tools. +pub const CACHE_VERSION: u64 = 2; +/// Filename sentinel stored in place of the real DTB path. +pub const CACHE_DTB_FILENAME: &str = "$dtb"; + +/// SHA-256 of a file's contents, as a lowercase hex string. +pub fn sha256_file(path: &Path) -> std::io::Result { + let mut f = fs::File::open(path)?; + let mut hasher = Sha256::new(); + let mut buf = [0u8; 1024 * 1024]; + loop { + let n = f.read(&mut buf)?; + if n == 0 { + break; + } + hasher.update(&buf[..n]); + } + Ok(hex(&hasher.finalize())) +} + +fn hex(bytes: &[u8]) -> String { + use std::fmt::Write; + let mut s = String::with_capacity(bytes.len() * 2); + for b in bytes { + let _ = write!(s, "{b:02x}"); + } + s +} + +/// The cache-key options object. +pub struct CacheOptions { + pub compatible_match: bool, + pub limit: Option>, + pub show_unmatched: bool, + pub verbose: bool, +} + +impl CacheOptions { + fn to_json(&self) -> Value { + json!({ + "compatible_match": self.compatible_match, + "limit": match &self.limit { + Some(l) => Value::Array(l.iter().map(|s| json!(s)).collect()), + None => Value::Null, + }, + "show_unmatched": self.show_unmatched, + "verbose": self.verbose, + }) + } +} + +/// A validation-diagnostics cache rooted at `cache_dir`. +pub struct ValidationCache { + cache_dir: PathBuf, + schema_hash: Option, + dtschema_version: String, + options: Value, +} + +impl ValidationCache { + /// Build a cache handle. `schema_file` is hashed into every key. + pub fn new( + cache_dir: PathBuf, + schema_file: Option<&Path>, + dtschema_version: String, + options: &CacheOptions, + ) -> std::io::Result { + let schema_hash = match schema_file { + Some(p) => Some(sha256_file(p)?), + None => None, + }; + Ok(Self { + cache_dir, + schema_hash, + dtschema_version, + options: options.to_json(), + }) + } + + fn cache_key(&self, filename: &Path) -> std::io::Result { + let dtb_hash = sha256_file(filename)?; + let key = json!({ + "cache_version": CACHE_VERSION, + "dtschema_version": self.dtschema_version, + "dtb_hash": dtb_hash, + "schema_hash": self.schema_hash, + "options": self.options, + }); + let data = canonical_json(&key); + let mut hasher = Sha256::new(); + hasher.update(data.as_bytes()); + Ok(hex(&hasher.finalize())) + } + + fn cache_file(&self, key: &str) -> PathBuf { + self.cache_dir.join(format!("{key}.json")) + } + + /// Load cached diagnostics for `filename`, rewriting the `$dtb` sentinel + /// back to the file's display path. Returns `None` on any miss/error. + pub fn load(&self, filename: &Path) -> Option> { + let key = self.cache_key(filename).ok()?; + let text = fs::read_to_string(self.cache_file(&key)).ok()?; + let doc: Value = serde_json::from_str(&text).ok()?; + let diags = doc.get("diagnostics")?.as_array()?.clone(); + let disp = display_path(&filename.to_string_lossy()); + Some( + diags + .iter() + .map(|d| map_diagnostic_filename(d, CACHE_DTB_FILENAME, &disp)) + .collect(), + ) + } + + /// Store `diagnostics` for `filename`, replacing the display path with the + /// `$dtb` sentinel. Best-effort: failures are silent. + pub fn store(&self, filename: &Path, diagnostics: &[Value]) { + let _ = fs::create_dir_all(&self.cache_dir); + let Ok(key) = self.cache_key(filename) else { + return; + }; + let disp = display_path(&filename.to_string_lossy()); + let mapped: Vec = diagnostics + .iter() + .map(|d| map_diagnostic_filename(d, &disp, CACHE_DTB_FILENAME)) + .collect(); + let doc = json!({ + "cache_version": CACHE_VERSION, + "diagnostics": mapped, + }); + // Write atomically via a temp file in the cache dir. + let tmp = self.cache_dir.join(format!(".dt-validate-{key}.tmp.json")); + if let Ok(mut text) = serde_json::to_string_pretty(&doc) { + text.push('\n'); + if fs::write(&tmp, text).is_ok() { + let _ = fs::rename(&tmp, self.cache_file(&key)); + } else { + let _ = fs::remove_file(&tmp); + } + } + } +} + +/// Replace `file == old` with `new`, and rewrite `formatted` line prefixes, +/// recursively. +fn map_diagnostic_filename(value: &Value, old: &str, new: &str) -> Value { + match value { + Value::Array(a) => Value::Array( + a.iter() + .map(|v| map_diagnostic_filename(v, old, new)) + .collect(), + ), + Value::Object(m) => { + let mut out = serde_json::Map::new(); + for (k, v) in m { + if k == "file" && v.as_str() == Some(old) { + out.insert(k.clone(), Value::String(new.to_string())); + } else if k == "formatted" { + if let Some(s) = v.as_str() { + out.insert( + k.clone(), + Value::String(crate::diagnostic::replace_filename_prefix(s, old, new)), + ); + } else { + out.insert(k.clone(), map_diagnostic_filename(v, old, new)); + } + } else { + out.insert(k.clone(), map_diagnostic_filename(v, old, new)); + } + } + Value::Object(out) + } + other => other.clone(), + } +} + +/// Serialize deterministically for cache-key hashing: sorted object keys and +/// no insignificant whitespace. +fn canonical_json(v: &Value) -> String { + let mut out = String::new(); + write_canonical(v, &mut out); + out +} + +fn write_canonical(v: &Value, out: &mut String) { + match v { + Value::Null => out.push_str("null"), + Value::Bool(b) => out.push_str(if *b { "true" } else { "false" }), + Value::Number(n) => out.push_str(&n.to_string()), + Value::String(s) => write_json_string(s, out), + Value::Array(a) => { + out.push('['); + for (i, e) in a.iter().enumerate() { + if i > 0 { + out.push(','); + } + write_canonical(e, out); + } + out.push(']'); + } + Value::Object(m) => { + let mut keys: Vec<&String> = m.keys().collect(); + keys.sort(); + out.push('{'); + for (i, k) in keys.iter().enumerate() { + if i > 0 { + out.push(','); + } + write_json_string(k, out); + out.push(':'); + write_canonical(&m[*k], out); + } + out.push('}'); + } + } +} + +fn write_json_string(s: &str, out: &mut String) { + // Cache-key strings are ASCII hashes, versions, and option keys. + out.push_str(&serde_json::to_string(s).unwrap()); +} diff --git a/rust/dtschema/src/diagnostic.rs b/rust/dtschema/src/diagnostic.rs new file mode 100644 index 0000000..794b99b --- /dev/null +++ b/rust/dtschema/src/diagnostic.rs @@ -0,0 +1,293 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Diagnostic formatting for `dt-validate`. +//! +//! Human-readable error lines, structured JSON diagnostics, and display-path +//! rewriting used by the CLI. + +use std::path::Path; + +use serde::Serialize; +use serde_json::Value; + +use crate::validator::{DtError, PathSeg}; + +/// Return the path as the CLI shows it — relative to CWD when that stays within +/// the tree, otherwise absolute. +pub fn display_path(filename: &str) -> String { + let abs = std::path::absolute(filename).unwrap_or_else(|_| Path::new(filename).to_path_buf()); + let cwd = std::env::current_dir().unwrap_or_default(); + match abs.strip_prefix(&cwd) { + Ok(rel) if !rel.as_os_str().is_empty() => rel.to_string_lossy().into_owned(), + _ => abs.to_string_lossy().into_owned(), + } +} + +/// Absolute path string used in formatted diagnostics. +fn abs_path(filename: &str) -> String { + std::path::absolute(filename) + .unwrap_or_else(|_| Path::new(filename).to_path_buf()) + .to_string_lossy() + .into_owned() +} + +/// Render a path segment list as `a:b:0:` (trailing colon per segment). +fn path_prefix(path: &[PathSeg]) -> String { + let mut s = String::new(); + for p in path { + s.push_str(&p.to_display()); + s.push(':'); + } + s +} + +/// Build the `file: node (compat): path: message` line, with a trailing +/// `from schema $id:` note. `note`/`context` sub-error expansion is not +/// reachable from the `dt-validate` path: notes are always `None` there and +/// nested `context` errors are flattened by the engine. +pub fn format_error( + filename: &str, + error: &DtError, + nodename: Option<&str>, + compatible: Option<&str>, +) -> String { + let mut src = format!("{}: ", abs_path(filename)); + + if let Some(nn) = nodename { + src.push_str(nn); + if let Some(c) = compatible { + src.push_str(&format!(" ({c})")); + } + src.push_str(": "); + } + + if !error.instance_path.is_empty() { + src.push_str(&path_prefix(&error.instance_path)); + src.push(' '); + } + + let mut msg = error.message.clone(); + if !error.schema_file.is_empty() { + msg.push_str(&format!("\n\tfrom schema $id: {}", error.schema_file)); + } + + src + &msg +} + +/// Rewrite `old:`-prefixed line starts (after leading whitespace) to `new`. +/// Used to turn absolute paths into display paths in already-formatted text. +pub fn replace_filename_prefix(text: &str, old: &str, new: &str) -> String { + let mut out = String::new(); + for line in text.split_inclusive('\n') { + let stripped = line.trim_start(); + let indent = &line[..line.len() - stripped.len()]; + let needle = format!("{old}:"); + if stripped.starts_with(&needle) { + out.push_str(indent); + out.push_str(new); + out.push_str(&stripped[old.len()..]); + } else { + out.push_str(line); + } + } + out +} + +/// Format an error, then rewrite the absolute path to the display path. +pub fn format_error_display( + filename: &str, + error: &DtError, + nodename: Option<&str>, + compatible: Option<&str>, +) -> String { + let text = format_error(filename, error, nodename, compatible); + replace_filename_prefix(&text, &abs_path(filename), &display_path(filename)) +} + +/// Render a path segment list as a JSON array (`_error_path`). +fn error_path_json(path: &[PathSeg]) -> Value { + Value::Array(path.iter().map(PathSeg::as_json).collect()) +} + +/// Diagnostic severity in JSON output. +#[derive(Clone, Copy, Serialize)] +#[serde(rename_all = "lowercase")] +pub enum DiagnosticLevel { + Error, + Warning, +} + +/// Structured diagnostics emitted by `dt-validate`. +#[derive(Clone, Serialize)] +#[serde(tag = "type")] +pub enum Diagnostic { + #[serde(rename = "validation")] + Validation { + level: DiagnosticLevel, + file: String, + line: Option, + column: Option, + node: Option, + nodename: Option, + compatible: Option, + property_path: Value, + schema_path: Value, + schema: String, + message: String, + #[serde(skip_serializing_if = "Option::is_none")] + formatted: Option, + }, + #[serde(rename = "unmatched")] + Unmatched { + level: DiagnosticLevel, + file: String, + line: Option, + column: Option, + node: String, + nodename: String, + compatible: Vec, + message: String, + #[serde(skip_serializing_if = "Option::is_none")] + formatted: Option, + }, + #[serde(rename = "decode")] + Decode { + level: DiagnosticLevel, + file: String, + line: Option, + column: Option, + message: String, + }, +} + +impl Diagnostic { + pub fn to_value(&self) -> Value { + serde_json::to_value(self).expect("diagnostic serialization should not fail") + } + + pub fn text(&self) -> String { + match self { + Diagnostic::Validation { + message, formatted, .. + } => formatted.clone().unwrap_or_else(|| message.clone()), + Diagnostic::Unmatched { + file, + node, + message, + formatted, + .. + } => formatted + .clone() + .unwrap_or_else(|| format!("{file}: {node}: {message}")), + Diagnostic::Decode { message, .. } => message.clone(), + } + } + + pub fn set_formatted_if_missing(&mut self, text: &str) { + match self { + Diagnostic::Validation { formatted, .. } | Diagnostic::Unmatched { formatted, .. } => { + if formatted.is_none() { + *formatted = Some(text.to_string()); + } + } + Diagnostic::Decode { .. } => {} + } + } +} + +/// Build the JSON record for a validation error. +pub fn error_diagnostic( + filename: &str, + error: &DtError, + nodename: Option<&str>, + fullname: Option<&str>, + compatible: Option<&str>, + formatted: Option<&str>, +) -> Diagnostic { + Diagnostic::Validation { + level: DiagnosticLevel::Error, + file: display_path(filename), + line: None, + column: None, + node: fullname.map(str::to_string), + nodename: nodename.map(str::to_string), + compatible: compatible.map(str::to_string), + property_path: error_path_json(&error.instance_path), + schema_path: error_path_json(&error.schema_path), + schema: error.schema_file.clone(), + message: error.message.clone(), + formatted: formatted.map(str::to_string), + } +} + +/// Build the JSON record for a node whose compatible matched no schema. +pub fn unmatched_diagnostic(filename: &str, fullname: &str, compatible: &[String]) -> Diagnostic { + let nodename = Path::new(fullname) + .file_name() + .map(|s| s.to_string_lossy().into_owned()) + .filter(|s| !s.is_empty()) + .unwrap_or_else(|| fullname.to_string()); + Diagnostic::Unmatched { + level: DiagnosticLevel::Warning, + file: display_path(filename), + line: None, + column: None, + node: fullname.to_string(), + nodename, + compatible: compatible.to_vec(), + message: unmatched_message(compatible), + formatted: None, + } +} + +/// The JSON `message` field for an unmatched compatible diagnostic. +/// +/// Keep the legacy list formatting in structured output even if stderr is only +/// expected to be similar. +fn unmatched_message(compatible: &[String]) -> String { + format!( + "failed to match any schema with compatible: {}", + py_list_repr(compatible) + ) +} + +/// Legacy single-quoted list representation: `['a', 'b']`. +fn py_list_repr(items: &[String]) -> String { + let inner: Vec = items + .iter() + .map(|s| format!("'{}'", s.replace('\\', "\\\\").replace('\'', "\\'"))) + .collect(); + format!("[{}]", inner.join(", ")) +} + +/// Build the JSON record for a byte-decode error. +pub fn decode_diagnostic(filename: &str, message: &str) -> Diagnostic { + Diagnostic::Decode { + level: DiagnosticLevel::Error, + file: display_path(filename), + line: None, + column: None, + message: message.to_string(), + } +} + +/// Return the stderr line for a diagnostic (`formatted` if present, else a +/// type-specific fallback). +pub fn diagnostic_text(d: &Value) -> String { + if let Some(f) = d.get("formatted").and_then(Value::as_str) { + return f.to_string(); + } + match d.get("type").and_then(Value::as_str) { + Some("unmatched") => format!( + "{}: {}: {}", + d["file"].as_str().unwrap_or(""), + d["node"].as_str().unwrap_or(""), + d["message"].as_str().unwrap_or(""), + ), + _ => d + .get("message") + .and_then(Value::as_str) + .unwrap_or("") + .to_string(), + } +} diff --git a/rust/dtschema/src/dtb.rs b/rust/dtschema/src/dtb.rs new file mode 100644 index 0000000..fdce024 --- /dev/null +++ b/rust/dtschema/src/dtb.rs @@ -0,0 +1,1137 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Schema-driven flattened-devicetree (DTB) decoder. +//! +//! Walks a DTB with the [`fdt`] crate, decoding each property's raw bytes into a +//! typed value using the property-type caches (`generated-types` / +//! `generated-pattern-types`) the schema pipeline produces, then reshapes GPIO, +//! interrupt, address, and phandle cell arrays into their validation form. +//! +//! The decoded tree is a [`DtValue`] rather than a raw `serde_json::Value` so +//! raw bytes, nodes, booleans, and integer bit widths stay distinguishable for +//! validation. +//! [`DtValue::to_json`] lowers a decoded tree to JSON for output / comparison. + +use std::collections::{BTreeMap, BTreeSet}; +use std::path::PathBuf; + +use fancy_regex::Regex as FancyRegex; +use serde_json::{Number, Value}; + +use crate::process::process_schemas; +use crate::types::get_prop_types; + +/// A decoded devicetree value. +#[derive(Clone, Debug, PartialEq)] +pub enum DtValue { + /// A present-but-empty property (`len(p) == 0`) — a boolean flag. + Bool(bool), + /// A `sized_int`: value plus its bit-width (8/16/32/64). + Int { val: i128, size: u32 }, + /// Undecoded raw property bytes (no known type, or a decode fallback). + Bytes(Vec), + /// A single decoded string. + Str(String), + /// An array / matrix row / string list. + List(Vec), + /// A node: named properties and child nodes share one namespace. + Node(BTreeMap), +} + +impl DtValue { + fn as_int(&self) -> Option { + match self { + DtValue::Int { val, .. } => Some(*val), + _ => None, + } + } + + fn as_node(&self) -> Option<&BTreeMap> { + match self { + DtValue::Node(m) => Some(m), + _ => None, + } + } + + /// Lower to `serde_json::Value` for output / comparison. Raw bytes become + /// `{"$bytes": [..]}` (a tag that can never collide with a decoded int + /// array); integer widths are omitted from JSON output. + pub fn to_json(&self) -> Value { + match self { + DtValue::Bool(b) => Value::Bool(*b), + DtValue::Int { val, .. } => Value::Number(int_to_number(*val)), + DtValue::Bytes(bytes) => { + let arr = bytes.iter().map(|b| Value::from(*b)).collect(); + let mut m = serde_json::Map::new(); + m.insert("$bytes".to_string(), Value::Array(arr)); + Value::Object(m) + } + DtValue::Str(s) => Value::String(s.clone()), + DtValue::List(l) => Value::Array(l.iter().map(DtValue::to_json).collect()), + DtValue::Node(m) => { + let mut o = serde_json::Map::new(); + for (k, v) in m { + o.insert(k.clone(), v.to_json()); + } + Value::Object(o) + } + } + } +} + +fn int_to_number(val: i128) -> Number { + if val < 0 { + Number::from(val as i64) + } else if val <= u64::MAX as i128 { + Number::from(val as u64) + } else { + // Out of u64 range shouldn't happen for DT (max is uint64); clamp. + Number::from(val as i64) + } +} + +/// `struct` size (bytes) and signedness for a base type name. +fn type_format(base: &str) -> Option<(usize, bool)> { + Some(match base { + "int8" => (1, true), + "uint8" => (1, false), + "int16" => (2, true), + "uint16" => (2, false), + "int32" => (4, true), + "uint32" => (4, false), + "int64" => (8, true), + "uint64" => (8, false), + "phandle" => (4, false), + "address" => (4, false), + _ => return None, + }) +} + +/// Unpack `data` big-endian into `sized_int`s of the given base type. +fn unpack(data: &[u8], size: usize, signed: bool) -> Vec { + let bits = (size * 8) as u32; + let mut out = Vec::with_capacity(data.len() / size); + for chunk in data.chunks_exact(size) { + let mut u: u128 = 0; + for &b in chunk { + u = (u << 8) | b as u128; + } + let val: i128 = if signed { + // Sign-extend from `size` bytes. + let shift = 128 - bits; + ((u as i128) << shift) >> shift + } else { + u as i128 + }; + out.push(DtValue::Int { val, size: bits }); + } + out +} + +// --------------------------------------------------------------------------- +// Property-type context (validator.property_get_type / _dim / has_fixed_dims). +// --------------------------------------------------------------------------- + +struct PatEntry { + regex: FancyRegex, + ptype: Option, + dim: Option, +} + +/// The subset of `DTValidator` state the decoder consults: the property-type +/// caches, used to resolve a property name to its candidate types and matrix +/// dimensions. +pub struct TypeContext { + /// Exact property name → type-entry list. + props: BTreeMap>, + /// `pat_props` compiled: pattern → first entry's type/dim. + pat: Vec, +} + +impl TypeContext { + /// Build from raw schema paths, always including the bundled core schemas. + pub fn new(schema_paths: &[PathBuf]) -> Self { + let schemas = process_schemas(schema_paths, true); + Self::from_schemas(&schemas) + } + + /// Build a decode context from an already-processed schema map (the + /// `generated-types`/`generated-pattern-types` entries and the individual + /// bindings), avoiding a second processing pass when the validator has + /// already built [`crate::process::ProcessedSchemas`]. + pub fn from_processed(schemas: &BTreeMap) -> Self { + Self::from_schemas(schemas) + } + + fn from_schemas(schemas: &BTreeMap) -> Self { + let (props, pat_props) = get_prop_types(schemas); + let pat = pat_props + .into_iter() + .filter_map(|(k, list)| { + let first = list.first()?; + let regex = FancyRegex::new(&k).ok()?; + Some(PatEntry { + regex, + ptype: first + .get("type") + .and_then(Value::as_str) + .map(str::to_string), + dim: first.get("dim").cloned(), + }) + }) + .collect(); + Self { props, pat } + } + + fn pat_matches(re: &FancyRegex, name: &str) -> bool { + re.is_match(name).unwrap_or(false) + } + + /// Return the candidate decoded types for a property. + fn get_type(&self, name: &str) -> BTreeSet { + let mut types: BTreeSet = BTreeSet::new(); + if let Some(list) = self.props.get(name) { + for v in list { + if let Some(t) = v.get("type").and_then(Value::as_str) { + types.insert(t.to_string()); + } + } + } + if types.is_empty() { + for p in &self.pat { + if let Some(t) = &p.ptype + && !types.contains(t) + && Self::pat_matches(&p.regex, name) + { + types.insert(t.clone()); + } + } + } + if types.len() > 1 { + types.remove("node"); + } + types + } + + /// Return the matrix dimensions for a property, if known. + fn get_type_dim(&self, name: &str) -> Option { + if let Some(list) = self.props.get(name) { + for v in list { + if let Some(dim) = v.get("dim") { + return Some(dim.clone()); + } + } + } + for p in &self.pat { + if p.ptype.is_some() + && let Some(dim) = &p.dim + && Self::pat_matches(&p.regex, name) + { + return Some(dim.clone()); + } + } + None + } + + /// Return whether a property has fixed matrix dimensions. + fn has_fixed_dimensions(&self, name: &str) -> bool { + match self.get_type_dim(name) { + Some(dim) => { + let d = |i: usize, j: usize| dim[i][j].as_i64().unwrap_or(0); + (d(0, 0) > 0 && d(0, 0) == d(0, 1)) || (d(1, 0) > 0 && d(1, 0) == d(1, 1)) + } + None => false, + } + } +} + +// --------------------------------------------------------------------------- +// Decoding (prop_value / get_stride / node scan). +// --------------------------------------------------------------------------- + +/// Pick a row stride for a matrix of `prop_len` scalars given +/// `dim = [[min,max],[min,max]]`. +fn get_stride(prop_len: i64, dim: &Value) -> i64 { + let g = |i: usize, j: usize| dim[i][j].as_i64().unwrap_or(0); + let mut outer_limit = g(0, 1); + if outer_limit == 0 { + outer_limit = prop_len; + } + let mut inner_limit = g(1, 1); + if inner_limit == 0 { + inner_limit = prop_len; + } + for outer in g(0, 0)..=outer_limit { + for inner in g(1, 0)..=inner_limit { + if outer * inner == prop_len { + return inner; + } + } + } + if g(1, 0) > 0 && g(1, 0) == g(1, 1) { + return g(1, 0); + } + if g(0, 0) > 0 && g(0, 0) == g(0, 1) { + return prop_len / g(0, 0); + } + prop_len +} + +/// Decode NUL-separated printable ASCII strings, or `None`. +fn bytes_to_string(b: &[u8]) -> Option> { + let s = std::str::from_utf8(b).ok()?; + if !s.is_ascii() { + return None; + } + let strings: Vec<&str> = s.split('\0').collect(); + let count = strings.len() as isize - 1; + if count > 0 && strings.last() == Some(&"") { + // Reject empty or non-printable interior strings; otherwise accept all + // strings before the trailing NUL. + for st in &strings[..strings.len() - 1] { + if st.is_empty() { + return None; + } + if !st.chars().all(|c| !c.is_control()) { + return None; + } + } + return Some( + strings[..strings.len() - 1] + .iter() + .map(|s| s.to_string()) + .collect(), + ); + } + None +} + +/// Decode one property's raw bytes. +fn prop_value( + ctx: &TypeContext, + decode_errors: &mut Vec, + nodename: &str, + name: &str, + data: &[u8], +) -> DtValue { + if data.is_empty() { + return DtValue::Bool(true); + } + + if name != "phandle" && (nodename == "__fixups__" || nodename == "aliases") { + return string_list(&data[..data.len().saturating_sub(1)]); + } + + let mut prop_types = ctx.get_type(name); + prop_types.remove("node"); + + if prop_types.is_empty() { + return DtValue::Bytes(data.to_vec()); + } + + let plen = data.len(); + // Filter out types impossible for this length. + if prop_types.len() > 1 { + let rm = |s: &mut BTreeSet, items: &[&str]| { + for it in items { + s.remove(*it); + } + }; + if !plen.is_multiple_of(8) { + rm( + &mut prop_types, + &["int64", "uint64", "int64-array", "uint64-array"], + ); + } + if !plen.is_multiple_of(4) { + rm( + &mut prop_types, + &[ + "int32", + "uint32", + "int32-array", + "uint32-array", + "phandle", + "phandle-array", + ], + ); + } + if !plen.is_multiple_of(2) { + rm( + &mut prop_types, + &["int16", "uint16", "int16-array", "uint16-array"], + ); + } + if plen > 4 { + rm(&mut prop_types, &["int32", "uint32", "phandle"]); + } else { + rm( + &mut prop_types, + &["int64", "uint64", "int64-array", "uint64-array"], + ); + } + if plen > 2 { + rm(&mut prop_types, &["int16", "uint16"]); + } else { + rm( + &mut prop_types, + &[ + "int32", + "uint32", + "int32-array", + "uint32-array", + "phandle", + "phandle-array", + ], + ); + } + if plen > 1 { + rm(&mut prop_types, &["int8", "uint8"]); + } else { + rm( + &mut prop_types, + &["int16", "uint16", "int16-array", "uint16-array"], + ); + } + if plen > 0 { + rm(&mut prop_types, &["flag"]); + } + + // Drop the unsigned type if both signed and unsigned exist. + for (s, u) in [ + ("int64", "uint64"), + ("int32", "uint32"), + ("int16", "uint16"), + ("int8", "uint8"), + ] { + if prop_types.contains(s) && prop_types.contains(u) { + prop_types.remove(u); + } + } + } + + let mut dim = ctx.get_type_dim(name); + let matrix_prop_types: BTreeSet = prop_types + .iter() + .filter(|t| t.contains("matrix") || *t == "phandle-array") + .cloned() + .collect(); + + let mut fmt: Option = None; + + if prop_types.len() > 1 { + if name == "dma-masters" { + fmt = Some("phandle-array".to_string()); + } else if name == "gpios" { + fmt = Some(if nodename.contains("hog") { + "uint32-matrix".to_string() + } else { + "phandle-array".to_string() + }); + } else if name == "mode-gpios" { + fmt = Some("phandle-array".to_string()); + } else if name == "cooling-levels" { + fmt = Some(if data[0] == 0 && data[1] == 0 { + "uint32-array".to_string() + } else { + "uint8-array".to_string() + }); + } else if prop_types.contains("string") || prop_types.contains("string-array") { + if let Some(strs) = bytes_to_string(data) { + return DtValue::List(strs.into_iter().map(DtValue::Str).collect()); + } + // Assume only one other type. + let mut rest: Vec = prop_types + .iter() + .filter(|t| *t != "string" && *t != "string-array") + .cloned() + .collect(); + match rest.pop() { + Some(t) => fmt = Some(t), + None => return DtValue::Bytes(data.to_vec()), + } + } else if !matrix_prop_types.is_empty() { + let scalar: Vec = prop_types.difference(&matrix_prop_types).cloned().collect(); + if scalar.len() == 1 { + let f = scalar[0].clone(); + let base = f.split('-').next().unwrap_or(&f); + let (fsize, _) = type_format(base).unwrap_or((4, false)); + let mut min_dim = if let Some(d) = &dim { + d[1][0].as_i64().unwrap_or(0) + } else { + 0 + }; + if min_dim == 0 { + min_dim = 1; + } + if let Some(d) = &dim { + min_dim *= d[0][0].as_i64().unwrap_or(0); + } else { + min_dim *= 0; + } + if (plen as f64) / (fsize as f64) >= min_dim as f64 { + // Pick the matrix type (arbitrary member of the set). + fmt = matrix_prop_types.iter().next().cloned(); + } else { + fmt = Some(f); + dim = Some(serde_json::json!([[1, 1], [1, 1]])); + } + } + } + } + + if fmt.is_none() && !prop_types.is_empty() { + if prop_types.len() > 1 { + eprintln!("{name}: property has multiple types: {prop_types:?}"); + } + // Pick the first one (BTreeSet: lexicographically smallest). This only + // affects genuinely ambiguous multi-type props, which the differential + // test guards. Remove the chosen type so the `flag` check below sees + // only the remaining candidates. + fmt = prop_types.iter().next().cloned(); + if let Some(f) = &fmt { + prop_types.remove(f); + } + } + + let Some(fmt) = fmt else { + return DtValue::Bytes(data.to_vec()); + }; + + if fmt.starts_with("string") { + if data.last() != Some(&0) { + return DtValue::Bytes(data.to_vec()); + } + return string_list(&data[..data.len() - 1]); + } + + if prop_types.contains("flag") { + if !data.is_empty() { + if fmt == "flag" { + decode_error( + decode_errors, + format!("{name}: boolean property with value {data:?}"), + ); + return DtValue::Bytes(data.to_vec()); + } + } else { + return DtValue::Bool(true); + } + } + + // Decode the integer(s). An unknown base type (e.g. `flag`) or a length + // that isn't a whole number of elements is a decode error: emit the size + // error, then re-unpack purely on total length (4→uint32, 2→uint16, + // 1→uint8, otherwise leave the bytes undecoded). + let base = fmt.split('-').next().unwrap_or(&fmt); + let mut val_int = match type_format(base) { + Some((size, signed)) if plen.is_multiple_of(size) => unpack(data, size, signed), + _ => { + decode_error( + decode_errors, + format!("{name}: size ({plen}) error for type {fmt}"), + ); + match plen { + 4 => unpack(data, 4, false), + 2 => unpack(data, 2, false), + 1 => unpack(data, 1, false), + _ => return DtValue::Bytes(data.to_vec()), + } + } + }; + + let is_matrix = + fmt.contains("matrix") || matches!(fmt.as_str(), "phandle" | "phandle-array" | "address"); + if is_matrix { + if let Some(dim) = &dim { + let stride = get_stride(val_int.len() as i64, dim).max(1) as usize; + let mut rows = Vec::new(); + let mut i = 0; + while i < val_int.len() { + let end = (i + stride).min(val_int.len()); + rows.push(DtValue::List(val_int[i..end].to_vec())); + i += stride; + } + DtValue::List(rows) + } else { + DtValue::List(vec![DtValue::List(val_int)]) + } + } else if !fmt.contains("array") && val_int.len() == 1 { + val_int.pop().unwrap() + } else { + DtValue::List(val_int) + } +} + +/// Decode NUL-separated ASCII into a list of strings (already trimmed of the +/// trailing NUL byte by the caller). +fn string_list(data: &[u8]) -> DtValue { + let s = String::from_utf8_lossy(data); + DtValue::List(s.split('\0').map(|p| DtValue::Str(p.to_string())).collect()) +} + +fn decode_error(errors: &mut Vec, msg: String) { + errors.push(msg); +} + +// --------------------------------------------------------------------------- +// Node scanning (fdt walk). +// --------------------------------------------------------------------------- + +struct Scanner<'a> { + ctx: &'a TypeContext, + decode_errors: &'a mut Vec, + phandle_loc: Vec, +} + +impl<'a> Scanner<'a> { + fn node_props( + &mut self, + node: &fdt::node::FdtNode, + nodename: &str, + ) -> BTreeMap { + let mut props = BTreeMap::new(); + for p in node.properties() { + let v = prop_value(self.ctx, self.decode_errors, nodename, p.name, p.value); + props.insert(p.name.to_string(), v); + } + props + } + + /// Decode a node's props, then recurse into subnodes. Returns `None` for + /// the special `__*__` nodes, which are consumed for their side effects. + fn scan_node(&mut self, node: &fdt::node::FdtNode, nodename: &str) -> Option { + if nodename == "__fixups__" { + self.process_fixups(node); + return None; + } + if nodename == "__local_fixups__" { + self.process_local_fixups(node, ""); + return None; + } + if nodename.starts_with("__") { + return None; + } + + let mut map = self.node_props(node, nodename); + for child in node.children() { + if let Some(sub) = self.scan_node(&child, child.name) { + map.insert(child.name.to_string(), sub); + } + } + Some(DtValue::Node(map)) + } + + /// Collect every fixup string into `phandle_loc`. + fn process_fixups(&mut self, node: &fdt::node::FdtNode) { + let props = self.node_props(node, "__fixups__"); + for v in props.values() { + if let DtValue::List(items) = v { + for it in items { + if let DtValue::Str(s) = it { + self.phandle_loc.push(s.clone()); + } + } + } + } + } + + /// For each property, append `path:name:offset` for every uint32 offset, + /// recursing with `/name` appended. + fn process_local_fixups(&mut self, node: &fdt::node::FdtNode, path: &str) { + for p in node.properties() { + for chunk in p.value.chunks_exact(4) { + let off = u32::from_be_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]); + self.phandle_loc.push(format!("{path}:{}:{off}", p.name)); + } + } + for child in node.children() { + let child_path = format!("{path}/{}", child.name); + self.process_local_fixups(&child, &child_path); + } + } +} + +// --------------------------------------------------------------------------- +// Fixup passes. +// --------------------------------------------------------------------------- + +/// phandle+args cell-count sources for properties that don't follow the +/// standard `foos` / `#foo-cells` convention. +enum CellName { + Named(String), + Fixed(i64), + None, +} + +fn phandle_args(name: &str) -> Option { + Some(match name { + "assigned-clocks" | "assigned-clock-parents" => CellName::Named("#clock-cells".into()), + "cooling-device" => CellName::Named("#cooling-cells".into()), + "interrupts-extended" => CellName::Named("#interrupt-cells".into()), + "interconnects" => CellName::Named("#interconnect-cells".into()), + "mboxes" => CellName::Named("#mbox-cells".into()), + "sound-dai" => CellName::Named("#sound-dai-cells".into()), + "msi-parent" => CellName::Named("#msi-cells".into()), + "msi-ranges" => CellName::Named("#interrupt-cells".into()), + "dma-masters" => CellName::Named("#dma-cells".into()), + "gpio-ranges" => CellName::Fixed(3), + "memory-region" => CellName::None, + _ => return None, + }) +} + +/// Return the number of argument cells for a referenced provider. +fn get_cells_size(node: &BTreeMap, cellname: &CellName) -> i64 { + match cellname { + CellName::Fixed(n) => *n, + CellName::None => 0, + CellName::Named(name) => node + .get(name) + .and_then(DtValue::as_int) + .map(|v| v as i64) + .unwrap_or(0), + } +} + +/// Return the number of cells for a directly-named cell property. +fn get_named_cells(node: &BTreeMap, name: &str) -> i64 { + node.get(name) + .and_then(DtValue::as_int) + .map(|v| v as i64) + .unwrap_or(0) +} + +/// Decode context carrying the `phandles` map built after scanning. +struct Fixups<'a> { + ctx: &'a TypeContext, + phandles: BTreeMap>, + phandle_loc: BTreeSet, +} + +impl<'a> Fixups<'a> { + /// Return whether the cell at `prop_path` is marked as a phandle. + fn check_is_phandle(&self, prop_path: &str, cell: i64) -> bool { + self.phandle_loc + .contains(&format!("{prop_path}:{}", cell * 4)) + } + + /// Return the width of a phandle group starting at `idx`. + fn phandle_arg_size( + &self, + prop_path: &str, + idx: i64, + cells: &[DtValue], + cellname: &CellName, + ) -> i64 { + if cells.is_empty() { + return 0; + } + let phandle = cells[0].as_int().unwrap_or(0); + if phandle == 0 || matches!(cellname, CellName::None) { + return 1; + } + if phandle == 0xffffffff { + if self.check_is_phandle(prop_path, idx) { + let mut cell_count = 1i64; + while (cell_count as usize) < cells.len() + && !self.check_is_phandle(prop_path, idx + cell_count) + { + cell_count += 1; + } + return cell_count; + } + return 0; + } + let Some(node) = self.phandles.get(&phandle) else { + return 0; + }; + get_cells_size(node, cellname) + 1 + } + + /// Reshape phandle-array properties into phandle argument groups. + fn fixup_phandles(&self, dt: &mut BTreeMap, path: &str) { + let keys: Vec = dt.keys().cloned().collect(); + for k in keys { + // Recurse into child nodes first. + if matches!(dt.get(&k), Some(DtValue::Node(_))) { + let child_path = format!("{path}/{k}"); + if let Some(DtValue::Node(child)) = dt.get_mut(&k) { + let mut child = std::mem::take(child); + self.fixup_phandles(&mut child, &child_path); + dt.insert(k.clone(), DtValue::Node(child)); + } + continue; + } + if !self.ctx.get_type(&k).contains("phandle-array") { + continue; + } + if k != "dma-masters" && self.ctx.has_fixed_dimensions(&k) { + continue; + } + // Not a matrix or already split, nothing to do. + let is_single_matrix = match dt.get(&k) { + Some(DtValue::List(rows)) => rows.len() == 1 && matches!(rows[0], DtValue::List(_)), + _ => false, + }; + if !is_single_matrix { + continue; + } + + let cellname: CellName; + let prop_path = format!("{path}:{k}"); + let val = match dt.get(&k) { + Some(DtValue::List(rows)) => match &rows[0] { + DtValue::List(v) => v.clone(), + _ => continue, + }, + _ => continue, + }; + + if let Some(cn) = phandle_args(&k) { + cellname = cn; + } else if k.ends_with('s') && !k.contains("gpio") { + let name = format!("#{}-cells", &k[..k.len() - 1]); + cellname = CellName::Named(name); + let i = self.phandle_arg_size(&prop_path, 0, &val, &cellname); + if i == 0 { + continue; + } + } else { + continue; + } + + // HACK: a dma-masters phandle in 1..=4 that doesn't resolve to a + // DMA provider is really a uint32, not a phandle. + let phandle = val[0].as_int().unwrap_or(0); + if k == "dma-masters" + && (1..=4).contains(&phandle) + && self + .phandles + .get(&phandle) + .map(|n| !matches!(&cellname, CellName::Named(nm) if n.contains_key(nm))) + .unwrap_or(true) + { + dt.insert( + k.clone(), + DtValue::Int { + val: phandle, + size: 32, + }, + ); + continue; + } + + let mut out: Vec = Vec::new(); + let mut i = 0i64; + while (i as usize) < val.len() { + let slice = &val[i as usize..]; + let mut cells = self.phandle_arg_size(&prop_path, i, slice, &cellname); + if cells == 0 { + break; + } + if k == "msi-ranges" { + cells += 1; + } + if k == "interconnects" { + let next = &val[(i + cells) as usize..]; + cells += self.phandle_arg_size(&prop_path, i + cells, next, &cellname); + } + let end = ((i + cells) as usize).min(val.len()); + out.push(DtValue::List(val[i as usize..end].to_vec())); + i += cells; + } + dt.insert(k.clone(), DtValue::List(out)); + } + } + + /// Reshape GPIO properties into phandle argument groups. + fn fixup_gpios(&self, dt: &mut BTreeMap) { + if dt.contains_key("gpio-hog") { + return; + } + let keys: Vec = dt.keys().cloned().collect(); + for k in keys { + if matches!(dt.get(&k), Some(DtValue::Node(_))) { + if let Some(DtValue::Node(child)) = dt.get_mut(&k) { + let mut child = std::mem::take(child); + self.fixup_gpios(&mut child); + dt.insert(k.clone(), DtValue::Node(child)); + } + continue; + } + let is_gpio = + (k.ends_with("-gpios") || k.ends_with("-gpio") || k == "gpio" || k == "gpios") + && !k.ends_with(",nr-gpios"); + if !is_gpio { + continue; + } + let val = match dt.get(&k) { + Some(DtValue::List(rows)) => match rows.first() { + Some(DtValue::List(v)) => v.clone(), + _ => continue, + }, + _ => continue, + }; + + let mut out: Vec = Vec::new(); + let mut i = 0i64; + while (i as usize) < val.len() { + let phandle = val[i as usize].as_int().unwrap_or(0); + let cells: i64 = if phandle == 0 { + 0 + } else if phandle == 0xffffffff { + // Next 0xffffffff in val[i+1 .. len-1], else len. + let mut found = None; + let start = (i + 1) as usize; + let stop = val.len().saturating_sub(1); + for (off, item) in val.iter().enumerate().take(stop).skip(start) { + if item.as_int() == Some(0xffffffff) { + found = Some(off as i64); + break; + } + } + let base = found.unwrap_or(val.len() as i64); + base - (i + 1) + } else { + match self.phandles.get(&phandle) { + Some(node) => get_named_cells(node, "#gpio-cells"), + None => 0, + } + }; + let end = ((i + cells + 1) as usize).min(val.len()); + out.push(DtValue::List(val[i as usize..end].to_vec())); + i += cells + 1; + } + dt.insert(k.clone(), DtValue::List(out)); + } + } + + /// Reshape interrupt properties using the active interrupt cell count. + fn fixup_interrupts(&self, dt: &mut BTreeMap, mut icells: i64) { + // interrupt-parent handling. + if let Some(DtValue::List(rows)) = dt.get("interrupt-parent") + && let Some(DtValue::List(first)) = rows.first() + { + let phandle = first.first().and_then(DtValue::as_int).unwrap_or(0); + if phandle == 0xffffffff { + dt.remove("interrupt-parent"); + } else if let Some(node) = self.phandles.get(&phandle) { + icells = get_named_cells(node, "#interrupt-cells"); + } + } + + let node_icells = get_named_cells(dt, "#interrupt-cells"); + let has_icells = dt.contains_key("#interrupt-cells"); + let ac = get_named_cells(dt, "#address-cells"); + + let keys: Vec = dt.keys().cloned().collect(); + for k in keys { + if matches!(dt.get(&k), Some(DtValue::Node(_))) { + let child_icells = if has_icells { node_icells } else { icells }; + if let Some(DtValue::Node(child)) = dt.get_mut(&k) { + let mut child = std::mem::take(child); + self.fixup_interrupts(&mut child, child_icells); + dt.insert(k.clone(), DtValue::Node(child)); + } + continue; + } + if k == "interrupts" { + if let Some(val) = first_row(dt.get(&k)) { + let mut out = Vec::new(); + let mut i = 0i64; + let step = icells.max(1); + while (i as usize) < val.len() { + let end = ((i + icells) as usize).min(val.len()); + out.push(DtValue::List(val[i as usize..end].to_vec())); + i += step; + } + dt.insert(k.clone(), DtValue::List(out)); + } + } else if k == "interrupt-map" + && let Some(val) = first_row(dt.get(&k)) + { + let imap_icells = node_icells; + let out = self.split_interrupt_map(&val, ac, imap_icells); + dt.insert(k.clone(), DtValue::List(out)); + } + } + } + + fn split_interrupt_map(&self, val: &[DtValue], ac: i64, imap_icells: i64) -> Vec { + let mut out = Vec::new(); + let phandle_idx = (ac + imap_icells) as usize; + let phandle = val.get(phandle_idx).and_then(DtValue::as_int).unwrap_or(0); + let mut i = 0i64; + if phandle == 0xffffffff { + // Uniform sizes: distance to the next 0xffffffff. + let start = (ac + imap_icells + 1) as usize; + let mut next = None; + for (off, item) in val.iter().enumerate().skip(start) { + if item.as_int() == Some(0xffffffff) { + next = Some(off as i64); + break; + } + } + let cells = match next { + Some(n) => n - (ac + imap_icells), + None => val.len() as i64, + }; + let step = cells.max(1); + while (i as usize) < val.len() { + let end = ((i + cells) as usize).min(val.len()); + out.push(DtValue::List(val[i as usize..end].to_vec())); + i += step; + } + } else { + while (i as usize) < val.len() { + let (p_icells, p_ac) = match self.phandles.get(&phandle) { + Some(node) => ( + get_named_cells(node, "#interrupt-cells"), + if node.contains_key("#address-cells") { + get_named_cells(node, "#address-cells") + } else { + 0 + }, + ), + None => (0, 0), + }; + let cells = ac + imap_icells + 1 + p_ac + p_icells; + let end = ((i + cells) as usize).min(val.len()); + out.push(DtValue::List(val[i as usize..end].to_vec())); + i += cells.max(1); + } + } + out + } + + /// Reshape address-like properties using parent bus cell counts. + /// + /// `ac`/`sc` are the *parent* bus's `#address-cells`/`#size-cells`; they + /// govern how this node's own `reg`/address-type properties reshape. The + /// node's *own* `#address-cells`/`#size-cells` (`node_ac`/`node_sc`) are + /// what get passed down to its children and, for `ranges`, supply the child + /// portion. `ac`/`sc` are only rebound when descending into a child node; + /// since FDT properties always precede subnodes, a node's own `reg` is + /// reshaped with the inherited (parent) cell counts, never its own. + fn fixup_addresses(&self, dt: &mut BTreeMap, ac: i64, sc: i64) { + let node_ac = get_named_cells(dt, "#address-cells"); + let node_sc = get_named_cells(dt, "#size-cells"); + // Cells handed to children: this node's own, or inherited if unset. + let child_ac = if dt.contains_key("#address-cells") { + node_ac + } else { + ac + }; + let child_sc = if dt.contains_key("#size-cells") { + node_sc + } else { + sc + }; + + let keys: Vec = dt.keys().cloned().collect(); + for k in keys { + if matches!(dt.get(&k), Some(DtValue::Node(_))) { + if let Some(DtValue::Node(child)) = dt.get_mut(&k) { + let mut child = std::mem::take(child); + self.fixup_addresses(&mut child, child_ac, child_sc); + dt.insert(k.clone(), DtValue::Node(child)); + } + continue; + } + if self.ctx.get_type(&k).contains("address") { + if let Some(val) = first_row(dt.get(&k)) { + let step = (ac + sc).max(1); + let mut out = Vec::new(); + let mut i = 0i64; + while (i as usize) < val.len() { + let end = ((i + ac + sc) as usize).min(val.len()); + out.push(DtValue::List(val[i as usize..end].to_vec())); + i += step; + } + dt.insert(k.clone(), DtValue::List(out)); + } + } else if (k == "ranges" || k == "dma-ranges") + && !matches!(dt.get(&k), Some(DtValue::Bool(_))) + && let Some(val) = first_row(dt.get(&k)) + { + let child_cells = node_ac + node_sc; + let step = (ac + child_cells).max(1); + let mut out = Vec::new(); + let mut i = 0i64; + while (i as usize) < val.len() { + let end = ((i + ac + child_cells) as usize).min(val.len()); + out.push(DtValue::List(val[i as usize..end].to_vec())); + i += step; + } + dt.insert(k.clone(), DtValue::List(out)); + } + } + } +} + +/// Extract `v[0]` as a row (list of scalars) when `v` is a single-row matrix. +fn first_row(v: Option<&DtValue>) -> Option> { + match v { + Some(DtValue::List(rows)) => match rows.first() { + Some(DtValue::List(row)) => Some(row.clone()), + _ => None, + }, + _ => None, + } +} + +/// Recursively populate the phandle map from the scanned tree: any node with a +/// scalar `phandle` property is recorded under its value. +fn collect_phandles(node: &DtValue, out: &mut BTreeMap>) { + if let DtValue::Node(map) = node { + if let Some(ph) = map.get("phandle").and_then(DtValue::as_int) { + out.insert(ph, map.clone()); + } + for v in map.values() { + if v.as_node().is_some() { + collect_phandles(v, out); + } + } + } +} + +/// Decode a whole DTB into its root node. +pub fn decode_dtb( + ctx: &TypeContext, + dtb: &[u8], + decode_errors: &mut Vec, +) -> anyhow::Result { + let fdt = fdt::Fdt::new(dtb).map_err(|e| anyhow::anyhow!("parsing DTB: {e:?}"))?; + let root = fdt + .find_node("/") + .ok_or_else(|| anyhow::anyhow!("DTB has no root node"))?; + + let mut scanner = Scanner { + ctx, + decode_errors, + phandle_loc: Vec::new(), + }; + let tree = scanner + .scan_node(&root, "/") + .ok_or_else(|| anyhow::anyhow!("root node decoded to nothing"))?; + + let mut phandles = BTreeMap::new(); + collect_phandles(&tree, &mut phandles); + + let DtValue::Node(mut map) = tree else { + return Ok(tree); + }; + + let fixups = Fixups { + ctx, + phandles, + phandle_loc: scanner.phandle_loc.into_iter().collect(), + }; + fixups.fixup_gpios(&mut map); + fixups.fixup_interrupts(&mut map, 1); + fixups.fixup_addresses(&mut map, 2, 1); + fixups.fixup_phandles(&mut map, ""); + + Ok(DtValue::Node(map)) +} diff --git a/rust/dtschema/src/fixups.rs b/rust/dtschema/src/fixups.rs new file mode 100644 index 0000000..9f633e1 --- /dev/null +++ b/rust/dtschema/src/fixups.rs @@ -0,0 +1,782 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Expand the compact binding-schema syntax into strict JSON Schema +//! (Draft 2019-09) that the validator can consume. +//! +//! The entry point is [`fixup_schema`], which mutates a loaded binding document +//! in place. Everything operates on `serde_json::Value`. Key ordering is not +//! significant: the golden differential test canonicalises both sides with +//! sorted keys. + +use regex::Regex; +use serde_json::{Map, Value}; +use std::sync::LazyLock; + +// ---- schema-shape helpers -------------------------------------------------- + +/// Return whether `subschema[key]` (or its first element, if a list) holds a +/// value matching the given predicate. +fn value_is_type(subschema: &Map, key: &str, pred: fn(&Value) -> bool) -> bool { + match subschema.get(key) { + None => false, + Some(Value::Array(a)) => a.first().map(pred).unwrap_or(false), + Some(v) => pred(v), + } +} + +fn is_integer(v: &Value) -> bool { + v.is_i64() || v.is_u64() +} + +fn is_string(v: &Value) -> bool { + v.is_string() +} + +/// Return whether a schema constrains integer values. +fn is_int_schema(subschema: &Value) -> bool { + let Some(obj) = subschema.as_object() else { + return false; + }; + ["const", "enum", "minimum", "maximum"] + .iter() + .any(|k| value_is_type(obj, k, is_integer)) +} + +/// Return whether a schema constrains string values. +fn is_string_schema(subschema: &Value) -> bool { + let Some(obj) = subschema.as_object() else { + return false; + }; + ["const", "enum", "pattern"] + .iter() + .any(|k| value_is_type(obj, k, is_string)) +} + +// ---- scalar/array/matrix fixups -------------------------------------------- + +const SCALAR_KEYWORDS: [&str; 6] = [ + "const", + "enum", + "pattern", + "minimum", + "maximum", + "multipleOf", +]; + +/// Pop scalar keywords out into a fresh object. +fn extract_single_schemas(subschema: &mut Map) -> Value { + let mut out = Map::new(); + for k in SCALAR_KEYWORDS { + if let Some(v) = subschema.remove(k) { + out.insert(k.to_string(), v); + } + } + Value::Object(out) +} + +/// Wrap string-valued schemas in an array item schema. +fn fixup_string_to_array(subschema: &mut Value) { + if !is_string_schema(subschema) { + return; + } + let obj = subschema.as_object_mut().unwrap(); + let inner = extract_single_schemas(obj); + obj.insert("items".to_string(), Value::Array(vec![inner])); +} + +/// Reshape `reg` scalar constraints into the matrix form used for DT data. +fn fixup_reg_schema(subschema: &mut Value, path: &[String]) { + if !path.iter().any(|p| p == "reg") { + return; + } + let Some(obj) = subschema.as_object() else { + return; + }; + + // Determine the item schema to reshape. + let item_is_int = if let Some(items) = obj.get("items") { + match items { + Value::Array(a) => a.first().map(is_int_schema).unwrap_or(false), + other => is_int_schema(other), + } + } else { + is_int_schema(subschema) + }; + + let has_items = obj.contains_key("items"); + if has_items && !item_is_int { + return; + } + if !has_items && !is_int_schema(subschema) { + return; + } + + let obj = subschema.as_object_mut().unwrap(); + let extracted = if let Some(items) = obj.get("items") { + // Extract from the item schema before overwriting the `items` entry. + let mut item_schema = match items { + Value::Array(a) => a + .first() + .and_then(|v| v.as_object()) + .cloned() + .unwrap_or_default(), + Value::Object(m) => m.clone(), + _ => Map::new(), + }; + extract_single_schemas(&mut item_schema) + } else { + // When scalar constraints live on the outer `reg` schema, extraction + // consumes them from that object. + extract_single_schemas(obj) + }; + let inner = serde_json::json!({"items": [extracted]}); + obj.insert("items".to_string(), Value::Array(vec![inner])); +} + +/// Return whether a schema already has matrix-like item constraints. +fn is_matrix_schema(subschema: &Value) -> bool { + let Some(obj) = subschema.as_object() else { + return false; + }; + let Some(items) = obj.get("items") else { + return false; + }; + let has_matrix_key = |m: &Value| { + m.as_object() + .map(|o| { + o.contains_key("items") || o.contains_key("maxItems") || o.contains_key("minItems") + }) + .unwrap_or(false) + }; + match items { + Value::Array(a) => a.iter().any(has_matrix_key), + other => has_matrix_key(other), + } +} + +/// Remove empty `items` arrays after preserving their fixed length. +fn fixup_remove_empty_items(subschema: &mut Value) { + let Some(obj) = subschema.as_object_mut() else { + return; + }; + match obj.get_mut("items") { + None => {} + Some(Value::Object(_)) => { + // recurse into the single items dict + let items = obj.get_mut("items").unwrap(); + fixup_remove_empty_items(items); + } + Some(Value::Array(_)) => { + let items_len = obj["items"].as_array().unwrap().len(); + let mut all_empty = true; + // Stop at the first non-empty item; only an all-empty list is + // collapsed. + { + let arr = obj.get_mut("items").unwrap().as_array_mut().unwrap(); + for item in arr.iter_mut() { + if !item.is_object() { + continue; + } + item.as_object_mut().unwrap().remove("description"); + fixup_remove_empty_items(item); + if item.as_object().map(|o| !o.is_empty()).unwrap_or(false) { + all_empty = false; + break; + } + } + } + if all_empty { + obj.entry("type") + .or_insert(Value::String("array".to_string())); + obj.entry("maxItems") + .or_insert(Value::Number(items_len.into())); + obj.entry("minItems") + .or_insert(Value::Number(items_len.into())); + obj.remove("items"); + } + } + _ => {} + } +} + +// Keep in sync with property-units.yaml +static UNIT_TYPES_ARRAY_RE: LazyLock = LazyLock::new(|| { + Regex::new(r"-(kBps|bits|percent|bp|db|mhz|sec|ms|us|ns|ps|mm|nanoamp|(micro-)?ohms|micro(amp|watt)(-hours)?|milliwatt|(femto|pico)farads|(milli)?celsius|kelvin|k?pascal)$").unwrap() +}); +static UNIT_TYPES_MATRIX_RE: LazyLock = + LazyLock::new(|| Regex::new(r"-(hz|microvolt)$").unwrap()); + +/// Apply implicit types for property names with known unit suffixes. +fn fixup_unit_suffix_props(subschema: &mut Value, path: &[String]) { + // Scan upward for the nearest container keyword, using the previous path + // segment as the property name. + let rev: Vec<&String> = path.iter().rev().collect(); + let mut propname: Option = None; + for (idx, p) in rev.iter().enumerate() { + if matches!(p.as_str(), "properties" | "$defs" | "definitions") { + let prev_idx = if idx == 0 { rev.len() - 1 } else { idx - 1 }; + propname = Some(rev[prev_idx].to_string()); + break; + } + } + let Some(propname) = propname else { + return; + }; + + if subschema.get("$ref").is_some() { + return; + } + + if UNIT_TYPES_ARRAY_RE.is_match(&propname) && is_int_schema(subschema) { + let obj = subschema.as_object_mut().unwrap(); + let inner = extract_single_schemas(obj); + obj.insert("items".to_string(), Value::Array(vec![inner])); + } else if UNIT_TYPES_MATRIX_RE.is_match(&propname) { + if is_matrix_schema(subschema) { + return; + } + let has_size_key = subschema + .as_object() + .map(|o| { + o.contains_key("items") || o.contains_key("minItems") || o.contains_key("maxItems") + }) + .unwrap_or(false); + if has_size_key { + let clone = subschema.clone(); + let obj = subschema.as_object_mut().unwrap(); + obj.insert("items".to_string(), Value::Array(vec![clone])); + obj.remove("minItems"); + obj.remove("maxItems"); + } else if is_int_schema(subschema) { + let obj = subschema.as_object_mut().unwrap(); + let inner = extract_single_schemas(obj); + let matrix = serde_json::json!({"items": [inner]}); + obj.insert("items".to_string(), Value::Array(vec![matrix])); + } + } +} + +/// Fill in fixed array sizes implied by `items`. +fn fixup_items_size(schema: &mut Value, path: &[String]) { + match schema { + Value::Array(arr) => { + for l in arr.iter_mut() { + fixup_items_size(l, path); + } + } + Value::Object(obj) => { + obj.remove("description"); + if obj.contains_key("items") { + obj.insert("type".to_string(), Value::String("array".to_string())); + + if let Some(Value::Array(a)) = obj.get("items") { + let c = a.len(); + if !obj.contains_key("minItems") { + obj.insert("minItems".to_string(), Value::Number(c.into())); + } + if !obj.contains_key("maxItems") { + obj.insert("maxItems".to_string(), Value::Number(c.into())); + } + } + + let mut new_path = path.to_vec(); + new_path.push("items".to_string()); + let items = obj.get_mut("items").unwrap(); + fixup_items_size(items, &new_path); + } else if !path.iter().any(|p| p == "then" || p == "else") { + let has_max = obj.contains_key("maxItems"); + let has_min = obj.contains_key("minItems"); + if has_max && !has_min { + let v = obj.get("maxItems").unwrap().clone(); + obj.insert("minItems".to_string(), v); + } else if has_min && !has_max { + let v = obj.get("minItems").unwrap().clone(); + obj.insert("maxItems".to_string(), v); + } + } + } + _ => {} + } +} + +/// Split legacy `dependencies` into `dependentRequired` / `dependentSchemas`. +fn fixup_schema_to_201909(schema: &mut Value) { + let Some(obj) = schema.as_object_mut() else { + return; + }; + let Some(Value::Object(deps)) = obj.remove("dependencies") else { + return; + }; + for (k, v) in deps { + if v.is_array() { + let dr = obj + .entry("dependentRequired") + .or_insert_with(|| Value::Object(Map::new())); + dr.as_object_mut().unwrap().insert(k, v); + } else { + let ds = obj + .entry("dependentSchemas") + .or_insert_with(|| Value::Object(Map::new())); + ds.as_object_mut().unwrap().insert(k, v); + } + } +} + +/// Apply value-level schema fixups. +fn fixup_vals(schema: &mut Value, path: &[String]) { + if let Some(obj) = schema.as_object_mut() { + obj.remove("description"); + } + fixup_reg_schema(schema, path); + fixup_remove_empty_items(schema); + fixup_unit_suffix_props(schema, path); + fixup_string_to_array(schema); + fixup_items_size(schema, path); + fixup_schema_to_201909(schema); +} + +/// Collapse simple `oneOf: [{const: ...}, ...]` forms into `enum`. +fn fixup_oneof_to_enum(schema: &mut Value, path: &[String]) { + let Some(obj) = schema.as_object() else { + return; + }; + + let list_key = if obj.contains_key("anyOf") { + "anyOf" + } else if obj.contains_key("oneOf") { + "oneOf" + } else if obj.get("items").map(Value::is_object).unwrap_or(false) { + let mut new_path = path.to_vec(); + new_path.push("items".to_string()); + let items = schema.as_object_mut().unwrap().get_mut("items").unwrap(); + fixup_oneof_to_enum(items, &new_path); + return; + } else { + return; + }; + + let sch_list = obj.get(list_key).unwrap().as_array().unwrap(); + let mut const_list = Vec::new(); + for l in sch_list { + let Some(lo) = l.as_object() else { + return; + }; + // This is a strict-superset test, not "has any key outside this set", + // so keys like `{const, deprecated}` still get converted. + let has_strict_allowed_superset = + lo.contains_key("const") && lo.contains_key("description") && lo.len() > 2; + if !lo.contains_key("const") || has_strict_allowed_superset { + return; + } + const_list.push(lo.get("const").unwrap().clone()); + } + + let obj = schema.as_object_mut().unwrap(); + obj.remove("anyOf"); + obj.remove("oneOf"); + obj.insert("enum".to_string(), Value::Array(const_list)); +} + +/// Walk property schemas and apply value-level fixups. +fn walk_properties(schema: &mut Value, path: &[String]) { + if !schema.is_object() { + return; + } + + fixup_oneof_to_enum(schema, path); + + for cond in ["allOf", "oneOf", "anyOf"] { + if schema.get(cond).map(Value::is_array).unwrap_or(false) { + let mut new_path = path.to_vec(); + new_path.push(cond.to_string()); + let arr = schema.as_object_mut().unwrap().get_mut(cond).unwrap(); + if let Value::Array(items) = arr { + for l in items.iter_mut() { + walk_properties(l, &new_path); + } + } + } + } + + if schema.get("then").is_some() { + let mut new_path = path.to_vec(); + new_path.push("then".to_string()); + let then = schema.as_object_mut().unwrap().get_mut("then").unwrap(); + walk_properties(then, &new_path); + } + + fixup_vals(schema, path); +} + +/// Apply interrupt-specific schema fixups. +fn fixup_interrupts(schema: &mut Value, path: &[String]) { + let Some(obj) = schema.as_object_mut() else { + return; + }; + + // properties handling + if let Some(Value::Object(props)) = obj.get("properties") { + let has_int_or_ctrl = + props.contains_key("interrupts") || props.contains_key("interrupt-controller"); + let has_parent = props.contains_key("interrupt-parent"); + let has_interrupts = props.contains_key("interrupts"); + let has_interrupts_ext = props.contains_key("interrupts-extended"); + let interrupts_clone = props.get("interrupts").cloned(); + + let props_mut = obj.get_mut("properties").unwrap().as_object_mut().unwrap(); + if has_int_or_ctrl && !has_parent { + props_mut.insert("interrupt-parent".to_string(), Value::Bool(true)); + } + if has_interrupts && !has_interrupts_ext { + props_mut.insert("interrupts-extended".to_string(), interrupts_clone.unwrap()); + } + } + + // required handling + let required_has_interrupts = obj + .get("required") + .and_then(Value::as_array) + .map(|a| a.iter().any(|v| v.as_str() == Some("interrupts"))) + .unwrap_or(false); + let last_is_oneof = path.last().map(|s| s == "oneOf").unwrap_or(false); + if obj.contains_key("required") && required_has_interrupts && !last_is_oneof { + let reqlist = serde_json::json!([ + {"required": ["interrupts"]}, + {"required": ["interrupts-extended"]} + ]); + if obj.contains_key("oneOf") { + let allof = obj.entry("allOf").or_insert_with(|| Value::Array(vec![])); + allof + .as_array_mut() + .unwrap() + .push(serde_json::json!({"oneOf": reqlist})); + } else { + obj.insert("oneOf".to_string(), reqlist); + } + // remove 'interrupts' from required + if let Some(Value::Array(req)) = obj.get_mut("required") { + if let Some(pos) = req.iter().position(|v| v.as_str() == Some("interrupts")) { + req.remove(pos); + } + if req.is_empty() { + obj.remove("required"); + } + } + } + + // dependentRequired handling + if obj.contains_key("dependentRequired") { + let dep_req = obj.get("dependentRequired").unwrap().as_object().unwrap(); + let has_interrupts = dep_req.contains_key("interrupts"); + let has_interrupts_ext = dep_req.contains_key("interrupts-extended"); + let interrupts_val = dep_req.get("interrupts").cloned(); + + if has_interrupts && !has_interrupts_ext { + obj.get_mut("dependentRequired") + .unwrap() + .as_object_mut() + .unwrap() + .insert("interrupts-extended".to_string(), interrupts_val.unwrap()); + } + + // Iterate props; first one whose value contains 'interrupts' triggers the + // dependentSchemas rewrite, then break after removing. + let prop_keys: Vec = obj + .get("dependentRequired") + .unwrap() + .as_object() + .unwrap() + .keys() + .cloned() + .collect(); + for prop in prop_keys { + let contains = obj["dependentRequired"][&prop] + .as_array() + .map(|a| a.iter().any(|v| v.as_str() == Some("interrupts"))) + .unwrap_or(false); + if !contains { + continue; + } + let ds = serde_json::json!({ + prop.clone(): { + "oneOf": [ + {"required": ["interrupts"]}, + {"required": ["interrupts-extended"]} + ] + } + }); + obj.insert("dependentSchemas".to_string(), ds); + // Move the interrupts dependency into dependentSchemas. + let dr = obj + .get_mut("dependentRequired") + .unwrap() + .as_object_mut() + .unwrap(); + if let Some(Value::Array(list)) = dr.get_mut(&prop) { + if let Some(pos) = list.iter().position(|v| v.as_str() == Some("interrupts")) { + list.remove(pos); + } + if list.is_empty() { + dr.remove(&prop); + break; + } + } + } + + if obj + .get("dependentRequired") + .and_then(Value::as_object) + .map(|o| o.is_empty()) + .unwrap_or(false) + { + obj.remove("dependentRequired"); + } + } + + // dependentSchemas handling + if let Some(Value::Object(ds)) = obj.get("dependentSchemas") + && ds.contains_key("interrupts") + && !ds.contains_key("interrupts-extended") + { + let v = ds.get("interrupts").unwrap().clone(); + obj.get_mut("dependentSchemas") + .unwrap() + .as_object_mut() + .unwrap() + .insert("interrupts-extended".to_string(), v); + } +} + +static KNOWN_VARIABLE_MATRIX_PROPS: [&str; 2] = ["fsl,pins", "qcom,board-id"]; +static PINCTRL_NUM_RE: LazyLock = LazyLock::new(|| Regex::new(r"^pinctrl-[0-9]").unwrap()); + +/// Apply fixups for a node or nested subschema. +fn fixup_sub_schema(schema: &mut Value, path: &[String]) { + if !schema.is_object() { + return; + } + + if let Some(obj) = schema.as_object_mut() { + obj.remove("description"); + } + fixup_schema_to_201909(schema); + fixup_interrupts(schema, path); + fixup_node_props(schema); + + if let Some(obj) = schema.as_object_mut() + && obj.get("additionalProperties") == Some(&Value::Bool(true)) + { + obj.remove("additionalProperties"); + } + + // Snapshot keys before mutating nested values. + let keys: Vec = schema.as_object().unwrap().keys().cloned().collect(); + + for k in keys { + if matches!( + k.as_str(), + "select" | "if" | "then" | "else" | "not" | "additionalProperties" + ) { + let mut new_path = path.to_vec(); + new_path.push(k.clone()); + let v = schema.as_object_mut().unwrap().get_mut(&k).unwrap(); + fixup_sub_schema(v, &new_path); + } + + if matches!(k.as_str(), "allOf" | "anyOf" | "oneOf") { + let mut new_path = path.to_vec(); + new_path.push(k.clone()); + if let Some(Value::Array(arr)) = schema.as_object_mut().unwrap().get_mut(&k) { + for subschema in arr.iter_mut() { + fixup_sub_schema(subschema, &new_path); + } + } + } + + if !matches!( + k.as_str(), + "dependentRequired" + | "dependentSchemas" + | "dependencies" + | "properties" + | "patternProperties" + | "$defs" + | "definitions" + ) { + continue; + } + + // Iterate the props under this container. + let prop_keys: Vec = match schema.as_object().unwrap().get(&k) { + Some(Value::Object(m)) => m.keys().cloned().collect(), + _ => continue, + }; + for prop in prop_keys { + let is_known_matrix = KNOWN_VARIABLE_MATRIX_PROPS.contains(&prop.as_str()); + let prop_is_dict = schema.as_object().unwrap()[&k] + .get(&prop) + .map(Value::is_object) + .unwrap_or(false); + if is_known_matrix && prop_is_dict { + let container = schema.as_object_mut().unwrap().get_mut(&k).unwrap(); + let prop_obj = container.get_mut(&prop).unwrap(); + let ref_val = prop_obj.as_object_mut().unwrap().remove("$ref"); + let mut replacement = Map::new(); + if let Some(r) = ref_val { + replacement.insert("$ref".to_string(), r); + } + container + .as_object_mut() + .unwrap() + .insert(prop.clone(), Value::Object(replacement)); + continue; + } + + let mut new_path = path.to_vec(); + new_path.push(k.clone()); + new_path.push(prop.clone()); + let prop_val = schema + .as_object_mut() + .unwrap() + .get_mut(&k) + .unwrap() + .get_mut(&prop) + .unwrap(); + walk_properties(prop_val, &new_path); + fixup_sub_schema(prop_val, &new_path); + } + } +} + +/// Add implicit node properties and pattern properties. +fn fixup_node_props(schema: &mut Value) { + let Some(obj) = schema.as_object_mut() else { + return; + }; + if !obj.contains_key("unevaluatedProperties") && !obj.contains_key("additionalProperties") { + return; + } + + let mut keys: Vec = Vec::new(); + if let Some(Value::Object(p)) = obj.get("properties") { + keys.extend(p.keys().cloned()); + } + if let Some(Value::Object(pp)) = obj.get("patternProperties") { + keys.extend(pp.keys().cloned()); + } + + if keys.iter().any(|k| k == "clocks") && !keys.iter().any(|k| k == "assigned-clocks") { + let props = obj + .entry("properties") + .or_insert_with(|| Value::Object(Map::new())) + .as_object_mut() + .unwrap(); + for name in [ + "assigned-clocks", + "assigned-clock-rates-u64", + "assigned-clock-rates", + "assigned-clock-parents", + "assigned-clock-sscs", + ] { + props.insert(name.to_string(), Value::Bool(true)); + } + } + + if keys.iter().any(|k| k == "ranges") { + let props = obj + .entry("properties") + .or_insert_with(|| Value::Object(Map::new())) + .as_object_mut() + .unwrap(); + props.entry("dma-ranges").or_insert(Value::Bool(true)); + } + + // If no restrictions on undefined properties, no implicit properties needed. + let addl_true = obj.get("additionalProperties") == Some(&Value::Bool(true)); + let uneval_true = obj.get("unevaluatedProperties") == Some(&Value::Bool(true)); + if addl_true || uneval_true { + return; + } + + let props = obj + .entry("properties") + .or_insert_with(|| Value::Object(Map::new())) + .as_object_mut() + .unwrap(); + for name in [ + "phandle", + "status", + "secure-status", + "$nodename", + "bootph-pre-sram", + "bootph-verify", + "bootph-pre-ram", + "bootph-some-ram", + "bootph-all", + ] { + props.entry(name).or_insert(Value::Bool(true)); + } + + let has_pinctrl_num = keys.iter().any(|k| PINCTRL_NUM_RE.is_match(k)); + if !has_pinctrl_num { + obj.get_mut("properties") + .unwrap() + .as_object_mut() + .unwrap() + .entry("pinctrl-names") + .or_insert(Value::Bool(true)); + let pp = obj + .entry("patternProperties") + .or_insert_with(|| Value::Object(Map::new())) + .as_object_mut() + .unwrap(); + pp.insert("^pinctrl-[0-9]+$".to_string(), Value::Bool(true)); + } +} + +/// Clone a schema value; `serde_json::Value` is already a plain data tree. +fn convert_to_dict(schema: &Value) -> Value { + schema.clone() +} + +/// Add a `select` schema from a constrained `$nodename` when possible. +fn add_select_schema(schema: &mut Value) { + let Some(obj) = schema.as_object() else { + return; + }; + if obj.contains_key("select") { + return; + } + let Some(Value::Object(props)) = obj.get("properties") else { + return; + }; + if props.contains_key("compatible") { + return; + } + let Some(nodename) = props.get("$nodename") else { + return; + }; + if nodename == &Value::Bool(true) { + return; + } + let nodename_conv = convert_to_dict(nodename); + let select = serde_json::json!({ + "required": ["$nodename"], + "properties": {"$nodename": nodename_conv} + }); + schema + .as_object_mut() + .unwrap() + .insert("select".to_string(), select); +} + +/// Apply all schema fixups in place. +pub fn fixup_schema(schema: &mut Value) { + if let Some(obj) = schema.as_object_mut() { + obj.remove("examples"); + obj.remove("maintainers"); + obj.remove("historical"); + } + add_select_schema(schema); + fixup_sub_schema(schema, &[]); +} diff --git a/rust/dtschema/src/lib.rs b/rust/dtschema/src/lib.rs new file mode 100644 index 0000000..aa38a58 --- /dev/null +++ b/rust/dtschema/src/lib.rs @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Devicetree schema validation library. +//! +//! The port keeps `serde_json::Value` as the universal representation for both +//! schema documents and decoded devicetree data. + +pub mod cache; +pub mod diagnostic; +pub mod dtb; +pub mod fixups; +pub mod lib_helpers; +pub mod process; +pub mod schema; +pub mod types; +pub mod validator; +pub mod yaml; + +use std::path::{Path, PathBuf}; + +/// Synthetic schema ID for "is this compatible documented anywhere?". +pub const GENERATED_COMPATIBLES_SCHEMA: &str = "generated-compatibles"; + +/// Locate the bundled `schemas/` and `meta-schemas/` data directories. +/// +/// The schema data lives in `dtschema/`; from the Rust workspace +/// (`rust/dtschema/`) it is at `../../dtschema/`. Allow an override via the +/// `DTSCHEMA_DIR` environment variable so the tools work from an installed +/// location too. +pub fn bundled_dir() -> PathBuf { + if let Ok(dir) = std::env::var("DTSCHEMA_DIR") { + return PathBuf::from(dir); + } + // rust/dtschema/src/lib.rs -> repo/dtschema + let manifest = Path::new(env!("CARGO_MANIFEST_DIR")); + manifest + .parent() // rust/ + .and_then(|p| p.parent()) // repo/ + .map(|p| p.join("dtschema")) + .unwrap_or_else(|| PathBuf::from("dtschema")) +} + +/// The dtschema version string used as a processed-schema cache key. +/// +/// A processed schema's `version` field must match to be reused, so read the +/// generated version module when present to keep processed schemas +/// interchangeable with the installed tools. Falls back to the crate version. +pub fn version() -> String { + let vpy = bundled_dir().join("version.py"); + if let Ok(text) = std::fs::read_to_string(&vpy) { + for line in text.lines() { + let line = line.trim_start(); + if line.starts_with("__version__") && line.contains('=') { + if let Some(start) = line.find('\'') + && let Some(end) = line[start + 1..].find('\'') + { + return line[start + 1..start + 1 + end].to_string(); + } + if let Some(start) = line.find('"') + && let Some(end) = line[start + 1..].find('"') + { + return line[start + 1..start + 1 + end].to_string(); + } + } + } + } + env!("CARGO_PKG_VERSION").to_string() +} diff --git a/rust/dtschema/src/lib_helpers.rs b/rust/dtschema/src/lib_helpers.rs new file mode 100644 index 0000000..6d05960 --- /dev/null +++ b/rust/dtschema/src/lib_helpers.rs @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Small schema-shape helpers shared outside the fixup pipeline. + +use std::collections::BTreeSet; + +use serde_json::Value; + +/// Return whether `const`/`enum`/`pattern` (or its first element) holds a +/// string. +pub fn is_string_schema(subschema: &Value) -> bool { + let Some(obj) = subschema.as_object() else { + return false; + }; + for key in ["const", "enum", "pattern"] { + let matched = match obj.get(key) { + Some(Value::Array(a)) => a.first().map(Value::is_string).unwrap_or(false), + Some(v) => v.is_string(), + None => false, + }; + if matched { + return true; + } + } + false +} + +/// Return `[min, max]` as a JSON array. +pub fn get_array_range(subschema: &Value) -> Value { + // Unwrap a single-element list. + let sub = match subschema { + Value::Array(a) => { + if a.len() != 1 { + return serde_json::json!([0, 0]); + } + &a[0] + } + other => other, + }; + let obj = match sub.as_object() { + Some(o) => o, + None => return serde_json::json!([1, 0]), + }; + + let items_is_list = matches!(obj.get("items"), Some(Value::Array(_))); + if items_is_list { + let max = obj["items"].as_array().unwrap().len() as i64; + let min = obj.get("minItems").and_then(Value::as_i64).unwrap_or(max); + serde_json::json!([min, max]) + } else { + let min = obj.get("minItems").and_then(Value::as_i64).unwrap_or(1); + let max = obj.get("maxItems").and_then(Value::as_i64).unwrap_or(0); + serde_json::json!([min, max]) + } +} + +/// Collect every value under `lookup_key`, recursively. +fn item_generator<'a>(json_input: &'a Value, lookup_key: &str, out: &mut Vec<&'a Value>) { + match json_input { + Value::Object(m) => { + for (k, v) in m { + if k == lookup_key { + out.push(v); + } else { + item_generator(v, lookup_key, out); + } + } + } + Value::Array(a) => { + for item in a { + item_generator(item, lookup_key, out); + } + } + _ => {} + } +} + +/// Extract compatible strings from one node schema. +pub fn extract_node_compatibles_pub(schema: &Value) -> BTreeSet { + extract_node_compatibles(schema) +} + +/// Extract compatible strings from one node schema. +fn extract_node_compatibles(schema: &Value) -> BTreeSet { + let mut compat: BTreeSet = BTreeSet::new(); + if !schema.is_object() { + return compat; + } + + let mut enums = Vec::new(); + item_generator(schema, "enum", &mut enums); + for l in enums { + if let Value::Array(a) = l + && a.first().map(Value::is_string).unwrap_or(false) + { + for v in a { + if let Some(s) = v.as_str() { + compat.insert(s.to_string()); + } + } + } + } + + let mut consts = Vec::new(); + item_generator(schema, "const", &mut consts); + for l in consts { + // Stringify const values even when they are not strings. + let s = match l { + Value::String(s) => s.clone(), + other => other.to_string(), + }; + compat.insert(s); + } + + let mut patterns = Vec::new(); + item_generator(schema, "pattern", &mut patterns); + for l in patterns { + if let Some(s) = l.as_str() { + compat.insert(s.to_string()); + } + } + + compat +} + +/// Extract compatible strings from every compatible schema in a document. +pub fn extract_compatibles(schema: &Value) -> BTreeSet { + let mut compat: BTreeSet = BTreeSet::new(); + if !schema.is_object() { + return compat; + } + let mut nodes = Vec::new(); + item_generator(schema, "compatible", &mut nodes); + for sch in nodes { + compat.extend(extract_node_compatibles(sch)); + } + compat +} diff --git a/rust/dtschema/src/process.rs b/rust/dtschema/src/process.rs new file mode 100644 index 0000000..bc448bb --- /dev/null +++ b/rust/dtschema/src/process.rs @@ -0,0 +1,272 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Schema processing pipeline: load raw binding YAML directories, meta-validate +//! and fix up each schema, then attach the `generated-types`, +//! `generated-pattern-types`, and `generated-compatibles` cache entries plus a +//! `version` marker. +//! +//! The result is the map that `dt-mk-schema` serializes and that the validator +//! consumes. + +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; + +use rayon::prelude::*; +use serde_json::Value; + +use crate::schema::DTSchema; +use crate::{bundled_dir, types}; + +/// A fully processed schema set, keyed by `$id` (trailing `#` stripped), plus +/// the generated cache entries and `version`. +pub struct ProcessedSchemas { + /// Insertion-ordered map of `$id` → processed schema value. Also holds the + /// `generated-*` and `version` keys once [`finalize`](Self::finalize) runs. + pub schemas: BTreeMap, + /// compatible string → schema `$id`. + pub compat_map: BTreeMap, + /// `$id`s of select-bearing schemas, applied unconditionally as `{if,then}`. + pub always_schemas: Vec, +} + +/// Recursively collect `*.yaml` files under `dir`, sorted for deterministic +/// output. File order only affects commutative type-list merging, so sorting is +/// a safe, reproducible choice. +fn collect_yaml(dir: &Path, out: &mut Vec) { + if let Ok(rd) = std::fs::read_dir(dir) { + let mut entries: Vec = rd.flatten().map(|e| e.path()).collect(); + entries.sort(); + for p in entries { + if p.is_dir() { + collect_yaml(&p, out); + } else if p.extension().and_then(|s| s.to_str()) == Some("yaml") { + out.push(p); + } + } + } +} + +/// Load one schema, check that meta-validation can run, fix it up, and tag it +/// with `type: object` and `$filename`. Any diagnostic lines are returned in +/// `warnings` so parallel processing can emit them in a deterministic file +/// order. +fn process_schema(path: &Path, warnings: &mut Vec) -> Option { + let dtsch = match DTSchema::load(path) { + Ok(s) => s, + Err(e) => { + warnings.push(format!("{}: ignoring, error parsing file", path.display())); + let _ = e; + return None; + } + }; + + match dtsch.check_schema_valid() { + Ok(()) => {} + Err(e) => { + warnings.push(format!( + "{}: ignoring, error in schema: {e}", + path.display() + )); + return None; + } + } + + let mut schema = dtsch.fixup(); + if let Some(obj) = schema.as_object_mut() { + obj.insert("type".to_string(), Value::String("object".to_string())); + obj.insert( + "$filename".to_string(), + Value::String(path.to_string_lossy().to_string()), + ); + } + Some(schema) +} + +/// Insert an already-processed schema, deduping by `$id`. +/// Warnings are appended to `warnings` for deterministic ordering. +fn add_processed( + schemas: &mut BTreeMap, + sch: Value, + warnings: &mut Vec, +) -> bool { + let Some(id) = sch.get("$id").and_then(Value::as_str) else { + return false; + }; + let id = id.trim_end_matches('#').to_string(); + if schemas.contains_key(&id) { + warnings.push(format!( + "{}: warning: ignoring duplicate '$id' value '{id}'", + sch.get("$filename").and_then(Value::as_str).unwrap_or("") + )); + return false; + } + schemas.insert(id, sch); + true +} + +/// Process explicit files and directories, plus the bundled core `schemas/` +/// tree when `core_schema` is set. Files are loaded, meta-validated and fixed +/// up in parallel (each is independent), then inserted in deterministic file +/// order so `$id` dedup and warning output stay stable. +pub fn process_schemas(schema_paths: &[PathBuf], core_schema: bool) -> BTreeMap { + let mut schemas: BTreeMap = BTreeMap::new(); + + // Explicit file arguments (in the given order), processed in parallel. + let explicit: Vec<&PathBuf> = schema_paths.iter().filter(|p| p.is_file()).collect(); + let processed: Vec<(Option, Vec)> = explicit + .par_iter() + .map(|p| { + let mut w = Vec::new(); + (process_schema(p, &mut w), w) + }) + .collect(); + for (sch, warns) in processed { + for line in &warns { + eprintln!("{line}"); + } + if let Some(sch) = sch { + let mut w = Vec::new(); + add_processed(&mut schemas, sch, &mut w); + for line in &w { + eprintln!("{line}"); + } + } + } + + let mut dirs: Vec = schema_paths + .iter() + .filter(|p| p.is_dir()) + .cloned() + .collect(); + if core_schema { + dirs.push(bundled_dir().join("schemas")); + } + + for dir in &dirs { + let mut files = Vec::new(); + collect_yaml(dir, &mut files); + // Load + fixup every file in parallel, preserving `files` order. + let processed: Vec<(Option, Vec)> = files + .par_iter() + .map(|f| { + let mut w = Vec::new(); + (process_schema(f, &mut w), w) + }) + .collect(); + let mut count = 0; + for (sch, warns) in processed { + for line in &warns { + eprintln!("{line}"); + } + if let Some(sch) = sch { + let mut w = Vec::new(); + if add_processed(&mut schemas, sch, &mut w) { + count += 1; + } + for line in &w { + eprintln!("{line}"); + } + } + } + if count == 0 { + eprintln!("warning: no schema found in path: {}", dir.display()); + } + } + + schemas +} + +impl ProcessedSchemas { + /// Build the full processed set from raw schema paths. + pub fn build(schema_paths: &[PathBuf], core_schema: bool, version: &str) -> Self { + let mut schemas = process_schemas(schema_paths, core_schema); + + types::make_property_type_cache(&mut schemas); + types::make_compatible_schema(&mut schemas); + + let (compat_map, always_schemas) = Self::assemble_dispatch(&schemas); + + schemas.insert("version".to_string(), Value::String(version.to_string())); + + Self { + schemas, + compat_map, + always_schemas, + } + } + + /// Reconstruct a processed set from a loaded processed-schema JSON document + /// (the `dt-mk-schema -j` output). The `generated-*` entries are kept as-is; + /// `compat_map`/`always_schemas` are rebuilt from the entries. + pub fn from_value(value: &Value, version: &str) -> anyhow::Result { + let obj = value + .as_object() + .ok_or_else(|| anyhow::anyhow!("processed schema is not a JSON object"))?; + if obj.contains_key("$id") { + anyhow::bail!("single schema is not a processed schema set"); + } + match obj.get("version").and_then(Value::as_str) { + Some(v) if v == version => {} + _ => anyhow::bail!("Processed schema out of date, delete and retry"), + } + + let mut schemas: BTreeMap = BTreeMap::new(); + for (k, v) in obj { + if k == "version" { + continue; + } + schemas.insert(k.clone(), v.clone()); + } + + let (compat_map, always_schemas) = Self::assemble_dispatch(&schemas); + schemas.insert("version".to_string(), Value::String(version.to_string())); + + Ok(Self { + schemas, + compat_map, + always_schemas, + }) + } + + /// Build the `compat_map` and `always_schemas` dispatch tables from the + /// non-generated schema entries. + fn assemble_dispatch( + schemas: &BTreeMap, + ) -> (BTreeMap, Vec) { + let mut always_schemas = Vec::new(); + let mut compat_map: BTreeMap = BTreeMap::new(); + for (key, sch) in schemas { + if key.starts_with("generated-") && key != crate::GENERATED_COMPATIBLES_SCHEMA { + continue; + } + let id = sch + .get("$id") + .and_then(Value::as_str) + .unwrap_or(key) + .trim_end_matches('#') + .to_string(); + if let Some(sel) = sch.get("select") { + if sel != &Value::Bool(false) { + always_schemas.push(id); + } + } else if sch + .get("properties") + .and_then(|p| p.get("compatible")) + .is_some() + { + let compat_sch = &sch["properties"]["compatible"]; + let mut compatibles: Vec = + crate::lib_helpers::extract_node_compatibles_pub(compat_sch) + .into_iter() + .collect(); + if compatibles.len() > 1 { + compatibles.retain(|c| c != "syscon" && c != "simple-mfd" && c != "simple-bus"); + } + for c in compatibles { + compat_map.insert(c, id.clone()); + } + } + } + (compat_map, always_schemas) + } +} diff --git a/rust/dtschema/src/schema.rs b/rust/dtschema/src/schema.rs new file mode 100644 index 0000000..6e8dd25 --- /dev/null +++ b/rust/dtschema/src/schema.rs @@ -0,0 +1,358 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Loading and meta-validation of a single binding schema (`DTSchema`). +//! +//! Loads binding YAML files, validates them against the meta-schema named by +//! `$schema`, and checks that `$id` and references resolve. + +use std::collections::HashMap; +use std::path::{Path, PathBuf}; +use std::sync::Mutex; + +use jsonschema::{Draft, Retrieve, Uri}; +use serde_json::Value; + +use crate::{bundled_dir, yaml}; + +const BASE_URL: &str = "http://devicetree.org/"; + +/// Retrieve `http://devicetree.org/...` references from the bundled schema +/// tree (or an override root). Falls back to arbitrary extra roots supplied by +/// the caller, used to resolve a binding's sibling files. +pub struct DtRetriever { + roots: Vec, + cache: Mutex>, +} + +impl DtRetriever { + /// Retriever rooted at the bundled `dtschema/` directory. + pub fn bundled() -> Self { + Self { + roots: vec![bundled_dir()], + cache: Mutex::new(HashMap::new()), + } + } + + /// Add an extra filesystem root to resolve `http://devicetree.org/schemas/` + /// references from (e.g. a binding's own directory). + pub fn with_root(mut self, root: PathBuf) -> Self { + self.roots.insert(0, root); + self + } + + /// Map a `http://devicetree.org/` URI to a filesystem path under each + /// known root and load the first that exists. + fn load_uri(&self, uri: &str) -> Option { + let uri = uri.trim_end_matches('#'); + let rel = uri.strip_prefix(BASE_URL)?; + // `roots` point at the bundled `dtschema/` dir; the URL path already + // carries the `schemas/` or `meta-schemas/` segment. + for root in &self.roots { + let candidate = root.join(rel); + if candidate.is_file() + && let Ok(v) = yaml::from_file(&candidate) + { + return Some(v); + } + } + // A binding's sibling-file roots point directly at the `schemas/` + // subtree, so also try mapping `schemas/` onto the root itself. + if let Some(schemas_rel) = rel.strip_prefix("schemas/") { + for root in &self.roots { + let candidate = root.join(schemas_rel); + if candidate.is_file() + && let Ok(v) = yaml::from_file(&candidate) + { + return Some(v); + } + } + } + None + } +} + +impl Retrieve for DtRetriever { + fn retrieve( + &self, + uri: &Uri, + ) -> Result> { + let key = uri.as_str().to_string(); + if let Some(v) = self.cache.lock().unwrap().get(&key) { + return Ok(v.clone()); + } + match self.load_uri(&key) { + Some(v) => { + self.cache.lock().unwrap().insert(key, v.clone()); + Ok(v) + } + None => Err(format!("no schema for {key}").into()), + } + } +} + +/// A single binding schema file loaded into memory. +pub struct DTSchema { + pub value: Value, + pub filename: PathBuf, +} + +impl DTSchema { + /// Load a binding schema from a YAML file. + pub fn load(path: &Path) -> anyhow::Result { + let value = yaml::from_file(path)?; + Ok(Self { + value, + filename: std::fs::canonicalize(path).unwrap_or_else(|_| path.to_path_buf()), + }) + } + + /// The `$id` with any trailing `#` removed. + pub fn id(&self) -> Option { + self.value + .get("$id") + .and_then(Value::as_str) + .map(|s| s.trim_end_matches('#').to_string()) + } + + /// The meta-schema URI named by `$schema`, trailing `#` stripped. + fn meta_schema_id(&self) -> Option { + self.value + .get("$schema") + .and_then(Value::as_str) + .map(|s| s.trim_end_matches('#').to_string()) + } + + /// Validate this binding against its `$schema` meta-schema. + /// + /// Returns the list of human-readable validation errors (empty ⇒ valid). + pub fn meta_validate(&self) -> anyhow::Result> { + let meta_id = self + .meta_schema_id() + .ok_or_else(|| anyhow::anyhow!("{}: missing $schema", self.filename.display()))?; + + let retriever = DtRetriever::bundled(); + let meta_schema = retriever + .load_uri(&meta_id) + .ok_or_else(|| anyhow::anyhow!("cannot load meta-schema {meta_id}"))?; + + let validator = jsonschema::options() + .with_draft(Draft::Draft201909) + .with_retriever(DtRetriever::bundled()) + .build(&meta_schema) + .map_err(|e| anyhow::anyhow!("building meta-schema validator: {e}"))?; + + let mut errors: Vec = validator + .iter_errors(&self.value) + .map(|e| format!("{}: {e}", e.instance_path())) + .collect(); + errors.sort(); + Ok(errors) + } + + /// True if the binding meta-validates with no errors. + pub fn is_valid(&self) -> anyhow::Result { + Ok(self.meta_validate()?.is_empty()) + } + + /// Check that the document is structurally valid JSON Schema. This only + /// rejects bindings that fail the JSON Schema meta-schema; it deliberately + /// does not compile `self.value` as a validator, because that would eagerly + /// resolve ordinary binding `$ref`s. + pub fn check_schema_valid(&self) -> anyhow::Result<()> { + jsonschema::draft201909::meta::validate(&self.value) + .map_err(|e| anyhow::anyhow!("{}", e.instance_path())) + } + + /// Return strict meta-validation errors rendered as `dt-doc-validate` + /// stderr lines: + /// `: : `. A DTB carries no source + /// positions, and neither does a YAML load here, so the legacy `line:col` + /// field is omitted (matching the `-n` obsolete flag). + pub fn format_errors(&self) -> anyhow::Result> { + let meta_id = self + .meta_schema_id() + .ok_or_else(|| anyhow::anyhow!("{}: missing $schema", self.filename.display()))?; + let retriever = DtRetriever::bundled(); + let meta_schema = retriever + .load_uri(&meta_id) + .ok_or_else(|| anyhow::anyhow!("cannot load meta-schema {meta_id}"))?; + let validator = jsonschema::options() + .with_draft(Draft::Draft201909) + .with_retriever(DtRetriever::bundled()) + .build(&meta_schema) + .map_err(|e| anyhow::anyhow!("building meta-schema validator: {e}"))?; + + let abs = std::path::absolute(&self.filename) + .unwrap_or_else(|_| self.filename.clone()) + .to_string_lossy() + .into_owned(); + + let mut errs: Vec = validator + .iter_errors(&self.value) + .map(|e| { + let mut src = format!("{abs}: "); + for seg in e.instance_path().iter() { + match seg { + jsonschema::paths::LocationSegment::Property(p) => { + src.push_str(&p); + src.push(':'); + } + jsonschema::paths::LocationSegment::Index(i) => { + src.push_str(&i.to_string()); + src.push(':'); + } + } + } + if e.instance_path().iter().next().is_some() { + src.push(' '); + } + format!("{src}{e}") + }) + .collect(); + errs.sort(); + Ok(errs) + } + + /// Run the fixup pipeline, returning the processed schema. The original + /// `value` is left untouched. + pub fn fixup(&self) -> Value { + let mut processed = self.value.clone(); + crate::fixups::fixup_schema(&mut processed); + processed + } + + /// Emit warnings for node subschemas that constrain no undefined + /// properties (missing `additionalProperties`/ + /// `unevaluatedProperties`). References are resolved against the bundled + /// tree and the binding's own directory. Warnings go to stderr. + pub fn check_schema_refs(&self) { + // Resolve `$ref`s against the bundled tree plus the binding's directory. + let mut retriever = DtRetriever::bundled(); + if let Some(dir) = self.filename.parent() { + retriever = retriever.with_root(dir.to_path_buf()); + } + let filename = self.filename.to_string_lossy().into_owned(); + check_refs_rec(&retriever, &filename, &self.value, None, None, None); + } +} + +/// Return whether a subschema describes a node: `type: object` or any of the +/// object-property keywords. +fn is_node_schema(schema: &Value) -> bool { + let Some(obj) = schema.as_object() else { + return false; + }; + if obj.get("type").and_then(Value::as_str) == Some("object") { + return true; + } + [ + "properties", + "patternProperties", + "additionalProperties", + "unevaluatedProperties", + ] + .iter() + .any(|k| obj.contains_key(*k)) +} + +/// Return whether the subschema forbids undefined properties, or is not a node +/// schema at all. +fn schema_allows_no_undefined_props(schema: &Value) -> bool { + if !is_node_schema(schema) { + return true; + } + let obj = schema.as_object().unwrap(); + let additional = obj.get("additionalProperties"); + let uneval = obj.get("unevaluatedProperties"); + let closed = |v: Option<&Value>| match v { + None => false, // default True ⇒ not closed + Some(Value::Bool(b)) => !*b, + Some(_) => true, // a dict constraint ⇒ closed + }; + closed(additional) || closed(uneval) +} + +/// Recurse the schema tree, warning about node subschemas that neither carry +/// nor inherit an `additionalProperties`/ +/// `unevaluatedProperties` constraint. +fn check_refs_rec( + retriever: &DtRetriever, + filename: &str, + schema: &Value, + parent: Option<&str>, + is_common: Option, + has_constraint: Option, +) { + // At the root, `is_common` is derived from the top-level schema. + let is_common = is_common.unwrap_or_else(|| !schema_allows_no_undefined_props(schema)); + let mut has_constraint = has_constraint.unwrap_or(false); + + match schema { + Value::Object(obj) => { + if matches!( + parent, + Some( + "if" | "select" + | "definitions" + | "$defs" + | "then" + | "else" + | "dependencies" + | "dependentSchemas" + ) + ) { + return; + } + + if is_node_schema(schema) && !matches!(parent, Some("oneOf" | "allOf" | "anyOf")) { + has_constraint = schema_allows_no_undefined_props(schema); + } + + let mut ref_has_constraint = true; + if let Some(Value::String(r)) = obj.get("$ref") { + let uri = r.trim_end_matches('#'); + if let Ok(ref_sch) = retriever.retrieve(&uri_of(uri)) { + ref_has_constraint = schema_allows_no_undefined_props(&ref_sch); + } + } + + let has_uneval = obj.contains_key("additionalProperties") + || obj.contains_key("unevaluatedProperties"); + if !(is_common || ref_has_constraint || has_constraint || has_uneval) { + eprintln!( + "{filename}: {}: Missing additionalProperties/unevaluatedProperties constraint", + parent.unwrap_or("") + ); + } + + for (k, v) in obj { + check_refs_rec( + retriever, + filename, + v, + Some(k.as_str()), + Some(is_common), + Some(has_constraint), + ); + } + } + Value::Array(items) => { + for v in items { + check_refs_rec( + retriever, + filename, + v, + parent, + Some(is_common), + Some(has_constraint), + ); + } + } + _ => {} + } +} + +/// Parse a bare URI string into the `jsonschema::Uri` the retriever expects. +fn uri_of(s: &str) -> Uri { + Uri::parse(s.to_string()).unwrap_or_else(|_| Uri::parse(format!("{BASE_URL}schemas/")).unwrap()) +} diff --git a/rust/dtschema/src/types.rs b/rust/dtschema/src/types.rs new file mode 100644 index 0000000..67f3693 --- /dev/null +++ b/rust/dtschema/src/types.rs @@ -0,0 +1,522 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Property-type extraction and generated compatible schema construction. +//! Produces the `generated-types`, `generated-pattern-types`, and +//! `generated-compatibles` cache entries that a processed schema +//! (`dt-mk-schema` output) carries. +//! +//! Each property-type entry is a `serde_json::Value` object: +//! `{"type": , "$id": [ids], "regex"?: , "dim"?: +//! [[a,b],[c,d]]}`. `regex` is a working-only pattern string and is stripped +//! before serialization. + +use std::collections::BTreeMap; +use std::sync::LazyLock; + +use fancy_regex::Regex as FancyRegex; +use regex::Regex; +use serde_json::{Map, Value}; + +use crate::lib_helpers::{extract_compatibles, get_array_range, is_string_schema}; + +/// Extract a known DT property type name from `$ref` text. +static TYPE_RE: LazyLock = LazyLock::new(|| { + Regex::new( + r"(address|flag|u?int(8|16|32|64)(-(array|matrix))?|string(-array)?|phandle(-array)?)", + ) + .unwrap() +}); +static MICROVOLT_RE: LazyLock = LazyLock::new(|| Regex::new(r"-microvolt$").unwrap()); +// '(^(?!opp)).*-hz$' — needs lookahead. +static HZ_RE: LazyLock = + LazyLock::new(|| FancyRegex::new(r"(^(?!opp)).*-hz$").unwrap()); +static REF_YAML_RE: LazyLock = LazyLock::new(|| Regex::new(r"\.yaml#?$").unwrap()); + +/// Map of property name → list of type-entry objects. +pub type PropMap = BTreeMap>; + +/// Merge two matrix dimensions. `dim` is `[[min,max],[min,max]]`. +fn merge_dim(dim1: &Value, dim2: &Value) -> Value { + let a = dim1.as_array().unwrap(); + let b = dim2.as_array().unwrap(); + let mut d = Vec::with_capacity(2); + for i in 0..2 { + let a0 = a[i][0].as_i64().unwrap(); + let a1 = a[i][1].as_i64().unwrap(); + let b0 = b[i][0].as_i64().unwrap(); + let b1 = b[i][1].as_i64().unwrap(); + let mut minimum = a0.min(b0); + let mut maximum = a1.max(b1); + if a1.min(b1) == 0 { + maximum = 0; + } + if maximum == 1 { + minimum = 1; + } + d.push(serde_json::json!([minimum, maximum])); + } + Value::Array(d) +} + +/// The `$id` list of a working prop entry contains `schema_id`. +fn id_list_contains(entry: &Value, id: &str) -> bool { + entry + .get("$id") + .and_then(Value::as_array) + .map(|a| a.iter().any(|v| v.as_str() == Some(id))) + .unwrap_or(false) +} + +fn push_id(entry: &mut Value, id: &str) { + if !id_list_contains(entry, id) { + entry["$id"] + .as_array_mut() + .unwrap() + .push(Value::String(id.to_string())); + } +} + +fn schema_id(schema: &Value) -> &str { + schema.get("$id").and_then(Value::as_str).unwrap_or("") +} + +/// Extract one property's type entry from a subschema. +fn extract_prop_type( + props: &mut PropMap, + schema: &Value, + propname: &str, + subschema: &Value, + is_pattern: bool, +) { + if propname.starts_with('$') { + return; + } + + let sid = schema_id(schema).to_string(); + + let Some(sub) = subschema.as_object() else { + // Non-object subschemas seed a default entry only for `true`. + if subschema == &Value::Bool(true) { + let mut default_type = Map::new(); + default_type.insert("type".to_string(), Value::Null); + default_type.insert("$id".to_string(), serde_json::json!([sid])); + if is_pattern { + default_type.insert("regex".to_string(), Value::String(propname.to_string())); + } + props + .entry(propname.to_string()) + .or_insert_with(|| vec![Value::Object(default_type)]); + } + return; + }; + + let mut prop_type: Option = None; + + // We only support local refs. + if let Some(Value::String(rf)) = sub.get("$ref") { + if rf.starts_with("#/") { + if let Some(existing) = props.get(propname) { + for p in existing { + if id_list_contains(p, &sid) { + return; + } + } + } + // Walk the local ref path. + let mut tmp = schema; + let mut ok = true; + for p in rf.split('/').skip(1) { + match tmp.get(p) { + Some(v) => tmp = v, + None => { + ok = false; + break; + } + } + } + if ok { + let tmp = tmp.clone(); + extract_prop_type(props, schema, propname, &tmp, is_pattern); + } + } else if rf.contains("/properties/") { + let last = rf.rsplit('/').next().unwrap_or(""); + if last != propname { + prop_type = Some(last.to_string()); + } + } + } + + // allOf/oneOf/anyOf recursion. + for k in ["allOf", "oneOf", "anyOf"] { + if let Some(Value::Array(arr)) = sub.get(k) { + let arr = arr.clone(); + for v in &arr { + extract_prop_type(props, schema, propname, v, is_pattern); + } + } + } + + props.entry(propname.to_string()).or_default(); + + let is_node = sub.get("type") == Some(&Value::String("object".to_string())) + || sub.contains_key("properties") + || sub.contains_key("patternProperties") + || sub.contains_key("additionalProperties"); + + if is_node { + prop_type = Some("node".to_string()); + } else { + // Infer the type name from a referenced core type schema. + let ref_type = sub + .get("$ref") + .and_then(Value::as_str) + .and_then(|r| TYPE_RE.find(r).map(|m| m.as_str().to_string())); + if let Some(t) = ref_type { + prop_type = Some(t); + } else if sub.get("type") == Some(&Value::String("boolean".to_string())) { + prop_type = Some("flag".to_string()); + } else if let Some(items) = sub.get("items") { + let items_is_string = match items { + Value::Array(a) => a.first().map(is_string_schema).unwrap_or(false), + other => is_string_schema(other), + }; + if items_is_string { + prop_type = Some("string-array".to_string()); + } else if MICROVOLT_RE.is_match(propname) { + // List-shaped matrix wrappers still infer matrix types from + // unit suffixes. + prop_type = Some("int32-matrix".to_string()); + } else if HZ_RE.is_match(propname).unwrap_or(false) { + prop_type = Some("uint32-matrix".to_string()); + } else { + prop_type = None; + } + } else if sub + .get("$ref") + .and_then(Value::as_str) + .map(|r| REF_YAML_RE.is_match(r)) + .unwrap_or(false) + { + prop_type = Some("node".to_string()); + } + } + + // Build new_prop. + let mut new_prop = Map::new(); + new_prop.insert( + "type".to_string(), + match &prop_type { + Some(t) => Value::String(t.clone()), + None => Value::Null, + }, + ); + new_prop.insert("$id".to_string(), serde_json::json!([sid])); + if is_pattern { + new_prop.insert("regex".to_string(), Value::String(propname.to_string())); + } + let mut new_prop: Option = Some(Value::Object(new_prop)); + + let Some(prop_type) = prop_type else { + // No type: seed the list only if empty. + let list = props.get_mut(propname).unwrap(); + if list.is_empty() { + list.push(new_prop.take().unwrap()); + } + return; + }; + + // Matrix dimensions. + let has_size = + sub.contains_key("items") || sub.contains_key("minItems") || sub.contains_key("maxItems"); + let dim = if (prop_type == "phandle-array" || prop_type.ends_with("-matrix")) && has_size { + let outer = get_array_range(subschema); + let inner = if let Some(items) = sub.get("items") { + match items { + Value::Array(a) => get_array_range(a.first().unwrap_or(&Value::Null)), + other => get_array_range(other), + } + } else { + serde_json::json!([0, 0]) + }; + let d = serde_json::json!([outer, inner]); + new_prop.as_mut().unwrap()["dim"] = d.clone(); + Some(d) + } else { + None + }; + + // Merge into existing entries. + let list = props.get_mut(propname).unwrap(); + let mut dup_idx: Option = None; + for (i, p) in list.iter_mut().enumerate() { + let ptype = p.get("type").cloned().unwrap_or(Value::Null); + if ptype.is_null() { + dup_idx = Some(i); + break; + } + let ptype_s = ptype.as_str().unwrap_or(""); + if let Some(dim) = &dim + && (ptype_s == "phandle-array" || ptype_s.ends_with("-matrix")) + { + if p.get("dim").is_some() { + let merged = merge_dim(&p["dim"], dim); + p["dim"] = merged; + } else { + p["dim"] = dim.clone(); + } + push_id(p, &sid); + new_prop = None; + break; + } + if ptype_s == prop_type { + push_id(p, &sid); + new_prop = None; + break; + } + if prop_type.contains("string") && ptype_s.contains("string") { + if prop_type == "string-array" { + p["type"] = Value::String(prop_type.clone()); + } + push_id(p, &sid); + new_prop = None; + break; + } + } + + if let Some(i) = dup_idx { + list.remove(i); + } + if let Some(np) = new_prop { + list.push(np); + } + + // Recurse into nested node props. + if sub.contains_key("properties") + || sub.contains_key("patternProperties") + || sub.contains_key("additionalProperties") + { + extract_subschema_types(props, schema, subschema); + } +} + +/// Recurse through a subschema and collect property type entries. +fn extract_subschema_types(props: &mut PropMap, schema: &Value, subschema: &Value) { + let Some(sub) = subschema.as_object() else { + return; + }; + + if let Some(ap) = sub.get("additionalProperties") { + let ap = ap.clone(); + extract_subschema_types(props, schema, &ap); + } + + for k in ["allOf", "oneOf", "anyOf"] { + if let Some(Value::Array(arr)) = sub.get(k) { + let arr = arr.clone(); + for v in &arr { + extract_subschema_types(props, schema, v); + } + } + } + + for k in ["properties", "patternProperties"] { + if let Some(Value::Object(m)) = sub.get(k) { + let entries: Vec<(String, Value)> = + m.iter().map(|(p, v)| (p.clone(), v.clone())).collect(); + for (p, v) in entries { + extract_prop_type(props, schema, &p, &v, k == "patternProperties"); + } + } + } +} + +/// Extract property type entries from every schema. +fn extract_types(schemas: &BTreeMap) -> PropMap { + let mut props: PropMap = BTreeMap::new(); + for sch in schemas.values() { + extract_subschema_types(&mut props, sch, sch); + } + + // Second pass: resolve propname-reference types (a type that isn't a known + // type name refers to another property). + let snapshot: BTreeMap> = props + .iter() + .map(|(k, v)| { + ( + k.clone(), + v.first() + .and_then(|e| e.get("type")) + .and_then(Value::as_str) + .map(String::from), + ) + }) + .collect(); + for prop in props.values_mut() { + for v in prop.iter_mut() { + let t = v.get("type").and_then(Value::as_str); + let Some(prop_type) = t else { continue }; + if prop_type == "node" { + continue; + } + if !TYPE_RE.is_match(prop_type) { + if let Some(Some(resolved)) = snapshot.get(prop_type) { + v["type"] = Value::String(resolved.clone()); + } + break; + } + } + } + + props +} + +/// Return exact-property and pattern-property type maps. +pub fn get_prop_types(schemas: &BTreeMap) -> (PropMap, PropMap) { + let mut props = extract_types(schemas); + let mut pat_props: PropMap = BTreeMap::new(); + + // Remove aliases/generic pattern. + props.remove(r"^[a-z][a-z0-9\-]*$"); + + // Remove all node types from each list. + for val in props.values_mut() { + val.retain(|t| t.get("type").and_then(Value::as_str) != Some("node")); + } + // Drop now-empty props. + props.retain(|_, v| !v.is_empty()); + + // Split out pattern properties (those whose first entry carries a regex). + let pat_keys: Vec = props + .iter() + .filter(|(_, v)| v.first().map(|e| e.get("regex").is_some()).unwrap_or(false)) + .map(|(k, _)| k.clone()) + .collect(); + for key in pat_keys { + let val = props.remove(&key).unwrap(); + // Only keep patternProperties with a non-null type. + if val[0].get("type").map(|t| !t.is_null()).unwrap_or(false) { + pat_props.insert(key, val); + } + } + + // Delete untyped entries matching a patternProperty. + let untyped_keys: Vec = props + .iter() + .filter(|(_, v)| v.first().map(|e| e["type"].is_null()).unwrap_or(false)) + .map(|(k, _)| k.clone()) + .collect(); + for key in untyped_keys { + for val in pat_props.values() { + let pat = val[0].get("regex").and_then(Value::as_str); + let typed = val[0].get("type").map(|t| !t.is_null()).unwrap_or(false); + if typed + && let Some(pat) = pat + && FancyRegex::new(pat) + .ok() + .and_then(|re| re.is_match(&key).ok()) + .unwrap_or(false) + { + props.remove(&key); + break; + } + } + } + + (props, pat_props) +} + +/// Strip working-only keys (`$id`, `regex`) from prop entries for +/// serialization. +fn strip_working_keys(props: &PropMap, keep_none: bool) -> Map { + let mut out = Map::new(); + for (k, list) in props { + let mut new_list = Vec::with_capacity(list.len()); + for entry in list { + let mut m = entry.as_object().unwrap().clone(); + m.remove("$id"); + m.remove("regex"); + new_list.push(Value::Object(m)); + } + let _ = keep_none; + out.insert(k.clone(), Value::Array(new_list)); + } + out +} + +/// Build the `generated-types` and `generated-pattern-types` schema entries and +/// insert them into `schemas`. +pub fn make_property_type_cache(schemas: &mut BTreeMap) { + let (props, pat_props) = get_prop_types(schemas); + + let types_props = strip_working_keys(&props, true); + schemas.insert( + "generated-types".to_string(), + serde_json::json!({ + "$id": "generated-types", + "$filename": "Generated property types", + "select": false, + "properties": types_props, + }), + ); + + let pat_types_props = strip_working_keys(&pat_props, true); + schemas.insert( + "generated-pattern-types".to_string(), + serde_json::json!({ + "$id": "generated-pattern-types", + "$filename": "Generated pattern property types", + "select": false, + "properties": pat_types_props, + }), + ); +} + +static COMPAT_SPECIAL_RE: LazyLock = + LazyLock::new(|| Regex::new(r".*[\^\[{\(\$].*").unwrap()); +static COMPAT_WILDCARD_RE: LazyLock = LazyLock::new(|| Regex::new(r"\.[+*]").unwrap()); + +/// Build the `generated-compatibles` entry. +pub fn make_compatible_schema(schemas: &mut BTreeMap) { + let mut enum_vals: Vec = Vec::new(); + let mut patterns: Vec = Vec::new(); + + let mut compatible_list: std::collections::BTreeSet = std::collections::BTreeSet::new(); + for sch in schemas.values() { + compatible_list.extend(extract_compatibles(sch)); + } + + for c in &compatible_list { + if COMPAT_SPECIAL_RE.is_match(c) { + if c != r"^[a-zA-Z0-9][a-zA-Z0-9,+\-._/]+$" + && !COMPAT_WILDCARD_RE.is_match(c) + && c.starts_with('^') + && c.ends_with('$') + { + patterns.push(c.clone()); + } + } else { + enum_vals.push(c.clone()); + } + } + + enum_vals.sort(); + // anyOf: enum first, then the fixed '^foo'/'^test,' patterns, then discovered. + let mut any_of: Vec = vec![serde_json::json!({ "enum": enum_vals })]; + any_of.push(serde_json::json!({"pattern": "^foo"})); + any_of.push(serde_json::json!({"pattern": "^test,"})); + for p in patterns { + any_of.push(serde_json::json!({ "pattern": p })); + } + + schemas.insert( + crate::GENERATED_COMPATIBLES_SCHEMA.to_string(), + serde_json::json!({ + "$id": crate::GENERATED_COMPATIBLES_SCHEMA, + "$filename": "Generated schema of documented compatible strings", + "select": true, + "properties": { + "compatible": { "items": { "anyOf": any_of } } + } + }), + ); +} diff --git a/rust/dtschema/src/validator.rs b/rust/dtschema/src/validator.rs new file mode 100644 index 0000000..8484524 --- /dev/null +++ b/rust/dtschema/src/validator.rs @@ -0,0 +1,1497 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Devicetree data validator built on the processed schema set from +//! [`crate::process::ProcessedSchemas`]. +//! +//! The instances being validated are decoded devicetree nodes ([`DtValue`]), +//! not `serde_json::Value`: a decoded integer carries its bit-width +//! (`sized_int`), which the custom `typeSize` keyword reads. To make the +//! `jsonschema` engine validate `DtValue` directly — so `typeSize` sees the +//! real width instead of a lossy JSON re-encoding — we implement a custom +//! in-memory representation ([`DtJson`]) over `&DtValue` and build validators +//! with `jsonschema::options_for::()`. + +use std::borrow::Cow; +use std::collections::BTreeMap; +use std::path::PathBuf; +use std::sync::{Arc, OnceLock}; + +use jsonschema::error::ValidationErrorKind; +use jsonschema::json::{Array, Json, JsonNumber, Node, NodeIdentity, Object}; +use jsonschema::{Draft, JsonType, Keyword, Retrieve, Uri, ValidationError, Validator}; +use regex::{Regex, RegexSet}; +use serde_json::{Map, Value}; + +use crate::dtb::{self, DtValue, TypeContext}; +use crate::process::ProcessedSchemas; + +// --------------------------------------------------------------------------- +// Custom `jsonschema` representation over `&DtValue`. +// --------------------------------------------------------------------------- + +/// The `Json` marker type: instances are borrowed [`DtValue`] trees. +pub struct DtJson; + +impl Json for DtJson { + type Node<'a> = &'a DtValue; + type PreparedKey = String; + type StringBuffer = DtValue; + + fn prepare_key(key: &str) -> String { + key.to_owned() + } + + fn with_string_node(buffer: &mut DtValue, string: &str, f: impl FnOnce(&DtValue) -> T) -> T { + *buffer = DtValue::Str(string.to_owned()); + f(buffer) + } +} + +impl Default for DtValue { + fn default() -> Self { + DtValue::Bool(false) + } +} + +/// A decoded integer presented as a JSON number. +pub struct DtNumber(i128); + +impl JsonNumber for DtNumber { + fn as_u64(&self) -> Option { + u64::try_from(self.0).ok() + } + fn as_i64(&self) -> Option { + i64::try_from(self.0).ok() + } + fn as_f64(&self) -> Option { + Some(self.0 as f64) + } + fn as_str(&self) -> Cow<'_, str> { + Cow::Owned(self.0.to_string()) + } + fn to_number(&self) -> Cow<'_, serde_json::Number> { + // DT integers fit in i128; i64/u64 cover the real range (max uint64). + let n = if self.0 < 0 { + serde_json::Number::from(self.0 as i64) + } else { + serde_json::Number::from(self.0 as u64) + }; + Cow::Owned(n) + } + fn is_integer(&self) -> bool { + true + } +} + +impl<'a> Node<'a, DtJson> for &'a DtValue { + type Object = &'a BTreeMap; + type Array = &'a [DtValue]; + type Number = DtNumber; + + fn as_object(&self) -> Option<&'a BTreeMap> { + match self { + DtValue::Node(m) => Some(m), + _ => None, + } + } + fn as_array(&self) -> Option<&'a [DtValue]> { + match self { + DtValue::List(l) => Some(l.as_slice()), + _ => None, + } + } + fn as_string(&self) -> Option> { + match self { + DtValue::Str(s) => Some(Cow::Borrowed(s.as_str())), + _ => None, + } + } + fn as_number(&self) -> Option { + match self { + DtValue::Int { val, .. } => Some(DtNumber(*val)), + _ => None, + } + } + fn as_boolean(&self) -> Option { + match self { + DtValue::Bool(b) => Some(*b), + _ => None, + } + } + fn is_null(&self) -> bool { + // Raw bytes are not JSON null. + false + } + fn json_type(&self) -> JsonType { + match self { + DtValue::Bool(_) => JsonType::Boolean, + DtValue::Int { .. } => JsonType::Number, + DtValue::Str(_) => JsonType::String, + DtValue::List(_) => JsonType::Array, + DtValue::Node(_) => JsonType::Object, + // Raw bytes have no JSON type. The `jsonschema` multi-type fast + // path has no "none" variant, but `JsonType::Number` with + // `as_number() == None` fails every numeric and non-numeric type + // check, so undecoded bytes fail every JSON type. + DtValue::Bytes(_) => JsonType::Number, + } + } + fn to_value(&self) -> Cow<'a, Value> { + Cow::Owned(self.to_json()) + } + fn identity(&self) -> Option { + Some(NodeIdentity::new( + std::ptr::from_ref::(*self) as usize + )) + } +} + +/// Iterator over object members exposing `&str` names. +pub struct DtMembers<'a>(std::collections::btree_map::Iter<'a, String, DtValue>); + +impl<'a> Iterator for DtMembers<'a> { + type Item = (&'a str, &'a DtValue); + fn next(&mut self) -> Option { + self.0.next().map(|(k, v)| (k.as_str(), v)) + } +} + +impl<'a> Object<'a, DtJson> for &'a BTreeMap { + type Node = &'a DtValue; + type MemberName = &'a str; + type MembersIter = DtMembers<'a>; + + fn len(&self) -> usize { + BTreeMap::len(self) + } + fn get(&self, key: &String) -> Option<&'a DtValue> { + BTreeMap::get(*self, key) + } + fn members(&self) -> DtMembers<'a> { + DtMembers(self.iter()) + } +} + +impl<'a> Array<'a, DtJson> for &'a [DtValue] { + type Node = &'a DtValue; + type ElementsIter = std::slice::Iter<'a, DtValue>; + + fn len(&self) -> usize { + <[DtValue]>::len(self) + } + fn elements(&self) -> std::slice::Iter<'a, DtValue> { + self.iter() + } +} + +// --------------------------------------------------------------------------- +// `typeSize` custom keyword. +// --------------------------------------------------------------------------- + +/// Require the decoded integer's bit-width to equal the schema value. Values +/// without an explicit width use the legacy 32-bit default. +struct TypeSizeKeyword { + expected: u64, +} + +impl<'i> Keyword<'i, DtJson> for TypeSizeKeyword { + fn validate(&self, instance: &'i DtValue) -> Result<(), ValidationError<'i>> { + if self.is_valid(instance) { + Ok(()) + } else { + let size = int_size(instance); + Err(ValidationError::custom(format!( + "size is {size}, expected {}", + self.expected + ))) + } + } + + fn is_valid(&self, instance: &'i DtValue) -> bool { + int_size(instance) == self.expected + } +} + +/// The effective bit-width of a decoded value for `typeSize` purposes. +fn int_size(instance: &DtValue) -> u64 { + match instance { + DtValue::Int { size, .. } => *size as u64, + _ => 32, + } +} + +/// A cheaply-cloneable retriever handle wrapping the shared processed-schema +/// map, so each per-schema validator build gets its own `Retrieve` without +/// deep-copying the map (`jsonschema`'s `with_retriever` takes ownership). +#[derive(Clone)] +struct SharedRetriever(Arc); + +impl Retrieve for SharedRetriever { + fn retrieve( + &self, + uri: &Uri, + ) -> Result> { + self.0.retrieve(uri) + } +} + +/// Build the validation options carrying the `typeSize` keyword and the +/// devicetree retriever, for the custom [`DtJson`] representation. +fn dt_options( + retriever: SharedRetriever, +) -> jsonschema::ValidationOptions<'static, Arc, DtJson> { + jsonschema::options_for::() + .with_draft(Draft::Draft201909) + .with_retriever(retriever) + .with_keyword( + "typeSize", + |parent: &Map, _schema: &Value, _path| { + let expected = parent.get("typeSize").and_then(Value::as_u64).unwrap_or(32); + Ok(Box::new(TypeSizeKeyword { expected }) as Box Keyword<'i, DtJson>>) + }, + ) +} + +// --------------------------------------------------------------------------- +// DTValidator. +// --------------------------------------------------------------------------- + +/// A single reported validation error, annotated with its originating schema +/// `$id`. +pub struct DtError { + /// Instance path in the decoded DT node. + pub instance_path: Vec, + /// Schema path inside the matching schema. + pub schema_path: Vec, + /// Human-readable validation message. + pub message: String, + /// The originating schema `$id`. + pub schema_file: String, + /// The instance at the error location, if it is a node (used for the + /// disabled-`status` suppression heuristic). + pub instance_is_disabled_node: bool, + /// True when this error, or an `anyOf`/`oneOf` child context, is a + /// missing-property-style error suppressed for disabled nodes. + pub has_suppressible_disabled_context: bool, +} + +/// A JSON-Pointer path segment: either a property name or an array index. +#[derive(Clone)] +pub enum PathSeg { + Key(String), + Index(usize), +} + +impl PathSeg { + /// Render as a JSON diagnostic path: strings stay strings, indices become + /// integers. + pub fn as_json(&self) -> Value { + match self { + PathSeg::Key(k) => Value::String(k.clone()), + PathSeg::Index(i) => Value::Number((*i).into()), + } + } + pub fn to_display(&self) -> String { + match self { + PathSeg::Key(k) => k.clone(), + PathSeg::Index(i) => i.to_string(), + } + } +} + +/// A lazily-built, shareable compiled validator slot. `None` inside the +/// `OnceLock` records a build failure (so it isn't retried or re-warned). +type ValidatorSlot = Arc>>>>; + +/// The devicetree data validator. +pub struct DTValidator { + schemas: Arc>, + compat_map: BTreeMap, + always_schemas: Vec, + type_ctx: TypeContext, + retriever: Arc, + always_dispatch: AlwaysDispatch, + vendor_prefixes: Option, + /// Compiled raw per-schema validators, built once and reused across every + /// node and DTB. The slots are allocated at construction time, so the hot + /// validation path only needs an immutable map lookup before entering the + /// `OnceLock`. + raw_validators: BTreeMap, + /// Compiled `{if: select, then: schema}` always-schema validators, aligned + /// with `always_schemas`. + always_validators: Vec, +} + +/// Retriever that resolves `$ref` URIs against the processed schema map. +struct DtSchemaRetriever { + schemas: Arc>, +} + +const VENDOR_PREFIXES_SCHEMA: &str = "http://devicetree.org/schemas/vendor-prefixes.yaml"; + +struct VendorPrefixesFastPath { + properties: std::collections::BTreeSet, + patterns: RegexSet, +} + +impl VendorPrefixesFastPath { + fn build(schema: &Value) -> Option { + if schema.get("additionalProperties") != Some(&Value::Bool(false)) { + return None; + } + let properties = schema + .get("properties") + .and_then(Value::as_object)? + .keys() + .cloned() + .collect(); + let patterns: Vec<&str> = schema + .get("patternProperties") + .and_then(Value::as_object)? + .keys() + .map(String::as_str) + .collect(); + let patterns = RegexSet::new(patterns).ok()?; + Some(Self { + properties, + patterns, + }) + } + + fn is_valid(&self, node: &DtValue) -> bool { + let DtValue::Node(map) = node else { + return false; + }; + map.keys() + .all(|key| self.properties.contains(key) || self.patterns.is_match(key)) + } +} + +#[derive(Default)] +struct AlwaysDispatch { + fallback: Vec, + compat_any_exact: BTreeMap>, + compat_first_exact: BTreeMap>, + compat_first_patterns: PatternCandidates, + compat_any_patterns: PatternCandidates, + nodename_exact: BTreeMap>, + nodename_patterns: PatternCandidates, + property_exact: BTreeMap>, + property_patterns: PatternCandidates, +} + +#[derive(Clone)] +struct SelectCandidate { + schema_index: usize, + required: Vec, +} + +enum SelectKey { + Never, + Fallback, + CompatibleAnyExact(Vec), + CompatibleFirstExact(Vec), + CompatibleAnyPattern(String), + CompatibleFirstPattern(String), + NodenameExact(String), + NodenamePattern(String), + PropertyInterest { + exact: Vec, + patterns: Vec, + }, +} + +#[derive(Default)] +struct PatternCandidates { + patterns: Vec, + candidates: Vec, + set: Option, +} + +impl PatternCandidates { + fn push(&mut self, pattern: String, candidate: SelectCandidate) { + self.patterns.push(pattern); + self.candidates.push(candidate); + } + + fn compile(&mut self) { + self.set = if self.patterns.is_empty() { + None + } else { + RegexSet::new(&self.patterns).ok() + }; + } + + fn insert_matches( + &self, + text: &str, + node: &BTreeMap, + selected: &mut Vec, + ) { + let Some(set) = &self.set else { + return; + }; + for idx in set.matches(text) { + let candidate = &self.candidates[idx]; + if candidate.required_present(node) { + selected.push(candidate.schema_index); + } + } + } +} + +impl AlwaysDispatch { + fn build(schemas: &BTreeMap, always_schemas: &[String]) -> Self { + let mut dispatch = Self::default(); + for (schema_index, schema_id) in always_schemas.iter().enumerate() { + let Some(schema) = schemas.get(schema_id) else { + continue; + }; + let candidate = SelectCandidate { + schema_index, + required: select_required(schema), + }; + match select_key(schema) { + SelectKey::Never => {} + SelectKey::Fallback => dispatch.fallback.push(candidate), + SelectKey::CompatibleAnyExact(values) => { + for value in values { + dispatch + .compat_any_exact + .entry(value) + .or_default() + .push(candidate.clone()); + } + } + SelectKey::CompatibleFirstExact(values) => { + for value in values { + dispatch + .compat_first_exact + .entry(value) + .or_default() + .push(candidate.clone()); + } + } + SelectKey::CompatibleAnyPattern(pattern) => { + dispatch.compat_any_patterns.push(pattern, candidate); + } + SelectKey::CompatibleFirstPattern(pattern) => { + dispatch.compat_first_patterns.push(pattern, candidate); + } + SelectKey::NodenameExact(name) => { + dispatch + .nodename_exact + .entry(name) + .or_default() + .push(candidate); + } + SelectKey::NodenamePattern(pattern) => { + dispatch.nodename_patterns.push(pattern, candidate); + } + SelectKey::PropertyInterest { exact, patterns } => { + for prop in exact { + dispatch + .property_exact + .entry(prop) + .or_default() + .push(candidate.clone()); + } + let mut has_unsupported_pattern = false; + for pattern in patterns { + if Regex::new(&pattern).is_ok() { + dispatch.property_patterns.push(pattern, candidate.clone()); + } else { + has_unsupported_pattern = true; + } + } + if has_unsupported_pattern { + dispatch.fallback.push(candidate); + } + } + } + } + dispatch.compat_any_patterns.compile(); + dispatch.compat_first_patterns.compile(); + dispatch.nodename_patterns.compile(); + dispatch.property_patterns.compile(); + dispatch + } + + fn candidates(&self, node: &BTreeMap) -> Vec { + let mut selected = Vec::with_capacity(self.fallback.len() + 8); + + for candidate in &self.fallback { + if candidate.required_present(node) { + selected.push(candidate.schema_index); + } + } + + let compats = compatible_strings(node); + for compat in &compats { + if let Some(candidates) = self.compat_any_exact.get(*compat) { + self.insert_matching_required(node, candidates, &mut selected); + } + self.compat_any_patterns + .insert_matches(compat, node, &mut selected); + } + if let Some(first) = compats.first() { + if let Some(candidates) = self.compat_first_exact.get(*first) { + self.insert_matching_required(node, candidates, &mut selected); + } + self.compat_first_patterns + .insert_matches(first, node, &mut selected); + } + + if let Some(nodename) = node_nodename(node) { + if let Some(candidates) = self.nodename_exact.get(nodename) { + self.insert_matching_required(node, candidates, &mut selected); + } + self.nodename_patterns + .insert_matches(nodename, node, &mut selected); + } + + for prop in node.keys() { + if let Some(candidates) = self.property_exact.get(prop) { + self.insert_matching_required(node, candidates, &mut selected); + } + self.property_patterns + .insert_matches(prop, node, &mut selected); + } + + selected.sort_unstable(); + selected.dedup(); + selected + } + + fn insert_matching_required<'a>( + &'a self, + node: &BTreeMap, + candidates: &'a [SelectCandidate], + selected: &mut Vec, + ) { + for candidate in candidates { + if candidate.required_present(node) { + selected.push(candidate.schema_index); + } + } + } +} + +impl SelectCandidate { + fn required_present(&self, node: &BTreeMap) -> bool { + self.required.iter().all(|key| node.contains_key(key)) + } +} + +fn select_required(schema: &Value) -> Vec { + schema + .get("select") + .and_then(|select| select.get("required")) + .and_then(Value::as_array) + .map(|items| { + items + .iter() + .filter_map(Value::as_str) + .map(str::to_string) + .collect() + }) + .unwrap_or_default() +} + +fn select_key(schema: &Value) -> SelectKey { + let Some(select) = schema.get("select") else { + return SelectKey::Fallback; + }; + if select == &Value::Bool(false) { + return SelectKey::Never; + } + if select == &Value::Bool(true) { + return property_interest_key(schema).unwrap_or(SelectKey::Fallback); + } + let Some(select_obj) = select.as_object() else { + return SelectKey::Fallback; + }; + + if let Some(compatible) = select_obj + .get("properties") + .and_then(|props| props.get("compatible")) + && select_requires(select, "compatible") + && let Some(key) = compatible_select_key(compatible) + { + return key; + } + + if let Some(nodename) = select_obj + .get("properties") + .and_then(|props| props.get("$nodename")) + && let Some(key) = nodename_select_key(nodename) + { + return key; + } + + SelectKey::Fallback +} + +fn property_interest_key(schema: &Value) -> Option { + let obj = schema.as_object()?; + for key in obj.keys() { + if key.starts_with('$') + || matches!( + key.as_str(), + "select" + | "title" + | "description" + | "maintainers" + | "examples" + | "type" + | "properties" + | "patternProperties" + | "dependentRequired" + | "dependentSchemas" + ) + { + continue; + } + return None; + } + + let mut exact = std::collections::BTreeSet::new(); + if let Some(properties) = schema.get("properties").and_then(Value::as_object) { + exact.extend(properties.keys().cloned()); + } + if let Some(deps) = schema.get("dependentRequired").and_then(Value::as_object) { + exact.extend(deps.keys().cloned()); + } + if let Some(deps) = schema.get("dependentSchemas").and_then(Value::as_object) { + exact.extend(deps.keys().cloned()); + } + + let mut patterns = Vec::new(); + if let Some(pattern_props) = schema.get("patternProperties").and_then(Value::as_object) { + for pattern in pattern_props.keys() { + patterns.push(pattern.clone()); + } + } + + if exact.is_empty() && patterns.is_empty() { + return None; + } + Some(SelectKey::PropertyInterest { + exact: exact.into_iter().collect(), + patterns, + }) +} + +fn select_requires(select: &Value, key: &str) -> bool { + select + .get("required") + .and_then(Value::as_array) + .is_some_and(|items| items.iter().any(|item| item.as_str() == Some(key))) +} + +fn compatible_select_key(schema: &Value) -> Option { + let (target, first_only) = if let Some(target) = schema.get("contains") { + (target, false) + } else { + let items = schema.get("items")?; + let target = match items { + Value::Array(items) => items.first()?, + Value::Object(_) => items, + _ => return None, + }; + (target, true) + }; + + if let Some(value) = target.get("const").and_then(Value::as_str) { + let values = vec![value.to_string()]; + return Some(if first_only { + SelectKey::CompatibleFirstExact(values) + } else { + SelectKey::CompatibleAnyExact(values) + }); + } + if let Some(values) = target.get("enum").and_then(Value::as_array) { + let values: Vec = values + .iter() + .filter_map(Value::as_str) + .map(str::to_string) + .collect(); + if !values.is_empty() { + return Some(if first_only { + SelectKey::CompatibleFirstExact(values) + } else { + SelectKey::CompatibleAnyExact(values) + }); + } + } + if let Some(pattern) = target.get("pattern").and_then(Value::as_str) { + return Some(if first_only { + SelectKey::CompatibleFirstPattern(pattern.to_string()) + } else { + SelectKey::CompatibleAnyPattern(pattern.to_string()) + }); + } + None +} + +fn nodename_select_key(schema: &Value) -> Option { + let items = schema.get("items")?; + let target = match items { + Value::Array(items) => items.first()?, + Value::Object(_) => items, + _ => return None, + }; + if let Some(value) = target.get("const").and_then(Value::as_str) { + return Some(SelectKey::NodenameExact(value.to_string())); + } + if let Some(pattern) = target.get("pattern").and_then(Value::as_str) { + return Some(SelectKey::NodenamePattern(pattern.to_string())); + } + None +} + +fn compatible_strings(node: &BTreeMap) -> Vec<&str> { + match node.get("compatible") { + Some(DtValue::List(items)) => items + .iter() + .filter_map(|item| match item { + DtValue::Str(s) => Some(s.as_str()), + _ => None, + }) + .collect(), + _ => Vec::new(), + } +} + +fn node_nodename(node: &BTreeMap) -> Option<&str> { + match node.get("$nodename") { + Some(DtValue::List(items)) => items.first().and_then(|item| match item { + DtValue::Str(s) => Some(s.as_str()), + _ => None, + }), + Some(DtValue::Str(s)) => Some(s.as_str()), + _ => None, + } +} + +impl Retrieve for DtSchemaRetriever { + fn retrieve( + &self, + uri: &Uri, + ) -> Result> { + let key = uri.as_str().trim_end_matches('#'); + if let Some(v) = self.schemas.get(key) { + return Ok(validation_resource(v)); + } + // A missed reference becomes a `false` schema that rejects everything. + Ok(Value::Bool(false)) + } +} + +impl DTValidator { + /// Build from raw schema paths (files and/or directories), always adding + /// the bundled core schemas. + pub fn new(schema_paths: &[PathBuf]) -> anyhow::Result { + let version = crate::version(); + if let [schema_file] = schema_paths + && schema_file.is_file() + && let Some(processed) = load_processed_schema_file(schema_file, &version)? + { + return Self::from_processed(processed); + } + let processed = ProcessedSchemas::build(schema_paths, true, &version); + Self::from_processed(processed) + } + + /// Build from an already-assembled processed schema set. + pub fn from_processed(processed: ProcessedSchemas) -> anyhow::Result { + let ProcessedSchemas { + schemas, + compat_map, + always_schemas, + } = processed; + let type_ctx = TypeContext::from_processed(&schemas); + let always_dispatch = AlwaysDispatch::build(&schemas, &always_schemas); + let vendor_prefixes = schemas + .get(VENDOR_PREFIXES_SCHEMA) + .and_then(VendorPrefixesFastPath::build); + let raw_validators = schemas + .keys() + .map(|schema_id| (schema_id.clone(), Arc::new(OnceLock::new()))) + .collect(); + let always_validators = always_schemas + .iter() + .map(|_| Arc::new(OnceLock::new())) + .collect(); + let schemas = Arc::new(schemas); + let retriever = Arc::new(DtSchemaRetriever { + schemas: schemas.clone(), + }); + Ok(Self { + schemas, + compat_map, + always_schemas, + type_ctx, + retriever, + always_dispatch, + vendor_prefixes, + raw_validators, + always_validators, + }) + } + + /// Decode a DTB into a devicetree tree (delegates to [`crate::dtb`]). + pub fn decode_dtb( + &self, + dtb: &[u8], + decode_errors: &mut Vec, + ) -> anyhow::Result { + dtb::decode_dtb(&self.type_ctx, dtb, decode_errors) + } + + fn retriever_handle(&self) -> SharedRetriever { + SharedRetriever(self.retriever.clone()) + } + + /// Keep a schema only if the `$id` contains one of the filter substrings + /// (or the filter is empty). + fn filter_match(schema_id: &str, filter: Option<&[String]>) -> bool { + match filter { + None => true, + Some(fs) => fs.is_empty() || fs.iter().any(|f| schema_id.contains(f.as_str())), + } + } + + /// Dispatch a node against the compatible-matched schema (first matching + /// `compatible` string) and, unless `compatible_match`, every + /// `always_schemas` entry as `{if:select, then:schema}`. + pub fn iter_errors( + &self, + node: &DtValue, + filter: Option<&[String]>, + compatible_match: bool, + show_unmatched: bool, + ) -> Vec { + let mut out = Vec::new(); + let node_map = match node { + DtValue::Node(m) => m, + _ => return out, + }; + + // Compatible dispatch: use the first `compatible` string with a schema. + if let Some(DtValue::List(compats)) = node_map.get("compatible") { + for c in compats { + if let DtValue::Str(cs) = c + && let Some(schema_id) = self.compat_map.get(cs) + { + if Self::filter_match(schema_id, filter) + && let Some(schema) = self.schemas.get(schema_id) + && let Some(slot) = self.raw_validators.get(schema_id) + { + self.collect(schema, schema_id, false, slot, node, &mut out); + } + break; + } + } + } + + if compatible_match { + return out; + } + + for schema_index in self.always_dispatch.candidates(node_map) { + let Some(schema_id) = self.always_schemas.get(schema_index) else { + continue; + }; + if !show_unmatched && schema_id.as_str() == crate::GENERATED_COMPATIBLES_SCHEMA { + continue; + } + if !Self::filter_match(schema_id, filter) { + continue; + } + let Some(schema) = self.schemas.get(schema_id) else { + continue; + }; + let Some(slot) = self.always_validators.get(schema_index) else { + continue; + }; + self.collect(schema, schema_id, true, slot, node, &mut out); + } + + out + } + + /// Compile `schema` (once, cached) and collect its errors against `node`, + /// tagging each with `schema_id`. `wrapped` selects the cache slot for the + /// `{if:select, then:schema}` always-schema form vs the raw compat form. + fn collect( + &self, + schema: &Value, + schema_id: &str, + wrapped: bool, + slot: &ValidatorSlot, + node: &DtValue, + out: &mut Vec, + ) { + if schema_id == VENDOR_PREFIXES_SCHEMA + && let Some(fast) = &self.vendor_prefixes + && fast.is_valid(node) + { + return; + } + let Some(validator) = self.cached_validator(schema, schema_id, wrapped, slot) else { + return; + }; + for err in validator.iter_errors(node) { + out.push(to_dt_error(&err, schema_id)); + } + } + + /// Return the compiled validator for `(schema_id, wrapped)`, building it on + /// first use and caching the result. Returns `None` if the schema failed to + /// compile (warned once). + fn cached_validator( + &self, + schema: &Value, + schema_id: &str, + wrapped: bool, + slot: &ValidatorSlot, + ) -> Option>> { + slot.get_or_init(|| match self.build_validator(schema, wrapped) { + Ok(v) => Some(Arc::new(v)), + Err(e) => { + eprintln!("{schema_id}: error building validator: {e}"); + None + } + }) + .clone() + } + + fn build_validator(&self, schema: &Value, wrapped: bool) -> anyhow::Result> { + let schema = if wrapped { + wrapped_validation_resource(schema) + } else { + validation_resource(schema) + }; + dt_options(self.retriever_handle()) + .build(&schema) + .map_err(|e| anyhow::anyhow!("{e}")) + } + + /// Return the compatibles that do not match the `generated-compatibles` + /// schema, i.e. are not documented by any binding. + pub fn get_undocumented_compatibles(&self, compatibles: &[String]) -> Vec { + let Some(schema) = self.schemas.get(crate::GENERATED_COMPATIBLES_SCHEMA) else { + return compatibles.to_vec(); + }; + let validator = match self.build_validator(schema, false) { + Ok(v) => v, + Err(_) => return compatibles.to_vec(), + }; + let mut undoc = Vec::new(); + for c in compatibles { + let instance = DtValue::Node(BTreeMap::from([( + "compatible".to_string(), + DtValue::List(vec![DtValue::Str(c.clone())]), + )])); + if !validator.is_valid(&instance) { + undoc.push(c.clone()); + } + } + undoc + } +} + +fn load_processed_schema_file( + path: &PathBuf, + version: &str, +) -> anyhow::Result> { + let text = std::fs::read_to_string(path)?; + let value = match serde_json::from_str::(&text) { + Ok(value) => value, + Err(_) => match serde_yaml::from_str::(&text) { + Ok(value) => value, + Err(_) => { + anyhow::bail!("preprocessed schema file is not valid JSON or YAML"); + } + }, + }; + if value.get("$id").is_some() { + return Ok(None); + } + ProcessedSchemas::from_value(&value, version).map(Some) +} + +/// Convert a `jsonschema` error into our annotated [`DtError`]. +fn to_dt_error(err: &ValidationError<'_>, schema_id: &str) -> DtError { + let instance_path = location_segments(err.instance_path()); + let schema_path = location_segments(err.schema_path()); + let instance_is_disabled_node = instance_status_disabled(err.instance()); + let has_suppressible_disabled_context = error_has_suppressible_disabled_context(err); + DtError { + instance_path, + schema_path, + message: err.to_string(), + schema_file: schema_id.to_string(), + instance_is_disabled_node, + has_suppressible_disabled_context, + } +} + +/// Split a `jsonschema` [`jsonschema::paths::Location`] into path segments. +fn location_segments(loc: &jsonschema::paths::Location) -> Vec { + loc.iter() + .map(|seg| match seg { + jsonschema::paths::LocationSegment::Property(p) => PathSeg::Key(p.to_string()), + jsonschema::paths::LocationSegment::Index(i) => PathSeg::Index(i), + }) + .collect() +} + +fn error_has_suppressible_disabled_context(err: &ValidationError<'_>) -> bool { + if schema_location_has_suppressible(err.schema_path()) { + return true; + } + match err.kind() { + ValidationErrorKind::AnyOf { context } + | ValidationErrorKind::OneOfMultipleValid { context } + | ValidationErrorKind::OneOfNotValid { context } => context + .iter() + .flatten() + .any(error_has_suppressible_disabled_context), + _ => false, + } +} + +fn schema_location_has_suppressible(loc: &jsonschema::paths::Location) -> bool { + loc.iter().any(|seg| { + matches!(seg, jsonschema::paths::LocationSegment::Property(p) + if p == "required" || p == "unevaluatedProperties") + }) +} + +/// True if the failing instance is a node carrying `status = "disabled"`. +fn instance_status_disabled(instance: &Value) -> bool { + instance + .as_object() + .and_then(|m| m.get("status")) + .map(|s| match s { + Value::String(st) => st.contains("disabled"), + Value::Array(a) => a + .iter() + .any(|v| v.as_str().is_some_and(|st| st.contains("disabled"))), + _ => false, + }) + .unwrap_or(false) +} + +const DRAFT_2019_09_SCHEMA: &str = "https://json-schema.org/draft/2019-09/schema"; + +/// The validation engine understands JSON Schema drafts, not dt-schema's +/// custom meta-schema URIs. Compile DT validators as Draft 2019-09 without +/// mutating processed output. +fn validation_resource(schema: &Value) -> Value { + let mut schema = schema.clone(); + normalize_dt_meta_schema(&mut schema); + schema +} + +fn wrapped_validation_resource(schema: &Value) -> Value { + let select = schema.get("select").cloned().unwrap_or(Value::Bool(true)); + validation_resource(&serde_json::json!({ "if": select, "then": schema })) +} + +fn normalize_dt_meta_schema(value: &mut Value) { + match value { + Value::Object(obj) => { + if obj + .get("$schema") + .and_then(Value::as_str) + .is_some_and(|s| s.starts_with("http://devicetree.org/meta-schemas/")) + { + obj.insert( + "$schema".to_string(), + Value::String(DRAFT_2019_09_SCHEMA.to_string()), + ); + } + for child in obj.values_mut() { + normalize_dt_meta_schema(child); + } + } + Value::Array(items) => { + for child in items { + normalize_dt_meta_schema(child); + } + } + _ => {} + } +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn test_validator(schema: &Value) -> Validator { + test_validator_with_schemas(schema, BTreeMap::new()) + } + + fn test_validator_with_schemas( + schema: &Value, + schemas: BTreeMap, + ) -> Validator { + let retriever = SharedRetriever(Arc::new(DtSchemaRetriever { + schemas: Arc::new(schemas), + })); + dt_options(retriever).build(schema).unwrap() + } + + #[test] + fn dtjson_contains_drives_if_then_branch_selection() { + let schema = json!({ + "$schema": "http://devicetree.org/meta-schemas/core.yaml#", + "allOf": [ + { + "if": { + "properties": { + "compatible": { + "contains": { + "enum": [ + "qcom,rpmcc-apq8060", + "qcom,rpmcc-ipq806x", + "qcom,rpmcc-msm8660" + ] + } + } + } + }, + "then": { + "properties": { + "clock-names": { + "items": [{ "const": "pxo" }] + } + } + } + }, + { + "if": { + "properties": { + "compatible": { + "contains": { + "const": "qcom,rpmcc-apq8064" + } + } + } + }, + "then": { + "properties": { + "clock-names": { + "items": [{ "const": "pxo" }, { "const": "cxo" }] + } + } + } + }, + { + "if": { + "properties": { + "compatible": { + "contains": { + "enum": [ + "qcom,rpmcc-mdm9607", + "qcom,rpmcc-msm8226", + "qcom,rpmcc-msm8916" + ] + } + } + } + }, + "then": { + "properties": { + "clock-names": { + "items": [{ "const": "xo" }] + } + } + } + } + ] + }); + let instance = DtValue::Node(BTreeMap::from([ + ( + "compatible".to_string(), + DtValue::List(vec![ + DtValue::Str("qcom,rpmcc-msm8916".to_string()), + DtValue::Str("qcom,rpmcc".to_string()), + ]), + ), + ( + "clock-names".to_string(), + DtValue::List(vec![DtValue::Str("xo".to_string())]), + ), + ])); + + let validator = test_validator(&schema); + let errors: Vec<_> = validator.iter_errors(&instance).collect(); + assert!( + errors.is_empty(), + "msm8916 should only select the xo branch, got: {errors:#?}" + ); + } + + #[test] + fn dtjson_contains_pattern_drives_if_then_branch_selection() { + let schema = json!({ + "$schema": "http://devicetree.org/meta-schemas/core.yaml#", + "allOf": [ + { + "if": { + "properties": { + "compatible": { + "contains": { + "pattern": "^qcom,adreno-305\\.[0-9]+$" + } + } + } + }, + "then": { + "properties": { + "clock-names": { + "items": [ + { "const": "core" }, + { "const": "iface" }, + { "const": "mem_iface" } + ] + } + } + } + }, + { + "if": { + "properties": { + "compatible": { + "contains": { + "pattern": "^qcom,adreno-306\\.[0-9]+$" + } + } + } + }, + "then": { + "properties": { + "clock-names": { + "items": [ + { "const": "core" }, + { "const": "iface" }, + { "const": "mem" }, + { "const": "mem_iface" }, + { "const": "alt_mem_iface" }, + { "const": "gfx3d" } + ] + } + } + } + } + ] + }); + let instance = DtValue::Node(BTreeMap::from([ + ( + "compatible".to_string(), + DtValue::List(vec![DtValue::Str("qcom,adreno-306.0".to_string())]), + ), + ( + "clock-names".to_string(), + DtValue::List(vec![ + DtValue::Str("core".to_string()), + DtValue::Str("iface".to_string()), + DtValue::Str("mem".to_string()), + DtValue::Str("mem_iface".to_string()), + DtValue::Str("alt_mem_iface".to_string()), + DtValue::Str("gfx3d".to_string()), + ]), + ), + ])); + + let validator = test_validator(&schema); + let errors: Vec<_> = validator.iter_errors(&instance).collect(); + assert!( + errors.is_empty(), + "adreno-306 should only select the six-clock branch, got: {errors:#?}" + ); + } + + #[test] + fn referenced_dt_schema_resource_keeps_validation_vocabularies() { + let referenced = json!({ + "$id": "http://example.com/schemas/gpu.yaml#", + "$schema": "http://devicetree.org/meta-schemas/core.yaml#", + "allOf": [ + { + "if": { + "properties": { + "compatible": { + "contains": { + "pattern": "^qcom,adreno-305\\.[0-9]+$" + } + } + } + }, + "then": { + "properties": { + "clock-names": { + "items": [ + { "const": "core" }, + { "const": "iface" }, + { "const": "mem_iface" } + ] + } + } + } + }, + { + "if": { + "properties": { + "compatible": { + "contains": { + "pattern": "^qcom,adreno-306\\.[0-9]+$" + } + } + } + }, + "then": { + "properties": { + "clock-names": { + "items": [ + { "const": "core" }, + { "const": "iface" }, + { "const": "mem" }, + { "const": "mem_iface" }, + { "const": "alt_mem_iface" }, + { "const": "gfx3d" } + ] + } + } + } + } + ] + }); + let schema = json!({ "$ref": "http://example.com/schemas/gpu.yaml#" }); + let instance = DtValue::Node(BTreeMap::from([ + ( + "compatible".to_string(), + DtValue::List(vec![DtValue::Str("qcom,adreno-306.0".to_string())]), + ), + ( + "clock-names".to_string(), + DtValue::List(vec![ + DtValue::Str("core".to_string()), + DtValue::Str("iface".to_string()), + DtValue::Str("mem".to_string()), + DtValue::Str("mem_iface".to_string()), + DtValue::Str("alt_mem_iface".to_string()), + DtValue::Str("gfx3d".to_string()), + ]), + ), + ])); + + let validator = test_validator_with_schemas( + &schema, + BTreeMap::from([( + "http://example.com/schemas/gpu.yaml".to_string(), + referenced, + )]), + ); + let errors: Vec<_> = validator.iter_errors(&instance).collect(); + assert!( + errors.is_empty(), + "referenced dt-schema resource should keep validation vocabularies, got: {errors:#?}" + ); + } + + #[test] + fn anyof_required_context_is_suppressible_for_disabled_nodes() { + let schema = json!({ + "anyOf": [ + { "required": ["foo"] }, + { "required": ["bar"] } + ] + }); + let instance = DtValue::Node(BTreeMap::from([( + "status".to_string(), + DtValue::Str("disabled".to_string()), + )])); + + let validator = test_validator(&schema); + let errors: Vec<_> = validator.iter_errors(&instance).collect(); + assert_eq!(errors.len(), 1, "expected one anyOf wrapper error"); + + let error = to_dt_error(&errors[0], "http://example.com/test.yaml"); + assert!(error.instance_is_disabled_node); + assert!( + error.has_suppressible_disabled_context, + "anyOf child required errors should be visible to disabled-node suppression" + ); + assert!( + !schema_location_has_suppressible(errors[0].schema_path()), + "test should exercise nested context, not a top-level required path" + ); + } + + #[test] + fn generated_compatibles_respects_show_unmatched() { + let version = crate::version(); + let processed = crate::process::ProcessedSchemas::from_value( + &json!({ + "generated-compatibles": { + "$id": crate::GENERATED_COMPATIBLES_SCHEMA, + "$filename": "Generated schema of documented compatible strings", + "select": true, + "properties": { + "compatible": { + "items": { + "anyOf": [ + { "enum": ["vendor,known"] }, + { "pattern": "^test," } + ] + } + } + } + }, + "version": version, + }), + &version, + ) + .unwrap(); + let validator = DTValidator::from_processed(processed).unwrap(); + let instance = DtValue::Node(BTreeMap::from([( + "compatible".to_string(), + DtValue::List(vec![DtValue::Str("vendor,missing".to_string())]), + )])); + + assert!( + validator + .iter_errors(&instance, None, false, false) + .is_empty(), + "generated-compatibles should stay suppressed unless -m is set" + ); + let errors = validator.iter_errors(&instance, None, false, true); + assert_eq!(errors.len(), 1); + assert_eq!(errors[0].schema_file, crate::GENERATED_COMPATIBLES_SCHEMA); + } + + #[test] + fn dtjson_bytes_fail_null_and_multi_type_checks() { + let bytes = DtValue::Bytes(vec![0x62, 0x61, 0x64, 0x00]); + + let null_validator = test_validator(&json!({ "type": "null" })); + assert!( + null_validator.iter_errors(&bytes).next().is_some(), + "raw bytes must not validate as JSON null" + ); + + let dt_core_type = test_validator(&json!({ + "type": ["object", "integer", "array", "boolean", "null"] + })); + assert!( + dt_core_type.iter_errors(&bytes).next().is_some(), + "raw bytes must fail dt-core's generic property type union" + ); + } +} diff --git a/rust/dtschema/src/yaml.rs b/rust/dtschema/src/yaml.rs new file mode 100644 index 0000000..62cf464 --- /dev/null +++ b/rust/dtschema/src/yaml.rs @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! YAML loading into `serde_json::Value`. +//! +//! Schema files are written in a JSON-compatible subset of YAML. `serde_yaml` +//! parses `0xff`-style hex scalars as integers, so no post-processing is +//! required. + +use serde_json::Value; + +/// Errors from loading YAML. +#[derive(Debug, thiserror::Error)] +pub enum YamlError { + #[error("YAML parse error: {0}")] + Parse(#[from] serde_yaml::Error), +} + +/// Load a YAML document from a string into a `serde_json::Value`. +pub fn from_str(text: &str) -> Result { + Ok(serde_yaml::from_str(text)?) +} + +/// Load a YAML document from a file. +pub fn from_file(path: &std::path::Path) -> anyhow::Result { + let text = + std::fs::read_to_string(path).map_err(|e| anyhow::anyhow!("{}: {}", path.display(), e))?; + from_str(&text).map_err(|e| anyhow::anyhow!("{}: {}", path.display(), e)) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn hex_scalars_become_integers() { + let v = from_str("a: 0xff\nb: 0x1234\nc: 10\n").unwrap(); + assert_eq!(v["a"], serde_json::json!(255)); + assert_eq!(v["b"], serde_json::json!(0x1234)); + assert_eq!(v["c"], serde_json::json!(10)); + } + + #[test] + fn hashed_keys_and_lists() { + let v = from_str("'#interrupt-cells':\n const: 2\nlist:\n - 1\n - foo\n").unwrap(); + assert_eq!(v["#interrupt-cells"]["const"], serde_json::json!(2)); + assert_eq!(v["list"], serde_json::json!([1, "foo"])); + } +} diff --git a/rust/dtschema/tests/diagnostic.rs b/rust/dtschema/tests/diagnostic.rs new file mode 100644 index 0000000..52e0953 --- /dev/null +++ b/rust/dtschema/tests/diagnostic.rs @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Unit tests for the diagnostic layer, mirroring the diagnostic-shaping cases +//! in the Python `test/test-dt-validate.py` (`test_json_error_diagnostic`, +//! `test_json_unmatched_diagnostic`, `test_format_error_rewrites_indented_paths`). +//! +//! The Python error diagnostics carry source `line`/`column` from +//! `error.linecol`; a decoded DTB carries no source positions, so the Rust port +//! reports `line`/`column` as `null` and we assert the remaining fields. + +use dtschema::diagnostic::{ + diagnostic_text, error_diagnostic, replace_filename_prefix, unmatched_diagnostic, +}; +use dtschema::validator::{DtError, PathSeg}; +use serde_json::json; + +fn key(s: &str) -> PathSeg { + PathSeg::Key(s.to_string()) +} + +#[test] +fn test_json_error_diagnostic() { + let error = DtError { + instance_path: vec![key("soc"), key("device@0")], + schema_path: vec![key("then"), key("required")], + message: "'foo' is a required property".to_string(), + schema_file: "http://devicetree.org/schemas/test.yaml#".to_string(), + instance_is_disabled_node: false, + has_suppressible_disabled_context: false, + }; + + let d = error_diagnostic( + "test.dtb", + &error, + Some("device@0"), + Some("/soc/device@0"), + Some("test,device"), + None, + ) + .to_value(); + + assert_eq!(d["type"], "validation"); + assert_eq!(d["level"], "error"); + assert_eq!(d["file"], "test.dtb"); + assert_eq!(d["node"], "/soc/device@0"); + assert_eq!(d["nodename"], "device@0"); + assert_eq!(d["compatible"], "test,device"); + assert_eq!(d["property_path"], json!(["soc", "device@0"])); + assert_eq!(d["schema_path"], json!(["then", "required"])); + assert_eq!(d["schema"], "http://devicetree.org/schemas/test.yaml#"); + assert_eq!(d["message"], "'foo' is a required property"); +} + +#[test] +fn test_json_unmatched_diagnostic() { + let d = + unmatched_diagnostic("test.dtb", "/soc/device@0", &["test,device".to_string()]).to_value(); + assert_eq!(d["type"], "unmatched"); + assert_eq!(d["level"], "warning"); + assert_eq!(d["file"], "test.dtb"); + assert_eq!(d["node"], "/soc/device@0"); + assert_eq!(d["compatible"], json!(["test,device"])); + assert_eq!( + d["message"], + "failed to match any schema with compatible: ['test,device']" + ); + assert_eq!( + diagnostic_text(&d), + "test.dtb: /soc/device@0: failed to match any schema with compatible: ['test,device']" + ); + + // A root node ("/") keeps "/" as its nodename. + let d = unmatched_diagnostic("test.dtb", "/", &["test,board".to_string()]).to_value(); + assert_eq!(d["nodename"], "/"); +} + +#[test] +fn test_format_error_rewrites_indented_paths() { + // Python's `_format_error` rewrites the absolute-path prefix to the display + // path on every line, including tab-indented sub-error lines. This is the + // core behaviour of `replace_filename_prefix`. + let text = "/abs/test.dtb:1:1: outer problem\n\t/abs/test.dtb:2:1: inner problem\n"; + let rewritten = replace_filename_prefix(text, "/abs/test.dtb", "test.dtb"); + + assert!(!rewritten.contains("/abs/test.dtb")); + assert!(rewritten.contains("test.dtb:1:1")); + assert!(rewritten.contains("\ttest.dtb:2:1")); +} diff --git a/rust/dtschema/tests/dtb_parity.rs b/rust/dtschema/tests/dtb_parity.rs new file mode 100644 index 0000000..48be700 --- /dev/null +++ b/rust/dtschema/tests/dtb_parity.rs @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Differential parity test for the DTB decoder (`decode_dtb`). +//! +//! For every `test/*.dts` fixture: compile it with `dtc -Odtb`, decode the blob +//! with the Rust [`dtschema::dtb`] pipeline and with the Python +//! `DTValidator.decode_dtb` reference, canonicalize both to JSON (raw bytes → +//! `{"$bytes":[..]}`, `sized_int` → plain int), and assert they are identical. +//! +//! Skipped (not failed) when a working Python `dtschema` or `dtc` isn't present. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +use dtschema::dtb::{self, TypeContext}; +use serde_json::Value; + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +fn find_python(repo: &Path) -> Option { + let venv = repo.join(".venv/bin/python3"); + for c in [venv, PathBuf::from("python3")] { + let ok = Command::new(&c) + .args(["-c", "import dtschema"]) + .status() + .map(|s| s.success()) + .unwrap_or(false); + if ok { + return Some(c); + } + } + None +} + +fn have_dtc() -> bool { + Command::new("dtc") + .arg("--version") + .output() + .map(|o| o.status.success()) + .unwrap_or(false) +} + +/// Python reference decode → canonical JSON string. +const PY_DECODE: &str = r#" +import json, sys, dtschema +def conv(v): + if isinstance(v, bool): return v + if isinstance(v, dict): return {k: conv(x) for k, x in v.items()} + if isinstance(v, list): return [conv(x) for x in v] + if isinstance(v, bytes): return {'$bytes': list(v)} + if isinstance(v, int): return int(v) + return v +data = sys.stdin.buffer.read() +val = dtschema.DTValidator([]).decode_dtb(data) +json.dump(conv(val[0]), sys.stdout, sort_keys=True) +"#; + +fn canon(v: &Value) -> Value { + match v { + Value::Object(m) => { + let mut keys: Vec<&String> = m.keys().collect(); + keys.sort(); + let mut o = serde_json::Map::new(); + for k in keys { + o.insert(k.clone(), canon(&m[k])); + } + Value::Object(o) + } + Value::Array(a) => Value::Array(a.iter().map(canon).collect()), + other => other.clone(), + } +} + +#[test] +fn decode_matches_python_oracle() { + let repo = repo_root(); + let Some(python) = find_python(&repo) else { + eprintln!("SKIP: no python with `dtschema` importable"); + return; + }; + if !have_dtc() { + eprintln!("SKIP: dtc not available"); + return; + } + + let ctx = TypeContext::new(&[]); + + let mut dts_files: Vec = std::fs::read_dir(repo.join("test")) + .expect("read test dir") + .flatten() + .map(|e| e.path()) + .filter(|p| p.extension().and_then(|s| s.to_str()) == Some("dts")) + .collect(); + dts_files.sort(); + assert!(!dts_files.is_empty(), "no .dts fixtures found"); + + let mut diffs: Vec = Vec::new(); + for dts in &dts_files { + // Compile to DTB. + let dtc = Command::new("dtc") + .args(["-Odtb", "-o", "-"]) + .arg(dts) + .output() + .expect("run dtc"); + if !dtc.status.success() { + // Some -fail fixtures may still compile; a dtc failure is a real + // problem for the parity comparison, so record and skip it. + diffs.push(format!("{}: dtc failed", dts.display())); + continue; + } + let dtb_bytes = dtc.stdout; + + // Rust decode. + let mut errs = Vec::new(); + let rust_tree = match dtb::decode_dtb(&ctx, &dtb_bytes, &mut errs) { + Ok(t) => t, + Err(e) => { + diffs.push(format!("{}: rust decode error: {e}", dts.display())); + continue; + } + }; + let rust_json = canon(&rust_tree.to_json()); + + // Python decode. + let py = Command::new(&python) + .args(["-c", PY_DECODE]) + .stdin(std::process::Stdio::piped()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .spawn() + .and_then(|mut child| { + use std::io::Write; + child.stdin.take().unwrap().write_all(&dtb_bytes).unwrap(); + child.wait_with_output() + }) + .expect("run python decode"); + if !py.status.success() { + diffs.push(format!( + "{}: python decode failed: {}", + dts.display(), + String::from_utf8_lossy(&py.stderr) + )); + continue; + } + let py_json: Value = serde_json::from_slice(&py.stdout).expect("parse python decode json"); + let py_json = canon(&py_json); + + if rust_json != py_json { + diffs.push(format!("{}: decoded tree differs", dts.display())); + // Emit a compact first-divergence hint. + eprintln!("--- {} ---", dts.display()); + eprintln!("rust: {}", rust_json); + eprintln!("py : {}", py_json); + } + } + + assert!( + diffs.is_empty(), + "{} fixtures diverged:\n{}", + diffs.len(), + diffs.join("\n") + ); +} diff --git a/rust/dtschema/tests/fixup_parity.rs b/rust/dtschema/tests/fixup_parity.rs new file mode 100644 index 0000000..cb8447e --- /dev/null +++ b/rust/dtschema/tests/fixup_parity.rs @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Differential parity test for the fixup pipeline. +//! +//! For every bundled and test schema, run the Rust [`DTSchema::fixup`] and diff +//! the canonical (sorted-key) JSON against the Python reference +//! `dtschema.DTSchema(f).fixup()`. Goldens are produced on the fly by invoking +//! the Python package, so the test tracks the oracle rather than a stale +//! checked-in snapshot. +//! +//! The test is skipped (not failed) when a working Python `dtschema` isn't +//! importable — CI without the Python venv still builds and runs the rest. + +use dtschema::schema::DTSchema; +use serde_json::Value; +use std::path::{Path, PathBuf}; +use std::process::Command; + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +/// Locate a python interpreter with `dtschema` importable. Prefers the repo +/// `.venv`, falls back to `python3` on PATH. +fn find_python(repo: &Path) -> Option { + let venv = repo.join(".venv/bin/python3"); + let candidates = [venv, PathBuf::from("python3")]; + for c in candidates { + let ok = Command::new(&c) + .args(["-c", "import dtschema"]) + .status() + .map(|s| s.success()) + .unwrap_or(false); + if ok { + return Some(c); + } + } + None +} + +fn canonicalize(v: &Value) -> Value { + match v { + Value::Object(m) => { + let mut keys: Vec<&String> = m.keys().collect(); + keys.sort(); + let mut out = serde_json::Map::new(); + for k in keys { + out.insert(k.clone(), canonicalize(&m[k])); + } + Value::Object(out) + } + Value::Array(a) => Value::Array(a.iter().map(canonicalize).collect()), + other => other.clone(), + } +} + +fn collect_yaml(dir: &Path, out: &mut Vec) { + if let Ok(rd) = std::fs::read_dir(dir) { + for e in rd.flatten() { + let p = e.path(); + if p.is_dir() { + collect_yaml(&p, out); + } else if p.extension().and_then(|s| s.to_str()) == Some("yaml") { + out.push(p); + } + } + } +} + +#[test] +fn fixup_matches_python_oracle() { + let repo = repo_root(); + let Some(python) = find_python(&repo) else { + eprintln!("SKIP: no python with `dtschema` importable"); + return; + }; + + let mut files = Vec::new(); + collect_yaml(&repo.join("dtschema/schemas"), &mut files); + collect_yaml(&repo.join("test/schemas"), &mut files); + files.sort(); + assert!(!files.is_empty(), "no schema files found"); + + // Generate all goldens in one Python invocation, keyed by absolute path. + let script = r#" +import sys, json, dtschema +out = {} +for f in sys.argv[1:]: + try: + out[f] = dtschema.DTSchema(f).fixup() + except Exception as e: + out[f] = {"__error__": str(e)} +json.dump(out, sys.stdout, default=str) +"#; + let file_args: Vec = files.iter().map(|f| f.to_string_lossy().into()).collect(); + let output = Command::new(&python) + .arg("-c") + .arg(script) + .args(&file_args) + .output() + .expect("run python oracle"); + assert!( + output.status.success(), + "python oracle failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + let goldens: std::collections::HashMap = + serde_json::from_slice(&output.stdout).expect("parse oracle json"); + + let mut mismatches = Vec::new(); + for f in &files { + let key = f.to_string_lossy().to_string(); + let Some(golden) = goldens.get(&key) else { + continue; + }; + if golden.get("__error__").is_some() { + continue; // Python couldn't process it; nothing to compare. + } + let sch = DTSchema::load(f).expect("load schema"); + let got = canonicalize(&sch.fixup()); + let want = canonicalize(golden); + if got != want { + let rel = f.strip_prefix(&repo).unwrap().display().to_string(); + mismatches.push(rel); + } + } + + assert!( + mismatches.is_empty(), + "fixup output diverged from Python for {} file(s):\n{}", + mismatches.len(), + mismatches.join("\n") + ); +} diff --git a/rust/dtschema/tests/mkschema_parity.rs b/rust/dtschema/tests/mkschema_parity.rs new file mode 100644 index 0000000..3d02e1b --- /dev/null +++ b/rust/dtschema/tests/mkschema_parity.rs @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Differential parity test for the processed-schema pipeline (`dt-mk-schema`). +//! +//! Builds the Rust processed schema set for `test/schemas` (+ bundled core) and +//! compares it against the Python `dtschema` reference as parsed JSON values. +//! The generated type caches and compatible enum/pattern lists are compared as +//! unordered sets, since their order depends on Python's `glob`/`set` +//! iteration. +//! +//! Skipped (not failed) when a working Python `dtschema` isn't importable. + +use std::collections::{BTreeMap, BTreeSet}; +use std::path::{Path, PathBuf}; +use std::process::Command; + +use dtschema::process::ProcessedSchemas; +use serde_json::Value; + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +fn find_python(repo: &Path) -> Option { + let venv = repo.join(".venv/bin/python3"); + for c in [venv, PathBuf::from("python3")] { + let ok = Command::new(&c) + .args(["-c", "import dtschema"]) + .status() + .map(|s| s.success()) + .unwrap_or(false); + if ok { + return Some(c); + } + } + None +} + +fn canon(v: &Value) -> Value { + match v { + Value::Object(m) => { + let mut keys: Vec<&String> = m.keys().collect(); + keys.sort(); + let mut o = serde_json::Map::new(); + for k in keys { + o.insert(k.clone(), canon(&m[k])); + } + Value::Object(o) + } + Value::Array(a) => Value::Array(a.iter().map(canon).collect()), + other => other.clone(), + } +} + +fn as_set(v: &Value) -> BTreeSet { + v.as_array() + .map(|a| a.iter().map(|x| canon(x).to_string()).collect()) + .unwrap_or_default() +} + +#[test] +fn mk_schema_matches_python_oracle() { + let repo = repo_root(); + let Some(python) = find_python(&repo) else { + eprintln!("SKIP: no python with `dtschema` importable"); + return; + }; + + // Golden: `dt-mk-schema -j test/schemas` (includes bundled core by default). + // mk_schema has no __main__ guard, so drive main() via argv. + let out = Command::new(&python) + .args([ + "-c", + "import sys; from dtschema.mk_schema import main; sys.exit(main())", + "-j", + ]) + .arg(repo.join("test/schemas")) + .current_dir(&repo) + .output() + .expect("run python dt-mk-schema"); + assert!( + out.status.success(), + "python mk-schema failed: {}", + String::from_utf8_lossy(&out.stderr) + ); + let golden: Value = serde_json::from_slice(&out.stdout).expect("parse golden json"); + let want = golden.as_object().unwrap(); + let version = golden.get("version").and_then(Value::as_str).unwrap_or("0"); + + let ps = ProcessedSchemas::build(&[repo.join("test/schemas")], true, version); + let got: BTreeMap = ps.schemas.clone(); + + let gk: BTreeSet<&String> = got.keys().collect(); + let wk: BTreeSet<&String> = want.keys().collect(); + assert_eq!( + gk, + wk, + "top-level key set differs: only_rust={:?} only_py={:?}", + gk.difference(&wk).collect::>(), + wk.difference(&gk).collect::>() + ); + + // Generated type caches: per-prop type-list compared as a set. + for genkey in ["generated-types", "generated-pattern-types"] { + let gp = got[genkey]["properties"].as_object().unwrap(); + let wp = want[genkey]["properties"].as_object().unwrap(); + let gpk: BTreeSet<&String> = gp.keys().collect(); + let wpk: BTreeSet<&String> = wp.keys().collect(); + assert_eq!(gpk, wpk, "{genkey}: property key set differs"); + for k in gpk { + assert_eq!( + as_set(&gp[k]), + as_set(&wp[k]), + "{genkey}: {k} type-list differs" + ); + } + } + + // generated-compatibles: enum + pattern lists as sets. + let ga = &got["generated-compatibles"]["properties"]["compatible"]["items"]["anyOf"]; + let wa = &want["generated-compatibles"]["properties"]["compatible"]["items"]["anyOf"]; + assert_eq!( + as_set(&ga[0]["enum"]), + as_set(&wa[0]["enum"]), + "compatibles enum differs" + ); + let gpat: BTreeSet<&str> = ga.as_array().unwrap()[1..] + .iter() + .map(|e| e["pattern"].as_str().unwrap()) + .collect(); + let wpat: BTreeSet<&str> = wa.as_array().unwrap()[1..] + .iter() + .map(|e| e["pattern"].as_str().unwrap()) + .collect(); + assert_eq!(gpat, wpat, "compatibles patterns differ"); + + // Per-schema entries: parsed-value match modulo absolute $filename. + let mut entry_diffs = Vec::new(); + for k in got.keys() { + if k.starts_with("generated-") || k == "version" { + continue; + } + let mut g = got[k].clone(); + let mut w = want[k].clone(); + for v in [&mut g, &mut w] { + v.as_object_mut().unwrap().remove("$filename"); + } + if canon(&g) != canon(&w) { + entry_diffs.push(k.clone()); + } + } + assert!( + entry_diffs.is_empty(), + "{} processed entries diverged from Python:\n{}", + entry_diffs.len(), + entry_diffs.join("\n") + ); +} diff --git a/rust/dtschema/tests/schema_valid.rs b/rust/dtschema/tests/schema_valid.rs new file mode 100644 index 0000000..b5738c2 --- /dev/null +++ b/rust/dtschema/tests/schema_valid.rs @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +//! Schema-set validity tests, mirroring the Python `test/test-dt-validate.py` +//! `TestDTMetaSchema.test_all_metaschema_valid` and `TestDTSchema` +//! (`test_binding_schemas_valid`, `test_binding_schemas_id_is_unique`). +//! +//! These run entirely against the bundled `dtschema/` tree — no Python or `dtc` +//! required. + +use std::collections::HashMap; +use std::path::{Path, PathBuf}; + +use dtschema::schema::{DTSchema, DtRetriever}; +use jsonschema::Draft; +use serde_json::Value; + +/// The bundled `dtschema/` data directory (honours `DTSCHEMA_DIR`). +fn dtschema_dir() -> PathBuf { + dtschema::bundled_dir() +} + +/// Recursively collect `*.yaml` files under `dir`. +fn collect_yaml(dir: &Path, out: &mut Vec) { + let Ok(rd) = std::fs::read_dir(dir) else { + return; + }; + let mut entries: Vec = rd.flatten().map(|e| e.path()).collect(); + entries.sort(); + for p in entries { + if p.is_dir() { + collect_yaml(&p, out); + } else if p.extension().and_then(|s| s.to_str()) == Some("yaml") { + out.push(p); + } + } +} + +#[test] +fn test_all_metaschema_valid() { + // Every meta-schema is itself a valid Draft2019-09 schema. + let mut files = Vec::new(); + collect_yaml(&dtschema_dir().join("meta-schemas"), &mut files); + assert!(!files.is_empty(), "no meta-schemas found"); + for f in files { + let value = dtschema::yaml::from_file(&f) + .unwrap_or_else(|e| panic!("{}: parse error: {e}", f.display())); + // Meta-schemas cross-reference each other via `http://devicetree.org/` + // URIs, so building requires the bundled retriever (the Python + // `check_schema` structural check needs no refs, but the Rust engine + // resolves them eagerly). + let built = jsonschema::options() + .with_draft(Draft::Draft201909) + .with_retriever(DtRetriever::bundled()) + .build(&value); + assert!( + built.is_ok(), + "{}: not a valid Draft2019-09 schema: {}", + f.display(), + built.err().unwrap() + ); + } +} + +#[test] +fn test_binding_schemas_valid() { + // Every bundled binding meta-validates cleanly against its `$schema`. + let mut files = Vec::new(); + collect_yaml(&dtschema_dir().join("schemas"), &mut files); + assert!(!files.is_empty(), "no bundled schemas found"); + for f in files { + let sch = DTSchema::load(&f).unwrap_or_else(|e| panic!("{}: load error: {e}", f.display())); + let errors = sch + .meta_validate() + .unwrap_or_else(|e| panic!("{}: meta-validate error: {e}", f.display())); + assert!( + errors.is_empty(), + "{}: unexpected meta-validation errors:\n{}", + f.display(), + errors.join("\n") + ); + } +} + +#[test] +fn test_binding_schemas_id_is_unique() { + // No two bundled bindings share a `$id`. + let mut files = Vec::new(); + collect_yaml(&dtschema_dir().join("schemas"), &mut files); + let mut seen: HashMap = HashMap::new(); + for f in files { + let value = dtschema::yaml::from_file(&f) + .unwrap_or_else(|e| panic!("{}: parse error: {e}", f.display())); + let id = value + .get("$id") + .and_then(Value::as_str) + .unwrap_or_else(|| panic!("{}: missing $id", f.display())) + .to_string(); + if let Some(prev) = seen.insert(id.clone(), f.clone()) { + panic!( + "duplicate $id {id}:\n {}\n {}", + prev.display(), + f.display() + ); + } + } +} diff --git a/test/address-cells-parent.dts b/test/address-cells-parent.dts new file mode 100644 index 0000000..f68074c --- /dev/null +++ b/test/address-cells-parent.dts @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: BSD-2-Clause +// Copyright 2026 dt-schema contributors +/dts-v1/; +/plugin/; // silence any missing phandle references +/ { + model = "none"; + compatible = "none"; + + #address-cells = <2>; + #size-cells = <1>; + + bus@1000 { + compatible = "simple-bus"; + reg = <0 0x1000 0x4000>; + #address-cells = <1>; + #size-cells = <1>; + ranges = <0 0 0x1000 0x4000>; + + child@100 { + reg = <0x100 0x20>; + }; + }; +};