Pre-publish docs sweep: README, AGENTS.md, licenses, inline doc fixes
- Add README.md reflecting the v0.1.0 state: 19 AlkType kinds, two layout modes, builder + AlkTypeEngine usage example (verified to compile and run), validation entry points, crate independence, untrusted-schemas guarantee, docs pointers. Mirrors the alkvault README structure. - Add AGENTS.md with alktype-specific git workflow, project conventions (no comments, AlkTypeError, untrusted schemas, overflow safety, no async, no feature flags, wasm-clean, preserve_order load-bearing, no unsafe), verification commands, and ADR/OQ index. Blocks auto-commit on semver-relevant public API changes per the crates.io 0.1.0 contract. - Add LICENSE-MIT and LICENSE-APACHE (dual MIT/Apache-2.0, matching alkvault and the Cargo.toml license field). - Cargo.toml: add readme, keywords, categories, rust-version = "1.85". - Fix broken intra-doc link in builder.rs: DiscriminatorKind -> crate::schema::DiscriminatorKind (cargo doc now warning-free). - N1 (review #002): document is_rfc3339_timestamp as non-strict in the function doc comment. Lists the specific gaps (day-of-month per month, seconds range, leap seconds) and points consumers needing strict validation to chrono/time. - N2 (review #002): document the FieldValue::Bytes-for-Record API asymmetry in the FieldValue enum doc and on read_record_value. - .opencode/agents/implementation-specialist.md: point to AGENTS.md for full convention details (matches the alkvault pattern). - review #002: mark N1/N2 resolved; all 7 findings now closed. Verification: - cargo test --release: 396 tests pass (310 crate + 86 integration) - cargo clippy --all-targets -- -D warnings: clean - cargo doc --no-deps: clean (no broken intra-doc link warnings) - cargo build --target wasm32-unknown-unknown --release: clean - cargo publish --dry-run --allow-dirty: clean
This commit is contained in:
glm-5.2 committed
1 parent
5f88bca0d8
commit
fb3a27f974
10 files changed
+641
-13
No files matched your search
@@ -207,7 +207,7 @@ This is especially important for complex tasks that span many file operations.
|
||||
|
||||
## Project Conventions
|
||||
|
||||
Key project conventions (no AGENTS.md — these are the canonical rules):
|
||||
Read `AGENTS.md` at project root for full details. Key rules:
|
||||
|
||||
1. **No comments in code** — Per project convention.
|
||||
2. **Error handling** — `AlkTypeError` (hand-rolled enum in `src/error.rs`) is
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
# AGENTS.md
|
||||
|
||||
Operating instructions for opencode agents working in this repo. opencode
|
||||
auto-loads this file as instructions, overriding the built-in defaults for
|
||||
this project. Custom agents in `.opencode/agents/` inherit these rules
|
||||
unless their own prompts say otherwise.
|
||||
|
||||
## Git Workflow
|
||||
|
||||
**Commit and push when reasonable.** When a change is complete and
|
||||
verified (build + lint + tests pass), commit and push to `origin/main`
|
||||
without asking. This overrides the built-in default of "only commit when
|
||||
explicitly asked."
|
||||
|
||||
The workflow:
|
||||
|
||||
1. Make the change
|
||||
2. Verify: `cargo test --release`, `cargo clippy --all-targets --
|
||||
-D warnings`, `cargo doc --no-deps` if docs changed, `cargo build
|
||||
--target wasm32-unknown-unknown --release` if layout/wasm-relevant
|
||||
code changed
|
||||
3. Inspect `git status` and `git diff` before staging — stage only the
|
||||
intended files, never secrets
|
||||
4. Write a concise commit message matching the repo style (see `git log
|
||||
--oneline -10`). For multi-point changes, use a summary line plus a
|
||||
body with bullet points and a verification block.
|
||||
5. `git push origin main`
|
||||
6. Report the commit hash and the verification summary
|
||||
|
||||
Exceptions — **do not** commit or push without asking:
|
||||
|
||||
- The change is exploratory / speculative (you're not sure the user wants
|
||||
it kept)
|
||||
- The user is actively reviewing the diff and may ask for changes
|
||||
- The change touches semver-relevant public API (this repo is on
|
||||
crates.io; the public surface is the 0.1.0 contract — `AlkTypeEngine`,
|
||||
`Schema`/`Definitions`/`Discriminator` builder types, `FieldValue`,
|
||||
`AlkTypeKind`, `Endian`, `VariableEncoding`, `DiscriminatorKind`,
|
||||
`LayoutMode`, `OffsetMap`/`ByteRange`, `LayoutBuilder`/`PackedLayout`/
|
||||
`FieldPosition`, `SequentialReader`, `UnionDispatch`, `AlkTypeError`
|
||||
variants, and the `data_access`/`schema`/`tunion`/`materialize`/
|
||||
`validation` public function signatures). Additive, non-breaking
|
||||
changes (new methods, new error variants, new builder setters) are
|
||||
fine to commit; renames, removals, signature changes, or behavioral
|
||||
shifts on existing public items are not.
|
||||
- You'd be force-pushing, amending a published commit, creating an empty
|
||||
commit, or skipping hooks
|
||||
|
||||
Never commit secrets, keys, or credentials. If a commit fails or hooks
|
||||
reject it, fix the issue and create a new commit — do not amend the
|
||||
failed one.
|
||||
|
||||
Git identity is preconfigured (`glm-5.2 <glm-5.2@alk.dev>`). Do not
|
||||
change `git config`, skip hooks, or use `git commit -i`.
|
||||
|
||||
## Project Conventions (Rust / binary struct engine)
|
||||
|
||||
This is a binary struct engine crate. The conventions below apply to all
|
||||
work in `src/` and `tests/`. They mirror `.opencode/agents/
|
||||
implementation-specialist.md` §Project Conventions and are repeated here
|
||||
so they apply to every session, not just spawned implementation agents.
|
||||
|
||||
1. **No comments in code** unless the user explicitly asks. This is a
|
||||
project-wide convention. Doc comments (`///`, `//!`) are fine and
|
||||
expected on public API. Inline `//` comments only when the user asks
|
||||
or when a non-obvious safety/correctness constraint would otherwise be
|
||||
missed (e.g., "the `u32` at offset 1 is unaligned — correct for
|
||||
protocol wire formats, which pack fields tightly").
|
||||
|
||||
2. **Error handling** — `AlkTypeError` (the hand-rolled enum in
|
||||
`src/error.rs`) is the library error type. No `anyhow` or `thiserror`
|
||||
— this is a library crate with a single error enum covering the three
|
||||
engine phases (schema, offset, access) plus validation. Never panic
|
||||
in library code. No `unwrap()` or `expect()` outside tests — if you
|
||||
reach for `unwrap`, the error path wasn't specified, stop and decide
|
||||
what should actually happen.
|
||||
|
||||
3. **Schemas are untrusted input** — every engine path that walks a
|
||||
schema must return `Err` on a malformed schema, never `panic!`/
|
||||
`unreachable!`. The downstream `alkcall` consumer accepts schemas
|
||||
from arbitrary internet peers in its hub/spoke topology, so a
|
||||
malicious or unsupported schema definition must produce a handleable
|
||||
error, not a crash. If you add a new `AlkTypeKind` variant, the
|
||||
`k if k.is_fixed_size()` guard pattern in `offset_map` and
|
||||
`layout_builder` will catch it at runtime via the `_ => Err(...)`
|
||||
arm; listing all fixed-size kinds explicitly to restore compile-time
|
||||
exhaustiveness is a separate cleanup, not a blocker (review #002, L2).
|
||||
|
||||
4. **Overflow safety** — use `checked_add`/`try_from` for any offset
|
||||
arithmetic or cast that can overflow on adversarial input. The
|
||||
`data_access::write_bytes` u32 truncation guard (review #002, M2) is
|
||||
the canonical pattern: validate the cast, return
|
||||
`AlkTypeError::Access` on failure. Do not regress to bare `as u32`
|
||||
or `+` in production paths.
|
||||
|
||||
5. **No `async`** — the engine is fully synchronous. No `tokio`, no
|
||||
`async`/`.await`, no async-sync primitives. Schema compilation,
|
||||
layout walks, read/write, and validation are all blocking,
|
||||
CPU-bound operations.
|
||||
|
||||
6. **No feature flags** — the crate has no feature flags and no
|
||||
optional dependencies. `default = []` in `Cargo.toml`. If a future
|
||||
need surfaces (e.g. `no_std` per OQ-002, or an optional strict
|
||||
RFC 3339 validator), raise it as an OQ/ADR before adding one.
|
||||
|
||||
7. **WASM-clean** — the only dependencies are `jsonschema` (with
|
||||
`default-features = false`) and `serde_json` (with `preserve_order`).
|
||||
No platform deps, no `std::time`, no filesystem, no threads. Must
|
||||
compile to `wasm32-unknown-unknown`. If you reach for a new
|
||||
dependency, first verify it's wasm-compatible and confirm it's worth
|
||||
the dependency cost.
|
||||
|
||||
8. **`serde_json`'s `preserve_order` is load-bearing** — field order in
|
||||
the schema JSON determines byte order in packed mode, and the
|
||||
`OffsetMap`/`PackedLayout` iteration order in both modes. Do not
|
||||
disable the `preserve_order` feature, and do not sort schema object
|
||||
keys anywhere in the engine.
|
||||
|
||||
9. **Naming** — Rust standard: `snake_case` for functions/variables/
|
||||
modules, `PascalCase` for types/traits, `SCREAMING_SNAKE_CASE` for
|
||||
constants.
|
||||
|
||||
10. **Module structure** — one module per file under `src/`, re-exported
|
||||
from `src/lib.rs`. Public API surface is the `lib.rs` re-exports; if
|
||||
a new public type or function needs to be visible to consumers, add
|
||||
it to the `pub use` block in `lib.rs`.
|
||||
|
||||
11. **No `unsafe`** — the crate has zero `unsafe` blocks and zero `unsafe
|
||||
extern` declarations. Bounds-checked slice access via
|
||||
`data_access::check_bounds` and `get(..)` with `ok_or_else` is the
|
||||
pattern. Do not introduce `unsafe` for performance; the
|
||||
bounds-check-eliding optimization belongs in the `jsonschema`/serde
|
||||
layer, not here.
|
||||
|
||||
## Verification Commands
|
||||
|
||||
Run these before committing. All must pass.
|
||||
|
||||
```bash
|
||||
cargo test --release # full suite (~396 tests: 310 crate + 86 integration)
|
||||
cargo clippy --all-targets -- -D warnings
|
||||
cargo doc --no-deps # if docs changed
|
||||
cargo build --target wasm32-unknown-unknown --release # if layout/wasm-relevant code changed
|
||||
cargo publish --dry-run --allow-dirty # before a release
|
||||
```
|
||||
|
||||
## Architecture Context
|
||||
|
||||
- `docs/architecture/` — the authoritative spec. Read it before
|
||||
non-trivial changes. ADRs are numbered; OQs (open questions) track
|
||||
resolved/deferred decisions.
|
||||
- ADR-001 — purpose, scope, "schema is the format" principle,
|
||||
jsonschema as the validation engine
|
||||
- ADR-002 — two layout modes (packed sequential vs aligned static); the
|
||||
most important architectural decision
|
||||
- ADR-003 — schema annotations (endianness, alignment, encoding, TUnion
|
||||
discriminators)
|
||||
- ADR-004 — error handling and validation strategy; the `AlkTypeError`
|
||||
enum, load-time build, access-time check
|
||||
- ADR-005 — Int64/Uint64 as first-class kinds; JSON precision caveat
|
||||
- ADR-006 — reject non-final inline length-prefixed variable fields in
|
||||
aligned mode (prevents silent data corruption)
|
||||
- ADR-007 — packed-mode read factory; `engine.sequential_reader()`
|
||||
returns an owned fresh reader (the reader has mutable cursor state)
|
||||
- ADR-008 — reject TUnion in aligned mode for v1 (broken semantics)
|
||||
- ADR-009 — builder API producing `serde_json::Value`; resolves OQ-003
|
||||
- ADR-010 — `validate_bytes` on `AlkTypeEngine`; materialize `Value`
|
||||
from bytes, then validate
|
||||
- If a TODO references a design direction that an ADR has since decided
|
||||
against, the TODO is stale — remove it and align with the ADR. Do not
|
||||
implement the rejected design.
|
||||
- OQ-001 (deferred): arrays of variable-length-element structs — blocked
|
||||
on a concrete consumer that needs interleaved variable-stride arrays.
|
||||
- OQ-002 (deferred): `no_std` + `alloc` support — blocked on an embedded
|
||||
use case; the core engine is already allocation-free, `jsonschema` is
|
||||
the only `alloc` consumer.
|
||||
@@ -2,9 +2,13 @@
|
||||
name = "alktype"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
rust-version = "1.85"
|
||||
license = "MIT OR Apache-2.0"
|
||||
description = "Binary struct engine: takes a JSON Schema with AlkType:* custom keywords and produces an offset map, read/write functions, and validation"
|
||||
repository = "https://git.alk.dev/alkdev/alktype"
|
||||
readme = "README.md"
|
||||
keywords = ["binary", "jsonschema", "wire-format", "serialization", "layout"]
|
||||
categories = ["encoding", "data-structures", "parsing"]
|
||||
|
||||
[lib]
|
||||
name = "alktype"
|
||||
|
||||
+192
@@ -0,0 +1,192 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name), or refer to, the Work.
|
||||
(Note: Derivative Works shall not include works that remain separable from,
|
||||
or merely link (or bind by name) to the interfaces of, the Work and
|
||||
Derivative Works thereof.)
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to the Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by the Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
Copyright 2025-2026 Alk Development
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025-2026 Alk Development
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,206 @@
|
||||
# alktype
|
||||
|
||||
The binary struct engine: a small Rust crate that takes a JSON Schema
|
||||
with `AlkType:*` custom keywords and produces an offset map, read/write
|
||||
functions, and validation — all driven by the schema. The schema is the
|
||||
format definition; the engine is generic.
|
||||
|
||||
`alktype` is a standalone crate with **two dependencies**: `jsonschema`
|
||||
(for validation) and `serde_json` (for schema parsing). No tokio, no
|
||||
platform deps, no `unsafe`. Compiles to `wasm32-unknown-unknown`.
|
||||
|
||||
## What it is
|
||||
|
||||
A JSON Schema annotated with `AlkType:*` custom keywords serves three
|
||||
roles simultaneously:
|
||||
|
||||
| Role | Mechanism | When |
|
||||
|------|-----------|------|
|
||||
| **Validation spec** | `jsonschema` custom keywords | Load time (build validator), access time (validate buffer) |
|
||||
| **Layout spec** | Offset computation from type sizes + field order | Load time (build offset map / packed layout) |
|
||||
| **Data access** | Read/write at computed offsets | Access time (read field, write field) |
|
||||
|
||||
No separate format definition, no separate parser, no separate
|
||||
validator. The schema is the single source of truth for the binary
|
||||
format. Adding a new field to a protocol is adding a property to the
|
||||
schema JSON — the engine computes the new offsets automatically.
|
||||
|
||||
This is the same principle as `#[repr(C)]` struct field access, but at
|
||||
runtime from a portable JSON Schema instead of at compile time from
|
||||
language-specific annotations. The schema is the ABI contract.
|
||||
|
||||
## Usage
|
||||
|
||||
Build the schema with the fluent Rust builder (ADR-009), compile it
|
||||
once into an [`AlkTypeEngine`], then read/write fields at computed
|
||||
offsets:
|
||||
|
||||
```rust
|
||||
use alktype::{AlkTypeEngine, Endian, LayoutMode, Schema, FieldValue};
|
||||
|
||||
// Channels' 8-byte chunk header: big-endian, packed mode.
|
||||
let mut schema = Schema::struct_()
|
||||
.endian(Endian::Big)
|
||||
.field("channel_id", Schema::uint32())
|
||||
.field("length", Schema::uint32())
|
||||
.build();
|
||||
|
||||
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Packed)?;
|
||||
|
||||
// Write a frame into a buffer. For fixed-size structs, the byte
|
||||
// positions are a direct read off the layout — channel_id at 0,
|
||||
// length at 4. (For variable-length fields, use LayoutBuilder to
|
||||
// compute positions from known data sizes.)
|
||||
let mut buf = vec![0u8; 8];
|
||||
alktype::data_access::write_u32(&mut buf, 0, 42, "channel_id", Endian::Big)?;
|
||||
alktype::data_access::write_u32(&mut buf, 4, 7, "length", Endian::Big)?;
|
||||
|
||||
// Validate the bytes against the schema in one call.
|
||||
engine.validate_bytes(&buf)?; // materializes a Value, then validates
|
||||
|
||||
// Read the frame back sequentially (packed mode is sequential by
|
||||
// construction — variable-length fields shift subsequent fields).
|
||||
let mut reader = engine.sequential_reader().expect("packed mode");
|
||||
let (name, value) = reader.read_next(&buf)?.expect("first field");
|
||||
assert_eq!(name, "channel_id");
|
||||
assert_eq!(value, FieldValue::U32(42));
|
||||
# Ok::<(), alktype::AlkTypeError>(())
|
||||
```
|
||||
|
||||
Schemas may also be authored as plain `serde_json::json!{...}` literals
|
||||
and passed directly to `AlkTypeEngine::compile` — the builder is a
|
||||
construction convenience, not a requirement.
|
||||
|
||||
## The 19 `AlkType:*` kinds
|
||||
|
||||
| Kind | Rust type | Size | Notes |
|
||||
|------|-----------|-----:|-------|
|
||||
| `AlkType:Int8` | `i8` | 1 | |
|
||||
| `AlkType:Int16` | `i16` | 2 | endian-sensitive |
|
||||
| `AlkType:Int32` | `i32` | 4 | endian-sensitive |
|
||||
| `AlkType:Int64` | `i64` | 8 | endian-sensitive; JSON precision caveat (ADR-005) |
|
||||
| `AlkType:Uint8` | `u8` | 1 | |
|
||||
| `AlkType:Uint16` | `u16` | 2 | endian-sensitive |
|
||||
| `AlkType:Uint32` | `u32` | 4 | endian-sensitive; also the enum/string/bytes length-prefix width |
|
||||
| `AlkType:Uint64` | `u64` | 8 | endian-sensitive; JSON precision caveat (ADR-005) |
|
||||
| `AlkType:Float32` | `f32` | 4 | endian-sensitive; NaN/inf rejected by validator |
|
||||
| `AlkType:Float64` | `f64` | 8 | endian-sensitive; NaN/inf rejected by validator |
|
||||
| `AlkType:Boolean` | `bool` | 1 | |
|
||||
| `AlkType:Enum` | `u32` index | 4 | index into the schema's `"enum"` array |
|
||||
| `AlkType:String` | length-prefixed UTF-8 | 4 + N | `[length: u32][bytes]` by default |
|
||||
| `AlkType:Bytes` | length-prefixed raw bytes | 4 + N | `[length: u32][bytes]` by default |
|
||||
| `AlkType:Timestamp` | length-prefixed RFC 3339 | 4 + N | non-strict string check (see inline docs) |
|
||||
| `AlkType:Struct` | record of fields | composite | nested; field paths are dotted (`"header.version"`) |
|
||||
| `AlkType:Union` | tagged union | composite | byte-offset or field-name discriminator |
|
||||
| `AlkType:Array` | repeated element | composite | fixed-size elements with stride, or variable count |
|
||||
| `AlkType:Record` | string-keyed map | composite | `[count: u32][key, value]...` |
|
||||
|
||||
The engine recognizes a kind when the schema object has a key starting
|
||||
with `AlkType:` whose value is `true` (the boolean shorthand) or an
|
||||
annotation object (e.g. `{ "AlkType:String": { "encoding": "offset-indirect" } }`).
|
||||
|
||||
## Two layout modes
|
||||
|
||||
The consumer selects the layout mode at engine construction time via
|
||||
`AlkTypeEngine::compile(schema, mode)`. The same schema can be compiled
|
||||
in either mode. Decided in ADR-002.
|
||||
|
||||
| Mode | Use case | Read API | Write API |
|
||||
|------|----------|----------|-----------|
|
||||
| **Packed** (`LayoutMode::Packed`) | Protocol wire formats (SFTP, channels, TTY) — fields packed with no alignment padding; variable-length fields shift subsequent fields | [`SequentialReader`] (walks fields in order) | [`LayoutBuilder`] (computes positions from known data sizes) |
|
||||
| **Aligned** (`LayoutMode::Aligned`) | mmap-friendly formats (metatensor, safetensors) — fixed positions with natural alignment padding; variable-length data lives outside the static layout | [`OffsetMap`] (random access by field path) | `OffsetMap` (write at known offsets) |
|
||||
|
||||
### Variable-length handling
|
||||
|
||||
- **Packed mode**: `[length: u32][data]` inline by default. The
|
||||
`LayoutBuilder` takes actual data sizes to compute positions; the
|
||||
`SequentialReader` reads the length prefix to find the data extent.
|
||||
- **Aligned mode**: a 4-byte length prefix sits at a known offset; the
|
||||
variable data is not part of the static layout. Offset indirection
|
||||
(the metatensor blob pattern: `{offset, length}` pointing into a
|
||||
separate data region) is opt-in via the `encoding` annotation.
|
||||
|
||||
### TUnion discriminators
|
||||
|
||||
`AlkType:Union` supports two discriminator kinds (ADR-003):
|
||||
|
||||
- **Byte-offset** — a fixed-size integer at a known byte offset. The
|
||||
SFTP `Packet` pattern: byte 0 is the type byte, bytes 1..N are the
|
||||
variant struct. Mapping keys are stringified integers.
|
||||
- **Field-name** — a named field within the struct. The TypeBox
|
||||
`typedef.ts` pattern. Mapping keys are string values matching the
|
||||
discriminator field's value.
|
||||
|
||||
## Endianness
|
||||
|
||||
Per-schema, default little-endian. Set `"endian": "big"` on the
|
||||
top-level schema (or via `Schema::endian(Endian::Big)`) and the engine
|
||||
byte-swaps every multi-byte read/write accordingly. SFTP consumers
|
||||
specify big-endian; channels' chunk header is big-endian.
|
||||
|
||||
## Validation
|
||||
|
||||
Two entry points on [`AlkTypeEngine`], one underlying `jsonschema`
|
||||
validator (ADR-010):
|
||||
|
||||
- `validate_json(&Value)` / `is_valid_json(&Value)` — for already-parsed
|
||||
JSON (call's payload schemas).
|
||||
- `validate_bytes(&[u8])` — materializes a `Value` tree from the bytes
|
||||
via the layout engine, then validates that `Value`. Single-call binary
|
||||
buffer validation.
|
||||
|
||||
The validator is compiled once at load time; access-time validation is
|
||||
a fast `is_valid()` check. High-throughput paths can skip validation;
|
||||
security-sensitive paths can validate every frame.
|
||||
|
||||
## Crate independence
|
||||
|
||||
`alktype` does **not** depend on any application or networking crate.
|
||||
It defines its own types (`AlkTypeError`, `AlkTypeEngine`, `FieldValue`,
|
||||
etc.) and is usable in contexts where networking doesn't exist — CLI
|
||||
tools, test harnesses, schema-building utilities, and WASM targets. The
|
||||
upcoming `alkcall` crate (the `alknet-call` + `alknet-channels`
|
||||
unification) depends on `alktype` for both binary layout and JSON
|
||||
payload schemas; `alktype` knows nothing about `alkcall`.
|
||||
|
||||
## Schemas as untrusted input
|
||||
|
||||
The crate treats schemas as untrusted input. A malformed schema
|
||||
returns `AlkTypeError::Schema` / `AlkTypeError::Offset` from any
|
||||
engine path — never a panic. This matters for hub/spoke topologies
|
||||
where the remote peer provides the schema (e.g. `alkcall` accepting an
|
||||
`OperationSpec` from an arbitrary internet peer). All `unreachable!()`
|
||||
sites in production code were converted to `Err` ahead of v0.1.0
|
||||
(review #002, L2).
|
||||
|
||||
## Documentation
|
||||
|
||||
Architecture documentation lives under [`docs/architecture/`](docs/architecture/):
|
||||
|
||||
- [Overview](docs/architecture/overview.md) — purpose, "schema is the
|
||||
format" principle, dependencies, consumers, scope boundaries
|
||||
- [Schema layer](docs/architecture/schema-layer.md) — the 19 kinds,
|
||||
jsonschema custom keyword integration, schema annotations
|
||||
- [Layout engine](docs/architecture/layout-engine.md) — offset
|
||||
computation, the two layout modes, alignment, endianness
|
||||
- [Data access](docs/architecture/data-access.md) — read/write
|
||||
functions, TUnion dispatch, field paths, zero-copy access
|
||||
- [Validation](docs/architecture/validation.md) — custom keyword
|
||||
validators, `AlkTypeError`, load-time vs access-time validation
|
||||
- [Builder](docs/architecture/builder.md) — fluent Rust API for
|
||||
constructing alktype JSON Schemas at runtime
|
||||
- [Architecture decisions (ADRs)](docs/architecture/decisions/) —
|
||||
purpose/scope, two layout modes, schema annotations, error handling,
|
||||
int64/uint64 kinds, packed-mode read factory, TUnion in aligned mode,
|
||||
builder API, `validate_bytes`
|
||||
|
||||
## License
|
||||
|
||||
Licensed under either of
|
||||
|
||||
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
|
||||
- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
|
||||
|
||||
at your option.
|
||||
|
||||
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: resolved (M1, M2, L1, L2, L3); open (N1, N2)
|
||||
status: resolved (M1, M2, L1, L2, L3, N1, N2)
|
||||
last_updated: 2026-08-11
|
||||
reviewed_artifacts:
|
||||
- src/lib.rs
|
||||
@@ -514,14 +514,20 @@ the `Validation` variant and `None` for the others. The existing
|
||||
and `source_returns_some_for_validation_variant` (new). ~6 lines +
|
||||
~10 lines of tests.
|
||||
|
||||
### Deferred
|
||||
### Deferred → Resolved (docs sweep)
|
||||
|
||||
- **N1** (non-strict `is_rfc3339_timestamp`): documented as "simple"
|
||||
in the existing doc comment. A strict implementation would add a
|
||||
`chrono` or `time` dependency, not worth it for 0.1.0. Will add an
|
||||
explicit "non-strict" note in the docs sweep.
|
||||
- **N2** (`FieldValue::Bytes` for `Record`): API asymmetry, not a
|
||||
bug. Revisit if the alkcall consumer finds it awkward.
|
||||
- **N1** (non-strict `is_rfc3339_timestamp`): documented as non-strict
|
||||
in the function's doc comment. Lists the specific gaps (day-of-month
|
||||
per month, seconds range, leap seconds) and points consumers needing
|
||||
strict validation to `chrono` or `time`. ~10 lines of doc in
|
||||
`src/validation.rs`. A strict implementation would add a dependency,
|
||||
not worth it for 0.1.0.
|
||||
- **N2** (`FieldValue::Bytes` for `Record`): documented as a known
|
||||
asymmetry in the `FieldValue` enum doc and on `read_record_value`.
|
||||
Notes that every other composite kind returns a typed descriptor while
|
||||
`Record` returns `Bytes`, and flags the possibility of a future
|
||||
`FieldValue::Record` variant. ~12 lines of doc across
|
||||
`src/sequential_reader.rs`.
|
||||
|
||||
### L2 (`unreachable!` → `Err`) — resolved (follow-up)
|
||||
|
||||
@@ -567,6 +573,5 @@ remaining hit at `offset_map.rs:688` is inside a `#[test]` fn,
|
||||
guarded by `assert!(matches!(...))` on the line above).
|
||||
|
||||
After M1, M2, L1, L2, and L3, the remaining open findings (N1, N2)
|
||||
are both deferrable to the docs sweep. The crate is ready for the
|
||||
pre-publish docs sweep (README, inline doc cleanup for docs.rs) and
|
||||
the final sanity check.
|
||||
were resolved in the pre-publish docs sweep. All 7 findings are now
|
||||
closed. The crate is ready for the final sanity check and publish.
|
||||
+2
-1
@@ -431,7 +431,8 @@ impl Schema {
|
||||
}
|
||||
|
||||
/// TUnion discriminator, builder-friendly form. Mirrors
|
||||
/// [`DiscriminatorKind`] but constructed via the builder API.
|
||||
/// [`crate::schema::DiscriminatorKind`] but constructed via the builder
|
||||
/// API.
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum Discriminator {
|
||||
/// Byte-offset discriminator. `offset` is the byte position; `disc_type`
|
||||
|
||||
@@ -21,6 +21,14 @@ use serde_json::Value;
|
||||
/// [`FieldValue::Array`]) return layout descriptors; the consumer
|
||||
/// recurses with a fresh [`SequentialReader`] scoped to the
|
||||
/// reported byte range.
|
||||
///
|
||||
/// **Known asymmetry**: `AlkType:Record` returns
|
||||
/// [`FieldValue::Bytes`] covering the record's byte range, not a typed
|
||||
/// `Record { ... }` variant. The consumer recurses into the record's
|
||||
/// value schema by walking the borrowed slice. Every other composite
|
||||
/// kind returns a typed descriptor; `Record` is the exception (review
|
||||
/// #002, N2). A future revision may add a `FieldValue::Record`
|
||||
/// variant; for v0.1.0 the `Bytes` form is stable.
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum FieldValue<'a> {
|
||||
/// `AlkType:Int8`.
|
||||
@@ -684,6 +692,10 @@ fn walk_variable_array_size(
|
||||
/// entries of `[key_len: u32][key_bytes][value]`. Returns the total size
|
||||
/// consumed. The reader does not decode the entries — the consumer
|
||||
/// recurses into the record's value schema.
|
||||
///
|
||||
/// Returns the record's byte range as [`FieldValue::Bytes`] (the one
|
||||
/// composite kind that does not return a typed descriptor — see the
|
||||
/// `FieldValue` enum doc for the known asymmetry).
|
||||
fn read_record_value<'a>(
|
||||
buffer: &'a [u8],
|
||||
root_schema: &Value,
|
||||
|
||||
@@ -348,6 +348,17 @@ impl Keyword for TimestampValidator {
|
||||
|
||||
/// Simple RFC 3339 / ISO 8601 datetime check: `YYYY-MM-DDTHH:MM:SS`
|
||||
/// optionally followed by `Z` or a timezone offset.
|
||||
///
|
||||
/// **Non-strict.** This is a structural sanity check, not a strict RFC 3339
|
||||
/// validator. It does NOT validate:
|
||||
/// - Day-of-month per month (Feb 31, Apr 31 pass).
|
||||
/// - Seconds range (only hour and minute are range-checked).
|
||||
/// - Leap seconds.
|
||||
///
|
||||
/// For strict RFC 3339 validation, use a dedicated crate (`chrono`, `time`).
|
||||
/// `AlkType:Timestamp` is a length-prefixed string at the binary level;
|
||||
/// strict datetime validation is the consumer's responsibility if they
|
||||
/// need it. See review #002, N1.
|
||||
fn is_rfc3339_timestamp(s: &str) -> bool {
|
||||
let parts: Vec<&str> = s.splitn(2, 'T').collect();
|
||||
if parts.len() != 2 {
|
||||
|
||||
Reference in new issue
Block a user