alkdev/alktype
6
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Pre-publish docs sweep: README, AGENTS.md, licenses, inline doc fixes

Browse Source 
- Add README.md reflecting the v0.1.0 state: 19 AlkType kinds, two
  layout modes, builder + AlkTypeEngine usage example (verified to
  compile and run), validation entry points, crate independence,
  untrusted-schemas guarantee, docs pointers. Mirrors the alkvault
  README structure.
- Add AGENTS.md with alktype-specific git workflow, project
  conventions (no comments, AlkTypeError, untrusted schemas, overflow
  safety, no async, no feature flags, wasm-clean, preserve_order
  load-bearing, no unsafe), verification commands, and ADR/OQ index.
  Blocks auto-commit on semver-relevant public API changes per the
  crates.io 0.1.0 contract.
- Add LICENSE-MIT and LICENSE-APACHE (dual MIT/Apache-2.0, matching
  alkvault and the Cargo.toml license field).
- Cargo.toml: add readme, keywords, categories, rust-version = "1.85".
- Fix broken intra-doc link in builder.rs: DiscriminatorKind ->
  crate::schema::DiscriminatorKind (cargo doc now warning-free).
- N1 (review #002): document is_rfc3339_timestamp as non-strict in the
  function doc comment. Lists the specific gaps (day-of-month per
  month, seconds range, leap seconds) and points consumers needing
  strict validation to chrono/time.
- N2 (review #002): document the FieldValue::Bytes-for-Record API
  asymmetry in the FieldValue enum doc and on read_record_value.
- .opencode/agents/implementation-specialist.md: point to AGENTS.md
  for full convention details (matches the alkvault pattern).
- review #002: mark N1/N2 resolved; all 7 findings now closed.

Verification:
- cargo test --release: 396 tests pass (310 crate + 86 integration)
- cargo clippy --all-targets -- -D warnings: clean
- cargo doc --no-deps: clean (no broken intra-doc link warnings)
- cargo build --target wasm32-unknown-unknown --release: clean
- cargo publish --dry-run --allow-dirty: clean
This commit is contained in:
glm-5.2 committed 2026-08-11 09:33:33 +00:00
1 parent 5f88bca0d8
commit fb3a27f974
10 files changed
+641 -13
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+206
View File
@@ -0,0 +1,206 @@
# alktype
The binary struct engine: a small Rust crate that takes a JSON Schema
with `AlkType:*` custom keywords and produces an offset map, read/write
functions, and validation — all driven by the schema. The schema is the
format definition; the engine is generic.
`alktype` is a standalone crate with **two dependencies**: `jsonschema`
(for validation) and `serde_json` (for schema parsing). No tokio, no
platform deps, no `unsafe`. Compiles to `wasm32-unknown-unknown`.
## What it is
A JSON Schema annotated with `AlkType:*` custom keywords serves three
roles simultaneously:
| Role | Mechanism | When |
|------|-----------|------|
| **Validation spec** | `jsonschema` custom keywords | Load time (build validator), access time (validate buffer) |
| **Layout spec** | Offset computation from type sizes + field order | Load time (build offset map / packed layout) |
| **Data access** | Read/write at computed offsets | Access time (read field, write field) |
No separate format definition, no separate parser, no separate
validator. The schema is the single source of truth for the binary
format. Adding a new field to a protocol is adding a property to the
schema JSON — the engine computes the new offsets automatically.
This is the same principle as `#[repr(C)]` struct field access, but at
runtime from a portable JSON Schema instead of at compile time from
language-specific annotations. The schema is the ABI contract.
## Usage
Build the schema with the fluent Rust builder (ADR-009), compile it
once into an [`AlkTypeEngine`], then read/write fields at computed
offsets:
```rust
use alktype::{AlkTypeEngine, Endian, LayoutMode, Schema, FieldValue};
// Channels' 8-byte chunk header: big-endian, packed mode.
let mut schema = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Packed)?;
// Write a frame into a buffer. For fixed-size structs, the byte
// positions are a direct read off the layout — channel_id at 0,
// length at 4. (For variable-length fields, use LayoutBuilder to
// compute positions from known data sizes.)
let mut buf = vec![0u8; 8];
alktype::data_access::write_u32(&mut buf, 0, 42, "channel_id", Endian::Big)?;
alktype::data_access::write_u32(&mut buf, 4, 7, "length", Endian::Big)?;
// Validate the bytes against the schema in one call.
engine.validate_bytes(&buf)?; // materializes a Value, then validates
// Read the frame back sequentially (packed mode is sequential by
// construction — variable-length fields shift subsequent fields).
let mut reader = engine.sequential_reader().expect("packed mode");
let (name, value) = reader.read_next(&buf)?.expect("first field");
assert_eq!(name, "channel_id");
assert_eq!(value, FieldValue::U32(42));
# Ok::<(), alktype::AlkTypeError>(())
```
Schemas may also be authored as plain `serde_json::json!{...}` literals
and passed directly to `AlkTypeEngine::compile` — the builder is a
construction convenience, not a requirement.
## The 19 `AlkType:*` kinds
| Kind | Rust type | Size | Notes |
|------|-----------|-----:|-------|
| `AlkType:Int8` | `i8` | 1 | |
| `AlkType:Int16` | `i16` | 2 | endian-sensitive |
| `AlkType:Int32` | `i32` | 4 | endian-sensitive |
| `AlkType:Int64` | `i64` | 8 | endian-sensitive; JSON precision caveat (ADR-005) |
| `AlkType:Uint8` | `u8` | 1 | |
| `AlkType:Uint16` | `u16` | 2 | endian-sensitive |
| `AlkType:Uint32` | `u32` | 4 | endian-sensitive; also the enum/string/bytes length-prefix width |
| `AlkType:Uint64` | `u64` | 8 | endian-sensitive; JSON precision caveat (ADR-005) |
| `AlkType:Float32` | `f32` | 4 | endian-sensitive; NaN/inf rejected by validator |
| `AlkType:Float64` | `f64` | 8 | endian-sensitive; NaN/inf rejected by validator |
| `AlkType:Boolean` | `bool` | 1 | |
| `AlkType:Enum` | `u32` index | 4 | index into the schema's `"enum"` array |
| `AlkType:String` | length-prefixed UTF-8 | 4 + N | `[length: u32][bytes]` by default |
| `AlkType:Bytes` | length-prefixed raw bytes | 4 + N | `[length: u32][bytes]` by default |
| `AlkType:Timestamp` | length-prefixed RFC 3339 | 4 + N | non-strict string check (see inline docs) |
| `AlkType:Struct` | record of fields | composite | nested; field paths are dotted (`"header.version"`) |
| `AlkType:Union` | tagged union | composite | byte-offset or field-name discriminator |
| `AlkType:Array` | repeated element | composite | fixed-size elements with stride, or variable count |
| `AlkType:Record` | string-keyed map | composite | `[count: u32][key, value]...` |
The engine recognizes a kind when the schema object has a key starting
with `AlkType:` whose value is `true` (the boolean shorthand) or an
annotation object (e.g. `{ "AlkType:String": { "encoding": "offset-indirect" } }`).
## Two layout modes
The consumer selects the layout mode at engine construction time via
`AlkTypeEngine::compile(schema, mode)`. The same schema can be compiled
in either mode. Decided in ADR-002.
| Mode | Use case | Read API | Write API |
|------|----------|----------|-----------|
| **Packed** (`LayoutMode::Packed`) | Protocol wire formats (SFTP, channels, TTY) — fields packed with no alignment padding; variable-length fields shift subsequent fields | [`SequentialReader`] (walks fields in order) | [`LayoutBuilder`] (computes positions from known data sizes) |
| **Aligned** (`LayoutMode::Aligned`) | mmap-friendly formats (metatensor, safetensors) — fixed positions with natural alignment padding; variable-length data lives outside the static layout | [`OffsetMap`] (random access by field path) | `OffsetMap` (write at known offsets) |
### Variable-length handling
- **Packed mode**: `[length: u32][data]` inline by default. The
`LayoutBuilder` takes actual data sizes to compute positions; the
`SequentialReader` reads the length prefix to find the data extent.
- **Aligned mode**: a 4-byte length prefix sits at a known offset; the
variable data is not part of the static layout. Offset indirection
(the metatensor blob pattern: `{offset, length}` pointing into a
separate data region) is opt-in via the `encoding` annotation.
### TUnion discriminators
`AlkType:Union` supports two discriminator kinds (ADR-003):
- **Byte-offset** — a fixed-size integer at a known byte offset. The
SFTP `Packet` pattern: byte 0 is the type byte, bytes 1..N are the
variant struct. Mapping keys are stringified integers.
- **Field-name** — a named field within the struct. The TypeBox
`typedef.ts` pattern. Mapping keys are string values matching the
discriminator field's value.
## Endianness
Per-schema, default little-endian. Set `"endian": "big"` on the
top-level schema (or via `Schema::endian(Endian::Big)`) and the engine
byte-swaps every multi-byte read/write accordingly. SFTP consumers
specify big-endian; channels' chunk header is big-endian.
## Validation
Two entry points on [`AlkTypeEngine`], one underlying `jsonschema`
validator (ADR-010):
- `validate_json(&Value)` / `is_valid_json(&Value)` — for already-parsed
JSON (call's payload schemas).
- `validate_bytes(&[u8])` — materializes a `Value` tree from the bytes
via the layout engine, then validates that `Value`. Single-call binary
buffer validation.
The validator is compiled once at load time; access-time validation is
a fast `is_valid()` check. High-throughput paths can skip validation;
security-sensitive paths can validate every frame.
## Crate independence
`alktype` does **not** depend on any application or networking crate.
It defines its own types (`AlkTypeError`, `AlkTypeEngine`, `FieldValue`,
etc.) and is usable in contexts where networking doesn't exist — CLI
tools, test harnesses, schema-building utilities, and WASM targets. The
upcoming `alkcall` crate (the `alknet-call` + `alknet-channels`
unification) depends on `alktype` for both binary layout and JSON
payload schemas; `alktype` knows nothing about `alkcall`.
## Schemas as untrusted input
The crate treats schemas as untrusted input. A malformed schema
returns `AlkTypeError::Schema` / `AlkTypeError::Offset` from any
engine path — never a panic. This matters for hub/spoke topologies
where the remote peer provides the schema (e.g. `alkcall` accepting an
`OperationSpec` from an arbitrary internet peer). All `unreachable!()`
sites in production code were converted to `Err` ahead of v0.1.0
(review #002, L2).
## Documentation
Architecture documentation lives under [`docs/architecture/`](docs/architecture/):
- [Overview](docs/architecture/overview.md) — purpose, "schema is the
format" principle, dependencies, consumers, scope boundaries
- [Schema layer](docs/architecture/schema-layer.md) — the 19 kinds,
jsonschema custom keyword integration, schema annotations
- [Layout engine](docs/architecture/layout-engine.md) — offset
computation, the two layout modes, alignment, endianness
- [Data access](docs/architecture/data-access.md) — read/write
functions, TUnion dispatch, field paths, zero-copy access
- [Validation](docs/architecture/validation.md) — custom keyword
validators, `AlkTypeError`, load-time vs access-time validation
- [Builder](docs/architecture/builder.md) — fluent Rust API for
constructing alktype JSON Schemas at runtime
- [Architecture decisions (ADRs)](docs/architecture/decisions/) —
purpose/scope, two layout modes, schema annotations, error handling,
int64/uint64 kinds, packed-mode read factory, TUnion in aligned mode,
builder API, `validate_bytes`
## License
Licensed under either of
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.