Add durable namespace control, Iroh-backed source transfer, session-independent publication lifetime, and wire managed Myelin jobs and bindings through the reusable data-plane API.
Architecture enforcement:
- Install a repository-owned rustc wrapper for ordinary cargo check,
build, and test commands. Resolve compiler item identities so renamed
imports and helper wrappers cannot hide spawning, timing, blocking,
polling, thread, or runtime-driving capabilities.
- Define the execution-owner crates and reject dependencies from those
substrates back into Myelin policy. Add compile-pass and compile-fail
contracts for actor helpers, execution owners, test waits, forbidden
capabilities, suppression attempts, and owner dependency inversions.
Execution ownership:
- Add engine-owned actor timers with cancellation and generation identity,
then migrate lifecycle deadlines and protocol ticks off application
tasks. Keep networking, process output, telemetry, and blocking provider
calls in their approved I/O substrates.
- Move process spawn, wait, signal, Unix listener, and output-following
mechanics into swactor-process. Isolate Vast.ai blocking HTTP mechanics
behind its adapter while actors retain retry, recovery, and provisioning
decisions.
Myelin control flow:
- Rework manual control, worker lifecycle, provisioning, provider recovery,
job deployment, distribution, edge orchestration, and shutdown as actor
state transitions and typed effects. Preserve durable provider adoption
and command outcomes across graceful and abrupt restarts.
- Replace controller loops and timer-forwarding tasks with actor messages;
leave substrate tasks as cancellable observation streams with no durable
policy state.
Properties and resource ownership:
- Add deterministic engine and component properties, a stateful mock-VastAI
lifecycle model, persisted regression cases, controlled fault injection,
and a bounded nightly workflow covering restart and teardown behavior.
- Terminate reply observers, cancel telemetry collectors, bound dashboard
projections, and release child observers, file descriptors, process
records, and inode-verified Unix sockets on every terminal path.
Verified with the compiler-policy contracts, 105 Myelin library tests, 32
swactor-process tests, telemetry cancellation contracts, randomized
stateful restart cases, cargo check, and formatting checks.
Add actor-backed manual node provisioning, control-plane endpoints, and fleet UI assets with durable provider lifecycle handling.
Simplify Myelin orchestration, node runtime, staging, and telemetry paths while removing obsolete engine-builder, dashboard-view, and local-mock implementations.
Align runtime delivery, data-plane, distribution, job-runner, process, telemetry, dashboard, Vast.ai integrations, and their tests with the revised actor and transport contracts.
Replace the single-purpose chat runtime with a persistent fleet daemon that provisions, adopts, and controls nodes through the dashboard.
Add distributed job-runner actors and provider-backed deployment so jobs can materialize workspaces, execute remotely, and return outputs over iroh.
The crate is the per-node metrics/logging pipe with a universal
subscriber endpoint, but "datastream" kept getting misread as a general
messaging plane. Rename crate, module paths, and public API
(`DatastreamEndpoint` → `TelemetryEndpoint`, etc.) so misuse is visible
on sight.
Renamed contracts (all in-repo producers/consumers migrated):
- env vars `MYELIN_DATASTREAM_*` → `MYELIN_TELEMETRY_*`
- artifact `datastream.ndjson` → `telemetry.ndjson`
- actor names `telemetry-publisher` / `telemetry-sink`
- wire ALPN `swactor/telemetry/0`
- `DATASTREAM_SPEC.md` → `TELEMETRY_SPEC.md`
Also fixes two latent test breaks: `process` and `iroh-driver` tests
imported `DatastreamEvent` from the crate root, which was never
re-exported; they now use the observer path `telemetry::frame::`.
Introduce a pure, level-triggered reconciler in `crates/provisioning`
that drives a declared cluster shape toward convergence over the
existing node lifecycle, replacing the edge-triggered imperative node
orchestration in `apps/myelin`.
- `reconcile`/`reconcile_node`/`observe`: pure decider and observation
folder with stable logical-node identity, per-attempt operation
identity, and deterministic retry backoff; `ClusterDriver` is the sole
writer of observed state, coalescing triggers, recording operations as
pending before dispatch, and scheduling timed requeues.
- `IdempotentEffectExecutor`: deduplicates submissions by
`(run_id, logical_node_id, attempt)` and runs provider work on the
engine-hosted blocking substrate, never blocking a reconcile pass.
- Myelin integration: `MyelinEffectBackend` bridges `ProvisionPlugin` to
the executor contract; `LocalProcessPlugin`/`LocalDockerPlugin`
provider adapters; `ProvisionedClusterGuard` pumps triggers,
observations, and due operations.
- Retire the imperative acquire/bootstrap/teardown sequencing across
`apps/myelin` orchestration, staging, observability, and provider
adapters in favor of the declarative driver.
- Move the reconciler specification to `docs/specs/archive`.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
