Integrate namespace and source-route lifecycle changes, contextual process cleanup, Python binding updates, and Myelin worker/orchestrator recovery. Keep the shared control contracts, deployment identity fencing, SSH bootstrap adapters, paid admission accounting, and VastAI cleanup implementation together with their consumers.
Migrate Iroh dependencies and telemetry transport/collection with dashboard and demo callsites, workspace build configuration, and actor-control-flow policy updates. This is an intermediate development checkpoint, not paid-provider qualification.
Review verification: contextual_process_guarantees (4 tests), telemetry_transport (4 tests), and shared control contracts (5 tests) passed. Historical five-node redeployment and campaign execution passed individually; complete ordered qualification remains pending.
Stream bounded provider logs and surface provisioning phases through telemetry and the dashboard. Preserve offer-search criteria, use live relay endpoints for remote bootstrap, and add GPU smoke coverage. Exercise orchestrator crash recovery and clean completed checkpoints without dropping active work.
Add durable namespace control, Iroh-backed source transfer, session-independent publication lifetime, and wire managed Myelin jobs and bindings through the reusable data-plane API.
Replace the single-purpose chat runtime with a persistent fleet daemon that provisions, adopts, and controls nodes through the dashboard.
Add distributed job-runner actors and provider-backed deployment so jobs can materialize workspaces, execute remotely, and return outputs over iroh.
The crate is the per-node metrics/logging pipe with a universal
subscriber endpoint, but "datastream" kept getting misread as a general
messaging plane. Rename crate, module paths, and public API
(`DatastreamEndpoint` → `TelemetryEndpoint`, etc.) so misuse is visible
on sight.
Renamed contracts (all in-repo producers/consumers migrated):
- env vars `MYELIN_DATASTREAM_*` → `MYELIN_TELEMETRY_*`
- artifact `datastream.ndjson` → `telemetry.ndjson`
- actor names `telemetry-publisher` / `telemetry-sink`
- wire ALPN `swactor/telemetry/0`
- `DATASTREAM_SPEC.md` → `TELEMETRY_SPEC.md`
Also fixes two latent test breaks: `process` and `iroh-driver` tests
imported `DatastreamEvent` from the crate root, which was never
re-exported; they now use the observer path `telemetry::frame::`.
Introduce the swactor engine: a swactor-owned composite that retains a
selected execution substrate, drives the core runtime, and hosts the
async/blocking/timer work that backs actors. Integrations receive one
cloneable EngineHandle and never construct or borrow a raw Tokio
runtime/handle.
Engine crate (crates/engine):
- The contract: spawn / spawn_blocking / timer / interval / now, a
per-implementation capability model with construction-time binding
(require()), and engine-owned time. The engine owns all progression;
actor handlers stay synchronous and never .await.
- TokioBackend owns the Tokio runtime and schedules core ticks and
supporting futures on it; SteppingBackend is a single-threaded
deterministic scheduler with virtual time (the non-Tokio portability
proof). Core is driven through its existing tick() surface; a
self-rescheduling CoreDriver is installed at construction and is the
sole place permitted to call try_tick.
iroh-driver:
- Receives an EngineHandle instead of a raw Tokio Handle. Accepts,
reads, dials, writes, endpoint construction, and teardown schedule
through it; required capabilities (tasks/timers/io) are validated
before the endpoint binds. Engine-hosted interval pumps drive
actor-bridge, datastream, and edge ingress.
myelin:
- One node/orchestrator engine owns core, protocol tick injection, and
transport progression; the application loop only drains
integration-owned queues. Stage-shard process readers, delayed actor
messages, helper stdout/stderr, prompt RPC, and CPU sampling all
schedule through the engine (spawn_blocking / engine tasks / timers).
- Removed the split-engine APIs: install_actor_bridge_pump(period) and
spawn_protocol_ticker(period) use each component's stored engine;
deleted the no-op pump_network callback and its plumbing; deleted the
dashboard raw-Tokio/standalone-runtime conveniences.
Enforcement:
- A clippy disallowed-methods boundary forbids direct runtime/scheduling/
time/core-driving bypasses, denied in swactor-engine, iroh-driver, and
myelin. Retained excluded uses (VastAI provider, provider process
supervision/log capture, OS-signal/stdin/process-control sequencing)
carry narrow allowances with reasons.
Verification:
- Engine contract + unit tests (incl. the SteppingBackend portability
proof), iroh integration tests (capability rejection before binding,
multi-node actor behavior), and a production execution-composition
smoke test that observes engine-driven actor progress with no ambient
Tokio runtime and no manual tick/pump. Workspace all-target/all-feature
clippy and tests are green.
Specs co-located with their crates: ENGINE_SPEC.md in crates/engine,
IROH_DRIVER_SPEC.md in crates/iroh-driver. VastAI remains explicitly out
of scope pending its separate redesign.
Move iroh_driver and the relay binary out of distribution into a dedicated
crates/iroh-driver (lib re-exports IrohDriver; relay bin renamed). Remove the node crate
and the single-gpu-inference example; drop the docker/datastream demo. Slim
pipeline-parallel vastai.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
