Add durable namespace control, Iroh-backed source transfer, session-independent publication lifetime, and wire managed Myelin jobs and bindings through the reusable data-plane API.
Architecture enforcement:
- Install a repository-owned rustc wrapper for ordinary cargo check,
build, and test commands. Resolve compiler item identities so renamed
imports and helper wrappers cannot hide spawning, timing, blocking,
polling, thread, or runtime-driving capabilities.
- Define the execution-owner crates and reject dependencies from those
substrates back into Myelin policy. Add compile-pass and compile-fail
contracts for actor helpers, execution owners, test waits, forbidden
capabilities, suppression attempts, and owner dependency inversions.
Execution ownership:
- Add engine-owned actor timers with cancellation and generation identity,
then migrate lifecycle deadlines and protocol ticks off application
tasks. Keep networking, process output, telemetry, and blocking provider
calls in their approved I/O substrates.
- Move process spawn, wait, signal, Unix listener, and output-following
mechanics into swactor-process. Isolate Vast.ai blocking HTTP mechanics
behind its adapter while actors retain retry, recovery, and provisioning
decisions.
Myelin control flow:
- Rework manual control, worker lifecycle, provisioning, provider recovery,
job deployment, distribution, edge orchestration, and shutdown as actor
state transitions and typed effects. Preserve durable provider adoption
and command outcomes across graceful and abrupt restarts.
- Replace controller loops and timer-forwarding tasks with actor messages;
leave substrate tasks as cancellable observation streams with no durable
policy state.
Properties and resource ownership:
- Add deterministic engine and component properties, a stateful mock-VastAI
lifecycle model, persisted regression cases, controlled fault injection,
and a bounded nightly workflow covering restart and teardown behavior.
- Terminate reply observers, cancel telemetry collectors, bound dashboard
projections, and release child observers, file descriptors, process
records, and inode-verified Unix sockets on every terminal path.
Verified with the compiler-policy contracts, 105 Myelin library tests, 32
swactor-process tests, telemetry cancellation contracts, randomized
stateful restart cases, cargo check, and formatting checks.
Subscribe to each worker telemetry pull endpoint during runtime readiness, retry interrupted streams, and remove the obsolete telemetry actor-address path.
Make shutdown state sticky, clean local Docker resources on Ctrl+C, validate fleet controls, and keep dashboard node state honest under failure. Wait for bidirectional iroh routes before job submission and render the orchestrator role as a card header.
Replace the per-attempt key-file side channel (DEMO_NODE_KEY_FILE,
read_key_report, JoinCheck) with a control-plane announce: node roles
send a tagged gossip frame {attempt, logical_node, key_hex,
endpoint_addr_json} after joining and every heartbeat thereafter.
- iroh-driver: IrohDriver::send_tagged_gossip — fire-and-forget
tag-routed gossip egress for bridge-less clients (reuses cached/join
connections, dials with backoff).
- provisioning: BootstrapMsg::Announce — first delivery while
bootstrapping completes the attempt (collector + exactly-once
Bootstrapped report); duplicates, misrouted attempts, and
terminal-phase announces drop. Unit-tested.
- xtask demo: AnnounceActor decodes the tag-routed frame and forwards
by attempt to the owning bootstrap actor; last_announce_ms is the
wire heartbeat. LocalProcessLogic keeps only process lifecycle.
- --docker: DockerProcessLogic (kind "docker") — attached
"docker run --rm" child on a per-run labeled bridge network
(foreign-node masking: per-container IPs, gateway-dialed
supervisor). Standalone scratch image from the static-musl xtask
binary (37MB), staged one-file build context. The container is
force-removed on every terminal path so a SIGKILLed docker CLI
cannot orphan a running container.
- Cleanup: no volumes/mounts; label-filtered exit sweep (SIGINT and
SIGTERM both drain first) + startup sweep of stale demo resources;
images persist per run token.
Verified live: process kind (kill -> replacement in 3.4s, provision/
remove/kill waves, zero orphans) and docker kind (8-node abuse waves
across docker kill, mid-provision control kills, CLI SIGKILL orphans
force-removed, SIGKILL-crash leftovers swept on restart, clean exits
leave zero containers/networks/CLIs). provisioning 22 + iroh-driver
13 tests pass.
The crate is the per-node metrics/logging pipe with a universal
subscriber endpoint, but "datastream" kept getting misread as a general
messaging plane. Rename crate, module paths, and public API
(`DatastreamEndpoint` → `TelemetryEndpoint`, etc.) so misuse is visible
on sight.
Renamed contracts (all in-repo producers/consumers migrated):
- env vars `MYELIN_DATASTREAM_*` → `MYELIN_TELEMETRY_*`
- artifact `datastream.ndjson` → `telemetry.ndjson`
- actor names `telemetry-publisher` / `telemetry-sink`
- wire ALPN `swactor/telemetry/0`
- `DATASTREAM_SPEC.md` → `TELEMETRY_SPEC.md`
Also fixes two latent test breaks: `process` and `iroh-driver` tests
imported `DatastreamEvent` from the crate root, which was never
re-exported; they now use the observer path `telemetry::frame::`.
2026-08-15 12:18:56 +04:00
Renamed from crates/iroh-driver/src/datastream_transport.rs (Browse further)
