Commit graph

3 commits

Author SHA1 Message Date
564c762be9 feat(data-plane): complete virtual blob namespace
Add durable namespace control, Iroh-backed source transfer, session-independent publication lifetime, and wire managed Myelin jobs and bindings through the reusable data-plane API.
2026-08-22 21:30:59 +04:00
0e2fc2b164 feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.

Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 18:45:10 +04:00
6f058a844c feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.

- Canonical header layout, shared parser, and a host writer that leases
  the header region and control ring disjointly under the arena's own
  placement law, zeroes the ring, stamps its generation, and returns the
  handoff (env map, inheritable arena and wake descriptors, host wake
  eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
  ground truth from fstat, validate through the shared parser, arm
  FD_CLOEXEC on the wake descriptor, close the arena descriptor after
  mapping, and drive main on asyncio with Context.data carrying the
  resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
  record the slice, invariants, and remaining bridge work in the handoff
  doc.

Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 22:33:19 +04:00