Commit graph

4 commits

Author SHA1 Message Date
b06583f598 refactor(datastream): cleanup
Trim the datastream crate to its dumb-pipe core: drop the frame-timing sidecar, make mux positions gap-free for accepted frames, and simplify the endpoint fanout.

- mux: defer position assignment from submit to drain so an overflowed submission no longer consumes a position (no synthetic gaps); submit now returns bool and the Mutex<Receiver> is removed since positions are assigned only to accepted frames
- endpoint/mux: switch from std::sync::mpsc to crossbeam-channel and drop per-event event_matches_request filtering — request filters now apply only to the initial catalog snapshot, and future events broadcast to all subscribers
- endpoint (DeliveryFanout): snapshot sender handles under the lock and deliver outside it via FanoutTarget/FanoutReport, so large batches or slow subscribers no longer block subscribe/snapshot control-plane ops
- emit/endpoint/producer: drop set_frame_timing_enabled/frame_timing_enabled and the Position return from submit_record/submit_text/submit_bytes, and add submit_text_owned taking owned String
- timing/lib/spec: delete the timing module and FRAME_TIME_CHANNEL/FRAME_TIME_CHANNEL_ID/FrameTimeSample re-exports (including the auto-registered timing channel in ChannelCatalogState) and renumber the DATASTREAM_SPEC.md section references across frame/ingest/store/mux
- tests: remove the 567-line shared datastream_support/mod.rs harness

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-18 13:38:16 +04:00
ce57012e11 feat: rework datastream into catalog events, add gguf metadata
- Rework the datastream endpoint into a catalog/event model (DatastreamEvent,
  SubscriptionRequest, channel/stream descriptors, DatastreamPublisherActor) across
  endpoint/frame/mux/wire/views.
- Add gguf_metadata planning reader, a dashboard hardware view, and iroh-driver
  datastream transport.
- Grow mvp-system orchestrator/worker_node bins and staging/provisioning; rename
  mvp_one_node_chat->mvp_chat; extend tinygrad worker.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-12 10:14:34 +04:00
a97864f7ec feat(mvp-system): expand one-node mvp binaries and provisioning
Flesh out mvp_node, mvp_one_node_chat, and mvp_orch_one_node binaries. Add node_image
and relay_provisioning; grow provisioning and vastai. Tune datastream mux/timing/emit.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-05 13:59:51 +04:00
0ffc5fd275 feat: add mvp provisioning subsystem and datastream transport
mvp-system: provisioner actor, provisioning module, node_agent, dashboard_view,
observability_surface; expand gpu_worker ctl/ingress/egress and run_plan.
iroh-driver: replace relay binary with datastream_transport; datastream gains endpoint
abstraction. Archive pipeline-parallel-inference app to old-pipeline-parallel-inference.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-25 16:30:18 +04:00