swactor/crates/datastream/Cargo.toml

[package]
name = "datastream"
version = "0.1.0"
edition = "2024"

[dependencies]
swactor = { path = "../..", features = ["serde", "transport"] }
swactor-transport = { path = "../transport" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
iroh = "0.98"
crossbeam-channel = "0.5"

[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
feat: datastream feature cleaning Promote pipeline-parallel-inference to a first-class app and consolidate observability on the datastream wire, decoupling the dashboard crate from `distribution`. - apps/pipeline-parallel-inference: move the example out of `examples/` into `apps/` as its own workspace, rename binaries to `pp-worker`/`pp-orchestrator`, and strip release binaries - cluster: add `ClusterNode`, a synchronous facade over the actorized distribution protocol (IrohDriver + per-node Runtime hosting Swim/Registry/Metadata/Directory actors with a `MembershipFanout`), replacing ad-hoc `driver.node()`/`tick()` call sites - fleet: add per-node fleet telemetry that ships identity/resource records as `DatastreamFrame`s over the cluster transport to the orchestrator's `DatastreamSink`, folded into a `FleetView` on a 3s tick - provision: add best-effort, opt-in SSH boot-phase telemetry (`PP_DEPLOY_KEY`) that streams rented-node boot logs onto the orchestrator's datastream as `proc.boot.<stage>.*` - dashboard: rewire the crate dependency from `distribution` to `datastream`, drop the standalone `swactor-datastream-dashboard` binary, and rewrite `datastream_source.rs` to demux per-node frames into Overview/Distribution/Fleet views with live-node TTL filtering - distribution: refresh dist/netmap plugin copy and README from "Kademlia routing" to gossip-directory terminology Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com> 2026-06-09 09:29:07 +00:00			`[package]`
			`name = "datastream"`
			`version = "0.1.0"`
			`edition = "2024"`

			`[dependencies]`
			`swactor = { path = "../..", features = ["serde", "transport"] }`
			`swactor-transport = { path = "../transport" }`
			`serde = { version = "1", features = ["derive"] }`
			`serde_json = "1"`
feat: rework datastream into catalog events, add gguf metadata - Rework the datastream endpoint into a catalog/event model (DatastreamEvent, SubscriptionRequest, channel/stream descriptors, DatastreamPublisherActor) across endpoint/frame/mux/wire/views. - Add gguf_metadata planning reader, a dashboard hardware view, and iroh-driver datastream transport. - Grow mvp-system orchestrator/worker_node bins and staging/provisioning; rename mvp_one_node_chat->mvp_chat; extend tinygrad worker. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com> 2026-07-12 06:14:34 +00:00			`iroh = "0.98"`
refactor(datastream): cleanup Trim the datastream crate to its dumb-pipe core: drop the frame-timing sidecar, make mux positions gap-free for accepted frames, and simplify the endpoint fanout. - mux: defer position assignment from submit to drain so an overflowed submission no longer consumes a position (no synthetic gaps); submit now returns bool and the Mutex<Receiver> is removed since positions are assigned only to accepted frames - endpoint/mux: switch from std::sync::mpsc to crossbeam-channel and drop per-event event_matches_request filtering — request filters now apply only to the initial catalog snapshot, and future events broadcast to all subscribers - endpoint (DeliveryFanout): snapshot sender handles under the lock and deliver outside it via FanoutTarget/FanoutReport, so large batches or slow subscribers no longer block subscribe/snapshot control-plane ops - emit/endpoint/producer: drop set_frame_timing_enabled/frame_timing_enabled and the Position return from submit_record/submit_text/submit_bytes, and add submit_text_owned taking owned String - timing/lib/spec: delete the timing module and FRAME_TIME_CHANNEL/FRAME_TIME_CHANNEL_ID/FrameTimeSample re-exports (including the auto-registered timing channel in ChannelCatalogState) and renumber the DATASTREAM_SPEC.md section references across frame/ingest/store/mux - tests: remove the 567-line shared datastream_support/mod.rs harness Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com> 2026-07-18 09:38:16 +00:00			`crossbeam-channel = "0.5"`
feat: datastream feature cleaning Promote pipeline-parallel-inference to a first-class app and consolidate observability on the datastream wire, decoupling the dashboard crate from `distribution`. - apps/pipeline-parallel-inference: move the example out of `examples/` into `apps/` as its own workspace, rename binaries to `pp-worker`/`pp-orchestrator`, and strip release binaries - cluster: add `ClusterNode`, a synchronous facade over the actorized distribution protocol (IrohDriver + per-node Runtime hosting Swim/Registry/Metadata/Directory actors with a `MembershipFanout`), replacing ad-hoc `driver.node()`/`tick()` call sites - fleet: add per-node fleet telemetry that ships identity/resource records as `DatastreamFrame`s over the cluster transport to the orchestrator's `DatastreamSink`, folded into a `FleetView` on a 3s tick - provision: add best-effort, opt-in SSH boot-phase telemetry (`PP_DEPLOY_KEY`) that streams rented-node boot logs onto the orchestrator's datastream as `proc.boot.<stage>.*` - dashboard: rewire the crate dependency from `distribution` to `datastream`, drop the standalone `swactor-datastream-dashboard` binary, and rewrite `datastream_source.rs` to demux per-node frames into Overview/Distribution/Fleet views with live-node TTL filtering - distribution: refresh dist/netmap plugin copy and README from "Kademlia routing" to gossip-directory terminology Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com> 2026-06-09 09:29:07 +00:00
			`[target.'cfg(target_os = "linux")'.dependencies]`
			`libc = "0.2"`