zacheryasc/swactor
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
swactor/crates/mvp-system/Cargo.toml

62 lines
1.4 KiB
TOML
Raw Normal View History

docs(mvp): lay out mvp-system spec and contracts Add MVP_SYSTEM_SPEC plus per-component contract docs (arena manager, device bridge, gpu worker ctl/ingress/egress/process-adapter, orchestrator run-fsm/token-endpoint, run plan, stage controller, tx_rx edge, weights). Scaffold the guarantee tests against the (empty) mvp-system crate. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-23 07:12:45 +00:00
[package]
name = "mvp-system"
version = "0.1.0"
edition = "2024"
publish = false
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
autobins = false
docs(mvp): lay out mvp-system spec and contracts Add MVP_SYSTEM_SPEC plus per-component contract docs (arena manager, device bridge, gpu worker ctl/ingress/egress/process-adapter, orchestrator run-fsm/token-endpoint, run plan, stage controller, tx_rx edge, weights). Scaffold the guarantee tests against the (empty) mvp-system crate. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-23 07:12:45 +00:00
refactor: consolidate dist tests, add mvp actors and e2e Fold the fragmented distribution swim/routing/gossip tests into swim_core, routing, and swim_actor. Prune the dashboard tui and command surfaces. Add mvp-system actors (node_agent, orchestrator, stage_controller), the local_e2e harness, and gpu worker e2e. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-24 09:30:29 +00:00
[features]
default = []
feat: add mvp provisioning subsystem and datastream transport mvp-system: provisioner actor, provisioning module, node_agent, dashboard_view, observability_surface; expand gpu_worker ctl/ingress/egress and run_plan. iroh-driver: replace relay binary with datastream_transport; datastream gains endpoint abstraction. Archive pipeline-parallel-inference app to old-pipeline-parallel-inference. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-25 12:30:18 +00:00
local-e2e = ["dep:dashboard"]
feat(mvp): implement mvp-system modules and in-crate tests Implement arena_manager, device_bridge, driver_pumps, edge_establisher, gpu_worker ctl/egress/ingress/process-adapter, orchestrator run-fsm and token-endpoint, run_plan, stage_controller, tx_rx_edge_actor, weight_lifecycle, and the remaining modules. Move guarantee tests from tests/mvp_system into crates/mvp-system/src/tests; add the tinygrad device-bridge backend helper. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-23 13:51:34 +00:00
[dependencies]
refactor: consolidate dist tests, add mvp actors and e2e Fold the fragmented distribution swim/routing/gossip tests into swim_core, routing, and swim_actor. Prune the dashboard tui and command surfaces. Add mvp-system actors (node_agent, orchestrator, stage_controller), the local_e2e harness, and gpu worker e2e. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-24 09:30:29 +00:00
datastream = { path = "../datastream" }
feat: add mvp provisioning subsystem and datastream transport mvp-system: provisioner actor, provisioning module, node_agent, dashboard_view, observability_surface; expand gpu_worker ctl/ingress/egress and run_plan. iroh-driver: replace relay binary with datastream_transport; datastream gains endpoint abstraction. Archive pipeline-parallel-inference app to old-pipeline-parallel-inference. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-25 12:30:18 +00:00
dashboard = { path = "../dashboard", optional = true }
feat(mvp): implement mvp-system modules and in-crate tests Implement arena_manager, device_bridge, driver_pumps, edge_establisher, gpu_worker ctl/egress/ingress/process-adapter, orchestrator run-fsm and token-endpoint, run_plan, stage_controller, tx_rx_edge_actor, weight_lifecycle, and the remaining modules. Move guarantee tests from tests/mvp_system into crates/mvp-system/src/tests; add the tinygrad device-bridge backend helper. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-23 13:51:34 +00:00
serde_json = "1"
refactor: consolidate dist tests, add mvp actors and e2e Fold the fragmented distribution swim/routing/gossip tests into swim_core, routing, and swim_actor. Prune the dashboard tui and command surfaces. Add mvp-system actors (node_agent, orchestrator, stage_controller), the local_e2e harness, and gpu worker e2e. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-24 09:30:29 +00:00
serde = { version = "1", features = ["derive"] }
feat: add mvp provisioning subsystem and datastream transport mvp-system: provisioner actor, provisioning module, node_agent, dashboard_view, observability_surface; expand gpu_worker ctl/ingress/egress and run_plan. iroh-driver: replace relay binary with datastream_transport; datastream gains endpoint abstraction. Archive pipeline-parallel-inference app to old-pipeline-parallel-inference. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-25 12:30:18 +00:00
swactor = { path = "../..", features = ["serde", "transport"] }
swactor-transport = { path = "../transport" }
distribution = { path = "../distribution" }
iroh-driver = { path = "../iroh-driver" }
iroh = "0.98"
feat(mvp-system): actor admin control, iroh relay debug Add node_agent actor and orchestrator/worker_node admin control; iroh-driver relay debugging; ACTOR_CONTROL_AUDIT_IDEAS notes. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-09 08:53:53 +00:00
tokio = { version = "1", features = ["rt-multi-thread", "macros", "process", "io-util", "sync", "time", "net"] }
feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
swactor-vastai = { path = "../../tools/vastai" }
parking_lot = "0.12"
feat(mvp-system): expand one-node mvp binaries and provisioning Flesh out mvp_node, mvp_one_node_chat, and mvp_orch_one_node binaries. Add node_image and relay_provisioning; grow provisioning and vastai. Tune datastream mux/timing/emit. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-05 09:59:51 +00:00
blake3 = "1"
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
toml = "0.8"
refactor: consolidate dist tests, add mvp actors and e2e Fold the fragmented distribution swim/routing/gossip tests into swim_core, routing, and swim_actor. Prune the dashboard tui and command surfaces. Add mvp-system actors (node_agent, orchestrator, stage_controller), the local_e2e harness, and gpu worker e2e. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-24 09:30:29 +00:00
docs(mvp): lay out mvp-system spec and contracts Add MVP_SYSTEM_SPEC plus per-component contract docs (arena manager, device bridge, gpu worker ctl/ingress/egress/process-adapter, orchestrator run-fsm/token-endpoint, run plan, stage controller, tx_rx edge, weights). Scaffold the guarantee tests against the (empty) mvp-system crate. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-23 07:12:45 +00:00
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
feat(mvp): implement mvp-system modules and in-crate tests Implement arena_manager, device_bridge, driver_pumps, edge_establisher, gpu_worker ctl/egress/ingress/process-adapter, orchestrator run-fsm and token-endpoint, run_plan, stage_controller, tx_rx_edge_actor, weight_lifecycle, and the remaining modules. Move guarantee tests from tests/mvp_system into crates/mvp-system/src/tests; add the tinygrad device-bridge backend helper. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-23 13:51:34 +00:00
feat(mvp-system): actor admin control, iroh relay debug Add node_agent actor and orchestrator/worker_node admin control; iroh-driver relay debugging; ACTOR_CONTROL_AUDIT_IDEAS notes. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-09 08:53:53 +00:00
[dev-dependencies]
iroh-relay = { version = "0.98", features = ["server", "test-utils"] }
feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
[[bin]]
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
name = "mvp-worker-node"
path = "src/bin/worker_node.rs"
feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
[[bin]]
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
name = "mvp-orchestrator"
path = "src/bin/orchestrator.rs"
feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
[[bin]]
feat: rework datastream into catalog events, add gguf metadata - Rework the datastream endpoint into a catalog/event model (DatastreamEvent, SubscriptionRequest, channel/stream descriptors, DatastreamPublisherActor) across endpoint/frame/mux/wire/views. - Add gguf_metadata planning reader, a dashboard hardware view, and iroh-driver datastream transport. - Grow mvp-system orchestrator/worker_node bins and staging/provisioning; rename mvp_one_node_chat->mvp_chat; extend tinygrad worker. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-12 06:14:34 +00:00
name = "mvp-chat"
path = "src/bin/mvp_chat.rs"
refactor: consolidate dist tests, add mvp actors and e2e Fold the fragmented distribution swim/routing/gossip tests into swim_core, routing, and swim_actor. Prune the dashboard tui and command surfaces. Add mvp-system actors (node_agent, orchestrator, stage_controller), the local_e2e harness, and gpu worker e2e. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-24 09:30:29 +00:00
[[test]]
name = "local_unmocked_mvp_e2e"
path = "tests/local_unmocked_mvp_e2e.rs"
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
harness = false
refactor: consolidate dist tests, add mvp actors and e2e Fold the fragmented distribution swim/routing/gossip tests into swim_core, routing, and swim_actor. Prune the dashboard tui and command surfaces. Add mvp-system actors (node_agent, orchestrator, stage_controller), the local_e2e harness, and gpu worker e2e. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-24 09:30:29 +00:00
required-features = ["local-e2e"]
[[test]]
name = "gpu_worker_node_e2e"
path = "tests/gpu_worker_node_e2e.rs"
required-features = ["local-e2e"]
refactor(dashboard): rebuild around swactor worker view Replace telemetry/history/plugin/topology/warnings layer with store/view and swactor worker_page/worker_view fed by datastream frames. Add mvp-system local_e2e_cluster harness; archive pipeline-parallel-inference to old-pipeline-parallel-inference. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-25 07:29:16 +00:00
[[test]]
name = "local-e2e-cluster"
path = "tests/local_e2e_cluster.rs"
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
harness = false
refactor(dashboard): rebuild around swactor worker view Replace telemetry/history/plugin/topology/warnings layer with store/view and swactor worker_page/worker_view fed by datastream frames. Add mvp-system local_e2e_cluster harness; archive pipeline-parallel-inference to old-pipeline-parallel-inference. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-25 07:29:16 +00:00
required-features = ["local-e2e"]
Reference in a new issue Copy permalink