zacheryasc/swactor
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
swactor/apps/mvp-node/Dockerfile

16 lines
767 B
Text
Raw Normal View History

feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
# MVP node CODE image — thin layer over the CUDA/tinygrad base.
# Build from workspace root after compiling the Rust binary:
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
# cargo build --release -p mvp-system --bin mvp-worker-node
feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
# docker build -f apps/mvp-node/Dockerfile.base -t swactor-mvp-node-base:cuda12.6 .
# docker build -f apps/mvp-node/Dockerfile --build-arg BASE_IMAGE=swactor-mvp-node-base:cuda12.6 -t swactor-mvp-node:latest .
ARG BASE_IMAGE=swactor-mvp-node-base:cuda12.6
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
ARG MVP_NODE_BIN=target/release/mvp-worker-node
feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
FROM ${BASE_IMAGE}
refactor: retire old pipeline-parallel app, restructure mvp-system - Drop the standalone apps/old-pipeline-parallel-inference app and its fleet/orchestrator/tests (~24k lines). - Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager, and vastai_offer_preview. - Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI provisioning; rework xtask runner. Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 10:40:02 +00:00
ARG MVP_NODE_BIN=target/release/mvp-worker-node
feat(mvp-chat): local e2e chat on cuda gpu Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP. - prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP - mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777) - mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts - mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink` - vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream - apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
COPY ${MVP_NODE_BIN} /usr/local/bin/mvp-node
COPY apps/mvp-node/tinygrad_worker.py /usr/local/share/mvp/tinygrad_worker.py
RUN chmod +x /usr/local/bin/mvp-node /usr/local/share/mvp/tinygrad_worker.py
Reference in a new issue Copy permalink