Zachery Aaron Shores-Chmielewski
6608824cb0
Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP.
- prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP
- mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777)
- mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts
- mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink`
- vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream
- apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
34 lines
1.3 KiB
Bash
Executable file
34 lines
1.3 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
set -u
|
|
|
|
if ! mkdir /run/mvp_entrypoint.lock 2>/dev/null; then
|
|
echo "mvp-entrypoint: already running (lock held); parking" >&2
|
|
exec sleep infinity
|
|
fi
|
|
|
|
mkdir -p /root/.ssh && chmod 700 /root/.ssh
|
|
: > /root/.ssh/authorized_keys
|
|
if [ -n "${PUBLIC_KEY:-}" ]; then printf '%s\n' "$PUBLIC_KEY" >> /root/.ssh/authorized_keys; fi
|
|
if [ -n "${SSH_PUBLIC_KEY:-}" ]; then printf '%s\n' "$SSH_PUBLIC_KEY" >> /root/.ssh/authorized_keys; fi
|
|
if [ -f /etc/mvp_deploy_key.pub ]; then cat /etc/mvp_deploy_key.pub >> /root/.ssh/authorized_keys; fi
|
|
chmod 600 /root/.ssh/authorized_keys
|
|
|
|
if [ ! -s /root/.ssh/authorized_keys ]; then
|
|
echo "mvp-entrypoint: WARNING no SSH public key found (PUBLIC_KEY/SSH_PUBLIC_KEY unset, no baked key); SSH will reject logins" >&2
|
|
fi
|
|
|
|
mkdir -p /run/sshd /var/log /var/cache/mvp-models
|
|
ssh-keygen -A 2>/dev/null || true
|
|
/usr/sbin/sshd -e
|
|
echo "mvp-entrypoint: sshd up on :22" >&2
|
|
|
|
NODE_LOG=/var/log/mvp-node.log
|
|
echo "mvp-entrypoint: launching mvp-node (log -> $NODE_LOG)" >&2
|
|
set -o pipefail
|
|
/usr/local/bin/mvp-node "$@" 2>&1 | tee "$NODE_LOG"
|
|
code=${PIPESTATUS[0]}
|
|
|
|
echo "mvp-entrypoint: mvp-node exited with code $code; NOT restarting (node held for postmortem)" >&2
|
|
echo "mvp-entrypoint: --- last 80 lines of $NODE_LOG ---" >&2
|
|
tail -n 80 "$NODE_LOG" >&2 || true
|
|
exec sleep infinity
|