119 lines
5.8 KiB
Markdown
119 lines
5.8 KiB
Markdown
# The simulator
|
|
|
|
We deploy distributed algorithms — SWIM today, others in the future — to
|
|
environments we don't fully control: WANs across NAT'd hosts, LANs, cloud
|
|
GPU rentals, whatever the next thing requires. Iterating on those
|
|
algorithms by deploying them is slow and expensive. We pay for cloud time,
|
|
we wait for runs to finish, we get one bundle of data per attempt, and the
|
|
environment is non-reproducible — two runs of the same code can produce
|
|
different outcomes.
|
|
|
|
The simulator exists to remove that bottleneck. It models the environments
|
|
swactor runs in, faithfully enough that algorithms developed against it
|
|
behave the same way when deployed. Once faithful enough, the development
|
|
loop becomes write, run in sim, iterate, deploy — instead of write, deploy,
|
|
wait, repeat.
|
|
|
|
## What it models, and what it doesn't
|
|
|
|
The simulator models the network and the transports that run over it. The
|
|
network covers arbitrary topology, per-link properties, NAT and
|
|
reachability, partitions, anything a real deployment might present.
|
|
Transports are modeled separately; iroh is the one we use today, others
|
|
will be added as needed.
|
|
|
|
The algorithms running on top — swactor and whatever it hosts — are not
|
|
simulated. They execute as code, against the simulated network and
|
|
transport, the same way they execute against the real ones in production.
|
|
Running the same code path in sim and in prod is the point.
|
|
|
|
## Properties it must have
|
|
|
|
The simulator must be deterministic. Same input, same output, byte for
|
|
byte. Tuning algorithms means searching a space of configurations and
|
|
comparing outcomes; if outcomes drift from run to run, the search has
|
|
nothing solid to compare against.
|
|
|
|
It must express any environment swactor could be deployed into. A closed
|
|
set of named topologies is not enough; real deployments aren't shaped that
|
|
way, and a sim limited to such shapes gives answers that don't generalize
|
|
past it.
|
|
|
|
It must be observably equivalent to production. Whatever data we collect
|
|
from a real deployment, the simulator must produce the same surface. If
|
|
production ever emits something the sim can't, the sim is incomplete —
|
|
that direction of asymmetry is always a bug. The form this data takes
|
|
today is a diag bundle, but the form will evolve as data collection
|
|
matures; the constraint is on the observable surface, not on whatever
|
|
artifact happens to carry it.
|
|
|
|
## The parity bar
|
|
|
|
Observable equivalence is stronger than matching the schema of whatever
|
|
artifact production emits. Every internal observation the process makes
|
|
— retry counts, peer-state distributions, timer firings, transport
|
|
stats, queue depths, the tails as well as the means — must be
|
|
statistically indistinguishable from the corresponding observation in a
|
|
real deployment under matched conditions, within the noise floor of the
|
|
real measurement. A schema-level match that hides a tail-behavior
|
|
mismatch is a parity failure. The process running on top of the sim
|
|
must have no means, statistical or otherwise, of detecting that it is
|
|
in a sim.
|
|
|
|
The model floor is whatever today's deployments can record. The model
|
|
grows as recording grows. If production ever observes something the
|
|
recording cannot capture or the sim cannot reproduce, both are
|
|
incomplete — extending recording and extending the sim are the same
|
|
project, pursued together.
|
|
|
|
## Two modes, one engine
|
|
|
|
The sim runs in two modes against one engine. In the generative mode
|
|
the environment is parameterized — topology, per-link policies, peer
|
|
populations — and the engine produces runs from that specification. In
|
|
the record-replay mode the engine ingests a captured deployment trace
|
|
and plays its environment back: the network conditions and the peer
|
|
behavior that surrounded the recorded run, rerun against whatever
|
|
algorithm code is under test now.
|
|
|
|
Replay is not a tape of the node-under-test; it is a restoration of
|
|
the world that node was in, so that a new version of the node can be
|
|
run against the same world. Peers in a replay are not stubs driven by
|
|
recorded outputs; they are the peer code, re-executed inside the sim
|
|
against the recorded network. The recording must carry enough to make
|
|
that re-execution faithful.
|
|
|
|
The recording schema is both the sim's input format and the sim's
|
|
output format. A sim run consumes the same shape it produces. That
|
|
shape is the bridge between deployment and sim.
|
|
|
|
## The hosted world
|
|
|
|
The sim hosts everything the node-under-test can touch on the wire.
|
|
Peers running our code are sim-native — same binary, same runtime
|
|
facade, different runtime implementation underneath. Infrastructure we
|
|
own — relays, signaling, whatever else lands on a node's path — is
|
|
also sim-native. Things we don't own, and that we can only get
|
|
faithful behavior from by running the real artifact — a peer on a
|
|
different version, a third-party node, a kernel that handles a packet
|
|
a particular way — are accommodated by an escape hatch: the sim drives
|
|
an opaque binary's I/O through a virtualized boundary, so the binary
|
|
still sees a real socket, real time, real interrupts, while the sim
|
|
controls what crosses the wire. "Anything it might interact with"
|
|
admits no exceptions in the long run; the engine is built to let any
|
|
of those interaction surfaces be plugged in.
|
|
|
|
## Calibration is ongoing
|
|
|
|
Every real deployment is evidence about how well the sim resembles
|
|
reality. The calibration loop is concrete: deploy, collect the
|
|
recording, feed it into the sim as the environment, run the same
|
|
algorithm version against it, diff the distributions of internal
|
|
observations. Where they diverge beyond noise, the sim has a gap to
|
|
fix. The sim's quality is measured against that loop, not against
|
|
itself.
|
|
|
|
This is not a one-time validation step. As transports change, as
|
|
algorithms evolve, as production collection matures, calibration
|
|
continues. A real run that the sim cannot reproduce is a sim bug, not
|
|
a curiosity.
|