Bug 1 (swim/node.rs): SwimProbe::check_suspicion_timeouts() calls
members.declare_dead() before translate_probe_actions() processes the
DeclareDead action. The second declare_dead() returned false (already dead),
so MembershipChanged{Dead} was never emitted and the death was never
enqueued for dissemination. Fix: remove the redundant declare_dead() call
in translate_probe_actions since the probe already performed the mutation.
Bug 2 (swim/node.rs): apply_membership_update() — which processes piggyback
on every ping/ack/ping_req — updated the internal member list but never
emitted NodeAction::MembershipChanged. This meant DistributedNode was blind
to all state transitions learned via gossip piggyback (e.g., a dead node
refuting via incarnation bump). Fix: return MembershipChanged actions from
apply_piggyback and propagate through handle_ping/handle_ack/handle_ping_req.
Bug 3 (node.rs): DistributedNode::handle_ping/handle_ack/handle_ping_req
never processed MembershipChanged actions from SwimNode — only tick() did.
Fix: extract process_membership_changes() helper and call it from all four
message paths (tick, handle_ping, handle_ack, handle_ping_req).
Additional fixes:
- registry.rs: add re_disseminate_all() for anti-entropy on partition heal
- node.rs: call re_disseminate_all on MemberState::Alive transitions so
registry state accumulated during partition reaches recovering nodes
- cluster_scenarios: enable dead_reprobe in 10% message loss test, since
correct death dissemination (now working) causes cascading false deaths
without a recovery mechanism
Authored by Claude, lovingly guided by Zachery Aaron Shores-Chmielewski
