P₄ce: Consensus over RDMA at Line Speed

P₄ce is the first replication protocol that exhibits the same latency and requires the same network capacity as sending data to a single server. P₄ce builds upon previous RDMA-based consensus protocols. They achieve consensus with a single network round-trip, but with a reduced network throughput. P₄ce also achieves consensus with a single round-trip, but without degrading throughput by decoupling the consensus decisions from the RDMA communications. The decision part of the consensus protocol runs on a commodity server, but the communication part of P₄ce is fully implemented on a programmable switch, which replicates data and aggregates the acknowledgements in the network, avoiding the throughput bottleneck at the leader. Although simple in its principle, the implementation of P₄ce raises many challenging issues, notably caused by the complexity of RDMA and the underlying network protocols, the intricacies of packet rewriting during replication and aggregation, and the restricted set of operations that can be implemented at wire speed in the programmable switch. We implemented P₄ce and deployed it on a commercially-available Intel Tofino switch, achieving up to 4x better through-put and better latency than state-of-the-art consensus protocols.