The case for Byzantine fault detection
Andreas Haeberlen, Petr Kouznetsov, Peter Druschel · Max Planck Digital Library · 2006
Distributed systems are subject to a variety of failures and attacks. In this paper, we consider general (Byzantine) failures [11], in which a failed node may exhibit arbitrary behavior. In particular, a failed node may corrupt its local state, send random messages, or even send specific messages aimed at subverting the system. Many security attacks can be modeled as Byzantine failures, such as censorship, freeloading, misrouting, or data corruption. Systems can be protected with Byzantine fault tolerance (BFT) techniques, which can mask a bounded number of Byzantine failures, e.g. using state machine replication [4]. BFT is a very powerful technique, but it has its costs. In a practical system that needs to tolerate up to f concurrent Byzantine failures, BFT cannot be implemented with less than 3f + 1 replicas [3]. Moreover, BFT scales poorly to large replica groups; as more servers are added, the throughput of the system may actually decrease [7]. In this paper, we explore an alternative approach that aims at detecting rather than masking faulty behavior. In this approach, the system does not make any attempt to hide the symptoms of Byzantine faults. Rather, each node is equipped with a detector that monitors the other nodes for signs of faulty behavior. If the detector determines that another node has become faulty, it notifies the local node, which can then take appropriate action. For example, it can cease to communicate with the faulty node; once all correct nodes have followed suit, the faulty node is isolated and the fault is contained. Specifically, we consider detection systems that are based on accountability [15]. With accountability, each action is associated with the identity of the node that has taken it, which allows the system to gather irrefutable evidence of faulty behavior. This has two important advantages: First, nodes can use the evidence to convince other nodes that a fault has occurred. Second, the evidence enables the system to resolve he-said-she-said situations in which two nodes accuse each other of having failed. Our goals in this paper are threefold: First, we examine the trade-offs between fault detection and traditional BFT. Second, we give a precise definition of the class of Byzantine faults that can be detected with this approach. Finally, we give a brief sketch of a practical system that implements such a detector.