As the Internet has become increasingly ubiquitous,
it has seen tremendous growth in the popularity of online
services. These services range from online CVS repositories
like sourceforge,
shopping sites, to online financial and administrative systems, etc.
It is critical for these services to provide correct and reliable
execution for clients. However, given their attractiveness
as targets and ubiquitous accessibility, online servers
also have a significant chance of being compromised,
leading to Byzantine failures.

Designing and implementing a service to run on a machine
that may be compromised is not an easy task, since
infrastructure under malicious control may behave arbitrarily.
Even worse, as any monitoring facility may also be subverted
at the same time, there is no easy way for system behavior
to be audited, or for malicious attacks to be detected.

We propose our solution to the problem by reducing the
trust needed on the server side in the first place. In
the other words, our system is designed specifically for
running on untrusted hosts. In this thesis, we realize
this principle by two different approaches. First, we design
and implement a new network file system -- SUNDR. In SUNDR,
malicious servers cannot forge users' operations or tamper
with their data without being detected. In the worst case,
attackers can only conceal users' operations from each other.
Still, SUNDR is able to detect this misbehavior whenever users
communicate with each other directly.

The limitation of the approach above lies in that the system
cannot guarantee ideal consistency with even one single
failure. In the second approach, we use replicated state
machines to tolerate some fraction of malicious server failures,
which is termed Byzantine Fault Tolerance (BFT)
in the literature. Classical BFT systems assume less than
1/3 of the replicas are malicious, to provide ideal consistency.
In this thesis, we push the boundary from 1/3 to 2/3.
With fewer than 1/3 of replicas faulty, we provide the same
guarantees as classical BFT systems. Additionally, we
guarantee weaker consistency, instead of arbitrary behavior,
when between 1/3 and 1/3 of replicas fail.