7 Answers
7

In general, an ABI is the interface between two program modules, one of which is often at the level of machine code. The interface is the de facto method for encoding/decoding data into/out of the machine code.

In Ethereum, it's basically how you can encode Solidity contract calls for the EVM and, backwards, how to read the data out of transactions.

The ABI, Application Binary Interface, is basically how you call functions in a contract and get data back.

An ABI
determines such details as how functions are called and in which
binary format information should be passed from one program component
to the next...

An Ethereum smart contract is bytecode deployed on the Ethereum blockchain. There could be several functions in a contract. An ABI is necessary so that you can specify which function in the contract to invoke, as well as get a guarantee that the function will return data in the format you are expecting.

If we wanted to call baz with the parameters 69 and true, we would pass 68 bytes in total, which can be broken down into:

0xcdcd77c0: the Method ID. This is derived as the first 4 bytes of the
Keccak-256 hash of the ASCII form of the signature baz(uint32,bool).
0x0000000000000000000000000000000000000000000000000000000000000045:
the first parameter, a uint32 value 69 padded to 32 bytes
0x0000000000000000000000000000000000000000000000000000000000000001:
the second parameter - boolean true, padded to 32 bytes

The 68 bytes is what would be specified in the data field of a transaction: a security note on that is here. (To summarise, be careful what you put in the data field, because it can have unintended, possibly malicious side-effects when passing it to the calling contract.)

To avoid a common pitfall when deriving the Method ID, the canonical types must be used, for example uint256 instead of uint.

Here is an example in Solidity of computing a Method ID for sam above:

bytes4(keccak256("sam(bytes,bool,uint256[])")

Using a higher-level library such as web3.js abstracts most of these details, but the ABI in JSON format still needs to be provided to web3.js.

Note: the ABI is an abstraction that is not part of the core Ethereum protocol. Anyone can define their own ABI for their contracts, and any callers of such contracts would have to comply with that ABI to get meaningful results. However, it is simpler for all developers to use Solidity, Serpent, and web3.js which all comply with the ABI above.

The point about "this" ABI not being part of the Yellow Paper and anyone being able to define "an" ABI is really interesting/important (and kinda shocking, to someone like me coming from the world of CPU/OS ABIs). Would be nice then to have some reference/history on where did this ABI come from and why is everyone standardizing on it. Is maybe everything so clear-cut that there is no reason to have different ABIs?
– hmijailNov 30 '18 at 5:15

Compiled Contract: The contract converted to byte-code to run on the Ethereum Virtual Machine (EVM), adhering to the specification. Note the function names and input parameters are hashed during compilation. Therefore, for another account to call a function, it must first be given the funtion name and arguments - hence the ABI:

Application Binary Interface - ABI: A list of the contract's functions and arguments (in JSON1 format). An account wishing to use a smart contract's function uses the ABI to hash the function definition so it can create the EVM bytecode required to call the function. This is then included in the data field, Td, of a transaction and interpreted by the EVM with the code at the target account (the address of the contract).

1Use of JSON is a de-facto standard; it's not in the formal spec but changing it would result in a need to amend many of the tools.

Think of it as the compiled version of an API (or as an API on the low level). As you know the contracts are stored as bytecodes in a binary format into the blockchain under a specific address. No one understands binary, so the ABI defines the structures and methods that you will use to interact with that binary contract (just like the API did), only on a lower level. The ABI indicates to the caller the needed information (functions signatures and variables declarations) to encode a meaningful(understood by the VM) call to the bytecode(contract).

additional info "from the official doc"

An Application Binary Interface (ABI) is intended to serve as the de
facto method for encoding & decoding data into & out of transactions

.

We assert that all contracts will have the interface definitions of
any contracts they call available at compile-time.

I was having a terribly hard time understanding the why part of this question myself, so I'd like to add one thing thanks to this excellent answer:

"How these bytes are interpreted into structured data is up to the program and thus up to the programming language used. In order to make it possible for two programs written in different programming languages to call each other, the compilers of such languages should implement the serialization and deserialization of data in the same way, i.e. they should implement the ABI specification, but they do not have to."

tl;dr a contract written in Solidity can interact with a contract written in Viper or Bamboo because they all implement and adhere to the ABI specification.