What algorithm is utilized for authentication in GSM networks?

The authentication algorithm used in the GSM system is known as the A3 algorithm.

Most GSM network operators utilize a version of the COMP128 algorithm as the implementation of the A3 algorithm.

A3’s task is to generate the 32-bit Signed Response (SRES) utilizing the 128-bit random challenge (RAND) generated by the Home Location Register (HLR) and the 128-bit Individual Subscriber Authentication Key (Ki) from the Mobile Station’s Subscriber Identity Module (SIM) or the Home Location Register (HLR).

A3 actually generates 128 bits of output. The first 32 bits of those 128 bits form the Signed Response.

The A3 algorithm is implemented in the Subscriber Identity Module (SIM).