Biometrics Performance: Practical Hints

Biometrics Performance: Practical Hints

Q. What does one need to be aware of regarding the FAR/FRR?

The measurement of biometric features as well as the features themselves are subject to statistical fluctuations. Therefore, every biometric recognition system has a built-in acceptance threshold, which when raised both decreases FAR and increases FRR. It should be clear that the given FAR and FRR values are belonging to the same threshold value. Stating only the FAR or only the FRR is thus misleading.

Additionally, even the Failure-to-Enrol Rate FER must be considered when comparing the FAR/FRR values of different systems. This is because the enrolment procedure can be parametrized in such a way that only best quality biometric features are approved for biometric templates while lower quality samples are dropped, thus contributing to a higher FER. Normally, the higher the FER forced by the biometric system, the better the FAR and FRR values, and vice versa!

In biometrics FAR/FRR are not theoretically ascertainable, instead they must be determined statistically in costly tests. Determining statistical significance is equally difficult. There were no standardized techniques, therefore results could vary due to differences in test conditions and sample size. Clarity was only provided by disclosure of the test conditions.

Q. Is a biometric system's performance dependent upon the user?

Generally, yes. This applies for false acceptance rate (FAR) as well as for false rejection rate (FRR). We experience this in our everyday lives -- some faces are easy to recognize and remember, whereas others are difficult. Therefore, the statistical means of FAR and FRR, typical indicators, are not very helpful for individual users. This dependence on the individual user is also responsible for the fact that statistical properties of FAR and FRR measurements are very difficult to quantify.

Q. Is Failure to Enroll a typical problem for biometric systems?

Every biometric characteristic can occasionally or permanently fail. Examples of temporary failures can be caused by worn down or sticky fingertips for fingerprints, medicine intake in iris identification (Atropin), hoarseness in voice recognition, or a broken arm affecting one's signature. Well known permanent failures are, for example, cataract, which makes retina identification impossible, or rare skin diseases which permanently destroy a fingerprint. Therefore, every biometric system needs a fall-back process. One also needs a fall-back if a key is lost or a PIN is forgotten; so not only are biometric systems affected by user failure, rather all authentication systems. In fact one can see that also here, biometric systems are preferable to conventional methods.

Q. How are the FAR and FRR minimized in a biometric system?

The false acceptance rate (FAR) can be adjusted in the recognition algorithm via the acceptance threshold - the higher the acceptance threshold, the lower the FAR. Raising the acceptance threshold, however also raises the FRR. Therefore, the goal must be to have as small an FAR as possible for any given FRR, and vice versa. There are certain factors which primarily influence the FAR, while others mainly affect the FRR. For a fixed FRR, FAR is dependent on the following factors:

choose the sensor with the best (picture) quality: this mainly reduces the FRR

eliminate false operations of the user: this also reduces the FRR

optimize the recognition algorithm

limit the number of biometric references in an identification system: this reduces the FAR and increases the FRR

Q. Is the Equal Error Rate a robust measure for system performance?

No. Using the threshold parameter, most practical biometric systems are not adjusted for FAR = FRR which defines the EER but for FAR << FRR. Since ROCs of different systems may behave completely different, two systems with the same EER may even differ by decades for other ROC points. To avoid such large errors, only the FAR - FRR pairs in the operating point are to be considered, e.g., by comparing the FARs at a common FRR. A consideration of the EER is only reasonable in those rare cases where the system uses the EER as operating point.
Source: http://www.bromba.com/faq/biofaqe.htm