Beginning a decade ago, mobile computing devices started to become powerful enough to be honestly robust key computing devices. With the advent of viable mobile devices coupled with biometric sensors suitable for mobile use, mobile biometric processing became practical. These practical devices, coupled with the early adopter mindset that law enforcement and the military have toward biometrics, caused these devices to start to appear in “real-world” production use. Throughout this past decade, many government agencies have actively deployed increasingly capable mobile biometric devices for use worldwide. As users gained experience with them in the field, those using these devices actively request new capabilities, driving rapid advancement of mobile biometric device design.

This chapter presents concepts that are not commonly synthesized and discussed in open forums. With tongue in cheek, perhaps the reason is that nobody is interested in what will be said. But it is far more likely the reason is that much of the information relates to companies that make at least a portion of their living by leveraging biometrics into solutions. Company-business practices and intellectual property are the keys to unlock these revenue streams and to protect competitive advantage. Yet, these items are rarely discussed. Much of what is presented is derived from the author's experiences and conclusions after having been involved in the biometric community for many years. Every effort has been made to give the reader an appreciation of the challenges in developing mass-market biometric sensors from a holistic perspective.

This chapter starts with a general and brief introduction of biometrics and audiovisual person recognition using mobile phone data. It begins with a discussion of what constitutes a biometric recognition system, and it then details the steps followed when audio-visual signals are used as inputs. This is followed by a review of the existing speaker and face recognition systems which have been evaluated on a mobile biometric database. We then discuss the key motivations of using deep neural network (DNN) for person recognition. We finally introduce a Deep Boltzmann Machine (DBM)- DNN, in short DBM-DNN, based framework for person recognition. An overview of the sections and sub-sections of this chapter is shown in Figure 4.1.

The focus of this chapter is on face-based biometric authentication methods. These methods first use the camera sensor images to detect the face of the users. Next, they extract low-level features from the face images, and then apply their algorithm to the extracted features to authenticate the user. These algorithms have access to some model of the enrolled user for comparison. In [11], Hadid et al. use Haar cascade and Adaboost of [12] to detect face components and then use Local Binary Pattern (LBP) histograms [13] and nearest-neighbor thresholding for authentication. In [7], Fathy et al. extract two types of intensity features from the full face and facial parts and compare four still image-based verification algorithms with four image set-based methods. The common pitfall of most of these algorithms is that they are very sensitive to changes in the low-level feature domain. They are sensitive in the sense that if two face images are under the same pose and lighting condition, they can perform well, but in unconstrained settings they become very inaccurate.

The article presented an HMM-based mm lip recognition for limited users of a handheld device. Tests are made on two small databases. The balanced accuracy in case of three, five, and ten classes are observed. While accuracy in case of three classes (two users) is approximately 99%, it falls to approximately 90% when ten classes (nine users) are considered. From the confusion matrices, it is evident that this fall of accuracy of 10% is due to the increase in number of classes. As this research focuses on use of handheld device by limited number of users, this limitation of scalability is not an issue. This approach can satisfactorily produce performance in considered situation. For practically using this methodology, any template replacement algorithm can be embedded into the biometric system to overcome slight challenges faced due to seasonal change of lip.

This chapter provides a broad overview of the advancements made on mobile platforms in regards to behavioral biometrics via usage data. Section 7.2 introduces the modules necessary for a complete biometric system. Sections 7.3 and 7.4 further elaborate on the data collection and feature extraction modules, respectively, as they relate to mobile device usage data. Section 7.5 provides an overview ofrelated research literature. Finally, Section 7.6 discusses several research challenges in this area and Section 7.7 summarizes the chapter.

This chapter aims to introduce and discuss continuous user authentication during natural user-phone interaction. Two types of user authentication are considered, including touch gesture-based and keystroke-based methods.We have described the threats encountered by the pervasive mobile devices, and that the goal of continuous user authentication is to balance security and usability duringnatural user-phone interaction. Literature on two types of user authentication has been reviewed, including touch gesture-based and keystroke-based authentication methods. General touch gesture features have also been elaborated as the feature basis for both types of methods. Three methods have been described: the first one using dynamic time warping, the second one using GTGF and statistical modelling, and the last one using virtual key typing features. The first two methods are good examples of continuous user authentication, which extract user identity from touch gestures throughout the usage sessions. The last method is a good example of user authentication, which takes effects when users apply virtual key typing. We have evaluated the effectiveness of the methods using collected databases as well as online testing user studies.

Although a reliable implementation of a smartwatch-based gait biometric system would usher in a new era of transparent authentication, the creation of such a system would not be without its own ethical problems. A primary problem of passwords is that users tend to reuse a small set of passwords (or, more realistically a single password; a primary responsibility (at least in the eyes of the commercial cybersecurity world) of biometrics is to replace these systems for that very reason (among others). The problem with an effective gait biometric system-much like many other forms of biometrics-is that once a user's training data is provided to the system that system can effectively impersonate that user; in essence, “reuse” is unavoidable. Thus, it becomes the responsibility of those designing gait biometric systems to create systems that are resistant to replay-style attacks. Similarly, work has to be done to ensure that the sensitive data collected from the smartwatch's sensors is not left in memory (where it could be recovered forensically or by a malicious application); much as traditional systems work to secure passwords. Gait biometrics might be the future of transparent authentication, but that does not allow conventional cybersecurity wisdom to be forgotten.

Gait is the unique human locomotion due to individual specific biophysical and behavior habits. With ubiquitous mobile devices in people's daily life nowadays, accelerometers and gyroscopes provided in these devices directly capture the dynamic motion characteristics and thus have great potential for nonobtrusive gait biometrics. In fact, inertial sensors have been exploited to perform highly accurate gait analysis under controlled experimental settings. However, their performance in realistic scenarios is unsatisfactory due to variations in data measurements affected by physiological, environmental, and sensor-placement-related factors. Practical mobile gait biometric algorithms need to be robust to these variations to achieve high authentication performance in the field. It is the focus of this chapter to address some of these issues for in-the-wild mobile gait biometrics applications. First, we propose a novel gait representation called gait dynamics image (GDI) for accelerometer and gyroscope data sequences. GDIs are constructed to be both sensor-orientationinvariant and highly discriminative to enable high-performing gait biometrics for real-world applications. Second, we show how to further compute walking pacecompensated GDIs that are insensitive to variability in walking speed. Third, we adopt the i-vector paradigm, a state-of-the-art machine learning technique widely used for speaker recognition, to extract gait identities using the proposed invariant gait representation. Fourth, we demonstrate successful fusion of accelerometer and gyroscope modalities for improved authentication performance. Performance studies using both the naturalistic McGill University gait dataset and the large Osaka University gait dataset containing 744 subjects have shown dominant superiority of this novel gait biometrics approach compared to state-of-the-art. Additional performance evaluations on a realistic pace-varying mobile gait dataset containing 51 subjects confirm the merit of the proposed algorithm toward practical mobile gait authentication.

In this chapter, we discuss a mobile fingerprint systems denoted as 4FTM-ID system. This system utilizes the built-in rear camera of the smartphone to capture a photo of four fingers, as shown in Figure 11.1. This new touchless fingerprint recognition system requires no extra hardware due to the use of fingers photo to perform authentication.

In mobile palmprint recognition, palmprint images are acquired by built-in cameras of mobile devices, such as smart phone, iPad. The preprocessing, feature extraction and storage/matching are implemented on the platform of mobile device.

In this chapter, we have explored presentation attacks in ocular biometric system on smartphones in the visible spectrum. We have discussed both kind of attacks - print attacks and electronic screen attacks. We have employed two publicly available databases that correspond to large-scale image-based artefact, MobiLive 2014 dataset, and for video-based artefacts, PAVID dataset.

This work focuses on the most common and cheapest face spoofing methods, i.e., photo attacks (including the printed photo on a paper or a photo demonstrated on an electronic screen). Many previous works [3-6] propose to classify genuine and fake samples based on frontal face images and achieve good performance on several face spoofing databases. However, in real applications, the imposter will try his best to fool the system and the texture difference between the genuine and fake samples is usually very small. In order to achieve robust face anti-spoofing performance, other cues like 3D face structure and motion pattern can be incorporated. In this work, we propose to detect spoofing photo attacks based on a sequence of rotated face images. Both the structure and texture information from the rotated face sequence are exploited. In practice, the users are only asked to take simple movement (i.e., rotate their faces). As pointed in [7], this head rotation requirement is much simpler than traditional challenge-response-based face anti-spoofing method, in which a combination of multiple movements is usually necessary. The proposed anti-spoofing method is applicable to face recognition applications such as face access control and remote authentication on mobile devices. The simple head rotation requirement is acceptable in these applications.

In the present chapter, after a thorough review of state-of-the-art in biometric antispoofing, we present a software-based spoof detection prototype for mobile devices, named MoBio_LivDet (Mobile Biometric Liveness Detection) that can be used in multiple biometric systems. MoBio_LivDet analyzes local features and global structures of face, iris and fingerprint biometric images using a set of low-level feature descriptors and decision-level fusion. In particular, we propose to use image descriptor classification algorithms Locally Uniform Comparison Image Descriptor (LUCID) [15], CENsus TRansform hISTogram (CENTRIST) [16] and Patterns of Oriented Edge Magnitudes (POEM) [17] for face, iris and fingerprint spoof detection. The proposed system allows user to choose “Security Level” (SL) against spoofing, between “low,” “medium“ and “high.” Depending on SL, the system selects unitdescriptor or multidescriptors-fusion-based liveness detection. These descriptors are computationally inexpensive, fast and novel approach to real-time image description, which are desirable requisites for mobile processors. Experiments on publicly available data sets containing several real and spoofed faces, irises and fingerprints show promising results.

Biometric technologies provide consumers with a long-awaited convenience to securely enter cyberspace on the front end. The biometric open protocol standard (BOPS), developed by Hoyos Labs, protects digital assets and digital identities on the backend. BOPS is a biometrics-agnostic standard that opens an application programing interface (API) for registered developers. Entering as a game-changer, BOPS communication architecture enables two-way secure sockets layer (SSL) or transport layer security (TLS) connection over the encryption mechanism to the server, which employs an intrusion detection system (IDS). The IDS is an external system responsible for blacklisting devices that are violating the replay portion of this specification.

Increased individual mobility has pushed the modern society needs for a reliable individual identity verification system as a critical component in many transactions in commercial industries, public sectors and government domains. The requirement for an ideal human identity verification is critical to security and prevention of identity fraud. Thus, trusted identity management has become an essential part of contemporary system infrastructure. It is now well accepted that biometrics-the science of identifying a person (or verifying their identity) based on their physiological or behavioral characteristics-can provide significant value when building such systems. Three key cornerstones in a trusted biometrics-based identity system include the following: (a) A trusted identity enrollment process (b) A trusted identity verification process (c) An identity credential management mechanism. In this chapter, we present several emerging developments in mobile biometrics technologies with particular focus on futuristic cognitive authentication systems for enabling large-scale trusted identity management systems based on biometrics and also biometrics identity services in the cloud. Biometrics is excellent mechanisms for the authentication or identification of individuals because of the credential's uniqueness and persistence almost over the lifetime of the person. Biometric identity services in the cloud enable mobile biometrics wide adoption economically and also take advantages of adjacent technology advancements.

Mobile biometrics is an emerging research area with some unique capabilities compared to the traditional biometrics. One aim of this book is to provide some unique views and experience from industrial partners. Their experience in developing mobile sensors or devices, algorithms, and systems could be useful and valuable to academic researchers. The angle from industry to view mobile biometrics may be different from academia, but the rich information and unique experience from industrial experts could inspire academic researchers greatly with some fresh ideas. In this book, there are five chapters from the industry experts, about one-third of the whole book. Progress can also be made faster in industrial mobile biometrics product development, given that important challenges and relevant issues are addressed by the academic researchers in a timely manner.