3
3 Mary Theofanos NIST 301 975-5889 mary.theofanos@nist.gov May 23, 2007 These tests were performed for the Department of Homeland Security in accordance with section 303 of the Border Security Act, codified as 8 U.S.C. 1732. Specific hardware and software products identified in this report were used in order to perform the evaluations described in this document. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products and equipment identified are necessarily the best available for the purpose. http://www.noblis.org/BiometricIdentificationClusterGroup.asp

5
What are our key operational issues? Auto-enrollment - enrollment must occur like verification; no time for multiple presentations Maximum device intuitiveness; no opportunity for training or instruction (potential language barriers) Device technology must be acceptable to the user population We felt we had set the bar for ourselves internally... We define throughput as the average transaction time for all transactions from insertion of ticket to time of next insertion, including failures to acquire, repeat attempts, etc.

11
What have we done? Magic Your Way! What have we done? Magic Your Way!

12
We require biometric verification of ALL guests age 10 or older. This population still includes elderly, foreigners, adolescents, disabled (and we still allow opting-out). Our guests must still be able to present their biometric feature while wearing sunglasses, hats, suntan lotion, and jewelry. In addition, they are still carrying children, tote bags, food, strollers, etc. Our guests are still often sweating, tired, anxious, and confused. The environment is the same; however now….

13
Allow me to introduce the Biometric Circus McGurkus: (apologies to Dr. Seuss) …The animals in the zoo are mainly relevant to system performance in terms of accuracy. I propose that there are other animals that are relevant mainly to throughput: The Flummox – This is the animal that Darwin might eventually take care, but that hasnt happened yet, so you will have to deal with them. This is also known as the throughput killer or 95 th percentile person The Juggling Jot – This is the animal with their off-spring in a pouch, food and water reserves in some adapted appendage, and quite often showing signs of excitement and stress leading to repeated presentation attempts Comments on Doddingtons Zoo: The Wink-hooded Hoodwink - This is the crafty, sneaky animal with the self-proclaimed, superior intellect that is going to test the system by switching fingers, simultaneously proclaiming innocence and ignorance

16
So, we held the vendor responsible for all of the technical development of the sensor package itself, and we took ownership of the sensor enclosure and ergonomic interface elements: Testing requirements would be established and milestones would be set in the schedule that allowed for go/no-go checkpoints Enclosure mock-ups would be created for visual acceptability Ergonomic interface mock-ups would be created and tested to compare effectiveness So, lets take a look at some examples of the intensity of the efforts that were involved…

17
In May of 2005, the vendor delivered ten (10) alpha prototypes that were retrofitted into commercially available optical TIR scanners We used these to conduct a Phase I - Technology test, as well as to study the human factors of what people would do when instructions were minimal

18
We also surveyed our test users, with some interesting findings: Q1: It would be easy to to figure out how to use this device without any instruction. > 70%

19
We also surveyed our test users, with some interesting findings: Q2:I would not be concerned about privacy issues when using this device. > 54%

20
We also surveyed our test users, with some interesting findings: Q4:What did you like about this device? Other Fast Easy

21
We also surveyed our test users, with some interesting findings: Q5:What did you not like about this device? Other Nothing (no improvement needed Security/ privacy Hygiene

22
and it supposedly came with full sunlight capabilities… In Sept. of 2005, we received the first beta test unit (it was nicknamed OCAM: One CAMera)

23
Early enclosure mock-ups made from foam-core to match what we might end up with for an package…

30
Degree of Yaw seemed to make no difference (15 degreses was decided upon to 10 degree forward pitch appeared to be best angle for reducing finger placement problems Categorizing sources of finger-scanning rejection We divided rejections into 7 categories of placement errors, where it was discernable : Fingertip presented Finger not flat Side of finger presented Finger motion during capture Early lift-off of finger during capture Finger rotated over 15 degrees from axis Finger too far forward We divided rejections into 7 categories of placement errors, where it was discernable : Fingertip presented Finger not flat Side of finger presented Finger motion during capture Early lift-off of finger during capture Finger rotated over 15 degrees from axis Finger too far forward

32
So we ended up with an ergonomic design that looked something like this… Novel enough to file for a patent…

33
Current distance from platen surface to surface of metal = 2.2 mm (0.0863) oSuggest small arc be added to prevent finger from sliding too far forward. oSuggest arc be ~ 2.8 mm high. Top of arc would be 5.0 mm from platen surface. oArc would come up to edge of platen.

38
So what was the end result of the project? Still a one-to-one verification Our enrollment is still transparent - single feature presentation with truly minimal operator intervention or feedback Verification is still anonymous; we do not know the identity of the passholder at the turnstile We are now operating at approx. 11.5 seconds average per transaction, which was our goal…and our accuracy is is in the 5-6% FAR range with a total rejection rate of 11- 12% and well below 1.0% FTA

39
Dont ask me – ask them… In our 1 st full year of new sensor operation weve processed over twice the number of people than were processed in the previous 10 years of operations! 49,000,000 x 70% = 34,000,000

40
What have we done since implementation? Put them in at Hong Kong Disneyland! But whats wrong with this picture?

41
Our lessons learned: There is still no magic bullet device for all applications, but we did build a better mousetrap A change in biometric technology does not solve all your problems, it exchanges them for other problems In conclusion: Implementing an enterprise-sized biometric system upgrade with a proprietary data interface is both costly and painful to accomplish