the majority of car accidents are caused by driversbeingdistracted ordriverfatigue.Twelvepercent

of the drivers distracted report

fatigue

issues causing this problem.This papertakes the approach of solving these concerns by looking at the technologies that can detect fatigued

driving through sensors and post processing. The sensor technologies that detect the driver’s fatiguecondition use either the driver’s optical behaviors or biometric signatures. In addition to be able to detecta fatigued

driver, an approach needs to be devised to respond to this issue to prevent an accident that mayharm the driver, car occupants,

breaks down the driver fatalities according to NHTSA.In comparison, thecombinedcausalities

total for both Operation Iraqi Freedom and Operation Enduring FreedomAfghanistan iscurrently7094 casualties since 2001 according toicasualties.org. That means there is afive

times greater

chance of death associated with driving under

the

presumablyless hostile roads

of theUnites Statesin aone

year period compared toten years of the Operation Freedoms across the world onroads full of Improvised ExplosiveDevises (IEDs)

in hostile territories.

TheNHTSA

estimates

thatover 56,000

police-reported accidents are due to driver fatigue. This results in1600 deaths, 71,000 injuries and 12.5 billion dollars monetary loss. This is conservative due to the factthat it is difficult to properly estimate how many accidents were really caused

by driver fatigue.

According to police,

following

afatigueddriver will exhibit the same behavioras a

drunk driver:

slowreaction times, swerving between lanes,

and unintentionally speeding or slowing down. Yet,there is no

lawfor

driving

fatigued

and often the driver doesnot realize how fatigued

they are

until it is too late. Thispaper will examine the behaviors of driver fatigue, ways to monitor the behavior, techniques to

integrate acontrol to prevent and notify the vehicle driver oftheir

behavior, and decisions to be made to the vehicleifthe driver fails to actwhenin this condition.

Figure2-1: United States Driver Fatality

3

Factors causing

Driving

Fatigue

Driver Fatigue is often caused by four main factors:

sleep, work, time of day,

and physical. Often peopletry to do much in a day and they lose precious sleep due to this. Often by taking caffeine or otherstimulants people continue tostay awake. The lack of sleep builds up over a number

of

days and

the

nextthing that happens is the body finally collapses and the person falls asleep.

work the body is tiredandready to relax. The driver puts the air conditioneron and listens to some soothing music,

and

the

next thingthat happens is the driver is in a vulnerableposition to be distracted

due to fatigue.

Time of day factors

can often affect the body.The human brain is trained to think there are times the bodyshould be asleep. These are often associated with seeing the sunrise and sunset. Between the hours of 2AM and 6 AM, the brain tells the body

it should be asleep. Extending the time awake will eventuallyleadto the bodycrashing.

The final factor is a person’s

physicalcondition.People sometimes

are on medications

that create

drowsiness

or have physical ailments that cause these issues. Being physically

unfit, by being either underoroverweight,

will cause fatigue. Additionally, being emotionally stressed will cause the body to getfatigued

quicker.

4

Background of Detection

of Fatigue

If car technologies are going to prevent or at

least

warn of driver fatigue,

what symptoms does the drivergive off that can be detected? According to research,

there seems to be three basic categories that candetect driver fatigue. The first is the useof cameras to monitor a person’s behavior. This includesmonitoring their pupils, mouth for yawning,

head position,

and a variety of other factors. The next ofthese technologies is voice recognition. Often a person’s

voice can give offclues

on how fatigued

theyare. The

final of these technologies is the biometrics the person gives off. A person’s

blood pressure, bodyimpedance,

and

pulse,

as well a variety

of

vitals,

will change if they are fatigued.

The question to be examined in this paper is which of the technologies are the most reliable. Additionally,even if

the technology is reliable

enough

to be accepted by the driver,

it has to be non-intrusive to the waythe driver feels comfortable. Finally, the cost to implement the technology is critical if

it is going to beaccepted.

5

Current Technologies

There are very few driver fatigue products on the market. The most commonly used product in the marketis the Driver Nap Zapper. This product retails

for about twenty five dollars and has been seen on late

nightinfomercials. The Driver Nap Zapper is nothing more than a head position sensor:

when it detectsthe position the head is tilted

it

givesoff a high pitch audio alarm in the person’s ear. The device is onlyeffective if

the person falls asleep withtheir head tilted forward and not backwards.The device operatesby using a reed switch that upon the head tilting

downward, forces the contacts on the switch to close tooperate the circuit. The circuit is nothing more than either an audio or vibration alert. The device fits overthe ear similar to a hearing aid.

The other products on the market are the Nap Alarm and the DD850 Driver Fatigue Monitor,

whichoperate and sell

at roughly the same cost; five hundred United States Dollars (USD). Basically the

devicemonitors the person’s eyes to detect the blink rate.The device,

upon detection, alarms the driver by either

blinking light and/or

loud audiowarning. This can createa distraction

to the driver. In addition,

only 80%of the timewill aperson’sblink patternbea key signal to their drowsiness.

A

new

product,

the Empath Wristwatch,

is probably the most effective product in detecting driver fatigueas it attached directly to the user.The product is somewhat bulky due to the integration of multiplesensors. Thewarning system is

attached to the watch so it could be ignored by the user unless the audio inthe watch is strong enough to wake up the driver

Retail Price (MSRP) is listed at about $50,000 USD and the driver assistfeature is additional $3000 MSRP USD. The way this device work it stores the driver’s behavior. If itnotices the driver’s behavior to be erratic it will notify the driver to take a nap. Such

According to Frost & Sullivan the consumer GPS market was 5.14 billion dollars in 2010. Thismarketcould

generate 10-20% of the GPS market unless forced mandatory by the NHTSA where themarket

could equal that of theconsumer GPS market.At one time seat belts and air bags were optional products.Mercedes is investing heavily in this market,

showing the high car manufacturer

sees

agrowth potential.

Table5-1: CurrentDriver FatigueProducts

Products

Price

Accurate

Non-Invasive

Effective

OverallScore

Company

DetectionType

Driver Nap Zapper

25

50%

3

3

5

No Nap

Motion

Nap Alarm (LS888)

500

80%

5

6

6

Leisure AutoSecurity

Optical

DD850 DriverFatigue Monitor

500

80%

5

6

6

Eye Alert

Optical

Exmovare EmpathWristWatch

1000

90%

6

5

6

Exmovare

Biometric

Driver Assist Package

3000

90%

7

7

7

Mercedes

Behavioral

6

Sensors

As described in Section 4 there are threeapproaches to the detection of driver fatigue: Optical, Voice, andBiometric monitoring and analysis. Since we are focusing on passive systems we will note that voiceanalysis requires the driver to be actively speaking while driving and we will spend our

time focusing onthe passive systems of Optical and Biometric detection.

6.1

Audio

Detection

Audio Detection is limited due to the driver constantly talking to a handheld device. As car technologiesand cell phones

get more integrated in automobiles,

this field could potentially be used. Currently mostdrivers

do little talking in their car unless there are other passengers. The way audio detectors work isbystoring

Another method currently use is the Head Position Detection. Basically thistechnologydetermines

theheadtilt angle. When the head angle goes beyond a certain angle,

an audio alarm is transmitted in thedriver’s ear.This sort of technology is most efficient in detecting onset of sleep,

which is

the last stage offatigue. However,

drivers not being focused on the road,

or other issues,

this technology cannot prevent.When a driver is in a fatigued

position theyare extremely vulnerable and the onset of sleep is too late.This technology was not researched any further due to its limited effectiveness.

Figure6-2

depicts the flow chart of the Head Angle Detector.

DetectHeadAngleIs HeadTilted?AudioAlarm

Figure6-2: Head Position Detection

6.3

Driver Behavior Detection

As seen earlier, Mercedes-Benz is investing in detecting fatigue drivers as a

feature in their cars. Themethod of detection is learning the driver’s behavior when it comes to operating the car.When it detectsabnormal driver behavior it alerts the driver to take a nap or drink caffeine.Figure6-3

depicts theflowchart how this system would work.

LearnDriver’sBehaviorIs there apattern?AudioAlarmMonitorDriver’sBehavior

Figure6-3: Driver Behavior Detection

6.4

Optical Detection

The most

common implementation of an optical sensor system uses infrared or near-infrared LEDs tolight the driver’s pupils, which are then monitored by a camera system. Computer algorithms analyzeblink rate and duration to determine drowsiness. The camera system may also monitor facial features andhead position for signs of drowsiness, such as yawning and sudden head nods.Figure6-4

Perhaps the most important element in optical detection is pupil detection and tracking. One effectivemethod uses a low-cost charge-coupled device

(CCD) micro camera sensitive to near infrared light

withnear-infrared LEDs for pupil illumination.

Pupil detection is simplified by the “bright pupil” effect,similar to the red-eye effect in flash photography. An embedded PC with a low-cost frame grabber isused for the video signal acquisition and signal processing. The pupils are detected by searching theentire image to locate two bright blobs that satisfy certain size and shape constraints. Once the pupils aredetected, information can be gathered relating to blink rate, blink duration, eye closure/opening speed,and conditions such as eyes being not fully open.

6.5

Biometric Detection

There are a number of biometric systems in development to detect driver fatigue. One of these uses acapacitive array on the vehicle’s ceiling to detect changes in the driver’s body position. This is used inconjunction with an optical system to increasethe accuracy of the results. One method being tested at theUniversity of Minnesota Duluth uses sensors on the steering wheel and driver’s seat to measure heart ratevariability to indicate drowsiness.

Another method of monitoring the driver’s vital signs

uses a wristwatch system that wirelessly transmitsthe data collected for further analysis of fatigue indicators. George Washington University is working ona system based on an artificial neural network. This detects drowsiness based on analysis of the driver’ssteering wheel behavior. The Johns Hopkins University Applied Physics Laboratory is developing asystem that uses a low power Doppler radar system and sophisticated signal processing to measure anumber of indicators of driver fatigue. These include changes in general activity, blink frequency andduration, general eye movement, heart rate, and respiration.Figure6-5

specific example of one system that has been tested uses sensors in both the seat and steering wheel.The sensors in the seat use capacitively-coupled-electrodes while the steering wheel uses a direct contactelectrode. The steering wheel collects the signal ground from contact with the driver’s bare hand. Onlyone hand contact is needed. The seat sensors collect the electrocardiogram (ECG) of the driver. Thesensors are placed under the buttocks for maximum contact pressure. A high-input impedance OP

amp isneeded to boost the ECG signal to a useful level. This system produced accurate ECG results exceptunder the conditions of driving over bumpy roads or periods of driver body movement.

7

Integration of Sensors for Fatigue Detection System

Integrating sensor systems into modern cars requires more than breakthrough technology; for any newsystem to thrive past infancy, it needs to be accepted into the market quickly. What would convince aconsumer to spend extra money on a new auto safety feature? To

be appealing enough, we propose that anew sensor system must have at least the following qualities:



It must be accurate.



It must have a fairly quick response time, which could be the difference between a near-miss and a tragic fatality.



It must be relatively inexpensive.



It must either be already integrated in the car design, or effortlessly adaptable, a la “plugand play.”



It must be discreet and noninvasive; a sensor that annoys the driver could potentiallyworsen the problem of distracted driving.



It must be adaptable to changes in driver attire, driver position, and driver style.



It must work with multiple users, as many different people may drive the same car.

Since the problem of drowsy driving is often not taken as seriously as other driving problems such asdrunk driving, making these systems appealing enough for the extra cost will likely be difficult. Extrasteps need to be taken to educate the public about the reality of drowsy driving and the importance ofmonitoring a driver’s condition.

Multiple methods of integrating biosensors into automobiles are currently in study, and have been forover a decade. Each method has obvious advantages and disadvantages that are the subject of ongoingresearch. Examples of some technologies are listed in the

following sections.

7.1

Head/Eye/MouthCamera

Mounted in a discreet corner of the car, this would monitor for any signs of the head tilting, the eyesdrooping, or the mouth yawning. The following figure shows possible camera locations within a car:

Figure7-1: Face camera locations within vehicle

As shown above, this technology would be very discreet and would need no physical user contact.However, its results can be skewed if the driver turns his face or makes other sudden movements, and thesystem will need to cope with rapid face tracking.[

An Evaluation of Emerging Driver Fatigue] Also, sucha system may only be useful once the driver has entered a severe and potentially dangerous state offatigue. The National Department of Transportation has reported that a fifth of people will not show eyeclosure as a sign of fatigue at all. An infrared (IR) source can be used to illuminate the driver’s eyes tomake them more pronounced to the camera[Active Facial Tracking for Fatigue Detection]. Sincesunglasses (particularly reflective sunglasses) can obstruct the view of a user’s eyes, this technology isbest suited for nighttime driving.

[An Evaluation of Emerging Driver Fatigue] For cameras that trackmultiple visual cues, however, even without view of the driver’s eyes, the system may be able to make ahelpful prediction based on head and mouth position. Some research has suggested that very subtlemovements such as nose wrinkling, chinrising, and jaw dropping, can also be used to predict a driver’scurrent state. [Driver fatigue Detection Using Sensor Network] The difficulty then, is in accuratelytracking a user’s face.

7.2

Wheel/Seat Sensor

A sensor system can be integrated in the steering wheel that would be able

to measure multiple factorsthat can be used as a measure of drowsiness. These factors are divided into two categories:pressuremeasurements such as grip force, pulse wave, and breathing wave, andelectrical

measurements like ECGreadings, skin conductance, and skin temperatures. To take ECG measurements, the sensors would takethe form of conductive fabric patches wrapped around the wheel, as shown:

Figure7-2: Steering wheel sensor

Taking the bio-signslisted could give a very accurate assessment of the user because physical cues areknown to be a better indication of fatigue than visual cues, and they can be used in any light condition.However, such a system would only work if the user was not wearinggloves and kept his hands in arelatively constant position on the wheel; in some ECG cases, both hands are required [Real-time non-intrusive]. Since standards for heart rate and heart rate variability can be different for differentindividuals, there needs to be an intelligent system with memory to adapt to its user, and possibly havethe option to select which user is driving the car. Furthermore, the vibrations of the car could tamper withthe data. For methods measuring pulse and breathing waves as pressure inputs, the gripping force of thedriver provides a high influence on the data and also needs to be accounted for. [An IntelligentNoninvasive Sensor]

Similar to the wheel sensor, two pieces of conductive fabric located at the backrest of the car seat couldtake ECG measurements. Such a system needs little care on the part of the driver. One difficulty in thismeasurement is the need for the driver to always lean back. Another obvious difficulty is the fact that thedriver will nearly always be wearing a shirt or coat, and as a result, there needs to be a very robustimpedance-matching circuit to compensate [Real-time non-intrusive]

There is also an ECG system proposed that uses a measuring electrode on the seat of the chair and isterminated by the steering wheel as ground. In this system, the test subjects were not required to use bothhands, but the effect of gloves was not explored either. The authors in this case acknowledged that extraresearch was needed to make the system robust to bumpy roads or changes in the driver’s position.

The following figure shows a summary of possible contact locations:

Figure7-3: Proposed ECG measurement locations

7.3

Wireless Wristwatch

An alternative to having one sensor per car, this sensor can be situated on the driver. An example of thistechnology is the Exmovere “Empath Watch”, which is designed to be worn 24 hours a day. This watchtakes multiple bio-signs:



Heart rate and heart rate variability



Skin temperature (and ambient temperature for comparison)



User acceleration



Skin conductance

Using these signs, the device can detect a wide variety of user emotions and conditions, including fatigue.The current design uses Bluetooth technology and can be used to send alerts via cell phone to healthproviders, etc. Such a watch could easily be adapted to interface with any car the wearer drives, as manycars do already have Bluetooth. Theoretically, the user would only need to press a button on the watchwhen entering the car, which would allow the ease equivalent of “plug and play”. Also, since the watch isalways with one user, it could be made to adapt to the user’s unique bio-signs. In other words, it could be‘trained’ to work well with any specific user, which could give it an advantage over sensors paired to anyspecific car. This is an emerging technology (currently in Version 1), however, and many improvementsneed to be made on size, battery life, and durability. This device in its current state would not beaesthetically acceptable to most users, as it is made of plastic and is much larger than conventionalwristwatches. It is approximately 3.3” long, 1.7” wide, and 1.3” tall [Exmovere PDF]. A similar devicewith these proportions is shown in the following figure:

Figure7-4: Large watch-like device on wrist

As shown in the above figure, not only would such a device be considered “ugly” and “bulky” by mostconsumers, but its size and height may also cause discomfortwhen the user’s wrist bends while driving.

Currently the Exmovere Empath is undergoing a redesign process which, along with battery anddurability improvements, would reduce the size by around 50%.

In conclusion, none of the technologies listed has been fine-tuned or used in widespread use.

8

Behaviors required to Prevent Accident

In case of the event, the CPU will assess the signals from the sensors and determine whether it is ahazardous

situation to the fatigued driver andhis or hersurroundings. The system will activate

built-inalerts gradually to wake up the driver,

and

not to startle him/her,

which might cause more harm than help.Most of the things that drivers do to fight off sleepinesswhiledriving are not effective

for

more than 10minutes. The alert system is useful to warn and provide drivers

the opportunity

to find safe place for rest.The first warning indicators a vehicle could give include:



Issue flashing lights or signs such as “Wake up”, “Attention”, etc.



Issue warning tone or voice



Recommend a short nap via recorded voice or signs

If the system detects repeated fatigue circumstances, stronger prevention actions would be carriedouttobring the driver to a safe condition. These actions require more complicated electronic circuits andmechanic systems to be integrated into the automobile.

These would calculate and counteract the symptoms of thefatigued driving

such as car swerving, lanedrifting,

and

speed change,

for example, the vehicle may:



Apply brake to slow down and turn on the emergency flashers



Enforce a break period using preset starter-kill circuit



Dispatch for help if no response or improvement over a period of time

Figure8-1

depicts a

flow chart of corrective action and driver prevention in the event of driver fatigue.

Figure8-1: Flowchart for Corrective Action and Driver Prevention during Fatigue State

Visual (LED’s) and audio warning technologies have been widely implemented in the fatigued-detectionsystems on the market. Auto-pilot for automobile has been developed and tested by manufacturers andother high-tech companies. When the technology becomes available (may be standard equipment forfuture automobiles), it can be implemented in the fatigued-detection systems depending on the productioncost.

9

Conclusion

As described throughout the paper,

many technologies exist to detect driver fatigue. This paper tries tolook at the emerging technologies and determine the best approaches in trying to prevent the number onecause of fatalvehiclecrashes.

In the coming months the methods and recommendation for future research will be analyzed.