Sign up to receive free email alerts when patent applications with chosen keywords are publishedSIGN UP

Abstract:

A human interface device with an inherent built-in feedback mechanism for
use by a user to remotely interface with a computer-simulated environment
is disclosed herein. The human interface device comprises at least one
sensor configured to sense a condition within the action of the
computer-simulated environment and also operable to generate a
communication concerning the sensed condition. At least one
micro-controller is positioned within the human interface device and
configured to receive the communication concerning the sensed condition
from the at least one sensor. The at least one micro-controller is
further configured to generate communication in reaction to the
communication from the sensor. At least one actuator is configured to
receive the communication from the at least one micro-controller and
provide a sensory experience in reaction to the communication.

Claims:

1. A human interface device for use by a user to interact with a
computer-simulated environment comprising:at least one sensor configured
to sense a condition within the computer-simulated environment and
generate a first communication concerning the sensed condition;at least
one micro-controller configured to receive the first communication
concerning the sensed condition from the at least one sensor, the at
least one micro-controller further configured to generate a second
communication in reaction to the first communication from the sensor;
andat least one actuator configured to receive the second communication
from the at least one micro-controller and provide a sensory experience
in reaction to the second communication, wherein the at least one
micro-controller is operable to independently determine the response of
the at least one actuator appropriate to the condition sensed by the at
least one sensor.

2. The human interface device of claim 1 wherein the at least one actuator
is configured to respond solely to sensor information processed by the at
least one micro-controller.

3. A gaming system comprising:a host device having a central processing
unit configured to control overall functions of the gaming system; anda
human interface device for use by a user of the gaming system and
comprising:at least one sensor configured to sense a condition within the
gaming system and generate a first communication concerning the sensed
condition;at least one micro-controller positioned within the human
interface device and configured to receive the first communication
concerning the sensed condition from the at least one sensor, the at
least one micro-controller further configured to independently process
the first communication and generate a second communication in reaction
to the first communication from the sensor; andat least one actuator
configured to receive the second communication from the at least one
micro-controller and provide a sensory experience in reaction to the
second communication.

4. The gaming system of claim 3 wherein the at least one controller does
not require communication from the host system for operation.

5. The gaming system of claim 3 wherein the at least one controller does
not communicate with the host system as to the processing of the first
communication.

6. A method of operating a human interface device to remotely interface
with a computer, the method comprising the steps of:sensing a condition
within the action of a virtual reality system;generating a first
communication concerning the sensed condition;sending the first
communication to a human interface device for processing;generating a
second communication solely within the human interface device in reaction
to the received communication;sending the second communication to at
least one actuator positioned with the human interface device;
andproviding a sensory experience with the at least one actuator in
reaction to the received human interface device communication.

Description:

BACKGROUND OF THE INVENTION

[0001]This invention relates in general to human interaction with
computer-simulated environments. More specifically, this invention
relates to multimodal devices having an inherent built-in feedback
mechanism that can be used to remotely interface human interaction with
computer-simulated environments, such as generated by computer-based
gaming and virtual reality systems.

[0002]Gaming and virtual reality systems allow a user or a number of users
to interact with a computer-simulated environment. A typical system
includes a computer that establishes the computer-simulated environment.
The computer-simulated environment can be "virtual reality" or based on a
real environment such as, for example, simulations for pilot or combat
training. The computer-simulated environment can also be for gaming or
based on an imagined environment such as, for example, imaginary
interplanetary worlds. Most computer-simulated environments are primarily
visual experiences, displayed either on a computer screen or through
special or stereoscopic displays, but some computer-simulated
environments established by computers include additional sensory
experiences, such as sound through speakers or headphones or vibration
through user input devices such as controllers. Users can typically
interact with such computer-simulated environments through the use of
standard input devices such as a keyboard and a mouse, or through
multimodal devices such as, for example, controllers, wired gloves,
joysticks or steering wheels.

SUMMARY OF THE INVENTION

[0003]According to this invention, there is provided a human interface
device with an inherent built-in feedback mechanism for use by a user to
remotely interface with a computer-simulated environment. The human
interface device comprises at least one sensor configured to sense a
condition within the action of the computer-simulated environment and
also operable to generate a communication concerning the sensed
condition. At least one micro-controller is positioned within the human
interface device and configured to receive the communication concerning
the sensed condition from the at least one sensor. The at least one
micro-controller is further configured to generate communication in
reaction to the communication from the sensor. At least one actuator is
configured to receive the communication from the at least one
micro-controller and provide a sensory experience in reaction to the
communication.

[0004]According to this invention, there is also provided a method of
providing a sensory experience to a user using the human interface device
described above. The method comprises the steps of sensing a condition
within the action of the gaming or virtual reality system established by
the computer, generating a communication concerning the sensed condition,
sending the communication to a human interface device for processing,
generating a communication within the human interface device in reaction
to the received communication, sending the human interface device
communication to at least one actuator; and providing a sensory
experience in reaction to the received human interface device
communication.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005]FIG. 1 is a schematic view of the major components of a conventional
gaming system.

[0006]FIG. 2 is a schematic view of the major components of a first
embodiment of the invention.

[0007]FIG. 3 is a side view, in elevation, of a first embodiment of the
human interface device of FIG. 2.

[0008]FIG. 4 is a side view, in perspective, of a second embodiment of a
human interface device.

[0009]FIG. 5 is a side view, in perspective, of a third embodiment of a
human interface device.

[0010]FIG. 6 is a schematic view of the components of the human interface
device shown in FIG. 2.

DETAILED DESCRIPTION OF THE INVENTION

[0011]For purposes of this patent application, the terms "gaming system"
and "virtual reality system" will be used interchangeably and are defined
to include any system, structure(s), or device(s) incorporating a
technology which allows a user to interact with a computer-simulated
environment.

[0012]Referring now to the drawings, there is illustrated in FIG. 1 a
gaming system, indicated generally at 10, that is conventional in the
art. A gaming system is one example of system operable to establish a
computer-simulated environment and the invention is not limited to gaming
systems. The basic structure and mode of operation of the gaming system
10 are well known in the art, and only those portions of the gaming
system 10 that are necessary for a complete understanding of the
invention will be described. Some examples of commercially available
gaming systems 10 include Wii® marketed by Nintendo, Playstation®
marketed by Sony Corporation, and Xbox® marketed by Microsoft
Corporation.

[0013]As shown in FIG. 1, a typical gaming system 10 includes at least one
human interface device (HID) 12, a plurality of sensors 14, at least one
host device or system or device 16, and a display (not shown).

[0014]The HID 12 is an input device used to affect the gaming system 10 or
govern the movement or actions of an entity within the computer-simulated
game/environment. While the HID 12 is typically connected via wires to
the host system 16, the HID 12 can also be operated in a wireless mode.
As will be explained in more detail below, the HID 12 can comprise many
different physical forms.

[0015]As shown in FIG. 1, the HID 12 typically includes, among other
things, a micro-controller 18 and at least one actuator 20. As will be
explained in more detail below, the micro-controller 18 is configured for
processing of information and communication of information to a variety
of other devices. The actuator 20 is configured to induce a sensory-based
reaction by the user of the HID 12. Examples of actuators 20 include
vibration inducing or motion causing motors, heat causing elements, sound
causing elements and motion causing solenoids. As shown in FIG. 1, the
actuator 20 is electrically connected to the micro-controller 18 and
configured to respond to signals from the micro-controller 18.

[0016]The sensors 14 are configured to provide input of the status of game
play or game conditions. The sensors 14 can sense a condition within the
action of the gaming system in that the sensors 14 can sense some
physical condition associated with the user, such as activity of the
user, the physical condition of the user, or the physical environment in
which the user is participating in the computer-simulated environment.
For example, one example of a sensor 12 is an accelerometer, which can
measure the magnitude, direction and force of movement of the user. Other
types of sensors 14 can be used to provide input regarding user-initiated
actions, such as pulling a trigger and movement of a virtual character.
Still other sensors 14 can be used to provide input regarding other
conditions, such as for example the heart rate of the user, the
temperature and humidity of the gaming environment, and the mental
consciousness of the user. While the plurality of sensors 14 illustrated
in FIG. 1 is shown as being positioned exterior to the HID 12, it should
be understood that the plurality of sensors 14 can be positioned in
various locations, including internal to the HID 12. While the embodiment
shown in FIG. 1 illustrates a quantity of four sensors 14, it should be
understood than more or less than four sensors can be used.

[0017]The host system 16 typically includes, among other things, a host
central processing unit (CPU) 22 and is configured to control the overall
functions of the gaming system. Examples of overall system functions
include loading of games software, start up, and shut down. The host CPU
22 typically processes input information from a variety of sources and
controls the play of the gaming system.

[0018]In operation, the sensors 14 sense a condition potentially affecting
the play of the gaming system 10 and generate a communication concerning
the sensed condition. The sensors 14 can communicate with the
micro-controller 18 positioned within the HID 12 or the host CPU 22. The
micro-controller 18 can be configured to receive the communications
generated by the individual sensors 14. The communication of the various
sensors 14 with the micro-controller 18 is shown in FIG. 1 as
communications C1a and the communication of the various sensors 14 with
the host CPU 22 is shown as communications C1b. The micro-controller 18
processes the information and communicates the information from the
sensors 14 to the host CPU 22 positioned within the host system 16 as
shown by communication C2. The host CPU 22 processes the sensor
information and determines an appropriate action and/or response. The
action and/or response is communicated from the host CPU 22 to the
micro-controller 18 positioned within the HID 12 by communication C3. The
micro-controller 18 receives the communication C3 and determines the
appropriate actuator 20 response. The micro-controller 18 then
communicates with the appropriate actuator 20 as shown by communication
C4. The actuator 20 can be configured to receive the communication C4 and
provide a sensory experience in reaction to the communication. For
example, the sensory experience can be a feeling of vibration or recoil.
The communication C4 instructs the actuator 20 to initiate and perform an
action, such as generate vibrations for a specified period of time. This
cycle is repeated as additional communications C1a and C1b are received
from the sensors 14.

[0019]As further shown in FIG. 1, additional communication of a general
nature, indicated as CG, occurs between the host CPU 22 and the
micro-controller 18 positioned within the HID 12. The communication CG
can include typical game play instructions.

[0020]Referring now to FIG. 2, there is illustrated an improved gaming
system, indicated generally at 40, in accordance with a first embodiment
of the invention. In the illustrated embodiment as shown in FIG. 2, the
gaming system 40 includes at least one HID 42, a plurality of sensors 44,
and at least one host device or system 46.

[0021]As shown in FIG. 2, the host system 46 can be substantially similar
to the host system 16 illustrated in FIG. 1. Similarly, the plurality of
sensors 44 shown in FIG. 2 can be substantially similar to the plurality
of sensors 14 illustrated in FIG. 1.

[0022]The HID 42 shown in FIG. 2 includes, among other things, a
micro-controller 50 and at least one actuator 48. The actuator 48 shown
in FIG. 2 can be substantially similar to the actuator 20 illustrated in
FIG. 1. The micro-controller 50 varies from the micro-controller 18
illustrated in FIG. 1 in that it performs the additional processing of
the sensor information C1a without communicating the sensor information
C1a with the host system 46. Accordingly, the HID 42 may include
different components and different electronic circuitry than the HID 12
illustrated in FIG. 1.

[0023]Generally, the improved gaming system 40 varies from the traditional
gaming system 10 shown in FIG. 1 in several ways. First, the HID 42 does
not communicate with the host system 46 as to the processing of the
sensor information C1a. Rather, the HID 42 performs the steps of
receiving the sensor information C1a, internally processing the sensor
information C1a, determining the appropriate actuator response and
communicating with the appropriate actuator without communicating the
sensor information C1a to the host system 46. The micro-controller 50 is
thus capable of independently determining the response of the actuator 48
that is appropriate or desirable to the condition that is sensed by any
of the sensors 44. Second, while the actuator 48 shown in FIG. 2 can be
substantially similar to the actuator 20 shown in FIG. 1, the actuator 48
is configured to respond solely to sensor information C1a processed by
the micro-controller 50. Accordingly, the actuator 48 does not require
communication from the host system 46 for operation.

[0024]Referring now to FIGS. 3-5, the human interface devices 42 can have
various physical embodiments. As shown in FIG. 3, a first physical
embodiment of one example of a HID 146 is illustrated. In this
embodiment, the HID 146 is in the form of a firearm. The firearm HID 146
is configured to interact with the gaming system or virtual reality
system such that the firearm HID 146 simulates the firing of a weapon.
Accordingly, in an effort to make the gaming system as realistic as
possible, the firearm HID 146 has all of the typical firearm components
including a stock 160, a muzzle 161, a magazine 162, a trigger 163, a
receiver 164, a sight 165 and a butt end 166. In the illustrated
embodiment, the firearm HID 146 also includes a recoil mechanism 170, a
sensor switch 171, a mode switch 172, a power supply 173, a
transmitter/receiver 174, and a motion control 175. While not illustrated
in FIG. 3, the firearm HID 146 also includes the micro-controller 50
shown in FIG. 2 and associated circuitry.

[0025]As illustrated in FIG. 3 and with reference to FIG. 2, the recoil
mechanism 170 is the same device as the actuator 48 shown in FIG. 2. The
recoil mechanism 170 is configured to provide a realistic "backward kick"
or force resulting from the act of firing the firearm HID 146. In the
illustrated embodiment, the recoil mechanism 170 is a solenoid. However,
recoil mechanism can be other suitable actuator devices. In operation, as
the trigger 163 is pulled, a sensor 144, illustrated as positioned
internal to the firearm HID 146, senses the motion of the trigger 163 and
sends information to the micro-controller 50 as shown in FIG. 2. The
micro-controller receives the sensor information, internally processes
the sensor information and communicates with the recoil mechanism 170,
thereby producing the recoil motion. Accordingly, the recoil mechanism
170 produces an immediate, real time action in the firearm HID 146 in
response to sensor input without the interaction with the host system 46.
While the actuator shown in FIG. 3 is a recoil mechanism 170, the
actuator can be other suitable devices, and non-limiting examples are
provided below.

[0026]The sensor switch 171 is configured to enable or disable
communication from the sensor 144 to the micro-controller 50. In the
illustrated embodiment, the sensor switch 171 is an on/off rocker switch
which is conventional in the art. However, the sensor switch 171 can be
other suitable devices, such as for example, a slide switch, capable of
enabling or disabling the communication from the sensor 144 to the
micro-controller 50.

[0027]The mode switch 172 is configured to provide the firearm HID 146
with various modes of game play. The mode switch 172 includes settings
for single shot, semi-automatic and full automatic firing rate. In
operation, as each mode is selected, the sensor 144 and recoil mechanism
170 react accordingly. In other embodiments, the mode switch 172 can be
configured to provide various modes of game play in accordance with the
nature of the virtual reality scenario. For example, in a scenario in
which the virtual reality game involves fishing, the HID 42 can be
configured as a fishing pole and the mode switch 172 could be configured
to provide the amount of bait used.

[0028]Referring again to FIGS. 2 and 3, the power supply 173 is configured
to provide sufficient power to the firearm HID 146 such that the firearm
HID 146 is independent of the host system 46 for power. As shown in FIG.
3, the power supply 173 is a rechargeable battery pack. However, the
power supply 173 can be other suitable devices sufficient to provide
power to the firearm HID 146. In other embodiments, the power supply 173
can be connected to a conventional electrical outlet (not shown).

[0029]The transmitter/receiver 174 is configured to transmit and receive
information to and from the host system 46. The transmitted and received
information can include general communications CG, as shown in FIG. 2, as
well as other necessary information. In the illustrated embodiment, the
transmitter/receiver 174 operates on radio frequencies, although other
methods of operation are possible.

[0030]The motion control 175 is an input device used to affect the gaming
system 46 or govern the movement or actions of an entity within the
computer-simulated game. As one example, the motion control 175 can be
used to move an entity within the game forward, backward or sideways. In
the illustrated embodiment, the motion control 175 is knob having
360° of available motion. However, the motion control 175 can be
any mechanism or device suitable to govern the movement or actions of an
entity within the computer-simulated game.

[0031]While not illustrated in FIG. 3, it should be understood that the
firearm HID 146 can project images onto the display of the gaming system.
The images can include any suitable visual affect, such as for example a
cross-hair and bullet tracers.

[0032]As shown in FIG. 4, a second physical embodiment of a HID 246 is
illustrated. In this embodiment, the HID 246 is in the form of a
joystick. Non-limiting examples include a joystick 246 that can be used
in gaming systems involving the control of helicopters or antique
aircraft. The HID 246 illustrated in FIG. 4 includes the sensor,
micro-controller, and feedback mechanism and these components operate in
the same manner as described above.

[0033]As shown in FIG. 5, a third physical embodiment of a HID 346 is
illustrated. In this embodiment, the HID 346 is in the form of a steering
wheel. Non-limiting examples where the steering wheel 346 can be used in
gaming systems involving the control of race cars or boats. The HID 346
illustrated in FIG. 5 includes all of the components and operates in the
same manner as described above.

[0034]While the illustrated embodiments of the human interface devices
include a firearm, joystick, and steering wheel, it should be understood
the human interface devices can represent any control mechanism, such as,
for example, a fishing pole, a tennis racket, or a surf board suitable to
affect the gaming system 10 or govern the movement or actions of an
entity within the computer-simulated game.

[0035]One example of suitable internal components and the circuitry within
the HID 42 are shown in FIG. 6. In the illustrated embodiment, the HID 42
includes the micro-controller 50, feedback mechanism (or actuator) 48, a
receiver 80, a transmitter 82, voltage conversion circuitry 84, a power
supply 86, memory 88, a time device 89, a variable signal generator 90,
various user switches, 92a and 92b, and sensor output 94.

[0036]The HID 42 is activated through a switch, such as for example user
switch 92a. Activating the HID 42 enables the micro-controller 50. The
micro-controller 50 awaits a user event, a sensor event or sensor input.
All sensor outputs 94 and inputs from user switches 92a and 92b interact
directly with and only with the micro-controller 50.

[0037]After receiving input, the micro-controller 50 communicates with the
host system 46. The communications from the micro-controller 50 to the
host system 46 are transmitted via industry standard communication
protocols and devices, such as for example the receiver 80 and the
transmitter 82. The HID 42 does not require custom protocols, additional
or special hardware, or custom or special drivers in addition to the
standard protocols, hardware and drivers currently residing on the host
system 46.

[0038]The micro-controller 50 can interpret certain sensor outputs 94 and
certain inputs from user switches 92a and 92b as input that require user
feedback through the feedback mechanism 48. In those situations, the
micro-controller 50 enables the power supply 86 to supply power to the
feedback mechanism 48. The micro-controller 50 further directs the
feedback mechanism 48 to provide a level of feedback as limited according
to the user setting 92a. As described above, the feedback mechanism 48
can apply the feedback to the user in any desired form. In one
embodiment, the user input 92a can be channeled to the micro-controller
50, causing the micro-controller 50 to enable the variable signal
generator 90. The signal generator 90 communicates a signal to a power
amplifier which amplifies the signal so that the signal may vary the
state of the feedback mechanism 48 accordingly. The user's settings 92a
and 92b have an effect on the type of feedback the user receives. In one
embodiment, the user can adjust the variable signal generator 90 through
a user switch 92b to get a different form of signal wave and,
accordingly, a different form of feedback. While the illustrated
embodiment shown in FIG. 6 provides one example of suitable internal
components and the circuitry within the HID 42, it should be appreciated
that the HID 42 could include different internal components and different
circuitry that operate and function in a different manner.

[0039]The improved HID 42 provides many benefits over a traditional HID
12. First, because the sensor information processing is performed within
the HID 42 and communication with the host system 46 is not necessary for
processing the sensor information, the HID 42 is compatible with any
gaming or virtual reality system without software or hardware changes to
the host system 46. For example, a common HID can be used with different
host systems through adaptors. Second, because the HID 42 is independent
of the host system 46, any number of HIDs 42 can be used simultaneously
with the host system 46. Third, the HID 42 is customizable to communicate
with various types of sensors and provide various types of sensory
feedback. Fourth, the HID can have a wide variety of physical
embodiments. Other advantages are also apparent from a reading of the
specification and claims and from a study of the Figures.

[0040]It is also noted that a sensor can communicate directly with the
feedback mechanism, bypassing the micro-controller. For example, a
photo-sensor could trigger the feedback mechanism when the photo-sensor
is exposed to light. Alternatively, a pressure sensor button can cause
feedback to occur upon being pushed. In such an arrangement, the sensor
and feedback mechanism can be connected by circuitry.

[0041]The principle and mode of operation of this invention have been
described in its preferred embodiments. However, it should be noted that
this invention may be practiced otherwise than as specifically
illustrated and described without departing from its scope.