Typical interactions in games include shooting, picking up objects, and opening doors. From an implementation point of view, these apparently different interactions are surprisingly similar: The user first aims and selects a target in the 3D scene, and then triggers an action on it. We call this process picking.

You can pick something by either pressing a key on the keyboard, or by clicking with the mouse. In either case, you identify the target by aiming a ray –a straight line– into the scene. This method to implement picking is called ray casting (which is not the same as ray tracing).

You should see four colored cubes floating over a gray floor, and cross-hairs. Aim the cross-hairs and click, or press the spacebar to shoot. The hit spot is marked with a red dot.

Keep an eye on the application’s output stream, it will give you more details: The name of the mesh that was hit, the coordinates of the hit, and the distance.

Understanding the Helper Methods

The methods makeCube(), makeFloor(), initMark(), and initCrossHairs, are custom helper methods. We call them from simpleInitApp() to initialize the scenegraph with sample content.

makeCube() creates simple colored boxes for “target practice.

makeFloor() creates a gray floor node for “target practice.

initMark() creates a red sphere (“mark). We will use it later to mark the spot that was hit.

Note that the mark is not attached and therefor not visible at the start!

initCrossHairs() creates simple cross-hairs by printing a “+ sign in the middle of the screen.

Note that the cross-hairs are attached to the guiNode, not to the rootNode.

In this example, we attached all “shootable objects to one custom node, Shootables. This is an optimization so the engine only has to calculate intersections with objects we are actually interested in. The Shootables node is attached to the rootNode as usual.

Understanding Ray Casting for Hit Testing

Our goal is to determine which box the user “shot (picked). In general, we want to determine which mesh the user has selected by aiming the cross-hairs at it. Mathematically, we draw a line from the camera and see whether it intersects with objects in the 3D scene. This line is called a ray.

Here is our simple ray casting algorithm for picking objects:

Reset the results list.

Cast a ray from cam location into the cam direction.

Collect all intersections between the ray and Shootable nodes in the results list.

Use the results list to determine what was hit:

For each hit, JME reports its distance from the camera, impact point, and the name of the mesh.

Sort the results by distance.

Take the closest result, it is the mesh that was hit.

Implementing Hit Testing

Loading the scene

First initialize some shootable nodes and attach them to the scene. You will use the mark object later.

Tip: Notice how you use the provided method results.getClosestCollision().getContactPoint() to determine the closest hit’s location. If your game includes a “weapon or “spell that can hit multiple targets, you could also loop over the list of results, and interact with each of them.

Picking Action Using Mouse Pointer

The above example assumes that the player is aiming crosshairs (attached to the center of the screen) at the target. But you can change the picking code to allow you to freely click at objects in the scene with a visible mouse pointer. In order to do this you have to convert the 2d screen coordinates of the click to 3D world coordinates to get the start point of the picking ray.

Use this together with inputManager.setCursorVisible(true) to make certain the cursor is visible.

Note that since you now use the mouse for picking, you can no longer use it to rotate the camera. If you want to have a visible mouse pointer for picking in your game, you have to redefine the camera rotation mappings.

Exercises

After a hit was registered, the closest object is identified as target, and marked with a red dot.
Modify the code sample to solve these exercises:

Exercise 1: Magic Spell

Change the color of the closest clicked target!
Here are some tips:

Go to the line where the closest target is indentified, and add your changes after that.

To change an object’s color, you must first know its Geometry. Identify the node by identifying the target’s name.

Use Geometry g = closest.getGeometry();

Create a new color material and set the node’s Material to this color.

Look inside the makeCube() method for an example of how to set random colors.

Exercise 2: Shoot a Character

Shooting boxes isn’t very exciting – can you add code that loads and positions a model in the scene, and shoot at it?

Tip: You can use Spatial golem = assetManager.loadModel(“Models/Oto/Oto.mesh.xml); from the engine’s jme3-test-data.jar.

Tip: Models are shaded! You need some light!

Exercise 3: Pick up into Inventory

Change the code as follows to simulate the player picking up objects into the inventory: When you click once, the closest target is identified and detached from the scene. When you click a second time, the target is reattached at the location that you have clicked. Here are some tips:

Create an inventory node to store the detached nodes temporarily.

The inventory node is not attached to the rootNode.

You can make the inventory visible by attaching the inventory node to the guiNode (which attaches it to the HUD). Note the following caveats:

If your nodes use a lit Material (not “Unshaded.j3md), also add a light to the guiNode.

Size units are pixels in the HUD, therefor a 2-wu cube is displayed only 2 pixels wide in the HUD. – Scale it bigger!

Position the nodes: The bottom left corner of the HUD is (0f,0f), and the top right corner is at (settings.getWidth(),settings.getHeight()).

Link to user-proposed solutions: jme3:solutionsBe sure to try to solve them for yourself first!

Conclusion

You have learned how to use ray casting to solve the task of determining what object a user selected on the screen. You learned that this can be used for a variety of interactions, such as shooting, opening, picking up and dropping items, pressing a button or lever, etc.

Use your imagination from here:

In your game, the click can trigger any action on the identified Geometry: Detach it and put it into the inventory, attach something to it, trigger an animation or effect, open a door or crate, – etc.

In your game, you could replace the red mark with a particle emitter, add an explosion effect, play a sound, calculate the new score after each hit depending on what was hit – etc.

Now, wouldn’t it be nice if those targets and the floor were solid objects and you could walk around between them? Let’s continue to learn about Collision Detection.