For the longest time I've wanted to make a true 4D video game. Sure lots involve higher dimensions and some have really well done 4D elements, but there are no games that really take this to the next level.

I want a game that doesn't treat the 4th physical dimention as different from the first 3. It would be just like how in Mario 64 you can't think of it as 2D mario with depth, since it's a true 3D game.

So how would this be possible?

No clue. What way would you do it?

The big hurdle imo is graphics. 4D objects are hard to display on 2D screens. The common method is to use wireframe 4d objects and cast them into physical 3D shadows and then render the shadows like standard 3D objects onto the screen. I don't like this because it makes the scene feel 3D not 4D, and the graphics are limited to just lines.

I'd prefer to project the 4D scene into a 3D volume exactly like how a camera works. This volume would be thick and could maybe be viewed in VR if it was slightly transparent. Another, stranger, method would be to somehow map this 3D volume to a 2D space that would then just act as an image on the screen. Could mayber use a hilbert curve to map the 3D volume to a line and then another hilbert curve to map it into a 2D picture.

This resulting image would look like a random mess of colors. My hope would be that it'd visually give enough feedback though for the brain to figure things out. Not a "good" solution but would be cool.

If there was a true 4D game, we would have played it in the past, but it wouldn't have been made yet :O

Though there is that 4D rubiks cube.

4D rubiks cubes are probably the closest thing to what I'm wanting since they def have 4D gameplay, but they still aren't that good at rendering them or treating the 4th dimension as identical to the other 3. (Fun fact, 4D 2x2x2x2 puzzles can be played with as physical 3D objects. I guess it could happen for any nD puzzle, but 2x2x2x2 is common)

If you want to have the 4th dimension be time, I'm pretty sure every game ever would qualify

4D rubiks cubes are probably the closest thing to what I'm wanting since they def have 4D gameplay, but they still aren't that good at rendering them or treating the 4th dimension as identical to the other 3. (Fun fact, 4D 2x2x2x2 puzzles can be played with as physical 3D objects. I guess it could happen for any nD puzzle, but 2x2x2x2 is common)

If you want to have the 4th dimension be time, I'm pretty sure every game ever would qualify

Yeah I've seen that one but it's still presented as 3D+1D, not full 4D.

Imagine a table, a ball, a light, and a camera. This setup in 2D would look like a 1D line going color of table->color of shadow->color of ball with a bright spot from the light->color of table. IE:

In 3D this setup would look like this:

In 4D, the image is a full 3D volume with every part colored in, the base color being the color of the table. Inside the cube is a 3D looking sphere. Next to that sphere is a stretched sphere with the color of the shadow. Inside the first sphere, the colors get brighter the closer they are to a point away from the shadow sphere. This point is the specular highlight from the light.

Games like the 4D toy box you posted are only showing a 3D slice of the 4D world. The physics behind it are definitely 4D, but it's rendered and interacted with as 3D+1D. It's comparable to a side scrolling platformer that allows you to move in and out. I'd love something like my ball+table example above where it's actually attempting to show you a 4D world.

Something I was thinking about earlier, was having a game use both analog sticks for the camera. Each dimension is important just like all the others.

On your 2d tv, you'd get your 3d representation of a 4d space.

Yes exactly.

It could probably work somewhat decently in VR. Put you in a virtual "control" room with a volumetric 3D display ie:

Then you'd also be able to use the VR controllers to each act like a 3D analogue stick and control the game in a "relatively" easy to understand way.

Hmm I like this idea a bit too much. After cube royale I might try making a prototype and see how realistic it is for me to make. 4D graphics could be done with raycasting and signed distance fields. Adding another dimension would only be a minor change. Getting it running at a smooth 60fps would be the hard part. Could either make the volumetric display only 124*124*124 voxels or try to do something dynamic based off camera position.

Something I was thinking about earlier, was having a game use both analog sticks for the camera. Each dimension is important just like all the others.

On your 2d tv, you'd get your 3d representation of a 4d space.

Thinking about this more, a Doom style game in 4D could map to a regular controller.

Left joystick left/right would move you left/right. Up/down would move you 4D+/4D-. R2/L2 would move you forward/backward. You move along 3 dimensions just like how in Doom you move along 2 dimensions. Moving up and down would just use a jump button.

I like this setup because it'd work well with the 3D picture of the 4D world. Left joystick would push all objects in the 3D image the opposite direction, with closer objects moving faster. Moving forward/backward would be look zooming in and out from the center of the 3D image. Pressing jump would move things in the 3D image down and then back up.

Camera would just be the right joystick. Since it's Doom like and you can't look up or down, it restricts the camera to spinning around the axis perpendicular to ground. In 4D an axis can spin in 2 different ways (pretty sure just 2) so that fits the joystick. Left/right would turn left/right and up/down would turn 4D+/4D-.

How would this game feel to play? The 3D picture of the 4D world would be positioned right in front of the player. Up would be up, left would be left, and forward away from the player's position would be 4D+.

If the game was set on an infinite plane with the camera looking at the horizon (but in 4D), the resulting 3D image would be divided in half. Top half has the sky color, bottom has the ground color. If you had a slow moving hypersphere bullet and fired from the center directly forward, the 3D image would get a sphere put in the center but large enough to fill the volume. Everywhere the sphere is, it cuts away at the sky and ground color. (A spot in the 3D picture can only belong to one solid object). As the bullet flys forward, this sphere in the image would get smaller and smaller. Eventually it'll get so small that it can't be seen any more (it's really far away in 4D). If the bullet fired fast, it'd only be visible in the 3D image for a few frames.

Now imagine there is a target. This will also be a hypersphere to make things easy. The hypersphere will be twice as tall as you and resting on the ground, so in the 3D image it looks like a sphere positioned right between the sky and ground. If it is close to the player it is large and if it is far away it is small. It can be at any point along the sky/ground boundry in the 3D image, just like how an enemy in doom can be at any point to the left or right in the screen. This hypersphere could be outside the edge of the 3D image and be behind you.

If the hypersphere is moving, you'd want to keep it visible within the 3D image. This would mean rotating the 4D camera. If in the 3D image the sphere image of the hypersphere is to the left of center, pushing the right joystick left would bring it back to center. If the sphere in the image is forward of the center, pushing the right joystick forward would bring it back.

This on its own could be the basis of a 4D game. Shoot hyperspheres at moving targets and rotate your camera to aim at them. Would probably need a high field of view and/or a tracking system since finding enemies off screen would be hard. A normal 3D game just has you swing the camera around to see everything yet here that method would miss a lot of area. Also having objects within the scene would really help to keep the player from losing track of their direction.

The graphics would be really important. With a 2D image of a 3D scene we understand size and distance but with a 3D image of a 4D scene this would be very hard. Is a hypersphere large or is it close? A checkerboard like pattern on objects would help with this. If you always know the pattern is the same size, then 3D images with small patterns would be far away. Shadows and lighting would also really help with this.

On the other hand, the 3D image itself will be hard to understand if there's too much detail. A dot like texture for all objects could work really well both for understanding the scene and understanding the 3D image. Would make it more abstract though.

A rasterization like method might be a smarter approach. Break the scene down into tetrahedrons and rasterize them one at a time onto a 3D pixel grid. Originally I dismissed this idea because I was thinking of rasterizing them into vector like 3D graphics to then be rendered again by the final regular camera, because such a method would involve costly boolean operations.

However if it's done as a true analogue to rasterization onto a 2D screen, then that'd be avoided. The 512x512x512 or so grid would have a color value and a depth value for each voxel. When rasterizing a new tetrahedron onto the grid, it'd do a depth check per voxel and only change the color if the new object is in front of the old object.

This method would be relatively fast for simple scenes and I imagine the resolution would be limited more by memory than processing power.

A hybrid approach could potentially work even better. Drop the grid's resolution compared to the previous method and don't record color per pixel. Instead record object ID per pixel. Then only when the final 2D image of the 3D image is being calculated would the textures and lighting be calculated. That way it's a per pixel effect on the final TV image and the initial grid's resolution would only matter for depth testing. Changing its resolution would only affect the borders between the 3D images of the 4D objects which while important, isn't that important.

This hypothetical 4D game is coming along nicely!

On paper it does seem like something I could potentially manage, so maybe I should figure out the 4D gameplay possibilities? Would this just be a little sandbox for the player to explore and observe a truly 4D world, or could it offer fun gameplay that isn't just mimicking what players are already used to?

The above example about a Doom style shooter would be almost like a isometric top down game in its basic form. Without motion it'd literally feel like playing a game where you're a satellite or something moving to chase people down. The world looping back around on itself like if it was a super small planet wouldn't really bring much to the table either. With motion and a maze like setting it could be really weird. Occlusion would probably become the big game element as it'd look like enemies disappearing into other objects. You would have to move to make them reappear again. Maybe some sort of x ray vision might help with this to stop it from being too confusing.

From a mechanics point of view, on the 3D image of the 4D scene you'd see the shape representing the cover/wall and the sphere representing the enemy moving towards that first shape. When it gets there if it is closer to you (is less dense with its texture than the other shape's texture) then it cuts into the other shape and stays visible. However if it is farther away from you in 4D, then the other shape cuts into it and the sphere stops being visible once it's fully inside. To make it come outside and thus become visible again, you'd push the left joystick in the opposite direction of where the wall shape is compared to the center of the 3D image. This would move the camera in 4D which would result in pushing everything on the 3D image screen in the opposite direction of the joystick. However, the 3D representations of 4D objects would move faster the closer they are to the camera, so the wall obstructing the enemy would have its 3D image move out of the way revealing the image of the enemy again. You'd also want to push the right joystick in the opposite direction so that the enemy and wall's images stay centered on the 3D screen.

So that gameplay wouldn't be too bad. Sure it'd be a simple 4D shooter but it'd be a good way to push the player to move about the world.

How about a race game? The track would be 3D in the 4D world. If we have the same camera setup as before with the infinite ground and leveled camera, that'd look like the 3D image box with the top half sky color, the bottom half ground color, and a pyramid rising from the bottom through the ground color to the exact center, assuming the camera was aligned with the track. Going forward more or less means expanding the 3D image as if you were zooming in on it. The 4D car could turn left/right and 4D +/- and so the racetrack could as well. In the 3D image the track turning left would look like the pyramid twisting its top left and then extending it off the image. Turning the camera/car left would just drag the whole 3D image right (some distortion but not based off 4D distance). Keeping the car on the road would just mean keeping the center bottom of the 3D image within the base of the road's pyramid. Doesn't sound that fun to me and would be pretty comparable to just flying in a 3D game. Sure you could spice it up with visuals along the track, other cars, and maybe even some jumps, but that wouldn't have a large impact on how it actually plays.

What about a 4D platformer? Again using the example scene of infinite flat ground, but this time with a pit right in front of the camera. The 3D image of the 4D scene would be the color of sky for the top half and color of ground for the bottom half, but within the ground would be an image of a shape for the pit. It'd be kinda hard to tell it's a pit and not some other object on the ground unless it was fully shaded in. If the player jumps, the entire 3D image moves down with closer objects moving faster, and then moves back up as the player lands. Moving forward makes the whole 3D image expand from the center with closer objects expanding faster. So if you moved forward without jumping, the image of the pit would just move down and off the screen as you reach it. If you jump and move forward, the pit also moves down off the screen but doesn't return when you land. A player avatar doesn't really simplify this since without shadows and a solid understanding of 4D depth, it's just as abstract. IE the 3D image of the player could be right next to the 3D image of the pit yet in the 4D space could be very far apart.

Also a 4D platformer would be really hard to design. In 2D, a hole/death zone must be jumped over no matter what since by default the hole blocks player progression. In 3D, the player can walk around the hole. This is why 3D platformers can't really do holes and instead focus on the platform itself being a safe space. In 4D it'd be even more extreme where you essentially have to make everywhere a kill zone and just a few dots of safe space. Would be very hard to design without it just feeling like jumping from one safe space to another. Plus it wouldn't feel that unique? Sure it'd require setting up the correct camera angle, but once that's done it'd just be a linear jump.

So a simple shooting game seems like it might actually be the best option. First level could just be a big empty plane, while additional levels would put more and more around you until you're inside a 4D building.

Ok we've figured out potential ways to render the graphics. How would the game engine side of things work?

For starters I'd have to essentially create my own game engine. Unity is very customizable but none of its default things could be modified to 4D. Instead I'd need to build absolutely everything myself through scripts.

Some of these things would be pretty easy. I understand vector math and position/scale are simple to expand to 4D. Rotations however would suck. I barely understand how quaternions work, much less the math behind them, so building my own 4D versions would be hard. 4D objects have 6 degrees of freedom for rotation so I'd need two quaternions to define 1 rotation or use some other way to sort it out. A good solution is important since rotation will be used for graphics and physics too.

Speaking of physics, I'd need to program my own methods from scratch. I've already done this for VizionEck with some things but I'd need to heavily optimize it and add standard functions like raycasting and spherecasting. 4D objects are very "dense" so I might need to fake a lot of things to make it run reasonably fast. Would definitely help if I built the scene out of primitives instead of tetrahedron meshes.

Designing and building the game would require overhauling the unity editor window and making it run in 4D as well. Again I have a little bit of experience with modifying it so this won't be easy but should be doable. Could sync the editor with my custom list of objects and just make it an inactive view of what's really happening since during runtime the unity scene would be empty.

Programing an engine from scratch would probably be more efficient but I think it'd be smarter to start with Unity as a framework. I'd want to get this to a prototype state as fast as possible so I can really assess the idea.

With how fast you seem to be putting this together, one would think VizionEck would've released months ago at least.

Spoiler for Hidden:

Jk I know you haven't done that much.

Haha the early stages of game dev are the best.

Yeah the only actual work I've done was last night. 4D rotations are core to practically everything so I was trying to sort the details out.

Still not sure when I'll actually start a prototype. I really don't want to spend much time on this since it's not that interesting from a gameplay perspective. Might even just release it as a tech demo only.

We play games on TVs with controllers and don't need an explanation for that, so maybe there shouldn't be an explanation for viewing the 3D volumetric display. It'd just be a mechanic.

The game would be exclusively 4D with only lore/worldbuilding relating to whatever story I want to tell in that 4D world. You wouldn't play as a person controlling a 4D object, you'd be playing directly as a 4D being. Would be cooler and also decrease rendering complexity.