Hey, Davidobot, if I could change the subject back to raycasting for a bit, I'd appreciate if you could help me solve a few problems I'm having with my own raycaster.

Now the main problem is since my code is much different from yours, things are done differently to get the same outcome. You seem to have solved a few of my problems so maybe you can help me solve them over here. We'll start with a simple one I guess...

I'm having trouble trying to get sprites to sort properly with the wall strips. Obviously their "distance" from the camera is calculated differently. I tried looking at your code but you calculate both your wall segment and sprite distances differently from mine which are calculated completely differently from each other. Here's the problem I'm having:

As you see, it seems that the sprite distance isn't properly "corrected". I've been ripping my hair out trying to figure out a good way to do it. Of course it only shows its ugly head when the sprite is next to a wall.

Here's my code. It's a dang mess I know. Some stuff is commented out because I just pulled it from my old project and haven't gotten around to implementing some stuff like doors yet.

function castSingleRay(rayAngle, stripIdx, checkOnly)
-- first make sure the angle is between 0 and 360 degrees
rayAngle = rayAngle % twoPI
-- if (rayAngle < 0) then rayAngle = rayAngle + twoPI end
-- moving right/left? up/down? Determined by which quadrant the angle is in.
local right = (rayAngle > twoPI * 0.75) or (rayAngle < twoPI * 0.25)
local up = (rayAngle < 0) or (rayAngle > pi)
-- only do these once
local angleSin = math.sin(rayAngle)
local angleCos = math.cos(rayAngle)
local vHit = false
local dist = 0 -- the distance to the block we hit
local xHit = 0 -- the x and y coord of where the ray hit the block
local yHit = 0
local textureX -- the x-coord on the texture of the block, ie. what part of the texture are we going to render
local wallX -- the (x,y) map coords of the block
local wallY
-- first check against the vertical map/wall lines
-- we do this by moving to the right or left edge of the block we're standing in
-- and then moving in 1 map unit steps horizontally. The amount we have to move vertically
-- is determined by the slope of the ray, which is simply defined as sin(angle) / cos(angle).
local tileNum = nil
local tileNum_u = nil
local hitDoor = false
local slope = angleSin / angleCos -- the slope of the straight line made by the ray
-- we move either 1 map unit to the left or right
local dX
if right then dX = 1 else dX=-1 end
local dY = dX * slope -- how much to move up or down
-- starting horizontal position, at one of the edges of the current map block
local x
if right then x = math.ceil(camera.x) else x = math.floor(camera.x) end
-- starting vertical position. We add the small horizontal step we just made, multiplied by the slope.
local y = camera.y + (x - camera.x) * slope
local floorTileName
while (x >= 0 and x < mapWidth and y >= 0 and y < mapHeight) do
local wallX
if right then wallX = math.floor(x) else wallX = math.floor(x -1) end
local wallY = math.floor(y)
local inside = wallX > 0 and wallX <= mapWidth and wallY > 0 and wallY <= mapHeight
floorTileName = "fb" .. wallX+1 .. "-" .. wallY+1
if not floorBoxList[floorTileName] then
floorBoxList[floorTileName] = { wallX+1, wallY+1 }
end
if inside then
-- if doorMap[wallX+1][wallY+1].kind ~= "" and doorMap[wallX+1][wallY+1].orientation == "v" then
-- visibleDoors[doorMap[wallX+1][wallY+1].id] = { active = true, orientation = "v" }
-- end
end
-- is this point inside a wall block?
local k = 1+(math.floor(wallY))*mapWidth+math.floor(wallX)
if map[k] > 0 then
wallBoxList["wb" .. wallX+1 .. "-" .. wallY+1] = { wallX+1, wallY+1 }
floorBoxList["fb" .. wallX+1 .. "-" .. wallY+1] = nil
tileNum = map[k]
-- tileNum_u = map_upper[k]
local distX = x - camera.x
local distY = y - camera.y
dist = distX*distX + distY*distY -- the distance from the player to this point, squared.
-- where exactly are we on the wall? textureX is the x coordinate on the texture that we'll use when texturing the wall.
textureX = y % 1
-- if we're looking to the left side of the map, the texture should be reversed
if (not right) then textureX = 1 - textureX end
xHit = math.floor(x)
yHit = math.floor(y)
ix = x
iy = y
vHit = true
if right then
-- if doorMap[wallX][wallY+1].kind ~= "" and (xHit == wallX and yHit == wallY) and doorMap[wallX][wallY+1].orientation == "h" then
-- tileNum = 12
-- end
elseif not right then
-- if doorMap[wallX+2][wallY+1].kind ~= "" and (xHit == wallX+1 and yHit == wallY) and doorMap[wallX+2][wallY+1].orientation == "h" then
-- tileNum = 12
-- end
end
break
end
if inside then
-- if spriteMap[wallX][wallY] > -1 then visibleSprites[spriteMap[wallX][wallY]] = true end
end
x = x+dX
y = y+dY
end
-- now check against horizontal lines. It's basically the same, just "turned around".
-- the only difference here is that once we hit a map block,
-- we check if there we also found one in the earlier, vertical run. We'll know that if dist != 0.
-- If so, we only register this hit if this distance is smaller.
local slope = angleCos / angleSin
local dY = up and -1 or 1
local dX = dY * slope
local y
if up then y = math.floor(camera.y) else y = math.ceil(camera.y) end
local x = camera.x + (y - camera.y) * slope
while (x >= 0 and x < mapWidth and y >= 0 and y < mapHeight) do
local wallY
if up then wallY = math.floor(y - 1) else wallY = math.floor(y) end
local wallX = math.floor(x)
local inside = wallX > 0 and wallX <= mapWidth and wallY > 0 and wallY <= mapHeight
floorTileName = "fb" .. wallX+1 .. "-" .. wallY+1
if not floorBoxList[floorTileName] then
floorBoxList[floorTileName] = { wallX+1, wallY+1 }
end
if inside then
-- if doorMap[wallX+1][wallY+1].kind ~= "" and doorMap[wallX+1][wallY+1].orientation == "h" then
-- visibleDoors[doorMap[wallX+1][wallY+1].id] = { active = true, orientation = "h" }
-- end
end
local k=1+(math.floor(wallY))*mapWidth+math.floor(wallX)
if map[k] > 0 then
wallBoxList["wb" .. wallX+1 .. "-" .. wallY+1] = { wallX+1, wallY+1 }
floorBoxList[floorTileName] = nil
local distX = x - camera.x
local distY = y - camera.y
local blockDist = distX*distX + distY*distY
if (dist==0 or blockDist < dist) then
dist = blockDist
tileNum = map[k]
-- tileNum_u = map_upper[k]
xHit = math.floor(x)
yHit = math.floor(y)
ix = x
iy = y
vHit = false
textureX = x % 1
if (not up) then textureX = 1 - textureX end
end
-- if up then
-- if doorMap[wallX+1][wallY+2].kind ~= "" and (xHit == wallX and yHit == wallY+1) and doorMap[wallX][wallY+1].orientation == "v" then
-- tileNum = 12
-- end
-- elseif not up then
-- if doorMap[wallX+1][wallY].kind ~= "" and (xHit == wallX and yHit == wallY) and doorMap[wallX][wallY+1].orientation == "v" then
-- tileNum = 12
-- end
-- end
break
end
-- if inside then
-- if spriteMap[wallX][wallY] > -1 then visibleSprites[spriteMap[wallX][wallY]] = true end
-- end
x = x+dX
y = y+dY
end
if checkOnly then
return xHit, yHit, stripIdx, height, vHit, textureX, dist, tileNum
else
if (dist>0) then
dist = math.sqrt(dist)
dist = dist * math.cos(camera.rot - rayAngle)
local height = (viewDist / dist)-- * ceilingHeight
createStrip(stripIdx, height, vHit, textureX, dist, tileNum)
end
end
end

Which calls this function to actually create the strip itself: (Note I have to invert the "layer" which is the equivalent to the "z" value since I use my own drawPool library which sorts differently from the original. Just ignore it.)

Hey, Davidobot, if I could change the subject back to raycasting for a bit, I'd appreciate if you could help me solve a few problems I'm having with my own raycaster.
...
Any help would make me so happy. I've tried everything except completely rip your code and put it in my version. I can probably provide the actual project itself if I need to.

I'm always happy to help. I do feel like this is going a bit off-topic in this particular thread. Would you like to maybe continue these and similar discussions on discord or something similar? If you'd like to add me, you can find me under "RinaLovesRobots [Davidobot]#9863" (without quotations marks of course)

Now, onto your code. The "getScreen2DPosition(e)" function just tells me how you get the distance to a point on your map. I don't think it shows how you actually draw or sort your sprites. In my project, I draw the sprites strip-by-strip. The distance to the center of the sprite is used to:
- Scale the sprite correctly and sort the sprites between one another.
- For each strip of the sprite, to check if that one strip is behind a wall or not, which is done by having a global zBuffer table with the closest wall distance at each point. So I check the appropriate strip against this buffer with zBuffer[strip] and check if that's less than the distance of the sprite (one distance per sprite, from the center of it). If the distance is greater than that of the buffer, I drop that one strip and continue.

I'm not sure if you render your sprite all at once, or strip-by-strip. If you have the former, try the latter. Otherwise, there is nothing in the code that instantly jumps out to me as wrong. I would gladly poke around in your whole project and help you find the error if you provide it. <- Ignore that, upon closer inspection of the video, it is obvious that you do it strip-by-strip. Since that's the case, look at the below recommendation.

Actually, I see that you're adjusting your wall distance to make it perpendicular to the camera with

However, I do not see you doing the same thing to your sprite distance, only for your sprite size. This perpendicular distance is what prevents a "fish-eye" effect of the sprites. It could be that it does cause a visual distortion because you adjust the sprite size for it, but it could be causing a distance fish-eye effect of sorts, which could be your problem. Try changing that part of getScreen2DPosition(e) to:

Success! It worked! One problem solved. I knew it was a "correction" problem I just couldn't figure out how to properly do it.

For the record, no I don't render sprites column by column like you do. I didn't feel it really necessary and never thought of it. I saw you do it in your code and wonder if there's a reason? Does it have any benefits? I'd think it would just add extra unnecessary slowdown to an already slow process? Maybe you know something I don't. lol

Anyway, my next problem is with determining what floor/ceiling tiles to render. You might notice some odd code in my example above. Lines that begin with something like floorBoxList["fb" .. wallX+1 .. "-" .. wallY+1]. This is my hacky way of "marking" tiles as "visible" so I can run through them and draw floor and ceiling tiles, or "flats" as I refer to them (Borrowing a term from DOOM. I'll also refer to sprites as "things" in the future.) but it seems to be buggy as it consistently marks non-visible tiles as visible.

For example:

In the video you see all the flats that are being rendered. Including some that are hidden behind walls and should not be seen at all. This adds a lot of unnecessary render time since the texture shader is already slightly slow.

For the record, no I don't render sprites column by column like you do. I didn't feel it really necessary and never thought of it. I saw you do it in your code and wonder if there's a reason? Does it have any benefits? I'd think it would just add extra unnecessary slowdown to an already slow process? Maybe you know something I don't. lol

Well, my original version didn't really sort the sprites and the walls. It just drew everything that should be there. So it would discard unneeded strips and just draw everything. Hence, eliminating the need for sorting. Though, I think I have some sort of sorting now tho so I guess it's just a relic.

In the video you see all the flats that are being rendered. Including some that are hidden behind walls and should not be seen at all. This adds a lot of unnecessary render time since the texture shader is already slightly slow.
How would I properly mark only visible flats?

Right then. I'll first explain my raycasting procedure -
- Increment the position of the ray on a tile-by-tile basis.
- If there is no wall, add the floor tile to a table and continue to increment.
- If there is a wall, add to the the wall strips and break the while loop.

Something odd about your code is that you loop through the system twice? I don't really know why you need to do that for "vertical" and "horizontal" lines separately. Also, you seem to add things and then remove them, I would recommend you just add the floor tile if there is no wall there.

If that doesn't solve the problem, there are multiple other possibilities:
- You don't clear floorBoxList after finishing your draws and hence tiles stay selected, unless they are "nilled" out by the code.
- Try adding floor only on one of the runs. As in you add them on both the vertical and horizontal ones, but try just adding them on the vertical, or vice versa.
- Your code flags up tiles visible on either of the runs, but not ones visible on both of them, as it should. So maybe make sure that the tile you're flagging is visible in both of your runs.

If I had to take a guess, your problem would be the last of the "other possibilities", as in that the tile needs to be visible from both runs.

If you have a better way of doing the raycasting loop, let me know. The one I have is also a relic of the original converted code I got from another member long ago. It was originally a JavaScript engine. I don't know why it goes through twice. It seems if I disable one, then it only renders opposite corners of the map but leaves the others empty. So who knows. I don't really understand much about how it works, just that it works.

Also, floorTileName is just shorthand for the "fb" .. wallX+1 .. "-" .. wallY+1 part. I just didn't replace it completely.

I'd really love to figure out a better way to do it to make sure the flats are optimized. I hate wasting time rendering unseen tiles.

I also need to figure out a way to do transparent walls like you do. The code I have doesn't account for them. But yours does. No idea how you did it. I'd like to have fences and grates you can see through. In my experience trying to modify the code I have, it wouldn't work and would just stop rendering when it hit a transparent wall like a normal wall.

I also need to reimplement doors properly. My old method had "problems" with broken seams. I never did understand how to do them and the JavaScript code didn't go far enough to talk about doors. (Though it didn't do flats either. Hence why I needed to come up with my own method.)

My original floor rendering method was much different. It was two canvases. One with the whole floor. One for rendering. It would rotate and draw the floor onto the second canvas and render that canvas as horizontal strips. But it looked bad. It worked, but it looked bad as rotating pixels would look weird and wonky. The new shader method works much better, but is much slower. I just need to optimize it.

Mine is quite similar to yours. I just calculate which way the ray is generally going (left or right; up or down) and then I have "delta" values for both the X and Y directions that get incremented every step, by the respective cos/sin of the angle. If the delta for X exceeds 1, then it's time to step left or right, which is does and then resets the delta value. Same thing for delta of Y.
This way, I am stepping one tile every loop of the while loop. When this happens, I check if the tile the ray is in, if it is:
- empty: add the tile to the floor list.
- a normal wall: add strip to the drawing list and terminate.
- a door or fence: add strip to drawing list but do not terminate.

I also need to figure out a way to do transparent walls like you do. The code I have doesn't account for them. But yours does. No idea how you did it. I'd like to have fences and grates you can see through. In my experience trying to modify the code I have, it wouldn't work and would just stop rendering when it hit a transparent wall like a normal wall.

I think this will be easier to do with a better-written raycasting loop.

Would you like to send me the project file so that I can try and fix the floor bug?

I tried it last night. With the way I do it (The dual checks against horizontal and vertical walls separately) it's not going to work the way I thought. I might need to just borrow your method and modify it to work with my code. Did you create yours from scratch or use another source initially? Because mine was obviously a really old school method. We'll figure it out I'm sure. I'll look into modifying your raycast loop to see if it works for me. I think as long as it returns the strips in the right format with the right "distance" for sorting that matches the format of the sprite distance, it'll be fine. And if it helps with floor/ceiling culling, even better.

I whipped up a .love project for you. It's all the relevant code for the raycasting engine. All the code you need to see is in states/fps.lua. The rest is just the game wrapping, state stack, libraries, etc. I had to actually do a bunch of work to transplant it from my working project (Where it was just a separate state I was toying with) but it works.

Press F1 to show the performance graph. Blue is draw time, green (Which you can't see any because all the stuff is currently done in draw for performance reasons) is update, the darker blue is the texture draw time. You'll note that it takes up half the total draw time depending on how big a room is.

F3 shows the debug grid marking all the flats that are being rendered including unseen ones.

Press Tab to toggle mouse look on and off if you don't like using the A and D keys for turning. (Arrows are for movement and strafing. Standard FPS controls.)

Press = (equals) to change the wall height between 32, 48 and 64 pixels. (I am partial to the 48 height.) I'm going a different direction than you with higher walls. (The GunGodz direction)

Press 0 (zero) to toggle ambient occlusion on the flats.

Press F (Flats), W (Walls) or T (Things) to toggle between textures and no textures. Press / (slash) to toggle all at once. It's just for debugging. But it does help show that drawing a wall with an image or a rectangle is the same and has no impact on rendering. Textures on flats of course does. Try pressing these keys when the performance graph is open)

(Sorry it's a .love file in a .zip file because for some reason I can't upload a .love anymore?)

Thanks for helping! It's my dream to make a GunGodz style tribute. Basically it has everything I want to do myself. Especially transparent "fence" style walls which you do yourself, but GunGodz also has a second type where the fence is halfway between tiles instead of just on the edge. But I guess if you make a "door" that doesn't actually open, it'll do the same thing. The taller ceilings are great too which is why I implemented them in mine.

I tried it last night. With the way I do it (The dual checks against horizontal and vertical walls separately) it's not going to work the way I thought. I might need to just borrow your method and modify it to work with my code. Did you create yours from scratch or use another source initially? Because mine was obviously a really old school method. We'll figure it out I'm sure. I'll look into modifying your raycast loop to see if it works for me. I think as long as it returns the strips in the right format with the right "distance" for sorting that matches the format of the sprite distance, it'll be fine. And if it helps with floor/ceiling culling, even better.

I looked into your code. I think you'll have to change the raycasting loop to mine or something else, because I cannot, for the life of me, figure out how to make it work properly. I nearly got it to work, but if a floor tile if between two walls, like in the picture below, it won't render.

Thanks for helping! It's my dream to make a GunGodz style tribute. Basically it has everything I want to do myself. Especially transparent "fence" style walls which you do yourself, but GunGodz also has a second type where the fence is halfway between tiles instead of just on the edge. But I guess if you make a "door" that doesn't actually open, it'll do the same thing. The taller ceilings are great too which is why I implemented them in mine.

I wish you luck! I want to make a polished small game using my engine too, eventuallyTM!

So about your next step, I would recommend you look into my "single-run" raycast loop and try to understand it and adapt it to suit your engine. Also man, I really like the Ambient Occlusion, just sayin'.

Yeah, I definitely need to use your loop. Mine is so old school. And with it doing the horizontal and vertical walls separately, it makes it a lot harder to implement other stuff. I'll see what I can do. It shouldn't be too hard.

Maybe.

Edit: Yeah it's a bit harder than I thought. You have a lot of different methods than I do. I have no equivalent for your camera.dirX/Y or camera.planeX/Y variables. My camera "dir" is a radian direction and I don't know how to properly convert that to the required dirX/Y values. And I don't even know what planeX/Y does. Your rayCast function accepts a single value of x to get its information which seems to be equivalent to whatever column of the screen it's scanning while mine accepts the angle of the camera. Yours also seems to only render in 4:3 instead of dynamically resizing to the aspect of the screen like mine. There's a lot of stuff I'm going to have to figure out.

Edit 2: Well it's a start but I got something working. It's not 100% though. I need to figure out a lot of things about your code. But hopefully I can make it work. Here's what I have so far with your raycasting overlaid on mine:

It.... does not work. Like at all. There's no "perspective". I am sure it's because I don't know how to properly translate some of the values you use that I don't have equivalents for. Here's the code I have so far:

Edit 2: Well it's a start but I got something working. It's not 100% though. I need to figure out a lot of things about your code. But hopefully I can make it work. Here's what I have so far with your raycasting overlaid on mine:

Oh yeah, you definitely need to know a few things about my raycasting method before you start tinkering.dirX and dirY do indeed control the angle orientation of the camera.
However, planeX and planeY are also the angle orientation.. Kind-of.

This is the code to rotate the orientation, where dx is the amount to rotate. That cos/sin mess is just a rotation matrix.

The aspect ration is actually determined by the starting value of planeY, which is also the FOV. The 0.66 value there is because I use a 4:3 ratio. The formula for finding the aspect ratio from planeY and vice versa is:

Also, you need to declare the dirX/dirY and planeX/planeY in the exact form they are now. If you want to start at a custom rotation (not math.pi) then start with the values I have and just rotate them using the rotation code.

EDIT: Btw, just so you know, after that fence/door comment, it's just a copy/paste of the strip-adding code from further down. I didn't move it into a separate function as I should have.

The eye-height is the player.zHeight value in lines 72, 73, 153, 154 of raycasting.lua; if you have a different eye-height, you need to change that. It's value in mine is 0 and that corresponds to eye-level being at exactly half the screen width. If you set zHeight to h, the player will be exactly level with the floor; -h and you'll be twice the current height. So if your eye-height is at 0.75 wall height, you need to set zHeight to -h/2.