Two awesome buddies and I are developing our first game, a top-shooter inspired by manic-shooters such as Touhou Project, Ikaruga. But we do like other games, like narrative, role playing games or 3D action RPG.

Anyway, we are a bit discovering Lua and LÖVE2D all at once, and as the dude in charge of organizing the programming, I was wondering about code organisation and resource management.

First, levels serialization. Our levels are composed of fine-drawed background, evil enemies with their patterns, blocking obstacles and the like. All of those are placed on certain coordinates, some are moving... We also have events and levels with differents phases (for instance, you kill a certain number of enemy type "bob", then "superbob" is coming to avenge them and replace them). Those do represent reasonable challenges enough for us, programming-speaking, but I would be happy to have your inputs. How do you store levels as complex entities? Are there standards formats (XML, CSV)? What tools exist to generate and read them? Especially, are there any map generator tools, or level generation?

Second, code organization. We separated our code in four main files:
player.lua, managing movement, drawing and bullets of players
enemy.lua, doing the same but for enemies. We split them from players because they follow patterns, which are special structures that drive how they move and shoot
map.lua storing our maps. For now we hardcoded our first level with its obstacles and spawn zones, but I would like to load them from the outside.
main.lua, doing a bunch of stuff, including animations and collisions.

This is a bit messy and our code is having some sort of existential crisis, not knowing were it belongs. Do you have any inputs on how to organize our code better?

I personally think it's well worth the effort of writing an in-engine level editor. Specifically, the editor should operate by letting you place and manipulate entities inside a frozen game instance.

Then, your level editor's format should ideally be the result of looping through each game entity and calling :serialize(). You should be able to take a serialized entity and reconstruct it exactly as it was before being serialized. If you satisfy this condition, then your level format also becomes the ability to losslessly snapshot a game-state, which is useful for making save files, debugging, replays, and many other things.

Obviously, when making an STG (or any game genre) you'll probably run into some things which might not fit exactly into this model. That's fine, tackle things as you see fit. Still, I think the basis of a coherent level format starts with rock-solid serialization.

JSON, XML, etc. Don't matter as they're all pretty interchangeable and have no real implications, but as others have pointed out, serializing straight to Lua is fastest and will have the fewest dependencies. I like Serpent. Just don't use CSV

This is entirely subjective, but I think the best option is using Tiled (http://www.mapeditor.org/). It entirely depends on what you want to do and how complex your game is, for really complicated projects I'd recommend tiled. But if you are doing a simple platformer making your own might be a better option because it could allow user created levels even

I have been working with Tiled for 5 months now and mostly like it, there are some draw backs though

Your game looks like it will have very strict rules and laws in your game world. This makes your game to a good candidate for an ECS bone structure (Entity-Component-System). So far I have experience with tiny-ecs and Concord. I would suggest you to go for Concord, because it has less limits and is faster than tiny-ecs, but there is also Lovetoys, HooECS and nata to consider. You should also implement a structure for gamestates by using hump.gamestate or using your own.

Having only 4 lua files in your whole game will cause some serious trouble as your game grows. Your main.lua should be only the entrance to your first gamestate and the systems of your ECS bone structure should bring the real logic into your game, actually. Entities and their components are only data structures without logic, then.

Once you adapt ECS, you will get rid of all the mess, because your systems are kinda modular and independent of each other, making it easier to find bugs and change code afterwards.

This is a good tutorial to understand ECS even more: https://www.youtube.com/watch?v=SE2Ded01lUU
However, this sample is made with tiny-ecs, so you should not look at the code too much and instead understand the sense and logic behind an ECS.