2 months ago a friend caught a swarm and we placed it in my new brood box containing new frames with foundation. This colony has been slow and somewhat inactive compared to my other hives. I inspected 3 weeks ago, they looked healthy, so I placed the QE and super on.
However they never did any work above the QE. They would go up there, but no waxing whatsoever.
Yesterday I inspected the hive and there was no brood at all, but lots of honey. I noticed a few small, empty queen cells, and I could not find the queen. Is she gone? dead? What to do now?

If there is no brood or capped worker cells you can assume the hive is queenless at this time of year. Whether she has gone or died is not important.
I would buy a new mated queen and introduce her into the hive for a couple of days so that she is accepted by the colony.
Your other option is to buy a nuc and combine the colony using the newspaper method placing the nuc at the bottom of the hive, keeping the QX in place above what will be the brood box.
Hope that explains what I would do clearly Ric
Cheers

Hard to know, but if your other hives can afford it, I would put a frame of very young brood in the hive, preferably with eggs or very early larvae. If they are queenless, they can then make another queen.

should I remove a frame of honey and replace it with a new foundation frame?

I would remove a frame of honey and replace it with a frame of brood from another hive. If they have been queenless for 2 months, the workers in that hive will be getting a bit old to look after a mated queen. You will need new young workers to feed and care for her and her brood properly.

Both suggestions are good- either give them a frame with eggs and the smallest larvae- or get that new queen ASAP. If your lucky they only went queenless recently- in which case the remaining bees won’t all be that old.

If there are hardly any bees you’ll be better off combining them with another hive.

It sounds possible your hive swarmed- there is a chance there is a virgin queen in there somewhere- check regularly to see if any eggs appear

UPDATE.
Just did another inspection and found capped brood on 2 frames… there was also a good amount of lavae. woo hoo! The bees seemed calmer and they seemed to be more active. feeeeew!
Didn’t find the queen, but will inspect in a week to watch progress.

I don’t know if you are familiar with the old cartoon serious, The Wacky Races? The lead villain often said to his accomplice, “Don’t just stand there, Muttley! DO something!!” Well, I used to work in an intensive care environment. One of my best mentors used to say, “Don’t just DO something, STAND there!” In other words, patience is golden.

Well done, in any case. You did the right thing, and your bees are fine.