A year and a half later I have found my solution. This was mostly because Scan Tailor outputs 600 dpi TIFFs and I always resized the covers so it wouldn't impact the filesize without any perceivable difference, which looked something like this:

[cover]

[_____page_____]
[_____page_____]
[_____page_____]
[_____page_____]

If you look at the above post, the cover resolution was maybe half of the rest of the pages, which doesn't fit too well with the rest of the PDF.

Here's how:

After you straighten the cover, crop it, apply the convulsion matrix filter in Gimp or whatever to remove the printing dots, and so on, open it with Inkscape (embed, link doesn't matter, you're saving it as a PDF anyway).

Go to File - Document Properties and set the width (don't worry about height) to whatever pixels/inches/centimetres you want and hit enter. You will immediately notice the bottom shadow shrinking if you look closely. Good.

Click the image and drag the top right corner using the Ctrl key (to preserve the aspect ratio). Drag it inside the new page.

View - Grid (# key - or Shift+3), zoom in and drag the top right corner until it snaps to the margin with a little red X. Make sure you zoom in close enough.

Go to File - Document Properties again and click "Resize page to content... - Resize page to drawing or selection."

File - Save As - PDF (PDF 1.4 but don't worry, Acrobat will save it with a newer spec when you'll merge it with the rest of the document. Also, 90 dpi by default, everything else unchecked).

So there you have it. As a final touch up, you can use Inkscape to add text behind the image (Object - Lower to Bottom) to make the entire PDF searchable, which is pretty cool, especially for the back cover.

PS: You will also notice that the Inkscape PDF is huge. I think it's because Inkscape saves it as BMP... Thing is, if it saves your JPG cover as BMP and then you let Acrobat save it as JPG again, you're basically reducing the quality of a reduced quality picture, which is bad. Best would be to initially save the cover as PNG (after you're happy with how it looks) and let Acrobat compress to JPEG - Maximum Quality when you optimise the PDF, else you're gonna lose quality. Big time.