+1 on Rich's commentary. You're paying for someone to put the logic together to make the hardware perform those tasks. Even if you got an Arduino, there would would be some programming to have this work. There is really no way to get around the soldering unless you purchase an LED matrix...

Well, the problem here is that USB isn't really intended to be more than power and data lines. There's no logic to talk to an LED directly, so unless you are planning on cycling the power to that port on and off "by hand", you're going to need some logic living between the port and the LED. There are lots of small ARM SBCs that could do this sort of thing easily for you, but you're looking at varying amounts of cost there. I've been thinking of setting up an RPi Zero to do exactly this sort of thing, but via multiple LEDs. Soldering is very likely to be involved, though. You had said 30$ seemed high and that you don't want to get your iron out, but what other criteria are you working from?

I was looking at the ThingM blink(1) which is a simple USB notification LED. I appreciate that it has some features and there's even integration with IFTTT for instance, but $30 seems a little steep for such simple hardware. Does anyone know of a cheaper alternative? I'm not interested in breaking out the soldering iron.

I have a monitor with USB ports on the side, but I use it in portrait orientation, which would put the light on the top, which is nice, but I don't think I'd be able to use a dumb LED and turn it off by cutting off power to the port.