Alright, i figured out the transport fail error. Guess lufa or the avr usb stack doesnt like there being an IN endpoint and an OUT endpoint on the same address, so i just swapped the OUT address to 0x02. The firmware can now read incoming requests from the switch, and react accordingly.
The only problem now is that the switch doesn't want to detect the firmware as a controller, so it seems i'm still missing something.http://www38.zippyshare.com/v/PjZ1M3gX/file.html
Filter for this dump is (usb.addr contains "1.87.")
The communication LOOKS okay, the firmware is reading the input data just fine, and replying to it correctly, so im kinda stumped here. I'm taking a break for now and trying again tomorrow.

Alright I got it working. The switch correctly detects the firmware as an official Switch Pro Controller. The only things left to do now are figure out how to encode the thumbstick/gyro/accel data into a format the switch expects. I'm not that good at figuring out how bitshifting/bitmasks work (which apparently are crucial as thumbstick data is encoded by shifting and masking certain bits), so i'm struggling quite a bit here. If you'd like to help me figure out exactly how to encode this thumbstick/gyro/accel data I would greatly appreciate it.

Thanks for that! The switch is now correctly detecting the joysticks using the formula you gave me! I'll try and mess with the gyroscope values now and see what I can come up with.

Edit:
Early celebrations wew. Left stick is properly detected, right stick seems to trigger certain buttons. Seems like i messed up somewhere, gonna try and dump communications and see where i went wrong
Edit edit:
disregard me i slightly messed up a memcpy on a serial routine

The current goal now is to get sensible gyroscope values. I'll see what I can do about that.

well, its done. updates at the expected maximum rate of 120 updates per second, with full gyroscope/accelerometer support. rumble info is not sent over serial to the hosting application since i personally dont feel thats necessary, but it can be added by parsing specific packets. i made a simple c# program that acts as a mini-gimx to test with, since the current firmware uses my own protocol. i even tested it out in splatoon 2 to check out how well it controls and, other than small issues with the proper Z value to use (which i'm currently attempting to fix), it works quite well!
i'll go ahead and start cleaning up/commenting my code and i'll send it over to you to implement with gimx as you see fit!