So i am totally new to Dolphin got it today and tried Super Mario Galaxy, things were fine at start i could play the game with keyboard, but was not very convenient so i hooked up my PS3 controller and emulated it to x360 controller.

Since i did that i keep getting this error message to connect the nunchuck i have looked around and found that changing to hybrid controller should solve this but did not work for me.

Changing Hybrid Wii Remote should not solve this. Set the controller to Emulated Wii Remote instead of Hybrid, then open the configuration window for it and change the Extension dropdown (I think it's in the second right-most column) to Nunchuk. After that, press the Configure button below the Extension dropdown and map the controls of the Nunchuk.