Si I have a theme called Oxygen that comes with Fancybox (similar to carousel). I like Carousel better so I went on and disabled Fancybox then enabled Carousel on my jetpack, well nothing happens (image opens at full size).
Yep - I have Carousel activated in my Jetpack.

Hi, I ran with the same issue running the plugin gallery carousel without jetpack. Oxygen for some reason the theme comes with it's own carousel style ("cleaner gallery") which I don't like and always overrides ANY other carousel. To solve this issue in the dashboard I went to Apperance - Editor - functions.php and look for

/* Add theme support for framework extensions. */.

Not to be confused with

/* Add theme support forcoreframework features. */

Remove this line of code

add_theme_support( 'cleaner-gallery' );

Click update file and you should be able to run the carousel style you want. Just to let you know I also made sure to disable "fancy box" from Appearance-Theme Settings. I hope this helps and good look.