First of all I have to mention that grouping wasn't intended to be used for bigger amounts of images but just a few ones instead. For example to group RAW+JPEG, or the different exposures for a HDR shot.

That being said I know that there is a significant speed problem when changing a group (this also happens when changing the representative). The problem is that we are caching the group_id in the image struct for quick access when drawing the lighttable, however the problem is that updating the group needs to iterate over all images, get a write lock from the internal cache, change the stuff and clean up. It used to be a single SQL query which was blazing fast, but it made drawing the lighttable somewhat slower. So in the end it's a trade-off we have to do.