Open your reference, tap on the Organize button from the toolbar to get to the Groups Management window, tap on the "All" group there.
Note: because of a bug the "All" group isn't visible in the Groups Management. Please temporary move your reference to any of your static groups, then find this reference in the library, call the Groups Management window again and unselect your group there.