The nucleolus is though to separate itself from the rest of the nucleoplasm via physical mechanisms resembling phase separation (like how oil droplets can separate from water). This is a very active area of research with many new results in the past couple of years. Here's a recent review on the topic: http://www.sciencedirect.com/science/article/pii/S0955067415000435