Getting the fan to rotate requires energy. In this case, the energy comes from the magnetic energy stored in material's aligned magnetic domains. The domains can be created via natural processes or the magnetism can be artificially produced by magnetizing the material in an electromagnetic field. In permanent magnets the domains must also be aligned. Either way, energy must be first put in to magnetize the material (=creating the domains) and that doesn't come free.

Furthermore, when the fan rotates, it consumes energy. As a result, the magnetic domains providing the energy for the fan will slowly shrink in size and lose their alignment. Eventually the permanent magnetic properties including extractable energy are lost. It's like a battery running dry, except that here the energy is stored in a magnetic field instead of an electric field like in a normal battery.

Yes, these are called permanent magnets, but permanent does not mean truly permanent. You can dissipate the energy or mix up the alignment of domains in a permanent magnet and thus destroy its permanent magnetism: extract its magnetic energy, heat it up, hit it hard with a hammer or use a demagnetizer, for instance.