Body text size when designing books

I am new to design and was wondering how to decide what size the body text should be when creating a book. Are there any methods or rules to work it out? Or is it just a matter of preference depending on the font?

There's no formula, but some factors that are particularly important to consider are the size of the page (or width of the text column) and the x-height and width of the font. Also budget constraints, printing method and paper choice, the kind of book you want it to look like and, as Riccardo mentioned, the target readership.