They're called flourishes. While they can be done in Photoshop I think, most of the tutorials point towards vector programs such as Illustrator or Inkscape. The easier method would be googling for free flourish brushes for photoshop.

/edit

Here's a tutorial for creating flourishes with the pen tool in photoshop.