OLEACEAE FAMILY
Known as the Olive Family, it comprises 600 species in 24 genera (one extinct) and occur on all continents. Other Genera in the family Oleaceae are: Ligustrum (Privet), Syringa (lilac), Fraxinus (ash) and Olea (olive). From Johnson 1957:

OLEA GENUS
Olea is a genus about 20 species in the family Oleaceae, native to warm temperate and tropical regions of southern Europe, Africa, southern Asia and Australasia. ? There is only one species in this genus, europaea which produces the edible olives and oil commonly consumed. It is by far the most important species, native to the Mediterranean region. O. paniculata is a larger tree, attaining a height of 15-18 m in the forests of Queensland, and yielding a hard and tough timber. The yet harder wood of the Black Ironwood O. laurifolia, an inhabitant of Natal, is important in South Africa.

EUROPAEA SPECIES
There are many varieties of this species. Some, such as Ascolano and Sevillano are large with small pits and low oil content that are commonly used for making table olives. Others such as Frantoio, Arbequina and Lucca are small but have high oil content and are used for making olive oil. For a large list of common varieties go to: Olive Varietals.

DISTANT COUSINSThe Russian Olive or Oleaster
The Russian Olive is not used to make olives or olive oil. Elaeagnus Angustifolia is only remotely related to the olive tree. They share the same class, Magnoliopsida (Dicotyledons) but different order, species, etc. There is evidence of cross-reactivity between olive, ash, privet, and Russian olive tree pollen allergens. (Annals of Allergy 69(6): 493-496) so if you are allergic to olive pollen don't plant Russian olive. Russian olive is a native of southern Europe and western Asia. It was introduced into the United States in the early 1900's and has now escaped cultivation and is extensively naturalized in 17 western states. According to the US forest service, once established, Russian olive is hard to control and nearly impossible to eradicate. Control efforts have included mowing, cutting, burning, spraying, girdling, and bulldozing, most with limited success. It does produce a small fruit that is nutritious to deer, cattle, birds and rodents but when Russian olive displaces natural species the resultant habitat is generally considered inferior.

The Sweet Olive Tree
The Sweet olive, Osmanthus Fragrans, also known as tea olive, fragrant olive, is in the same Family (Oleacea) as the olive oil olive. Other Genera in the family are: Ligustrum (Privet), Syringa (lilac), Fraxinus (ash)
Synonyms: Olea fragrans, Olea ovalis, Osmanthus longibracteatus, Osmanthus macrocarpus
The sweet olive does not make a seed with appreciable oil and is not used for making oil. Note that olive oil that is late harvest and has a sweeter, more ripe flavor is sometimes referred to as "sweet olive oil" on restaurant menus, as opposed to an astringent, bitter Tuscan style oil. Bland refined olive oil is also referred to as "sweet olive oil" when it is sold in drug stores for softening earwax, etc.