The discrimination in horizontal and vertical caves is useful only in areas with rather small caves.
This caves consist normally of a single tunnel or shaft.
In other karst areas with larger caves any cave is a cave system.

by the time they were formed, in relation to the forming of the rocks they are in:

primary caves formed together with the surrounding rocks.
This are typically lava tubes or gas bubbles or tufa caves.

most caves are secondary caves.
After the formation of the rock there is a time when part of the rocks are removed.
This secondary stage formed the cave.
The mechanism of the transport of removed material is not relevant for this classifikation.

tertiary caves are the result of the collapse of other caves.

by the way they were formed:

solutional caves or Karst caves:
Most caves are in rocks which can be dissolved by a weak natural acid (usually carbonic acid).
This acid forms when rainwater absorbs CO2 from the air and the upper layers of the soil.
The forming of Gypsum Caves does not require CO2, Gypsum has a very high solubility.

lava caves or lava tubes:
First a crust hardens on a lava flow.
When the crust gets thick enough, the lava flow is underground.
When the eruption end, the lava keeps flowing and the empty tunnel-like passage remains.
The length of this tubes depends on the distance from the lava source to the drain, a depression or the sea.
It can be hundreds or even thousands of meters long.Example:Hana Cave

sea caves:
This caves are created by the erosion of waves.
The waves force water into cracks in the rock, breaking of the rock and forming caves.
Often this caves follow less resistant rock layers.Example:Sea Lion Cave

earthquake cave:
Formed by the movement of rock along a fault.
Its just a natural crack in the rock and the big ones are very rare.Example:Seneca Caverns

glacier caves:
Melting water moving through glaciers creates glacier caves.
This caves are formed inside the ice.
But: Ice caves on the other side are caves that are filled with ice, but the cave itself is formed in rock.
Most ice caves are formed as solutional caves in limestone!
Examples can be found in Canada, Alaska, and high on Mt. Ranier in Washington.

soil tubes:
In desert areas, flash floods can move through the soil and hollow out openings.
Examples can be found in the Mohave Desert in California.

by the age of the rock:
This is useful for limestone caves.
Limestone is a sedimentary rock and is characterized by the time it was formed.