The youngest of six children, José Luis Cantero was born in Madrid, close to a famous bullring. As a boy he would play truant from school, preferring to spend time partying with Gypsies imitating his idol, the traditional "copla" singer Rafael Farina. It was from Farina that Cantero adopted his stage name of "El Fary". Resolute that his shortness would not stand in the way of success he was determined that his dream of following in his idol's footsteps would one day come true. It was during the military service (late 1950s) that he would finally learn to read and write.

Before hitting the big time El Fary worked hard as a gardener and a taxi driver in order to make the money to record his first songs. As a taxi driver, he claimed to have driven the American actress Ava Gardner around Madrid.

Once he had recorded and pressed these songs - rumour has it - he sold them to the public himself from a stall in the Rastro (Madrid's weekly street market). At this time he also took part in singing contests on the radio and performed at local fiesta. Only when he was in his early thirties did El Fary start to make money as a singer when he was called to stand in for Pepe Blanco at a show in Pozoblanco, Córdoba, Andalusia. Later, the legendary Antonio Molina hired him for a two-month tour.