The shift from hard o-stem to the soft jo-stem, as outlined in the chronology above, has several theories of origin:

Holzer explains the j-suffix as originating from a substantivized possessive adjective

Schenker suggests analogical replacement after agent nouns ending in *-teljь or other words denoting leaders such as *cěsarjь and *kъnędzь

Pronk-Tiethoff suggests the final *-ljь is due to the fact that Proto-Slavs likely perceived the Old High German final consonant as soft, similarly as in the loanword *grędeljь.

The word has been described as "without doubt the most famous Germanic loanword in Slavic" (Pronk-Tiethoff 2013) due to the fact that it's the only loanword in Slavic that can actually be dated, thus giving clues to the absolute dating of Proto-Slavic phonological developments. The fact that it regularly underwent historical Proto-Slavic sound laws, and that it's reflected in all three branches, is one of the chief indications to date Late Proto-Slavic (Common Slavic) to the ninth century.

However, this is comparatively late (only a century before Old Church Slavonic manuscripts were written), so other etymologies have been suggested:

Holzer derives it from the name of the Frankish ruler Charles Martel (688-741). The issue with this theory is that Charles Martel was not particularly important to the contemporary Slavs.

Stender-Petersen derives it from Proto-Germanic*kar(i)laz ‎(“free man”) (Old High German karl ‎(“man”)) with a semantic shift explained as "very ordinary".

These theories are generally thought of as less convincing than from Karl "Charlemagne", who was an actual king of (some) Slavs.