“Sake” means not just rice wine but also alcohol generally. Amazake doesn’t have any (or much?) alcohol in it, and you’ll often see children drinking it at festivals and shrines, but its name means “sweet rice wine” or “sweet alcohol”.

According to the Wikipedia amazake page, it is made from the same yeast (koji) and a similar process as sake, and according to Pink Samurai, the heating of it then boils off the rest of the alchohol.