Bake in top and bottom thirds of 275°F (140°C) oven, rotating and switching pans halfway through, until firm to the touch and bottoms are light golden, 40 to 50 minutes. Let cool on pans on racks for 10 minutes; transfer to racks and let cool completely.

Icing Paint: Whisk sugar with water until spreadable, adding more water if necessary. Divide among small bowls; tint with colouring as desired. Using palette or small knife, spread thin coat on cookies. Let dry about 4 hours. Meanwhile, scoop remaining icing into separate resealable plastic bags; seal and set aside.