Here's another possibility. Matte black photofoil (or cinefoil) is inexpensive and might be useful for boundary markers. It's claimed to be light absorbing for use in photography when stray light needs to be contained. Supporting that claim, one review on Youtube said it can be used by photographers to reduce the brightness of adjacent items or surfaces.

Youtube videos show slight reflections when the material is wrapped around very bright photography lights, but maybe the material will absorb enough light to fool the cliff/drop sensors of robots. The material is heavy aluminum foil that can be cut or molded to suit your needs. I searched but couldn't find a reference to this material on Robotreviews.

Cost: Less than $20 including shipping, for a roll 12 inches x 10 feet.

The IR spectrum characteristics are not known, and visibly black is not indicative. Need a smaller piece at less cost for testing. Matte black paint even made for IR absorption does not work, unless very expensive. (Except on Roomba's which seem to have unusual proprietary IR sensors, probably made for other, military products where they work for different applications.)