Yoon Ha Lee(click on names to see more mathematical fiction
by the same author)

...

(quoted from Counting the Shapes)

How many shapes of pain are there? Are any topologically equivalent? And is one of them death?

This is a fantasy story in which magic is achieved through mathematics, and hard work. For example,

(quoted from Counting the Shapes)

"Do you have any battle spells?" he asked gravely.

"Not many." She leaned over and tapped the nearest pile of paper. "I was in the middle of this proof when I discovered that I'd have to review one of Yverry's theorems. I fell asleep trying to find it."

A central idea seems to be a development in magical "theory" analogous to the development of fractal geometry.

Lee is a former math teacher who "hated math for the longest time" (see this interview). The story was originally published in The Magazine of Fantasy & Science Fiction, vol 100, no 6 (June 2001) and reprinted in Conservation of Shadows.