Is the rule that mathemations just avoid this "function" and engineers just use it without and problems?

No its that we don't bother to tell engineers about generalised functions, the
theory of which justifies most of the antics of engineers and physicist with
the delta function and its relations (like the derivative of the delta function!).

There are a number of theories of generalised functions is that of Schwartz
Distributions. You can start here though its does not look as comprehensive
as I would like its a good place to start.