First of all, I should point out that normally you don't want to do this. If you want a short message, then croak. If you want to provide full debugging information, there is little reason not to provide a little extra. And there is usually good reason not to, because when an error is thrown it can be useful for the developer to have the option of seeing the actual code that decides to throw the message. (Just to trace what has to be fixed.)

Now one solution is the old internal style. If you want to exclude 4 levels of call exactly, you can play games with $Carp::CarpLevel. Unfortunately whenever one person plays games with that, they break it for everyone else. Also its usage with shortmess messages is obscure at best. Furthermore that gets you into the game of counting how many levels of your module you use.

The more robust solution is the new internal style. Just stick the line: $Carp::Internal{$_}++ for 'Log::Dispatch::Output', 'Log::Dispatch'; in your module. And now those two packages will be ignored whenever Carp crawls back, looking for where to start reporting the error from.