Without the Biblical assertion that humans have “dominion” over all other creatures, this issue does require a bit of thought.

The difficulty is that work is an almost exclusively human concept: the idea of regularly performing an action for another creature which has no direct benefit to oneself, but will be rewarded later in a physically unrelated way. There are some practical relationships in the animal kingdom where one animal appears to be working for another (e.g. the birds eating insects off a hippo), but in all the cases that come to mind the action is itself beneficial to the “workers” (the insects are nutritious).

So, when an animal works for a human we try to apply human standards to the arrangement. The critical question from your perspective is, therefore, does it seem more like employment or slavery? One is voluntary and gainful, the other is forced and unrewarding. One doesn’t raise any major ethical questions, the other is a moral outrage.

Rather unsatisfyingly, the answer depends heavily on the details of the situation. Looking at all of history, there have been and still are plenty of deplorable cases where animals are whipped, starved, placed in extreme danger and killed as soon as they outlive their usefulness. In other cases, the animals are bred and raised not just to be good at a particular job but to love it. They’re well fed, praised, looked after, given days off and live to a ripe old age in relaxing retirement. It’s hard to begrudge them that because of some lofty idea that an animal without the intelligence to understand an employment contract cannot fully consent to work. They do the work willingly and come home happy at the end of the day. That’s downright enviable.

Taking all of this into account, my opinion on working animals hinges almost entirely on the welfare of the animals themselves. If they prosper and rejoice in their day’s activities, it’s fine. If they suffer, it’s not fine. There’s nothing intrinsically ethical or unethical about an activity or arrangement that produces such varied quality of life.