I'm trying to find a way to create a task that doesn't escalate for our Critical priority incidents.

For a critical priority, we have a follow up task that's created from a workflow after the incident is resolved. Since it's created on a critical incident and our tasks follow the incident priority, it becomes a critical task. In the workflow that creates this task, I've set the priority to overwrite the value as high, but it's still a critical and it breaches after 15 minutes.

We don't want these follow up tasks to breach, but I don't know how to accomplish that since we already have an escalation exception defined on the critical priority for something else.

The Task Priority is set on the relationship IncidentAssocTaskAssignment on the Task.Assignment Object. You can either remove the relationship then the default task.assignemnt priority will come form the initialisation rules on the object (default is 3) and add your own editing rules to compensate for the values not copying over from the relationship.