I can't determine a pattern that represents a pair which does not also match {{a,b},{c,d}}, but I can think of two options. The first is not to use pattern matching at all, which may be more efficient anyway: