That's a bit restrictive. The set {a} is the same as the set {a,a}. It's just pointless to write the latter, not wrong (multiplicity is ignored, not illegal). Your {a,b} where a=b is still a set that only contains a (or b depending on your labelling preference).
Asymmetric relations are indeed those which do not contain the pairs (a,b) and (b,a). You disprove the inclusion of elements like (a,a) by the trivial fact that if a=b, then (a,a) = (a,b) = (b,a) cannot be in the relation by definition.

Grrrr, there's a typo in my original post. I meant antisymmetric not asymmetric. Sorry.

edit: My lecturers textbook has this written in it: "... {a,a,b,c} and {{a,b},{b,a}} do not denote sets, because the elements of a set are distinct." I have read about multiplicity the way you write about it, but my lecturer doesn't seem to take this approach. It's a first year course for students in the maths department, so it doesn't cover advanced axiomatic set theories (but its still pretty hard, we do nothing but proofs).

A relation is "asymmetric" if and only if it is not symmetric. That is, there exist some pair (a,b) in the relation for which (b,a) is not in the relation. There might be some other pair, say (x,y) such that (y,x) is in the relation. A relation is "anti-symmetric" if and only if whenever (a,b) is in the relation, (b,a) is not in the relation. It is impossible for another (x,y) to be in the relation with (y,x) also in the relation.

If I read this correctly, you are talking about relation on the two member set U= {a, b}. The cross-product UxU contains exactly 4 members: UxU= {(a,a), (a,b), (b,a), (b,b)}. Any relation must be a subset of that. In order to be anti-symmetric, a relation cannot contain (a,a) or (b,b) (since, in that situation, it would also contain the reverse: (a,a) or (b,b)!). It may contain (a,b) but then it can't contain (b,a). Conversely, it may contain (b,a) but then it can't contain (a,b). The only antisymmetric relations on {a,b} are
{(a,b)} and {(b,a)}.

PS:
My understanding is that (a,b) and (b,a) can't be both in the relation, because that would imply a=b, however that is not true. The lecturer has made it clear verbally and in his book, that to him things like {a,a} and {a,b}, where a=b, are not sets at all. Hence a=b can't be true (since the problem assumes {a,b} is a set) and if a=b isn't true, then either (a,b) or (b,a) must be excluded from an anti-symmetric relation (my understanding of definition 2).