Peers keep contacting me

With any BitTorrent client, you are connected to at least one tracker (either a central tracker server or a distributed tracking system). The tracker stores your IP address and port, and passes it to other peers. Similarly, you get information about other seeds/peers from the tracker. Peers can also exchange information about other peers in the torrent through Peer Exchange (PEX).

Other clients can obtain your IP address and port from the tracker or from other peers, but may not use it right away. Your client will tell the tracker when you log off, but other clients may try to contact you before being notified about that by the tracker.

Providing that the peer has a correctly behaving client, it will stop the contacts it learns from the tracker that you have gone.

It is also possible that they have been told by the tracker that you are no longer connected, but due to a buggy or rogue client they are ignoring this.

In addition, many bittorrent clients save information about active peers when stopping, and then use the stored information to speed up launch next time. Thus, a client may have stored your address a few hours/days/weeks earlier and then uses it later.