How accurately can a domain name's availibility be determined by checking for the existence of NS or SOA (Start of Authority) records?

If not, how can I determine this as accurately as possible without resorting to WHOIS? For example, is it worth checking for MX records if no NS records exist?

I am specifically interested in .co.za domains, but the only official, fool-proof way to check availability in the .co.za namespace is to use http://co.za/whois.shtml, which limits consecutive checking by IP address.

6 Answers
6

The only 100% reliable way to check for domain availability is to query the registrar's database. I wouldn't trust whois.

You can use DNS to get an estimate. Instead of looking for an SOA record, I would just look to see if anything at all is listed with the TLD name server. dig is a good tool for this (runs on Windows, too), although I guess you could use nslookup too. For example:

dig co.za. NS

will provide a list of the name servers for .co.za. One of those servers is ns0.is.co.za.

Next, query that server directly to see if they have anything listed for your domain of interest:

dig ibm.co.za @ns0.is.co.za

That query returns NS records, but no SOA record since SOA records are provided by the domain's name server (which may or may not be online). The NS records indicate the domain name is in use.

The reason for going direct is that it's usually much faster than relying on recursive queries from your local name server.

Hence, the existence of an NS record is a better indication that it is taken? Can you elaborate on "going direct"?
–
pateNov 23 '09 at 2:20

Yes, NS records are a better indication. If they don't exist, you can't get the SOA record or anything else meaningful using DNS. By "going direct", I mean querying only the TLD server. The normal query process goes something like this: your app queries the local DNS, which forwards to a shared DNS (optional; such as on a router) which forwards to your ISP's DNS which queries root servers to find the right TLD server, then queries the TLD server to find the domain's server, then queries the domain server. To determine domain validity, just query the TLD server; nothing else is needed.
–
RickNZNov 23 '09 at 7:58

Is there an overarching .co.za TLD nameserver? If not, which DNS server will provide the quickest answer for .co.za queries? Bulk checking for "ANY" DNS records seems to find virtually all registered .co.za domains.
–
pateNov 23 '09 at 15:25

The first dig query above will return a list of all of the .co.za TLD servers, any one of which can handle queries for the entire domain. If you're planning a bulk query process where perf is important, you could ping all of them and see which one is topologically closest.
–
RickNZNov 23 '09 at 23:04

Good suggestion. ns0.is.co.za. seems to have the least hops inbetween.
–
pateNov 24 '09 at 0:51

The only way to determine if a domain is available or not is to query the registry database for that domain. Usually you can do that by performing a WHOIS query on the registry WHOIS interface.

There are several reasons to explain why a DNS check is not safe.

The most simple explanation is because you can register a domain without pointing it to any location. It's like a house. You can buy a house, but you are not forced to build a road to reach it.

The most part of available domain providers force you to point a DNS, otherwise they set a default one. But you're not forced, it's not a kind of mandatory domain requirement.

The second reason is because there are actually cases when a domain is registered but not available. One of this cases is the period often known as redemption period. This period lasts for some days after the domain is expired and the current owner don't renew it. Several registries disable the domain, it means it is no longer reachable via DNS, but the domain still exists and can be redeemed (usually) only by the original owner.

The DNS check is cheap and sometimes faster. If you need to check whether the domain exists, then you can try a DNS query first and, in case of failure, fallback to the WHOIS query. In other words, if the DNS query succeed the WHOIS query is almost pointless. But you cannot do the opposite because if a DNS query fails, it doesn't mean the domain is not registered.

Whois is the canonical way to check for domain availability. The rate limiting is there deliberately to keep folks from overloading the whois servers. Existence of SOA records could be a good guess, but it isn't a certain answer.

Yes, checking for SOA isn't perfect. But is there a better way, perhaps in conjunction with other records, to improve accuracy?
–
pateNov 23 '09 at 1:00

While it may be the canonical way, whois is also a notoriously unreliable way to check for domain availability, since each registrar operates their own databases, with their own non-uniform policies.
–
RickNZNov 23 '09 at 1:12

RickNZ: it depends on the TLD. For the vast majority of the TLD, the whois server is operated by the registry, not the registrars.
–
bortzmeyerNov 23 '09 at 8:24

Within .com, for example, there are multiple sources / databases of whois records (the so-called "thin" model), which are located with and run by registrars. Many of those servers have a reputation for being offline regularly. Also, some TLDs don't even have whois servers (including .co.za).
–
RickNZNov 23 '09 at 23:18

In practice, the DNS lookup works 99% of the time. If you writing something like Ajaxwhois, I'd suggest going this route. If you have a million names to check, I would do a first pass with a DNS lookup, and then maybe do a second pass on the ones which you may consider purchasing.

My tests show that checking for the existence of NS
records or ANY DNS records in bulk
seems to be the most accurate way of
guessing whether a .co.za domain is available for registration. SOA records work too, but not as well.

I've also been using the dig method to test for the existence of a domain before registering but about 10% of the time, it returns domains that are NOT available as available. This is because the owner of the domain hasn't set up any records for the domain. So what I've decided to do is:

First check via dig method.

If domain appears to be available, then check using whois.shtml

People will go through a bunch of domains that are not available and this doesn't waste the allocated requests that UniForum provides. Only if something appears to be available do you check on co.za itself. It seems to me to be the only reliable way to do this WITHOUT losing access to co.za/whois.shtml due to sending them a million requests :)