The post talks about how some of the robots.txt files are telling search engines not to index the jobs on the sites.

For those of you who I lost at robots.txt, let me help explain this in plain English. Google, and the other search engines, have computer programs that go out and look at your website, or in the case of a careers site, it looks at your careers information and your job listings. These programs are known as “spiders”, “crawlers”, “bots”, etc. These “spiders” take a look at the information that is on each of your pages, and record it in their “index” or database. When you run a keyword search in Google, or any other search engine, the results you see are the made up of web pages that have been looked at and recorded by the spider, and the listings are ranked based on how well that page matches up to the keyword you search for.

As a web programmer, I can add a little text file to a website named robots.txt. That file allows me to tell a spider what pages to look at or not look at. By telling the spider not to look at a web page, it means it won’t record it in the index, and that page won’t appear in the search engine when someone runs a keyword search. There are various reasons why someone might want to do this, but for employer web pages and job listings, I can’t think of a good reason to block it.

Jason, the person who posted the article linked above, did some digging around and found that a few major ATS vendors were telling spiders not to index the job listings or careers sites of their clients. I’m not sure why they would do this, I’m sure that the software programmers had a valid reason in their mind for doing this. Since software programmers are generally not web marketers, they might not see the potential problems caused by blocking the spiders.

The blocking of the search engine spiders is just the tip of the iceberg when it comes to Career Sites and Search Engine Optimization. Even if the robots.txt file wasn't excluding the jobs from being indexed, it really wouldn't matter much for most ATS career sites. Most vendors simply aren't building SEO friendly career sites, for some reason SEO is simply not a priority. To make matters worse, many of their clients don’t want to send their job seekers off of the company website over to the ATS careers website, so they try to use a frame to display the job listings. In plain English, you land on www.samplecompany.com/jobs and that page shows you the job listings from your ATS system on the page. If you click on a job, the page content changes, but the url remains the same as www.samplecompany.com/jobs. This is bad for the search engines and stops your jobs from being indexed or harms the rankings of those jobs. The end result, it keeps you from getting free applicants directly from the search engines, and everyone loves free job applicants.

This ATS verses Search Engine Optimization has been something that I’ve thought over quite a few times in the last 3 years. I have come up with a few ideas of why this is happening, but they are just my own speculation:

#1 ATS companies are made up of, and ran by, software folks, not web marketers. For some reason there is a huge difference in how these 2 groups build things from the ground up. Software guys tend to follow what they have always done, and those habits continue to get in the way of SEO for the systems they design.

#2 There is little to no demand for search engine optimization by the clients using ATS systems. Ask any HR manager, especially at smaller companies under 2000 employees, what SEO is, and 95% don't know. Even those that do, don't understand it enough to pitch it to their bosses to get money for it. If they did get buy in from the boss, without web analytics they couldn't prove that they were getting visitors or applicants from the keywords, and the cycle just keeps repeating. Without proof, you can’t get funding increased to be more aggressive with SEO, etc, etc.

#3 Problem 2 results in there being no money in it for the ATS vendor. Since clients don't demand it, they definitely won't pay for it, and vendors therefore won't built it.

#4 It is definitely not in the best interests of the major job boards, recruiters, headhunters, etc to have jobs showing up in the search results of Google with direct links to the employer’s careers page. This would mean that the job boards would be competing directly with the employers for the traffic. Since most speakers talking about online recruiting at events and seminars are from one of these sources, SEO is simply not part of the conversation, and so SEO is not gaining traction with employers.

This has been one of the most amazing things that I have seen in this industry. We built iApplicants to be search engine friendly simply because of my background. When you come from the Internet Marketing world, SEO is always part of the design of a website. If you could build a system that would provide a valuable tracking tool to your clients, and generate free traffic from the search engines just because you built it the right way, why wouldn't you do it? I knew that our clients wouldn't understand SEO or pay for it, but I knew it would pay off in the end.

As of now our search engine friendly careers pages combined with our pushes to the free job boards like InDeed.com account for an average increase in applicant flow to our client’s sites of more than 20% each month. That is a number that HR understands. We use that increase to show that buying iApplicants will generally pay for itself just off of the value of the increased applicants, and the other benefits will just be the icing on the cake.

2 comments:

Anonymous
said...

One issue at hand is the ever increasing number of aggregator sites that use content from other sites to artificially increase content and take direct traffic from the posting sites. Traffic drives many of these sites either in additional advertising or in jusitfiying cost to post. While it might be good to have jobs indexed in the general search engines, it does not always make business sense to allow employment specific search engines take that traffic.

Great comparison, although I wish you could update it. I just installed both for two different sites because I am finding it difficult to see which one will suit better my interests. SEO is important for me, so I might choose Drupal. Thanks for the very extensive review.

Subscribe to our Blog

Is this site not the site you were expecting to see?

If you were expecting to see an employers careers site, please check the spelling of the url. My guess is that you missed an "s" at the end of iapplicants.com. Our clients sites are in the form of yourcompany.iapplicants.com. Please check the spelling of the url and try again.

Online Recruiting & Tracking

iApplicants™ blog is intended to provide a discussion on ideas and tips to help employers be more productive with their hiring process. Including helping them expand their applicant pool, reduce their recruiting costs, and better manage their employment applications.

About iApplicants™

For Employers of 20 to 2000iApplicants™ provides Employer Careers Websites and Application Management Systems designed to be Flexible, Easy to Use, and Inexpensive. Even the smallest employer can afford to leave their paper application behind for a web-based hiring system. Our system is powerful enough to get the job done, but easy enough that your receptionist could run it!