I really appreciate you reading this. I have spent close to 6 months putting my soul into creating my first ruby on rails app, and now my site is all buggered up SEO-wise and I'm really bummed. I'm going to try to (simply) explain what happened and I really hope someone can help me. Thanks in advance!!

I examined webmaster tools like a hawk during my launch, to make sure my sitemaps/SEO was all good, updating properly and so on. Everything is good for about a week.

I then notice (after refining my title tags) that they haven't changed for a solid week, despite Google seeing my new updated sitemap daily. Up until then Google had been updating the index with my changes after only 12 hours- 2 days so I thought this was kind of odd but I was like "meh, whatever. it will happen eventually"

I then look in analytics and see I get a hit for a certain keyword. I Google the keyword myself and low and behold. I see it in the index, but its indexed as this:

ns2.deluxetracking.com

This is one of my nameservers. I'm like what!!

I Googled site:ns2.deluxetracking.com and I see now that ALL my pages are indexed there, and they're ALL UPDATED with the right meta titles/descriptions that I was waiting to see changed! Google had been updating this information instead of my pineapple.io domain.

I talk to my host, and they told me that because I set a new document root with apache, that cPanel doesn't like that and.. I dont know the details but basically ns2.deluxetracking.com was showing an exact duplicate of my site.

I had them fix this issue, so that when you go to ns2.deluxetracking.com it would show the proper default page for a nameserver (Cgi-sys default webpage).

I then realized that all my pages in Google's index would be broken now, and I didn't want that as I wanted all that fresh meta information to transfer over.

So underneath the virtual host fix that my host did, I added another section which redirects EVERYTHING from ns2.deluxetracking.com/whatever/page over to --> pineapple.io/whatever/page

This is how it is now.

I then went into webmaster tools with the intent to add ns2.deluxetracking.com, and fetch as Google but then resubmit to index (so it would see all my 301s). In order to verify my account I had to temporarily break my site again and remove the 301. After I verified, I fetched as Google bot, submitted to index and quickly re-enabled the 301's

I waited for about 2 days, and now when i do site:ns2.deluxetracking.com, it actually has MORE results than it did, even WITH the 301. It used to have about 900 and now it has 1300 after I did that

What gives? How can I fix this?

I have several specific concerns:

Why would Google index something that is a different domain? Under normal conditions, how does Google work with a nameserver? I thought the entire point of a nameserver is just to locate an IP of the host, and then get a domain name (the proper one), and Google just uses that nameserver to locate the real site. Why is it actually accessing get requests on port 80 for a real web request to my nameserver? Furthermore, Why would it ever index that? Traditionally I never (obviously) needed to supply a 301 redirect on a nameserver to have it not be indexed on all my other sites. So what is different now? What I'm saying is, on any of my other websites, if you visit ns1.deluxetracking.com you get the 'cgi-sys default webpage'. Google knows not to index that page because its a nameserver. So why did it suddenly choose to index whats on my nameserver THIS time? EVEN though my entire website showed up on that nameserver (my fault), I still dont know why it would ever choose to index that.

How can I remove all the existing SEO results for ns2. Will Google just naturally over time see my 301's and remove the old site from the index? I can't really use robots.txt because ns2 is a subdomain thats in use already somewhere, not to mention thats kind of complicated iwth virtualhosts to try and hack it so I can upload a robots.txt.

I'm pretty sure it's bigger on pineapple because when I first put it up I didn't use the URL params feature. So it indexed a lot of urls with ?sort=. On the ns2 one I'm pretty sure it was slow to pick those up. On ns2 the index has been growing though as opposed to shrinking
–
TallboyAug 13 '12 at 1:07

2 Answers
2

It appears as though you did not re-enable the 301 redirects (I'm looking at pineapple.io content under the nameserver's domain right now).

WHY would google index something that is a different domain? ... Why is it actually ACCESSING get requests on port 80 for a real web request to my nameserver? Furthermore, why would it ever index that??

Googlebot doesn't know which domain content is intended to reside under - you have to make that indication, preferably by ensuring that the content is not served under any other domains (or from a bare IP address).

This means that Apache's default vhost should either return an error, a blank page, or a permanent redirect and your application's vhost should be configured to respond similarly if accessed by a non-canonical domain name, subdomain, or IP address.

Google KNOWS not to index that page because its a nameserver.

No, it really doesn't - your nameserver's domain name points to an IP address and that IP address is responding with content.

... i still dont know why it would ever choose to index that.

There are many sites which automatically generate links to domain names and even IP addresses - Google has indexed many of these sites and will follow the links.

How can i remove all the existing seo results for ns2. Will google
just naturally over time see my 301's and remove the old site from the
index?

Your initial plan would have worked well enough if the redirects had actually been implemented - I'd recommend that you revisit your configuration to correct the missing redirects.

Wow that's insane that it still isnt redirecting. Hmm... so you mean if you paste this into a fresh chrome incognito window it wont redirect? ns2.deluxetracking.com/tags/ruby
–
TallboyAug 12 '12 at 23:47

Okay, that's what I thought. In all my tests it redirects, but it's definitely possible I have missed something. So what else should I do? Thanks for the replies guys :)
–
TallboyAug 13 '12 at 0:33

If I were you I would just start over since it is a new domain/project. On your ns2.deluxetracking.com domain just add a robots.txt with global disallow and then add that domain to webmaster tools seperately and submit a removal request from there. This will/should remove the ns2 links from the index. It takes time to do it the slow 301 way
–
Anthony HatzopoulosAug 13 '12 at 0:33

Unfortunately that option isnt possible ;/ However, someone recommended I use this: web-sniffer.net, and then choose HTTP/1.0 (without Host header) (to emulate google). When I do that its returning a cgi-sys page
–
TallboyAug 13 '12 at 0:35