6.6.8 To exclude all robots from the entire server User-agent: * Disallow: /

6.6.9 To exclude all files except one This is currently a bit awkward, as there is no “Allow” field. The easy way is to put all files to be disallowed into a separate directory, say “stuff”, and leave the one file in the level above this directory: User-agent: * Disallow: /~joe/stuff/

7 Sitemap.xml

7.1 complete?

7.2 correct?

7.3 General recommendations

7.3.1 XML file named “Sitemap.xml”

7.3.2 file must be no larger than 50MB when uncompressed

7.3.3 no more than 50,000 URLs for a single sitemap

7.3.4 If you have more than one Sitemap, you can list them in a Sitemapindex.xml file and then submit the Sitemap index file to Google. You don’t need to submit each Sitemap file individually.

7.3.5 If your site is accessible on both the www and non-www versions of your domain, you don’t need to submit a separate Sitemap for each version. However, we recommend picking either the www or the non-www version, and using recommended canonicalization methods to tell Google which version you are using.

7.4.4 <loc> – Required – Specifies the URL. For images and video, specifies the landing page (aka play page, referrer page). Must be a unique URL.

7.4.5 <lastmod> – Optional – The date the URL was last modifed, in YYYY-MM-DDThh:mmTZD format (time value is optional).

7.4.6 <changefreq> – Optional – Provides a hint about how frequently the page is likely to change. Valid values are: – always. Use for pages that change every time they are accessed. – hourly – daily – weekly – monthly – yearly – never. Use this value for archived URLs.

7.4.7 <priority> – Optional – Describes the priority of a URL relative to all the other URLs on the site. This priority can range from 1.0 (extremely important) to 0.1 (not important at all). Does not affect your site’s ranking in Google search results. Because this value is relative to other pages on your site, assigning a high priority (or specifying the same priority for all URLs) will not help your site’s search ranking. In addition, setting all pages to the same priority will have no effect.

11.3.1.1 The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible. This response is cacheable unless indicated otherwise.

11.4 Deleted page

11.4.1 404

11.4.1.1 404 Not Found: The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.

11.5 Temporary Redirect

11.5.1 302

11.5.1.1 302 Found (HTTP 1.1) / Moved Temporarily (HTTP 1.0) A 302 redirect is a temporary redirect. It passes 0% of link juice (ranking power) and, in most cases, should not be used. The Internet runs on a protocol called HyperText Transfer Protocol (HTTP) which dictates how URLs work. It has two major versions, 1.0 and 1.1. In the first version, 302 referred to the status code “Moved Temporarily.” This was changed in version 1.1 to mean “Found.”

11.5.2 307

11.5.2.1 307 Moved Temporarily (HTTP 1.1 Only) A 307 redirect is the HTTP 1.1 successor of the 302 redirect. While the major crawlers will treat it like a 302 in some cases, it is best to use a 301 for almost all cases. The exception to this is when content is really moved only temporarily (such as during maintenance) AND the server has already been identified by the search engines as 1.1 compatible. Since it’s essentially impossible to determine whether or not the search engines have identified a page as compatible, it is generally best to use a 302 redirect for content that has been temporarily moved.

11.6 if the page you are removing has a suitable alternative page on your web site, then 301 it. Do not always 301 the page to your home page. If there is no suitable, and by suitable I mean, a page that is very similar to the page you are removing, then 404 the page. 301 if there is a related and similar page to the page you are removing. 404 if there is not.

15.4.3.3.1 Download a back link report to see if you’re missing out on links pointing to orphaned, 302 or incorrect URLs on your site. If you find people linking incorrectly, add some 301 rules on your site to harness that link juice

30.6.1 Meta refreshes are a type of redirect executed on the page level rather than the server level. They are usually slower, and not a recommended SEO technique. They are most commonly associated with a five-second countdown with the text “If you are not redirected in five seconds, click here.” Meta refreshes do pass some link juice, but are not recommended as an SEO tactic due to poor usability and the loss of link juice passed.

31.5 Additionally: – the image filename is related to the image’s content; – the alt attribute of the image describes the image in a human-friendly way; – HTML page’s textual contents as well as the text near the image are related to the image.

35.4 1. Post it on your website with no strings attached. It’s free and you require no personal information from prospects 2. Blog about it 3. E-mail your in-house database 4. Post it on your social media profiles 5. Publish a press release (pitch it to the media too) 6. Create an ad campaign using banner and text ads 7. Reach out to popular and respected bloggers in your industry and get them to blog about it 8. Mention it in your next monthly newsletter 9. Use it as a basis for a webinar or podcast episode 10. Produce a video about it