http://nickberardi.com/Ghost 0.5Sun, 15 Sep 2019 13:18:10 GMT60As I alluded in a post a couple of weeks ago, I have been a bad blogger. And I have neglected my community of readers. However I would like to tell you what I have been doing in the last couple of months while I have been neglecting my blog.

I recently got involved in creating a startup as the lead developer for an online classifieds site called Anything For Sale By Owner. From the ground up this was conceived as a middle-ground between craigslist and ebay where every listing would be charged at a static rate of $1.00/month. The $1.00 is a way to week out the crap from craigslist and the death-by-fees from ebay.

The description is pretty straight forward but the more interesting tid-bits that I know my readers are interested in are as follows:

.NET/C#

We choose .NET mostly because it is what I knew and .NET for me has always been stable, predictable, and performed really well on server-side applications. The alternative was PHP and even through we wrote many of the processing layers by our self (i.e. REST Web Service Handler) the time to deployment was greatly accelerated because of all the work the Microsoft ASP.NET Team has put in to the product. The user of Master Pages and Web Services made for an easy separation between content and display.

MySQL

We choose MySQL for a whole host of reasons mostly based around the costs associated with a single Microsoft SQL Server Standard Edition license. Other reasons we choose MySQL was for scalability, because not only could we install 5 database servers (hardware included) for the same price we could purchase 1 MSSQL database server for but also because the master/slave replication of the databases seemed to be an easier process when we needed to scale horizontally.

REST Web Services/AJAX

We followed the Digg Model for exposing web services and each web service could be changed around to provide output through JSON, RSS, ATOM, KML (where applicable), and XML. I even did a write up about a month ago on the JSON Serializer that I developed for this website. This was very important for the AJAX we needed to control many aspects of the user experience.

Open Search

Open search was one of the value add features that wasn't in the original spec but was an easy add-on because we already had the Web Services in place to leverage. You can view our open search XML definition file here. If you are unfamiliar with Open Search this is how A9.com defines it:

OpenSearch is a set of simple formats for the sharing of search results. Any website that has a search feature can make their results available in OpenSearch format. Other tools can then read those search results.

So that little search box in the upper right hand corner of your browser is an example of an Open Search client.

SEO

Search Engine Optimization is a very important part of any website today. Because for most new sites and even some of the older ones, Google is going to be the largest most ignored user of the site. Not only will Google look at every single page on your site every week, which I dare any human to try and accomplish, they will also be the largest organic promoter of your site.

One of the corner stones of SEO is easily readable URL's that contain descriptive keywords. This means you have to have a good URL Rewriter in place. We choose Ionic's ISAPI Rewriter that integrated nicely with IIS 6.0. However that left a big gap between using Visual Studio's Intigrated Web Server (which doesn't support ISAPI) and a full blown IIS Server. The benifits are pretty obvious for running the Intigrated Web Server that comes with Visual Studio, for one you don't always have to have IIS that comes with Windows XP always running and the host of security problems that comes with it, two I was running Windows Vista and IIS 7.0 has some huge differences from IIS 6.0.

So I sat down one night and wrote my own Apache mod_rewrite compatible HttpModule for running the same rule-set that I defined for Ionic's ISAPI Rewriter, to fill in the gap and make developing on my local machine as close as running the live web site on IIS 6.0.

If anybody is interested I offer the Url Rrewriter as a free download:

I am a software developer and usually don't get invovled in the artsy end of the web site design. So I will let my co-worker Tom Lauck describe how he developed the front-end for Anything For Sale By Owner.

So all in all I believe that this website has a very good chance of making in the Wild Wild West that the Internet is, however that is probably just the ramblings of a proud father. Be on the look out for some major marketing campaigns in the NYC Times Square region.

]]>http://nickberardi.com/anything-for-sale-by-owner/b1f0f013-a79e-487f-83ee-ab0e9a238ea3Mon, 01 Oct 2007 17:46:04 GMTGoogle has extended its support for Google Bot restriction by giving us web developers a new tool to stick in our belt. It was announced today on the Google Blog that you can now control access to your non-HTML files on your website with a simple header. The header X-Robots-Tag will allow you to do everything the normal Robots Meta tag will, but now you can do it for the PDF, Word, Image, and any other document you can think of that is served via HTTP. They also announced on the same post a new type of exclusion cause that lets you set when the document will be unavailable, see below for more information on this new feature as well as currently supported ones for use with X-Robots-Tag:

INDEX|NOINDEX - Tells whether the page may be indexed or not

FOLLOW|NOFOLLOW - Tells whether crawlers may follow links provided on the page or not

ALL|NONE - ALL = INDEX, FOLLOW (default), NONE = NOINDEX, NOFOLLOW

NOODP - tells search engines not to use page titles and descriptions from the ODP on their SERPs.

NOYDIR - tells Yahoo! search not to use page titles and descriptions from the Yahoo! directory on the SERPs.

UNAVAILABLE_AFTER: RFC 850 formatted timestamp - Removes an URL from Google's search index a day after the given date/time

So how can X-Robots-Tags help you better control the content that is indexed by Google? Well you can now tell the Google Bot that you do not want specific non-HTML documents like PDF, Word, and Image documents that you don't want them cached on the Google Server or that a paper you have released on your website in PDF format should only be good until a specific date. So now you just need to force you server to include an addition X-Robots-Tag in the header which can be done with any of the modern languages and server, the header would look something like this:

You can do this with anything that can be served over HTTP now, so this is a huge boost for any of us control freaks that like to have our content easily organized and controlled on what is searchable on Google.

]]>http://nickberardi.com/control-google-bot-with-the-new-x-robots-tag/f5f58219-2207-4d47-a5c7-20f66402f60dTue, 31 Jul 2007 23:26:57 GMTA major update has been pushed out for Google Analytics, as described in a post on Google Webmaster:

Webmaster tools from Google are indispensable for people who optimize their site for indexing in Google. Eighteen months ago, Google launched another free tool for webmasters - Google Analytics - which tells you about your visitors and the traffic patterns to your site using a JavaScript code snippet to execute tracking and reporting. This past Tuesday, Google Analytics launched a new version, with an easier-to-use interface that has more intuitive navigation and greater visibility for important metrics. We also introduced some collaboration and customization features such as email reports and custom dashboards.

I simply love this tool, and the data it provides is invaluable to my day to day operations of this website.

]]>http://nickberardi.com/a-blog-owners-best-friend-google-analytics/ec1de36f-4142-4f39-bed0-7c8cb43ca917Tue, 15 May 2007 08:53:20 GMTOnly a geek would find this funny. And if you really don't understand it then you may want to brush up on your HTTP Status Codes.]]>http://nickberardi.com/unfortunate-placement-of-yahoo-ad/9294545c-053a-4e10-9c80-e9dfffa45ffdFri, 27 Apr 2007 12:54:09 GMTIn a follow up to my previous post on Proper URL Construction, I am going to dive more deeply in to the Status Codes that control the redirects that were talked about in my previous article.

Most developers are familiar with the HTTP 1.0 Status Codes, that have been recently popularized by the SEO guys. We have all heard that you should use 301 Moved Permanently instead of 302 Temporary Redirect. What many of the SEO guys won't tell you, because they don't know any better, is that they are using the RFC 1945 HTTP/1.0 Standard that was released in May 1996, that is right it is about 12 years old. The newest HTTP/1.1 Standard, RFC 2616, was released in June 1999, and made some pretty drastic changes the the 3xx Redirect Status Codes. The goal of this post is to inform and familiarize developers with the HTTP/1.1 Standard, specifically the 3xx Redirect Status Code changes. This can have drastic effect on how you handle requests on your website and optimize your site for search engines.

History

In the middle-to-late 1990's 302 Moved Temporarily was the most popular redirect code, but also an example of industrial practice contradicting the standard. HTTP/1.0 specification (RFC 1945) required the client to perform a temporary redirect (the original describing phrase was "Moved Temporarily"), but popular browsers implemented it as though it was a 303 See Other.

Note from 302 Found:RFC 1945 and RFC 2068 specify that the client is not allowed to change the method on the redirected request. However, most existing user agent implementations treat 302 as if it were a 303 response, performing a GET on the Location field-value regardless of the original request method. The status codes 303 and 307 have been added for servers that wish to make unambiguously clear which kind of reaction is expected of the client.

Therefore, HTTP/1.1 added status codes 303 and 307 to disambiguate between the two behaviors. However, majority of Web applications and frameworks still use the 302 status code as if it were the 303.

Proper Use of HTTP Redirects

The next part will be a guide of the conditions that should be met in order to use the specific redirect.

301 Moved Permanently

The URL (or page) is going to permanently reside in a differently location

This should be used for most static redirects that are not generated programmatically.

*NEW* This status was mostly designed to be used with GET and HEAD requests.

303 See Other

This is going to be the most common type of redirect that you want to use when you are programmatically changing where the user is located in your site during a POST back.

Any time you want to redirect a user to another URL after a POST from a form has occurred (i.e. The visitor to your site registers with your site and after they are done registering you want to direct them back to the home page, this is when you would use a 303 redirect).

*NEW* This status was designed to be used with POST requests specifically, so it should not be used for GET or HEAD requests.

307 Temporary Redirect

Anytime that you want to put up a temporary page (i.e. your site is under construction and you want all traffic temporarily redirected to a static HTML page).

*NEW* This should be used when you want to redirect a GET request to different location each time the URL is requested.

*NEW* This should be not be used with POST requests, because of this statement in the specification:

the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

302 Found

Use this for any condition not met above.

This should be used sparingly because there is a search engine penalty if used too much, because of some spammers that used an Exploit called Page Hijacking.

This is sort of the antithesis to 404Not Found and should be used in a similar way. So if you have a page that is referenced but no longer exists, but you do not want to return a 404 and just redirect the user to a random (not static as defined in a 301) site you would use a 302 redirect. (note this argument is very weak and there is very little reason in a HTTP/1.1 world to use a 302 redirect)

*NEW* This status should be used during GET requests for any semi-static URL's that may change in the future, but don't change with each and every request. A good example of this on Coder Journal is my Essential Software Every Developer Needs which I publish annually, and is located at http://www.coderjournal.com/essential-software/. It changes but it only changes once a year, so it is semi-static in terms of the internet.

*NEW* The 302 Found falls right between 301 Moved Permanently and 307 Temporary Redirect in terms of how permanent the URL is for GET requests.

An example of an HTTP Redirect Response will look something like the following, this was take from my own site when somebody queries www.coderjournal.com:

If you would like to learn more about how to perform these redirects, that I have talked about above, in your favorite language please read this article from Steven Hargrove.

Update (2008-5-20): I have updated my understanding of the different types of redirects that developers may want to use. See above for my new understandings.

]]>http://nickberardi.com/world-of-http11-status-codes/d921d8aa-9eca-4d03-b0ce-2e897a336eebTue, 24 Apr 2007 02:30:48 GMTLast week all the major search engine providers, announced that they were going to support a new specification at sitemap.org that allows them to auto discover your sitemap without you having to submit it:

Yahoo did a good job at summing up the advantages to putting your sitemap location in the robots.txt file.

All search crawlers recognize robots.txt, so it seemed like a good idea to use that mechanism to allow webmasters to share their Sitemaps. You agreed and encouraged us to allow robots.txt discovery of Sitemaps on our suggestion board. We took the idea to Google and Microsoft and are happy to announce today that you can now find your sitemaps in a uniform way across all participating engines.

This directive is independent of the user-agent line, so it doesn't matter where you place it in your file. If you have a Sitemap index file, you can include the location of just that file. You don't need to list each individual Sitemap listed in the index file.

]]>http://nickberardi.com/sitemap-auto-discovery-and-you/fbeb5fd2-c8f4-4e1e-b242-987cde4806b4Tue, 17 Apr 2007 17:15:15 GMTFor many developers the URL Address is just a means to an end, so very little time is actually spent on creating and planning a URL that is both functional and user friendly. We have all seen the URLs that seem to go on forever, I am not going to dwell on those URLs because you can find them anywhere. I am going to go over what a good URL consists of, and some easy ways to increase your search engine ranking with your already developed application.

Search Engines Crawlers are like People

One thing that a web developer has to understand is that Search Engine Crawlers are like people. Everybody understands that if your site content is not laid out in a way that is readable, people will not spend much time on your site. The same goes for Search Engine Crawlers, if your site doesn't conform to XHTML or at the very least HTML standards the search engine crawler isn't going to spend much time indexing your site.

Having these multiple URLs reference the same content decreases your Search Engine Ranking, specifically PageRank is calculated per-URL. So the best idea is to do a 301 Redirect for the different patterns I listed above. In my case of Coder Journal I have URLs 2,3,4 all redirecting to URL 1.

Combine Domains
Most people don't know but this blog has multiple domains that get you to the same point.

Just like what we previously went over about Duplicate URLs the same applies to domain names. So it is wise to also do a 301 Redirect from the domains. In the case of this blog I have the .net and .org domains doing a 301 redirect to my .com domain name.

The 3 tips that I gave you above are just the tip of the SEO iceberg. However implementing one or all of these should increase your Search Engine Ranking, without effecting the functionality of your application. What more could you ask for?

]]>http://nickberardi.com/a-guide-to-proper-url-construction/a1e0ef3b-1d9d-46a5-9e1e-d6c6664c9855Tue, 10 Apr 2007 13:19:05 GMTDo you hate having to go to umpteen sites just to download your essential software to get your computer running?Do you hate then having to again go to umpteen sites again to check for software updates, and then downloading and installing them?

Well I have the answer for you, it is call the Google Pack. Not only does the Google Pack include a wealth of Google software it also includes many non-Google software titles such as, Skype, RealPlayer, Adobe Reader, Norton AntiVirus 2005 SE, Ad-Aware SE Personal, Mozilla Firefox, and all the software is downloaded and installed according to your preferences. And as an added bonus there is a service that runs in the background called Google Updater and it will keep all of the supported installed programs updated to their latest and greatest version. See image to the right for a screen shot of Google Updater.

So if you would like to check out the Google Pack just click the button below:

The following wealth of programs is one of the main reasons I recommend it along with Firefox on the left side of my site.

List of Software in Google Pack

Google Earth

Google Desktop

Picasa

Google Pack Screensaver

Google Toolbar for Internet Explorer

Mozilla Firefox with Google Toolbar

Norton AntiVirus 2005 Special Edition

Ad-Aware SE Personal

Adobe Reader

Google Talk

Google Video Player

RealPlayer

GalleryPlayerHD Images

Skype

]]>http://nickberardi.com/why-i-love-google-pack/36042e89-93c9-43ee-a070-26c9fb766584Mon, 26 Mar 2007 12:18:11 GMTMany of the modern and sophisticated traffic monitoring software can track everything from number of visits, where a visitor came from, what search terms were used to find your site, where the visitor came from geographically, to what type of browser they are using on what platform.

Many of the JavaScript based solutions such as Google Analytics offer much more information about the browser than ones that sit between the web and the web server such as AWStats. This is because there is much more information provided via JavaScript about the screen resolution and color depth, however the down side is that if JavaScript is disabled you don't get any information. So it is usually a wise idea to use a combination of both Client and Server based traffic monitoring.

However one feature that they all seem to lack is tracking if a user has cookies enabled on their browser. For the life of me I cannot understand why this is the case for the JavaScript based Client side solutions. So I developed a small web service in .NET that allowed me to track this information via an AJAX call. You can accomplish the same thing with about an hour of your time and the ASP.NET AJAX Library provided by Microsoft.

First you need to create a web service called CookieSupport.asmx with the following code.

You may have noticed that CookieTest.CookieSupport.Supported(cookieEnabled); is the same namespace as the C# web service I listed in the first step. They took the idea of C# namespaces and translated it in to JavaScript to make it easy to remember and call.
This is facilitated though the ScriptServiceAttribute which creates a JavaScript file that is imported into your website, that is based on the methods of the WebService. You can view this generated JavaScript by visiting http://www.yoursite.com/CookieSupport.asmx/js, I have also included it below:

This came in very useful for me tracking the number of users that have cookies enabled. This may come in very useful for anybody else who wants to join their client side script via AJAX with a web service. This new technology in ASP.NET has limitless possibilities of joining your existing web services with the client side browser with out having to refresh the content of your page with each and every call that needs to be made.

As always happy coding.

]]>http://nickberardi.com/track-cookie-usage-with-a-web-service-via-ajax/1fa0fffe-d31a-42fb-af79-f483210bc072Thu, 08 Mar 2007 13:01:11 GMTGoogle Maps Germany has a new feature: if you have a BMW car that includes a navigation system and you happen to live in Germany, it's easy to send the address of a local business to your car's navigation system.

The "send" link from every Google Maps page will open a dialog that lets you fill your BMW account name and send an address plus some notes to your car. This service is free and it works only for businesses in Germany. Source