Tech

Why the Web Hasn't Birthed a Prettier Craigslist

If I told you these Eagles lyrics described a certain website, you'd probably think it was Facebook. After all, public exclamations of quitting Facebook are so common it's cliche, but by and large the people still stay. The same is true on other networks that angered users with product or business changes. But even when a better alternative arrives — such as Path to Facebook or App.net to Twitter, we still don't see the users walk away. The two times a major exodus did come to fruition — Myspace to Facebook and Digg to Reddit — it was largely based on weaknesses in ease of use rather than philosophy.

The weaknesses in Craigslist are painfully obvious: The site is ugly, any listing you post almost guarantees spam to your inbox, the apartment listings are full of scams, the site was once a known destination for sex trafficking (albeit minimized now) and lastly, people have died from engaging in a Craigslist transaction.

Yet, we continue to rely on Craigslist.

In my own experience, I've found 13 roommates across five apartments, one studio sublet, purchased a bunk bed and two queen-size mattress sets (new), and bought and subsequently sold a bed frame and a yellow Honda scooter.

As one commenter on Hacker News says, once people get to learn a platform, they develop muscle memory and stay as long as the platform maintains its internal rules.

But the question remains: Are we content trudging along, searching classified listings like it's 1999? The answer lies not just in the strength of product design, but also the limitations native to every network.

The Network Problem

Many startups have what's called a "chicken and egg” problem. In the case of Facebook, it wasn't useful until a person had about ten friends. The challenge is to attract someone's friends to the network so they'll come, too, and vice versa. You can see why this can be difficult. Most social networks have a threshold of connections a user needs in order for the user to return.

Entrepreneur Andrew Chen explains that it's difficult to even demo a product without existing users. This is why Instagram originally allowed users to find social connections via other networks, to enable this minimum threshold early on, and also why Twitter shut it off. Network strength is a valuable asset — not something you pass along to a competitor. But for an up-and-coming network, standing on the backs of giants is common.

For a commerce site such as Craigslist, the ratio of buyers to sellers is what's most important. The buyers won't come without items to be sold while the sellers won't come unless there's a market for their goods. If a marketplace was just starting out, it might want to link out to products in a similar marketplace to enhance its own offerings, in order to mimic the look of activity — the single measure which dictates if a social product works.

Undercover Fixes

Sometimes, a nascent network will stray from traditional growth tactics to build its network. At Reddit, the founders admitted to creating hundreds of accounts and posting through them to deal with the "ghost town" problem. Airbnb is accused of programmatically targeting Craigslist users to get its service off the ground (on the flip side, early Airbnb also allowed users to duplicate posts to Craigslist). For a new website looking to be a destination for apartment hunting, roommate finding, used car sales, jobs listings or any other facet of what Craigslist offers — or all of it — the ability to start with a basic critical mass is key. A Craigslist API would enable this for an up-and-coming site. But it doesn't exist.

How to Scrape a Website

An API can allow two seperate platforms — such as Google Maps with Yelp or Twitter with Tumblr — to interact with each other. For example, Goodreads' API would enable users on another site to add books to their shelves, find authors, view books owned by a user, and other actions native to Goodreads. A Craigslist API could allow users through a separate service to post, respond to, and see listings on a map and more. If it was available.

Since it's not, there is another way. Sites including Wikipedia, Twitter and Craigslist have standard structures that allow engineers to write a basic program that can target specific information. And, of course, do that as many times as needed.

For a developer working in Rails, there's also a Gem called "craig” that's done the work of demystifying Craigslist's site structure for you. These days, everyone can be a developer, thanks to learning programs including Flatiron School, Treehouse, Skillshare and Dev Bootcamp, all of which offer courses in seeker-friendly Rails — therefore, more beautiful, Craigslist-killer apps should be a dime a dozen.

But they're not. The sites that did make use of Craigslist data, large and small, each recieved a cease-and-desist letter from Craigslist in June 2012. From Craigslist's Terms Of Use (TOU): "Any copying, aggregation, display, distribution, performance or derivative use of Craigslist or any content posted on Craigslist whether done directly or through intermediaries (including but not limited to by means of spiders, robots, crawlers, scrapers, framing, iframes or RSS feeds) is prohibited.”

Image courtesy of AGBeat

Rentscoper, a site aimed at improving decision making for real estate investors, featured a heat map relative to real estate pricing, Yelp reviews and Craigslist listings on a map but complied with Craigslist's cease and desist. Rentscoper contact Jeremy Elser says "We are restricting our business model to our core strength, which is real estate analytics and visualization." He adds that the "home-scoping" aspect of the site will return pending changes in Craigslist's position.

Other cease and desist recipients include jaXed, Mapskrieg, Craigspal, Swappel and Padmapper. To avoid what one startup revealed would be a $15,000 fine per agreement violation, each of those sites seems to have complied with Craigslist's wishes, at least for now.

The API Ninja

With a licensing agreement and revenue share, a site could play nice with Craigslist. Alternatively, a third party could provide a workaround — which is Padmapper's solution.

3taps makes APIs. According to the founder, Greg Kidd, what his company does is no different than what Google bots do when they index websites — in fact, some of his data comes directly from Google. For reference, the company I used to work for repackaged news articles in a mobile app, and at one point pulled some of this data from Google's servers as well. While Google can cut off a site that accesses it too frequently, it is constantly indexing the web and everything posted publicly there. Currently, 3taps allows registered developers to use the APIs it creates on a pro bono basis and, in addition to the Padmapper users, has several hundred developers registered, although a much smaller number are active.

According to Eric DeMenthon of Padmapper, this workaround circumvents the TOU issue which brought him the cease and desist — however, both himself and 3taps are now faced with something more serious — a lawsuit.

Legal Troubles

But aside from requiring 3taps to stop harvesting Craigslist's data, what can the lawsuit really do? According to startups lawyer Damion Robinson, if Craigslist can prove its business was harmed, the other companies could owe it for damages. Kidd from 3taps notes Craigslist briefly changed its TOU to claim copyright for all user postings but quickly reversed the terminology to distance itself from ownership and responsibility for the content. Social networks and other platforms built with user generated content (like Yelp, Instagram and Twitter) generally do not claim copyright for content posted by users — in Craigslist's case, it seems the company would want to distance itself from ownership of that content, as it does not want to be liable for illegal postings the site has a knack for attracting.

The lawsuit is ongoing — and in September 2012, 3taps struck back with a lawsuit of its own.

What Is Public Data?

3taps sued Craigslist for anticompetitive business practices, or more simply, monopoly. Craigslist is the only classifieds site consumers can expect to find a generous supply of goods, and sellers can find numerous buyers, but Craigslist declines to add features (read: innovate), such as saving a listing to favorites, which are technically possible and users generally expect.

Image courtesy of Craigslist

3taps argues that the information posted on Craigslist is public data. Kidd explains that a product description and price are facts (not creative expression) and distribution should not be controlled by one company. This distinguishes the information on Craigslist from user-posted information on social sites such as Twitter, which is narrative rather than fact-based.

If 3taps wins its suit, will this set a precedent for sites in the future that may want to freely access data from one another? "It would be a step in the right direction," Kidd says.

Is what 3taps does all that different from what Google does — sorting and optimizing public information for users? Think of Padmapper as a glorified search engine: It pulls information from many apartment listing sites, and local business and walkscore information from other sources, but optimizes it with a map rather than a list ordered by keywords.

A terms of use sets expectations for users, and every site has the right to lay out whatever terms it deems to be good. One thing nearly all sites hosting user-generated content have in common is that it's expected users will redistribute posts elsewhere — for example, their own YouTube videos. Even for Twitter, users once took screenshots to post tweets elsewhere and still would, if the company hadn't innovated and introduced the embed tweets feature. Craigslist is the only site to want to limit redistribution.

But there's a difference between a user posting, say a link to his or her Craigslist listing on a Facebook page, and a company mass posting Craigslist listings on user's behalf without explicit permission. So I ask you: Do users of Craigslist have an expectation that their listings will only be viewed and accessed through Craigslist proper?

Solutions

An argument at the forefront of Craigslist's lawsuit is that the company "worked hard and invested heavily for many years” to build its community and business, and each Padmapper and 3taps grows "its business on the back of Craigslist and its users.” If users aren't getting what they want on Craigslist, it seems another company could do the same — simply offer a better experience and albeit slowly, build its own community from scratch to eventually take market share in the classified listings space.

Swappel and Zaarly are two such examples. Swappel, which is in beta and currently limited to the Dallas market, allows users to post goods for sale with a price and suggested items for which they'd trade. The site at one time offered a feature that allowed users to cross post to Craigslist without any mention promoting Swappel, but removed the feature due to Craigslist's cease and desist. Zaarly built up its local listings through user storefronts and has not integrated directly with Craigslist, but used tactics similar to Airbnb's wooing of Craigslist sellers directly through their listings. As mentioned earlier, Airbnb integrated with Craigslist briefly to enhance the usefulness of its separate marketplace. Krrb, a home goods listing site and mobile app, offers a feature to allow users to republish their Craigslist posts onto Krrb.

It would seem that it's impossible (or foolish) to try to replace Craigslist without integrating with it in some way.

Images courtesy the App Store

In the apartment listings space, Lovely is building a formidable competitor to Craigslist's real estate vertical, but got its head start from Craigslist via the 3taps API (Kidd is also one of Lovely's investors). Lovely was made up of 100% Craigslist information at launch but the proportion is now reduced to about 40%, says founder Blake Pierson, with the goal of removing 3taps completely by the end of this year. The company offsets what it gets from 3taps with other data sources, including rental sites such as apartments.com and programs used by property management companies such as AppFolio. This year, Lovely will continue attracting supply (i.e., apartment listings) by releasing digital tools for what Pierson calls "longtail landlords," or people who might have a day job and just rent out one or two properties on the side. These tools could better enable digitally collecting rent payments or managing maintenance requests.

Despite never releasing financial information, it is estimated that Craigslist earns more than $300 million per year (the company could not be reached for comment on this story). The site certainly has the funds to hire top-notch interaction designers to build a superior product, as the original Craigslist must have been for users in 1995. But instead, it focuses resources on fighting for legal domination over competitors. If Craigslist is to lose the lawsuit at hand, would it maintain its competitive edge simply on the size of its community? Would it finally change course and innovate? Or, would the site slowly fall into obscurity like other websites that display blue hyperlinks on a white background?

One anonymous Quora user claims the reason Craigslist has not been replaced is simply because its competitors focus on a subset of its offerings and and therefore cannot replicate the network effects of Craigslist.

Pierson points out that having 100% of a market is crucial, and even more so for an infrequent and important transaction as finding an apartment is — knowing there's inventory they may not find is too much of a disadvantage for users.

Kidd from 3taps says he expects progress from courts in mid-March regarding whether Craigslist carries any rights to its users' postings. Until then we will continue to anonymize our email addresses in exchange for the buying and selling of vintage dressers, and have a friend accompany us on transaction meetings for safety's sake.

Craigslist

1998

Craigslist first launched in 1995, but web archives of the site only go back to 1998.

1999

In 1999, Craigslist was still exclusively in the Bay Area and the site design featured a left rail, centered listings hyperlinks and company information aligned to the far left. The design almost feels adaptive ... but not really.

2000

In the early years, "new features" were often called out on the site, and as the "activist postings encouraged" copy suggests, norms for the community were still being set.

2001

2002

In a discussion of World Series ticket reselling, The Berekely Daily Planet writes, "But for real jawdroppers, check out Craigslist, an online bulletin board based in San Francisco. It’s the Internet, so not everything is what it seems. Still, postings reflect a sellers-market delirium."

2005

A Boston writer hypothesizes Google's strategy is to take market share from Craigslist, which he describes thus: "While bourgeoise civic life in its home town of San Francisco is unimaginable without Craigslist - it's the city's scrappy noticeboard for everything from accommodation to quickie knee-tremblers - the operation has little foothold beyond the coastal United States."

2008

From USA Today: "With 25 employees working out of Victorian houses in San Francisco's Inner Sunset neighborhood, the site has grown from 1 billion page views per month in 2004 to 9 billion per month now, according to Craigslist. It hosts 30 million new classifieds a month."

Mashable
is a global, multi-platform media and entertainment company. Powered by its own proprietary technology, Mashable is the go-to source for tech, digital culture and entertainment content for its dedicated and influential audience around the globe.