If you are considering adding user-generated content as a form of content marketing on your website to increase your traffic, like I was trying to do, here are a few things to keep in mind:

Lesson #1: Google knows what pages contain user-generated content

Over the last few months, I’ve continually received messages like this one from Google:

I didn’t realize how sophisticated Google’s algorithm was in detecting user-generated content until I received warning messages telling me which pages on my site contained spam.

The legitimate user-generated content never ranked as high as my blog posts did. It wasn’t because forum posts lacked links and the pages were indexed. It was because the quality of the forum’s content wasn’t as high as the blog’s.

Unless you have quality guidelines like Wikipedia does, it’s tough to encourage high quality user-generated content. And if you can’t get the quality up, your search traffic will be low.

Lesson #2: More pages doesn’t mean more traffic

The only issue is that I lack content. Over the last 7 years, I’ve only created 567 blog posts because it’s a personal blog. I knew I couldn’t write 10 blog posts a week, so I decided to encourage user-generated content as it would allow me to create thousands more pages and potentially rank for more terms on Google.

As you can see from the screenshot above, the forum was generating 43,467 pageviews a month. It may seem like a lot, but it is less than 4% of all the pageviews. Given that the forum makes up 89% of the pages on Quick Sprout, 4% is not very impressive.

If you are looking to generate more search traffic, adding more pages isn’t the solution unless those pages are exceptionally high in quality.

Lesson #3: Participating helps build brand loyalty

The beautiful part about user-generated content is that you, as the site owner, can participate in it. For example, I responded to almost every single thread on the Quick Sprout forum.

When a user had a question, I responded. People continually thanked me, and my participation helped build loyalty.

I even got emails from companies thanking me for my responses and telling me they wanted to work with me.

I never generated any customers as I wasn’t willing to entertain any offers, but I’m confident that I could have converted the forum into a profitable channel.

If you decide to create a forum, a questions-and-answers portal on your site, or anything similar, make sure you take the time to respond to your users’ queries. If you really care about your readers, they will feel it, and it will help you generate more brand loyalty.

Lesson #4: User-generated content creates a lot of spam

The forum contained 346,299 users, 2,788 threads, and 12,731 replies as you can see from the stats I published at the bottom of the pages.

The number of users may seem impressive, but over 95% of them were spam users. My developer and I kept fighting these users off, but they found more ways to sign up.

Luckily, through Akismet, very few of these spammers were able to participate. Every once in a while, however, a few would get through, and I would get notified by Google Webmaster Tools through a message similar to the one you saw at the beginning of the post.

Additionally, spam comments went through the roof. Quick Sprout used to generate 1,000 or so spam comments a day, so it was possible for me to have someone go through them to make sure legitimate comments weren’t marked as spam.

Now, I was generating 50,000 to 60,000 spam comments a day, and it’s nearly impossible for me to have someone go through them. This was preventing legitimate comments from being posted on the blog.

For this reason, I removed the forum as the traffic wasn’t substantial and it was creating too many warnings from Google.

Lesson #5: People are inherently lazy

There is nothing wrong with this as I am inherently lazy too.

But with user-generated content, it’s hard to get people to add large quantities of high quality text. This caused forum pages to have on average 327 words, which doesn’t help very much with search rankings.

In order to help boost the word count, I was responding to every single thread. In addition, I recruited moderators to help respond to user questions with high quality content.

But I still couldn’t get the word count high enough without adding tons of fluff, which I didn’t want to do.

For this reason, I wasn’t able to create detailed, Wikipedia-like user-generated content. When you consider that Quick Sprout gets at least 500,000 visitors on a bad month, it shows that you need a lot of traffic to build tons of high quality user-created content.

Lesson #6: Indexing is an issue

When you have thousands of pages on a sub-section, you’ll notice that those pages don’t get indexed fast or indexed at all.

To fix the indexing problem, I implemented the following:

Cross-link – I manually went into hundreds of forum pages and cross-linked different threads with each other to help increase the indexing rate. I also did this with blog pages, linking them to the forum.

Link building – I reached out to thousands of sites over the year to encourage them to link to the forum. More specifically, I wanted other sites to link to various threads within the forum.

Leveraged press – every time someone interviewed me and asked me a question that was already answered within my forum, I plugged it, which helped with rankings.

Modified meta tags – a lot of the title and description tags on forum pages were too short and not unique enough. I fixed this over time, which helped more pages to get indexed.

Although doing the above things helped get more pages indexed, it still didn’t increase the search traffic to forum pages. The forum was bringing only 1.44% of my total search traffic.

Lesson #7: If you can’t build it, you can buy it

The one thing I was able to do with the use of cash was encourage engagement. Every day, new replies and threads were created as people participated on the forum.

I paid bloggers to promote the forum – the most engaging users a blogger typically has are the ones subscribed to his or her email list. So, I was paying bloggers, such as JohnChow.com, $5,000 to announce the Quick Sprout forum to their lists. This helped encourage more signups and create more content. I did this with 32 popular bloggers in the marketing and design space.

I marketed the forum to new readers – every time people signed up to the Quick Sprout email list, one of the first emails they received encouraged them to sign up to the forum and create a thread. I also promised in that email to respond to their questions and help them out. This is where I got the most consistent engagement and new content.

If you have the money to spend, you can create more content by following the tactics I used above. They work and are still effective today. It’s just expensive.

Conclusion

I removed the Quick Sprout forum because of a few reasons:

It was causing too many spam complaints from Google.

It wasn’t generating enough traffic.

It took at lot of time to maintain, considering the low traffic volume it provided.

I still do think user-generated content is valuable. You just have to figure out how to get your community to engage. And more importantly, you have to make sure the generated content is extremely high in quality and quantity, or else you won’t see much of an increase in ranking.

Comments

I quite enjoyed browsing through the forum on a semi regular basis and I always assumed it generated you a lot of new visitors. It’s really interesting to see some of your actual figures and I’m quite shocked it wasn’t generating you more visits.

I hear what you are saying there Neil, but aren’t a forum and a blog two totally separate things for separate purposes – therefore providing different (and additional) levels of feedback and interaction?

I am pretty new to this, so I may be wrong, but I look at a blog as medium that provides some new information to the reader and has a “call-to-action” and provides the first step to helping the reader act on that information.

Whereas a forum is more of a “quick stop” place where a person can get a pretty quick response to a thought or a question on the fly without waiting for it to come up in a blog or posting off-topic in the blog comments.

Seems to me that keeping things separate like that would add to the quality rating of both formats.

But as I say, I am only new to this, so I would love to hear your thoughts.

I think another problem was you had no real ‘end game’ for it.. good test for USC and content marketing thing but if you had this as a private and paid community with access to you, it would have been a great recurring income source. And a very useful place, content quality behind closed doors is way way better than on public forums..
It’s also great way to get clients or have people who can become them.

Adrijus, while that is a great idea it’s ultimately against my philosophy. I believe information should be free, up to a certain level. A forum is a place where people go to ask any and all questions. Typically I was getting a lot of basic questions on there from beginners and people really looking to get into the field.

I get that. But as member of both kinds of forums..real value is in private ones. And you share plenty of info for free already, why repeat the same answers on forum again? Private community would probably attract more advanced/serious people who have more to gain from you than newbies can possibly gain. Different way of providing value (more action takers on it too). That’s the next level.

Anyway, not saying that you must. But just an option if you ever consider it. 😉

Thank you Neil for the really informative post (as usual) and Adrijus for your thoughts here.

I am still in the process of building my site but have planned to have both blogs and forums on it.

I have made it a private site right from the start and it has taken months of work to set that up with more than 75% of the 50 pages completed so far just devoted to management of the membership aspect.

I am obviously hoping for little or no spam with this and at the same time wanting to keep content at a very high level.

I also want to keep the site totally free for members with limited and not “in-your-face” advertising providing some revenue.

So based on your comments, I am hoping that I am on the right track.

The other thing that I just set up a few days ago is something that I called “TripleTweats” that allows members to add a 420 character (3×140) “TripleTweat” that is either informative, motivational or thought provoking for other members.

While members have to be logged in to post, the “TripleTweats” are displayed on a free public page.

I am looking to see if I can extend that sort of thing with my forums and blogs in a way that will make valuable and ranking content available to all, but not generate any rubbish or spam.

Yes, still a site very much under construction Adrijus with MyBB software installed and integrated with the membership program to avoid double log-in, but still a fair bit of work to do to totally customise it and set it up.

In fact, it is intended that there will be three separate forums covering three totally distinct and separate areas when it is all finalised.

This really hits home on the importance of quality long tail strategies. Long-tail searches account for the large majority of online searches and have a vastly higher conversion rate. And like you said, it’s about the quality. If the quality isn’t high, then your conversions will be low, so why bother?

Great post, I was thinking about starting a forum and I’m glad this helped me dodge a potential bullet.

Mike,
I’d have to agree with Christopher. What didn’t work for me may work for you. It’s all about finding the right fit. I found that my blog and other channels were providing much more valuable traffic and engagement. You may find that a forum does the trick better for you.

I had a forum on a site called toptenz.net (over 1.5 million visitors per month to the site itself) and removed it after a few months for the same reasons you mentioned. Spam was a killer and took all my time to manage even with apps to help keep the clutter down.

I am now wondering if comments should be hidden so they don’t cause in flags for Google. Some posts on my site have 600 comments that don’t really add much and the comment text is more than the post text in some places. Do comments play a role in search? It seems both positive and negative after reading this.

I was wondering what happened with it. I can imagine the time and resources it must have taken to maintain it.

I think the best example I’ve seen of an engage community with high quality content is Moz’s YouMoz community. When posts go crazy viral, they’ll promote it to their main blog as well as an incentive to produce great content.

Neil, it’s sad that you removed your forum.
But I totally agree with your reasoning.

Sanitation of user-generated content is one of the priorities indeed.
Technically it can be done with a system of user/messages ratings and the team of forum administrators. But I agree that it’s easier said than done.

One of the good examples of successful forums I know is webhostingtalk.com. Regardless of such a lucrative filed like web hosting, its community manages to maintain the forum from spam and deceits quite well (at least better than many other forums).

I guess the key to its success is an ethical and moral aim of its members to keep that place clean from scammers.

I think I will analyze it (and other forums) in more details some time in the future, because I want to understand deeper a healthy community building idea, as well as technical ways to make such a good and content-reliable place.

The leading existing marketing forums in many ways is just a bunch of spam, ads and scammers (with a portion of good and ethical users, but it does not change the overall picture).

Michael, don’t be sorry — it was a great test that provided some useful information. At the end of the day, as you mentioned, the spam was just overwhelming. I learned a lot during the process and my visitors did to. That is valuable data that I could never get anywhere else.

Community building is ultimately about having honest conversations — I think I get more of that on my blog which is why I am focused on it solely right now.

Thanks for being so honest. You spelled out all the rights and wrongs that you did, mistakes that you made and the lessons you learned from them – this is invaluable!

This post is also important to me because it’s been a couple of months that I too started a forum at the community on my blog. So far, things are pretty okay, but I can understand that the scenario is different with the users and their posts and comments are in such grand numbers as you mention. With having even just an iota of participants that you had at your forum, I still find it difficult to engage with everybody, as that eats up my time for the other blogging tasks.

So, I agree that maintaining forum is not an easy task as it requires more of your time and personal input, and if you have to hire people, then more money too. Quality and quantity engagement and interaction is the key in forums.

I’m sorry for what you had to undergo, as I’m sure you spent a lot of time on your forum, which sort of turned into a bad investment. Nevertheless, such life lessons learned are more precious and they do help us in someway or the other later in life.

As for the forum and community over at my blog, it caused the Alexa rank rise from 30,000 to 18,000 in about 3 months time. I didn’t analyze the other aspects and ranking facts, but I’m sure something good must be going on in the statistical background.

I do make sure that the forums and the community is spam-free, doing it manually as well as taking help of a few plugins. I guess having less members is beneficial because they are easy to manage and it is easy to create quality and maintain standards. People are loving the ABC (my blog community) as it is like a second home and family for them.

But I’ll take heart from your experiences and emphasize on the important things to do as you mention in this post. Indexing is a major issue for me and I can’t afford to pay so much to get the forum running.

Your post is just in the nick of time for me. Thanks once again and I hope my comment doesn’t vanish like it did in your previous post, so I’m saving it so I can repost it in such eventuality! 🙂

~ Harleena

P.S. I’d really love you to visit my blog community and see the forums in action. I’d appreciate very much if you can give me some of your advice to make it better. Thanks. 🙂

Harleena, glad I could share my experiences. Good luck on your forum — I am sure you will do well. Just be mindful of all the mistakes and pitfalls that I encountered. At the end of the day you have to make sure you are moderating and getting rid of spam. Give me a link and I’ll check your forum out.

I’m an amateur and not afraid to admit to it. I travel to your website at least 3-4 times a week and post your infographics on my website because they’re extremely valuable. I’m surprised you failed at content marketing, but lesson learned. Thank you for teaching me.

I want to dive in further to this article later today because there’s so much to learn, but I wanted to point out that I admire you not wanting to add a lot of fluff to your content, just to rank higher.

I think it’s so important to put readers before ranking strategies — if your readers don’t want to read a lot of fluff, and you don’t want to waste their time, it’s important to stick to this strategy. Losing readers would be a quick result of fluff and wouldn’t be worth “almost, maybe” ranking higher on Google.

We have a really successful forum on our Writers Workshop website. Our site offers various editorial services to writers (editorial consultancy, etc) but our forum – we call it a community – is structured as a place where we encourage writers to critique each other’s work, interact, support each other &c.

We have 8,000 members and those 8000 have generated well over 40,000 posts. I don’t think I’ve ever once had a spam warning from Google.

So how do we do it? First of all, we use software (from SocialGo, if you’re interested) which is about building communities, not just offernig places to ask questions. Secondly, we make the sign up form relatively challenging: we make people answer questions on their favourite writers, that kind of thing. If that form is filled in by a robot or a spammer, it’s instantly obvious and we just delete it: we delete 2-4 new members a day: a literally 1 minute job. Thirdly, if idiots do get through, we’ve given a few highly trusted site users limited admin abilities, so they can eject the spammers without asking us. The site has an incredible community focus, so much so that there’s been a real-life marriage arising out of two people who got to know each other on the site. Also offline meetups for people who only knew each other online to start with.

The biggest business benefit for us? Well, we’re careful with the community. We don’t hard-sell our product at all and we’d hurt the community if we did. But those 8000 email addresses are BY FAR our most loyal fans & customers. And they’re all the more loyal because we give them a genuinely supportive and non-commercial place to meet and exchange ideas. True, there are plenty of people on the community who are barely aware of us as a business – but we don’t care. Those guys probably wouldn’t buy from us anyway. For the others – the ones who are in the market for what we sell – we’ll be their first natural port of call.

A few years ago now I added a forum to my tiny little computer game blog, promoted it to my readers, and nothing but crickets…

Until the spammers found it.

I had so many spam signups it was ridiculous. I managed to set things up so that spam posts were blocked, but I just found that it wasted my time. I could have been writing posts, instead I was maintaining a useless forum. I ended up deleting it.

Thinking back now, I believe I approached it wrong. I was looking at it much as you did, Neil, as a means of increasing content, but also as a way to increase engagement on the site. My mistake was not planning it properly, or appreciating how much work was needed to keep the spammers at bay. It was probably also the wrong type of website for a forum. Gotta know your audience.

I guess I can understand your reasoning, but instead of hitting delete, wouldn’t it have been a better idea to move the forum to either a subdomain or it’s own domain? I only started looking at it a few weeks ago, but I think there was a significant amount of content there.

Surely you’d find someone willing to keep the forum going on a different name and perhaps a rebrand. That way all your efforts of growing the forum wouldn’t have been in vain. All you get out of it now is a good blog post 🙂

Bob, that was a suggestion I heard a lot when I was in the process of deletion. Ultimately I just wanted to do away with the forum and focus solely on my blog — that’s where people provide the best engagement.

When we were just setting up the website, I remember adding a forum page on it. We even came up with topics, I got moderators into the job, and was so excited about the whole idea. Until I thought about it actually going live. Thinking back to it now and after reading your post, it really is important to understand what you’re getting into. I’m happy with my website content now. We’re still improving – but I’m glad I removed the forums before we started.

Al, I think ultimately you have to do what works best for you. Everyone thinks there is a magic formula whereby you have a blog and a forum and everything will come into place. However, that’s not the reality. Certain things will work better than others.

With the popularity of blogs and increase in high-quality blog posts, forum posts have taken a back seat in terms of search rankings. I do still see some of the top forums ranking well though, especially for long-tails.

The key to a forum is user engagement and spam filtering. This will equate to high quality threads. Unfortunately, this takes a lot of time to do well as you have pointed out Neil.

Great post!
I implemented user generated content on my site a few years ago. I get about the same traffic as you. I also had SEO fantasies lol.

What happened in my case was I ended up moving the forum to a separate domain on proboards, because it was only a small number of users who wanted to chat. a lot. all the time. And if they didn’t have an outlet, they would take it out on game content pages and fill them with non relevant stuff (young user base). Also, my initial forum was coded from scratch and ridiculously inefficient.

What did work though was I charged $1/month for commenting privileges around the site. This eliminated spam 100%. Never has a spammer joined the site and paid lol. So that’s one way to think about it… Payment doesn’t have to be large. I bet even a one time fee of $1 would be enough to eliminate nearly all spam (possibly all).

But yeah, it’s hard to track, but I don’t think it’s really brought more traffic my way. At least the forum hasn’t. Do you find that comments under blog posts bring significant traffic compared to the post content?

You can’t depend on others to create your level of quality. I vibe with it Neil because recently I began posting once weekly. 3,000, or 6,000 word posts – or longer – are my gig now. Less posting, but posting only the highest quality, keyword-rich, astoundingly in-depth stuff is my goal because Google likes the highest quality, in-depth stuff.

Your realization about volume resonates here with me in Bali. I played the mental game for a bit. More, even if it’s really good, is better, to my old mind. But lil old Google and it’s mechanisms are so wise to our game, and catch us when we try to pull a low quality or not astoundingly high quality fast one on them.

Thank you for sharing this post. For new bloggers like me generating traffic is always a puzzle. Most of the time I am not sure about what is right and what is wrong. This post, cover some interesting tactics, which even though appears lucrative, didn’t generated much result. Thank you for putting up everything in such a great detail. Loved it!

For example, your blog doesn’t facilitate posting of pictures, video, links, or other cool things I might want to share to make my comments sexier.

I had the exact same issue a couple of years back with a forum I started. I found UI/UX of all forum platforms were not conducive to content discovery and sharing.

Social media platforms like Twitter, LInkedIN and Facebook are the forums of today. Makes content and discussion discovery much easier, given hashtags and the open nature of social media. If forums were to somehow integrate those features, webcasting, vlogingg and the UI/UX of social media, I think they could be more successful. Alas, haven’t seen anything come close to the likes of Twitter or LinkedIN groups. And those platforms have their limitations also.

Aaah Neil. 🙁 I wish I knew before that you were about to close the forum, so I could have saved a few threads I started or that I liked. I tend to grow an emotional attachment to forums, they’re like homes, or libraries.

But I know how stressing and counterproductive it can get when spammers are involved… There are forums I had to keep shut down for months until I managed to clean up all the junk.

Most of my new forums use SMF (Simple Machines Forum, http://www.simplemachines.org) that comes with easy-to-install anti-spam hacks that stop 99% of spam registrations. One of my younger forums only had 1 spam registration in 11 months. 🙂 Works pretty well.

(Long time lurker, first time post here.) This post was very timely for me as I’ve been going back and forth over the idea of adding a forum to two of my high traffic sites for the last month. But every time I get close to do it, the thought of spam stopped me in my tracks. I could not find one forum setup that would effective at the fight. I was still weighing my options as I stumbled upon this article.

The reason I had thought of adding a forum was that I already have extremely engaged readers who comment on my pages. I, like you, wanted more pages where they could create user-generated content for me. I have zero issues with spam because I dumped askimet and started using the cleantalk plugin (not free but well worth the few bucks).

So I have a question for you, regarding comments such as this one on this page: since I have such clean comments coming to my pages, with many of the pages having hundreds of comments, does Google consider that a positive for the pages? Or is there some negative (or neutral) aspect that I am unaware of. Every comment is real, and we reply to the majority of them (not sure if Google notices, but they seem to know everything). Can I keep encouraging and responding to the comments with the thought I’m actually doing something that makes the page more valuable to Google (thus helping the page’s ranking)?

I too started my forum with big expectation and paid forum posters to fill my forum which got my community a big kick start but soon after 6 month of operating things really get out of hand spammers filled my forum with garbage post while profit from ads wasn’t much to cover up the expenses.

So, it better to choose something you can control i guess how hard would be to monitor yahoo answers i wonder how they manage such big community site a automated monitor software?? might be.

“For this reason, I wasn’t able to create detailed, Wikipedia-like user-generated content.”

I know this was brought up earlier in the thread, but perhaps if you want Wikipedia-like content, you have to create a Wikipedia-like system. If you want an encyclopedia of marketing terms and practices on your site created by users, you have to create that system with editors and moderation, not just throw up a standard forum.

From what I remember (could be wrong) the forum was pretty basic and wasn’t build out like Moz or Stack Overflow. Anyone could post on yours. On Moz if you’re a free member you have to accrue a ton of community points to post.

Well i liked the forum. what i didn’t was the bbpress software you used for wp. you may have chosen the wrong one. a standalone forum software with a wp bridge could work wonders. sure most are not much better. one issue was the facebook sign-up. it didn’t let me update my profile and picture.

i have used also a wp-plug-in called simple:press and at the peak of my sites traffic of 30k uv’s/day, it ranked almost any topic that was created top three.

the good thing was, it not only showed the thread that was searched for, it also showed similar topics below it on google.

that was a time where a lot of topics were created on a daily basis. with the traffic and topic-creating decreasing, the rankings went down. now only a few threads still generate traffic.

when talking about spam, i have to say akismet, anti-spam bee, math for spam and simple captcha was the only way to stop it. i barely get 10 spam regs on a daily basis. none of them could post spam topics.

when looking for the best ranking sites with forum softwares, one shines trough the most. it’s called vbbulletin. the vbseo extension seems pretty good too. the issue for some could be that it’s not free though.

Hi! I was curious to know if setting up a web site such your own: %BLOGURL% is hard to do for inexperienced people? I have been hoping to create my own website for a while now but have been turned off mainly because I’ve always believed it demanded tons of work. What do you think? Many thanks

Leveraging your brand, name and existing user base gives you a huge advantage and has more of an impact that you will ever admit. You say this “While this does help out a bit, it’s not the main advantage I have, as I discussed above. It’s my experience that gives me a leg up.” If experience is more important why don’t you do a blind case study where you start with nothing, completely anonymous under a pen name and no connection to you or your present businesses or circles of influence?

Vincent — I think the main goal of all this is to help others. If I could start over I would definitely take that tact. With that being said I will continue on this way and provide as much value as I can to readers.