Research is always challenging, I’m required to stay objective, follow a consistent methodology, and inform the market who is strong and who’s not. Believe me, I’ve gotten my share of angry calls, emails, blogposts, comments, and conversations with brands and vendors as a result of my reports. I’ve found that one thing that really helps everyone to understand is to be open and transparent about the process.

In every Wave, like the Community Platform Wave I recently published, there are vendors that are elated and those that are disappointed, this is a normal outcome of reports that make the tough decisions to help brands make decisions. I’m empathetic to some of the vendors who were not in the Wave and want to make some clear explinations as they have to answer their colleagues, customers, and investors.

To start with, in a market of now 100 vendors, it’s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog (free after registration). We used this information to filter out which vendors would be appropriate for the Wave report. I made a few public calls on myblog, twitter, indicating my intent as a result over 50 vendors submitted. We generally do not follow up unless we’ve heard client demand from inquiry calls, and I turned to 3 analyst peers for their inclusion of any additional vendors.

You can understand the challenges in filtering a large market to just a few vendors, given the amount of hours required to spend on each vendor, Forrester limits the number of candidates involved in the Wave, for this one, nine was a manageable size and we capped it at that number.

Next, we filtered vendors to meet the needs of external communities used for customers (I serve the Interactive Marketer) as well as vendors were primarily serving enterprise class companies.

Some vendors chose not to participate as being involved in a research report takes time of the vendors and perhaps precious time away from clients. Vendors have to participate in meetings, plan travel to meet me in lab, and be involved in several communications throughout the process. In fact, a multi billion dollar company who owns a community platform chose not to participate despite my offer.

I’m required to be objective, without this, I lose credibility and then everything else will fall apart. Several vendors who are in the community space who are Forrester clients were not included in the Wave, in fact, some were livid and gave me an loud earful and I actually suspect they may leave us as a client. Despite this, I’m supported by my management, and our CEO to stay objective.

Now for the good news, while there is no immediate plans, Forrester is known for updating Wave’s in the space after a few years. Secondly, if you’re in this space, you know I publish a weekly digest giving updates to buyers about what’s happening in the market. I know for a fact the digest is read by senior decision makers at large corporations. If you are a vendor in this space, you can schedule a briefing with me and we can discuss your situation, I speak to clients, press, and media frequently and will always match market needs with vendors as deemed appropriate.

I believe in the open conversation, and will support the right for every vendor and customer to voice an opinion. I hope this helps provide insight to why some vendors did –and did not make the Wave, we tried every effort to be fair in our first report in a very large market.

Peter thank you, in a few cases, the lack of an AR strategy is to blame. I think there are some strong companies out there that did not get considered because they don’t get in front of analysts.

Lorenz

Hi Jeremy,

I don’t quite understand how you can give a balanced report if you exclude your own clients (I do understand that you are a stakeholder, but you are supposed to evaluate the BEST vendors, and you show a bias against your own). Earlier in the article you also mention that in a market of now 100 vendors, it’s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog. Does that mean you are biased towards those people that don’t hear your call, don’t follow you or don’t find your methodology the most ideal as it takes time away from their clients? It would appear that in essence it does.

In the end, in order to evaluate a product, you analyze the most prevalent needs of the market space, segment them into categories of needs, and test products in the marketplace in relation to those needs, then rank them per category. As long as you are transparent which clients are on your book, I have no problem with them being included in a study (in fact, they are right to be angry because they suffer an obvious disadvantage from the relationship).

I didn’t expect you to read it that way, there was no bias at all. Please re-read my post, I suggest that it doesn’t matter if they are a client or not for the selection process. I’m sorry you misunderstood, was my wording not clear?

You’re right, following a consistent process is what I did. The vendor product catalog as a filter for a wave is a process that has been used for other wave reports, and in a market as large as this one, that’s certainly a key way for me to be efficient with my time.

To suggest that I reach to 100 vendors via email or phone would quickly overtake my normal duties.

Your second paragraph is spot on, you get the process.

Thanks for the thoughtful comment.

Lorenz

Sorry, I misunderstood the following paragraph: Several vendors who are in the community space who are Forrester clients were not included in the Wave, in fact, some were livid and gave me an loud earful and I actually suspect they may leave us as a client.

I guess the confusing came from the understanding of the word ‘Wave’. I now understand that ‘Wave’ is an end result.

The other paragraph:

‘To start with, in a market of now 100 vendors, it’s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog (free after registration). We used this information to filter out which vendors would be appropriate for the Wave report. I made a few public calls on my blog, twitter, indicating my intent as a result over 50 vendors submitted.’

lead me to assume that vendors needed to actively submit to the Vendor Product Catalog before they get included.

I am not an analyst and have only been following your blog for a short while. I will have to get used to some of the terminology and read the blog posts you reference in each post to fully understand the article in question.

Glad that’s a bit more clear. Yes you are right, submitting to the vendor product catalog was indeed a key requirement to be included to the wave. We needed that information to determine who would be filtered.

Then, I gave a caveat in the post indicating that I leaned on client requests to analysts (there are a few of us here) to see if we missed anyone. If so, we contacted them if they did not submit to the wave.

I totally understand. I only saw it from the outside and it seemed like A TON of work. I can’t imagine what went into it from your end. Respect.

My point is, is a large, one off report like this worth the effort vs what expanding on you’re doing here? Is there a different format (respecting a similar rigor) that can serve your clients better and is more scalable to have fast the market moves?

Don’t answer until you’ve had your vacation.

Re: updates, we announced App Studio on Tuesday. Integrating 3rd part apps to our application building software. http://tinyurl.com/9obyut

As one of the vendors (LiveWorld) in the top 9,of course we were happy to be included. I can also say that all 9 of the vendors took major issue with the wave elements they didn’t like. So this wasn’t some special club or easy ride even for the included vendors. Jeremiah held strong and didn’t change anything. We rated tops in services, but just strong in applications. This forced us to come to grips with his negative as well as his positive comments, whether we agreed with them or not. Enabling us to then advance in both services and applications. The moral of the Wave is, whether or included or not a vendor chooses to whether to learn or just to gripe.

There is one thing I don´t quite understand though, and maybe one of you guys could help me to clarify this – what capabilities define a community platform? For me I have a bit of a hard time understanding the boundaries of such a product, and I would appreciate if somebody could enlighten me on this topic.

To clarify my quesstion – what makes for example LiveWorld or Jive a Community Platform, and lets say Microsoft Sharepoint or some similar product that has forums included – not?

Hi
Not sure if I understand still, sorry, I still don´t understand the difference between products like Alfresco or MS SharePoint, that have social tools as part of the suite so to say, and the products described here as community platforms.

Let me clarify with an example:
A company is looking to expand their web portfolio with an online forum to get a community going. When thinking about a technical product to support this there are two ways to go – either leverage the current web site content management platform (assuming that the platform has forums, wikis and the other social media tools), or use one of the products that you define as community platforms.

So. The question is – what is the difference? Why is Alfresco, or Sharepoint etc, not a community platform, what are they missing in terms of functionality/capabilities compared to the products defined as community platforms?

Ok, I think I understand better. This wave is for interactive marketers, so we found out which vendors were creating communities for customer-facing communities –not employee-facing communities. I expressed that in the 5th paragraph above.

I did understand that we are talking about customer facing communities, as my example aimed at describing such a scenario, but obviously it was not clear enough.

In short – if you select a product for setting up a customer forum – why can´t you use Alfresco, SharePoint or whatever major CMS suite out there (the ones that have forums, blogs and wikis in their offering)? What capabilities do the products in your report have that makes them into a “community platform”?

What can, for example, Jive or LiveWorld do for me that, for example, SharePoint can not? What are the differences between a “true” community platform and a CMS that has forums, blogs and wikis included?

Let me know if this is getting tedious for you, was intented to be just a short question from the beginning. I am just trying to understand.

Kind regards
/james

Lorenz

Hi Jeremy,

I don't quite understand how you can give a balanced report if you exclude your own clients (I do understand that you are a stakeholder, but you are supposed to evaluate the BEST vendors, and you show a bias against your own). Earlier in the article you also mention that in a market of now 100 vendors, it’s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog. Does that mean you are biased towards those people that don't hear your call, don't follow you or don't find your methodology the most ideal as it takes time away from their clients? It would appear that in essence it does.

In the end, in order to evaluate a product, you analyze the most prevalent needs of the market space, segment them into categories of needs, and test products in the marketplace in relation to those needs, then rank them per category. As long as you are transparent which clients are on your book, I have no problem with them being included in a study (in fact, they are right to be angry because they suffer an obvious disadvantage from the relationship).