LimeSurvey for Market Research, an invitation to participation

Poline replied the topic: LimeSurvey for Market Research, an invitation to participation

I'm new here and I was glad to find out about LimeSurvey for Market Research. I'm going to advise it to my boss. We make [url=nospam please]nospam please[/url] for the clients. I hope that it will make our work easier and more professional.

jelo replied the topic: LimeSurvey for Market Research, an invitation to participation

spacejanitor wrote: As such, there is a huge opportunity for LimeSurvey to become a very credible competitor to all of the licensed survey software out there, which takes in hundreds of millions of dollars per year in licensing alone.

What are the five most wanted/needed features from MR point of view?
Looping, Other Answer Text behind every item; No answer at the end of item list; Lists (predefined and dynamic constructed)..

How big is the difference to e.g. social science?
What amount of developer hours would be needed to implement them.
What amount of money should be raised to implement them.

Confirmit and Questback are moving away from MR directly to the customers of MR.
The growth rate in MR will be low if they don't lower there prices.
If MR is into Conjoint they need SawtoothSoftware. Alternatives are rare. Next year Sawtooth will only offer licenses on yearly base.

The biggest asset of limesurvey is easy selfhosting. Try this with Confirmit or Questback (Globalpark software is PHP based too and was years ago also offered for selfhosting. But with a few strings attached.).

Compliance may be a driver for LimeSurvey. If selfhosting and datacenter location is no issue, a service like SurveyGizmo is attacking Confirmit and Questback nicely.

Perhaps it is time to make a survey to ask everyone landing on Limesurvey.
I remember we had this once.

I second that Expression Manager was a big step forward.
When I first saw the EM I was flashed. But TMSWhite is no longer part of the team.

To me it is a bit of a wonder that Limesurvey is still evolving in that niche with such a quality. Would be interesting to know if any of the active developers ever used software like Confirmit, Questback/Globalpark or Sawtoothsoftware. And the secret to stay motivated in terms of coding.

holch replied the topic: LimeSurvey for Market Research, an invitation to participation

Hi Jelo,

I have seeen a few of your posts lately and I can feel a certain frustration with Limesurvey from your side.

While I can understand where it might come from, I would like to give you my point of view here.

I would love Limesurvey to compete with professional tools like Confirmit and Questback. But we need to face reality. From what I remember, our Globalpark license probably cost more per year than the yearly Limesurvey budget (good guess, I don't have any exact figures for both).

Globalpark / Questback and Confirmit play undoubtely in the premier league of questionnaire tools. I would be supprised if Limesurvey, a open source tool developed by a handfull of more or less fixed developers, none of which is doing this full time (from what I can see).

Of course we can compare the accomplishments of a project like Limesurvey with these companies that has various full time programmers on their staff is possible, as they are playing more or less in the same area, but it feels slightly "unfair". Saying that: I am at all suprised that Limesurvey is far from where these other tools are.

On the other hand, I remember that it took long until new features were in implemented in Globalpark as well. This was especially frustrating as we had/have quite a close relationship and we gave them a lot of free ideas and tips on how to improve their product. There were times where we had to program a "card sort" solution via Javascript to be able to implement it in Globalpark. This took a intern not very long, so imagine a full time programmer that is familiar with the tool and doesn't do anything else. Not sure if it is implemented yet (about 10 years later). So things are (or at least have been) not that easy with the commercial tools either.

And as I said, the tools you are mentioning are top notch solutions. There is a huge market with tools "in between" that are playing in a lower league and still survive and are used. This suprises me a lot more, as they are not sooo far away from Limesurvey.

Actually I am quite impressed how much you can do with Limesurvey, when you put things in relation.

Don't get me wrong: I feel that there are soo many things that can and should be improved in Limesurvey.

For example the GUI. While it might feel that new and different features are more important, I feel that Limesurvey could gain a lot more users in its own niche (I feel that Survey Monkey and Co are the real competitors to compare with). Limesurvey came a long way in terms of ease of installation and update. I feel this is working pretty well now. But GUI is important and while things work well, it is not the quickest and of course not very attractive compared to the new tendencies in GUI design. Right or wrong, people like to work with beautiful things and a nice, simple, modern and beautiful GUI will impress first time users and testers and motivate them to go forward. But this might be just my impression and I totally get that this would involve major work which might increase the gap in features to professional solutions even more.

But let me tell you something: A lot of questionnaires and surveys go over my desk per year, from the most diverse companies from Europe, the US and Asia. If I would have to guess, Limesurvey could cover 80% of the surveys that are applied out of the box. Another 10-15% will be possible with some kind of T-Partners workarounds in Javascript. The other 5% are either not possible or would need major work on the core. This is an estimate, but I don't think that I am too off.

When you say that there are almost no alternatives to tools like Sawtooth when it comes to Conjoint, I totally agree. But this gives me also the impression that there is not much market available, or it is a extremely difficult and costly to implement, otherwise another big one would have stepped in.

But there are myriads of different survey solutions out (low, mid and top range) there and they all seem to survive.

I am in no way involved or informed about the development of Limesurvey, but I would guess that there is so much small things to fix and keep up to date, that there is little time for "thinking big" and strategic thoughts. We also have to keep in mind what Limesurvey is and how it is organized. This is quite similar for many OS projects.

While they might benefit applying some techniques that are used in corporations, I don't think this is very easy to implement in OS projects that live of the work of voluntaries. As a project leader you can tell your team what to do, if you thing it is the right way. No mather what they think of it. It might not be the best style, but you can. Do that in an open source project and see after a few weeks how many active contributers are left....

That became quite long now. All in all, I would also love Limesurvey to progress quicker, with a clearer strategy. But to be honest, I am quite impressed and happy where Limesurvey is right now and how it has evolved since I have used it for the first time.

EM was a huge step forward, but we can't expect that to happen all the time. I don't know why TMSWhite left the team, but from what I understood it was because he felt that EM was what he wanted it to be at this point. And he comes back from time to time to the forum to respond...

What I feel is the most urgent future tendency: Make Limesurvey mobile ready (front end / surveys)! First step: a responsive template among the default templates shipped with Limesurvey.

Then for market research purposes a good reporting would be great. But to be honest, Globalpark's reporting was never very good either (haven't seen it for a while though - talking about 2008 and before).

jelo replied the topic: LimeSurvey for Market Research, an invitation to participation

holch wrote: I have seeen a few of your posts lately and I can feel a certain frustration with Limesurvey from your side.

Thanks for your extensive post.
This thread seems to be started to get MR companies/users/community on board and hopefully bringing support in terms of ideas and sponsorship. Spacejanitor might want to correct my if I got it wrong.

I cannot be frustrated with Limesurvey itself. I would have no right to do so.
It's an offer. I cannot look into the minds of the developers and project team. Not sure where their aims are in terms what Limesurvey should be in the next years.

Limesurvey is ahead of many products in the marketplace. I didn't wanted to mentioned bad examples. Not sure how this plays out in a public forum. The knowledge of user and the lockin costs are helping that crappy survey software is still widely used.
I think we will find many usercases where LS is the quickest tool to realize a survey.

As long as there is only a small subset of the userbase is joining this board and only a subset of them are interested beyond getting there problems solved I don't see a strong signal to what features are really wanted.
As a developer you shouldn't following the hardcoreusers or elephants in the forum, which demanding features, which nobody else is using in years. The problem might be that many users are only executing one survey per year

When looking into the feature requests in mantis I come to the conclusion that a main difference in the userbase is between "Reporting/Analytics inside Limesurvey" vs. "LimeSurvey only for datacollection".
I would expect that MR/Science is more focused on datacollection with limesurvey. The SMB/private segment is stronger in the demand for better reporting.

The offer of the fulltime position as a developer looks like a change.

holch replied the topic: LimeSurvey for Market Research, an invitation to participation

Hi Jelo

I think we are mostly on the same page.

The problem that I see with market research companies is that many are not interested in a open source tool, either because they are locked in, or because they don't want to "risk" it. This is not just a problem in the MR industry, but I think it is one of the main points.

It is so much "easier" and "saver" to go for the big player and pay US$ 2,5 for any response than to set up an open source tool. Imagine if something goes wrong during the survey? Who will they be able to blame? A small open source developer community? The clients will laugh in their face. If they go for Confirmit or Questback and something goes wrong, they can at least say: we went with the market leader. Who would expect that they screw this up? But I am sure you know what I mean. If I would have proposed my managing director back in the day to go for Limesurvey, he would have considered me crazy. Even if most of our surveys back than would have worked more than fine.

I agree about not just listening to those that are power users and dominate the forum. Like always: there are so many users not involved in this forum, that we would not get a very good overview.

However, how to get to those users that are not coming here? Maybe via the auto-update, asking the admin to participate. But then this might be just an IT guy who is responsible for the tool running and has actually no idea what it is used for, etc.

Research should be done, but I am not sure that a simple survey will do any good.

Problem is probably also that it is very difficult to have a look into the professional tools. First of all, they usually don't have any "trial" access and second it takes a lot of time to understand what really is good or bad in a tool when you really use it.

So we users need to give this feedback. But I would rather like to have a "qualitative phase" before. Some chats in a group of interested users from different areas and with important developers. To see what is possible and what not.

I just feel that there are not enough resources for a really strategic move forward like with EM. There was someone who had the concept basically ready and was a full time employee to implement that (he probably had some other tasks, but there was a company paying him to implement EM into Limesurvey, from what I understand).

Like in many OS projects, things only move forward, when there are certain parties with a special interest in something, that put resources on it.

I am not informed about anything about development, but what makes me think is how many universities use Limesurvey around the world and how little I see universities contributing. Wouldn't it be a easy thing to teach students basics of programming and doing little projects in Limesurvey. Not that this will give a huge push because I guess one experienced programmer will help a lot more than 20 students trying to learn to program, but I feel that they could support Limesurvey better. But maybe this is just my impression.

Mazi replied the topic: LimeSurvey for Market Research, an invitation to participation

holch wrote: I would love Limesurvey to compete with professional tools like Confirmit and Questback. But we need to face reality. From what I remember, our Globalpark license probably cost more per year than the yearly Limesurvey budget (good guess, I don't have any exact figures for both).

Globalpark / Questback and Confirmit play undoubtely in the premier league of questionnaire tools. I would be supprised if Limesurvey, a open source tool developed by a handfull of more or less fixed developers, none of which is doing this full time (from what I can see).

Limesurvey can always compete, at least when it comes to pricing

I don't really use other tools much because I know how to achieve nearly everything with LS. Thus, I am not aware of all features other tools offer but from what I can tel there are only some very few feature which are not be available at Limesurvey or can't be implemented easily using JavaScript.

I would even say that with the new Expression Manager features (scoring, complex conditions, ...) Limesurvey is capable of many stuff other tools can't deal with.

holch wrote: Actually I am quite impressed how much you can do with Limesurvey, when you put things in relation.

Don't get me wrong: I feel that there are soo many things that can and should be improved in Limesurvey.

For example the GUI. While it might feel that new and different features are more important, I feel that Limesurvey could gain a lot more users in its own niche (I feel that Survey Monkey and Co are the real competitors to compare with). Limesurvey came a long way in terms of ease of installation and update. I feel this is working pretty well now. But GUI is important and while things work well, it is not the quickest and of course not very attractive compared to the new tendencies in GUI design. Right or wrong, people like to work with beautiful things and a nice, simple, modern and beautiful GUI will impress first time users and testers and motivate them to go forward. But this might be just my impression and I totally get that this would involve major work which might increase the gap in features to professional solutions even more.

I totally agree. At every developer meeting dealing with new features I vote for working on the GUI first. while adjusting/improving a GUI sounds easy, it is really causing us problems for several reasons:

Developers like coding new features but they are not that into improving designs

We do not have a professional designer available who can contribute the required time to re-design all screens. -> Are there any volunteers!?

Remember that the whole admin interface consists of ~30-40 different screens. That takes weeks even for a full time developer!

Everyone has a different opinion of how a better design should look like. There have been some approaches in the past to collect example but we never got beyond that.

Improving the design isn't just about adjusting some lines of CSS to make things nicer. We are dreaming of having a new drag and drop survey designer (something like that was available at a beta version which was dropped) and doing a bigger refactoring of the whole admin GUI to make it simpler and more user friendly but that requires adjusting the underlying PHP core code which again takes weeks. Not to mention that whole work-flows would ned to be adjusted

holch wrote: But let me tell you something: A lot of questionnaires and surveys go over my desk per year, from the most diverse companies from Europe, the US and Asia. If I would have to guess, Limesurvey could cover 80% of the surveys that are applied out of the box. Another 10-15% will be possible with some kind of T-Partners workarounds in Javascript. The other 5% are either not possible or would need major work on the core. This is an estimate, but I don't think that I am too off.

I think these numbers are a pretty good guess and from my experience of >200 survey projects I can second this.

Often the biggest problem is not the lack of a feature but the stubbornness of the client ("But we have told our customer that we can have text + radio button + radios mixed at one question type!").

holch wrote: When you say that there are almost no alternatives to tools like Sawtooth when it comes to Conjoint, I totally agree. But this gives me also the impression that there is not much market available, or it is a extremely difficult and costly to implement, otherwise another big one would have stepped in.

I have used Limesurvey for conjoint studies successfully. With some additional coding we have been able to connect a DB with conjoint details (product comparison data) with Limesurvey and present the details at a Limesurvey survey. You can find a screenshot at
www.limesurvey-consulting.com/integration/

Data analysis was later done using R and special conjoint software.

holch wrote: I am in no way involved or informed about the development of Limesurvey,

There is a developer meeting every Tuesday afternoon European time, everyone who is interested is invited to join us

holch wrote: but I would guess that there is so much small things to fix and keep up to date, that there is little time for "thinking big" and strategic thoughts. We also have to keep in mind what Limesurvey is and how it is organized. This is quite similar for many OS projects.

While they might benefit applying some techniques that are used in corporations, I don't think this is very easy to implement in OS projects that live of the work of voluntaries. As a project leader you can tell your team what to do, if you thing it is the right way. No mather what they think of it. It might not be the best style, but you can. Do that in an open source project and see after a few weeks how many active contributers are left....

You are right. Open Source (OS) projects are organized differently and since every developer is spending his/her free time on this you can't organize this like at an IT company.
We have very limited resources and need to focus on the important things first. That is bug fixing and that already takes a lot of time. There have been > 700 fixes the last 365 days (see
bugs.limesurvey.org/summary_page.php
for more details).

We have a very long list of new features we want to develop but as said, our resources are limited. From my experience the last years this is something the community can help a lot. Either by contributing new stuff themselves (@those interested please check
manual.limesurvey.org/How_to_contribute_new_features
) or by sponsoring the development of new features.

Once we had a nice (but no longer maintained) tool for letting users add new features to a global wish list and let others vote for them. That had helped to get an overview of what the community is looking for when it comes to new features. Currently we are scanning the forums and keep an eye on IRC and bugracker to identify such needs but from my point of view that is far from perfect.
So if anyone had an idea of how to better connect community and development team, please share your thoughts.
As most long time users have already noticed we suck at communication

That became quite long now. All in all, I would also love Limesurvey to progress quicker, with a clearer strategy. But to be honest, I am quite impressed and happy where Limesurvey is right now and how it has evolved since I have used it for the first time.

holch wrote: EM was a huge step forward, but we can't expect that to happen all the time. I don't know why TMSWhite left the team, but from what I understood it was because he felt that EM was what he wanted it to be at this point. And he comes back from time to time to the forum to respond...

Tom is an Open Source enthusiast and thus took the time to implement EM at Limesurvey. He used similar features at a custom developed survey software but if I remember correctly it made sense to use a more powerful and flexible tool (=Limesurvey) for his surveys in the long run so he coded EM.
Luckily Tom hasn't left us completely. He shows up at the forums from time to time and helps answering development questions. But unfortunately he is not actively developing EM anymore which is a pity.

Tom had already spent > 1000 hours on this, just so you know what it takes a very experienced developer to code something that fantastic.

holch wrote: What I feel is the most urgent future tendency: Make Limesurvey mobile ready (front end / surveys)! First step: a responsive template among the default templates shipped with Limesurvey.

I have also seen some approaches for developing a bootstrap based template. Some of these look really nice but most do not yet support all question types.

I'd also love to see some kind of Limesurvey app especially for offline use but technically that is very hard to implement, especially when it comes to automatic data upload.

holch wrote: Then for market research purposes a good reporting would be great. But to be honest, Globalpark's reporting was never very good either (haven't seen it for a while though - talking about 2008 and before).

What reporting feature do you think is missing? The Limesurvey admin statistics support charts, complex filtering and export to different formats (BTW, I think we should improve the documentation, lots of users seem not to be aware of those capabilities).

From my point of view complex data analysis is nothing a survey tool implicitly has to deal with. Powerful tools like SPSS and R are made for this and Limesurvey support exporting data in these formats.

jelo replied the topic: LimeSurvey for Market Research, an invitation to participation

Wonder if Mark Zielinski (spacejanitor) is still using Limesurvey in his MR Business.
Seems to have lost interest in the discussion.
The company website mentioned here (
www.dataminingblog.com/guest-post-mark-zielinski/
) isn't working at the moment.

Mazi replied the topic: LimeSurvey for Market Research, an invitation to participation

jelo wrote: Wonder if Mark Zielinski (spacejanitor) is still using Limesurvey in his MR Business.
Seems to have lost interest in the discussion.
The company website mentioned here (
www.dataminingblog.com/guest-post-mark-zielinski/
) isn't working at the moment.

The vendors of the old, big suites are focusing on big companies. They more and more ignoring universities and market research firms. Market and/or budget them to be to small to be targeted.

The "newcomer" are SurveyGizmo and Qualtrics. Qualtrics is targeting universities heavily.
Just search for Limesurvey and Qualtrics and you will find examples, that it departments if universties are offering Qualtrics instead of LS.

The users won't recognize that hard work and the success of the OSS project "limesurvey".
When I talk to other survey creators (one or two surveys a year) they are seeing many features as a commodity, which LS is missing. E.g. when you restrict a numeric field to a range, that no hint of restriction is shown until the submit button is clicked. Or that error messages are customizable (not via PO-Files edit).

It still boils down to the question who collects the money and how is it distributed.

If I put a price tag under each of my feature request, what will happen?
I will offend developers because of low amount in comparison to the hours needed to implement the requested feature.

We could start with three categories 100 / 250 / 500 EUR . Features which are above 500 EUR would have to be split up into smaller requests.

Another source of income (which once seemed to be considered by LS) is the comfort update.
OSS project Contao is using that concept (
update.contao.org/en/
).
But the market for CMS is a lot bigger than for survey-tools.