Survey Results

Posted by Paul Withers on March 31, 2016

Thank you to both contributors and consumers
for your feedback to the recent survey. It's been a busy couple of weeks
with OpenNTF involved in Engage and the Business Partner Bluemix workshop
from the last two days. But the "early birds" at Engage last
week will have seen some high level highlights of the survey results. Here
they are in a bit more depth:

Consumer SurveyThere were 79 responses, which is in
line with surveys a couple of years ago about Collaboration Today. That
figure was the benchmark for me when setting up the survey.

1. Approximately how many OpenNTF
projects (full projects, XSnippets, controls, tools) have you used in production?Over 80% were in the middle range of
1 - 10. 11% used more than 10. Maybe there is some education we need to
do, because when I consider things like OpenNTF Domino API, OpenLog, XPages
OpenLog Logger, Extension Library, XPages Debug Toolbar, various XSnippets
like Fredrik Norling's Standby Dialog Custom Control, Phase Listeners,
DateConverters, not forgetting tools like Swiper and others, I think I'm
way past 10. There is certainly more that can be done to maximise use.

2. Do you agree OpenNTF projects
provide good code?Over 90% agreed strongly or somewhat,
which is encouraging.

3. Do you agree OpenNTF projects
provide useful techniques to support training?78% agreed with this. Certainly the
projects that I've developed and the session databases I've created aim
to train and provide learning techniques. After all, I'm usually developing
them to learn something myself and, if I'm going to spend time on them,
it makes sense to try to increase my knowledge at the same time. But maybe
there are things we can do to share that more (both from the contributors
and from consumers).

4. Do you agree OpenNTF projects
are easy to install?About two thirds agreed. Yes there is
work to do here and I've heard comments from one consumer recently who
had some problems with a project but will be putting together things he
learned to feed back to improve the project. I think that's an encouraging
and important point, that OpenNTF is not a shop for contributors to put
code and consumer to use it. It's a community, and if a consumer hits problems
with my project , I would want them to feed back "feature requests"
for improving the documentation as much as feature requests on the project.
They are projects we as a community own, not just donated code. Consumers
can contribute to projects in many ways, one of which is proposing improvements
on documentation or functionality to improve deployment experience, and
I would hope contributors would welcome that.

5. When installing OpenNTF software,
are you confident it will work on your server?Nearly 80% agreed. This question was
prompted by my own experience of struggling to install some projects. The
figure of 80% is encouraging, but does mean there is still room for improvement.
But the improvements can be made by OpenNTF as well as by contributors.

6. Do you agree defects on OpenNTF
projects are dealt with in a manner appropriate for open source software?About two thirds agreed, but there are
many ways we can improve this and I would hope to see some during this
year. This doesn't mean we'll regularly be expecting much more from contributors
and expecting more significant levels of support. And we're conscious that
some defects may be because a project is no longer supported for whatever
reason, but there's no way for the consumer to see that. But there's no
visible way for consumers to seek help other than posting to the project
and waiting for a response. There's also no easy way for potential contributors
to get involved or for project chefs to easily request more contributors.
And only recently have projects embraced open source, allowing developers
to fork the code and submit pull requests, if they are confident
in doing so. There is some education to be done here, we know.

7. Do you think OpenNTF projects
would benefit from larger project teams?Nearly 85% agreed with this question.
We don't want to take kudos away from contributors, but neither do we want
them to feel tied to a project forever. OpenNTF is a community and open
source should be managed by the community.

8. Do you find it easy to locate
OpenNTF projects relevant for you?Less than 60% agreed and that doesn't
surprise us. Look for some initiatives around this and look for requests
for involvement from consumers as well as contributors. After all, if you
use a project, you know what it does.

9. Do you feel aware of new OpenNTF
projects?10. Which of the following would
help you be more aware of new projects?60% agreed. 50% wanted more blog posts,
21% felt weekly tweets would help, 63% wanted a newsletter and 48% monthly
webinars. I only recently became aware that time was being donated to OpenNTF
to do some of these areas. The interest in a newsletter was a little surprising,
but after some digging into some administrative systems, it appears this
used to be done. Some work needs to be done to allow people to opt in /
out of newsletters, but it's something we will look at doing. I will also
re-iterate that if anyone wants to blog about their project, we're more
than happy to post guest blogs, so please get in touch with any of the
board on various social media channels or on the Slack chat.

CommentsThere was some useful feedback around
usability of the site. There is definitely work required based on a completely
different scope than the website redevelopment a couple of years ago. There
are some difficult decisions going forward and I don't think we'll be able
to have a single layout to cover older projects and yet also best support
brand new ones. There may be the opportunity for new contributors to get
involved, so stay tuned.There was a comment about fixing Team
MailBox. We'll take that on board and there's work to be done, but it won't
be forgotten though there will not be any immediate action.

1. When was a release last uploaded
for a project you work on?Just over a third have had a release
uploaded in the last year. That's a little disappointing, but is not surprising
based on what we're seeing on releases. It also fits in with many projects
being single developer projects, which means more effort required per capita
per release.

2. Do you agree your projects are
adequately tested?This is admittedly a subjective question,
but 78% agreed.

3. Do you agree your projects are
adequately documented?Two thirds agreed. Yes, we're developers:
what we do well is develop, documentation is not our best competence. So
in a single developer project, it can vary.

4. Are you involved in a multi-developer
project? This includes a project that uses a pre-existing OpenNTF project
or is used within another OpenNTF project.With the second sentence in the question,
I'm very much thinking of things like the Extension Library, OpenNTF, XPages
Debug Toolbar (presumably included in at least one project). Only 25% agreed.
This is an area we can help to do something about and avoid single point
of failure on projects, support projects where the developer moves to other
technologies, and turn consumers into contributors without the huge pressure
of having to come up with their own idea or feel the pressure of a level
of quality they may not yet feel totally comfortable with.

5. Do you feel you need help on a
project you're involve in (e.g. additional developers, testers, for better
documentation, someone to take over projects because you are no longer
working in that area)?45% agreed, 55% disagreed. This may
not be appropriate for all projects, but may help turn consumers into contributors,
help with mentoring etc. We're not saying single developer projects will
be banned, but the numbers of contributions and (regular) contributors
is not significant - as shown by question 1 - and to have a flourishing
open source community, we need to do all we can to increase contribution
and consumption.

6. Do you feel you respond in a timely
manner to defects and feature requests?Just over 70% agreed. It takes a lot
of honesty to admit if you don't and we're conscious there is more we can
do to support those contributors.

7. Have you received reviews or feedback
of any of your projects?About 50% have received feedback on
OpenNTF, about 60% outside. There is more we can do around to increase
that.

8. Would you feel confident about
large organisations using your project(s)?80% agreed, which is good to hear. Hopefully
larger organisations are becoming more open to the use of open source.
It's almost certain that they do in some form.

9. Do you use source control for
any of your projects?57% do, another 19% plan on doing so.
Admittedly, if you've got a traditional Notes Client solution, this is
difficult. This may be new for some developers, but there are some great
NotesIn9 episodes covering it and Cameron Gregor's Swiper tool (also on
OpenNTF) really helps.

10. Would integrated build management
be of benefit to your development process (processing of test suites, automated
build and releases)?40% said yes, 22 % didn't know enough
about it. Again, traditional Notes Client solutions may not benefit from
this. There's also some education we need to do on the benefits and the
infrastructure for it is very new, but it's something we would like to
offer. Look out for more about this in the future.

CommentsOne of the comments about working in
groups was admitting a lack of understanding of the tools like Git, Maven,
JUnit etc. We'll take that on board for webinars. We're conscious there's
a lot that's new to Domino developers and one of the roles of OpenNTF is
to educate on tools some of us are starting to consider better practice.Another comment was about increasing
feedback rather than just consumption, something we also feel is important.

Next StepsThe feedback we've received, combined
with our own thoughts, means there is a lot of work ahead. There are some
clear infrastructure and communication enhancements that the OpenNTF board
should make. We need to agree on those as a board and action. Like you
all, we also have day jobs, and our next board meeting is not for two weeks,
so do not expect immediate announcements.

Beyond that, there are project governance
enhancements that we need to discuss, draft and solicit feedback on. OpenNTF
is a community so agreement needs to come as a group. We don't want to
drive developers away from OpenNTF. We want to increase contribution and
consumption, while promoting and valuing what OpenNTF can offer over and
above open source repositories like GitHub and BitBucket, both to contributors
and to consumers. Details will come out in due course (weeks not days),
but if you have any concerns, please reach out to us on the Slack channels
or via other social media.