Revision as of 09:51, 28 February 2008

(Work in progress)

The following defines the quality levels for OWASP TOOLS and DOCUMENTATION (Projects). Rating projects against these criteria aid in recognizing excellent contributions and identifying projects in need of further work.

WebGoat would not be appropriate for example since it would light up like a Christmas tree :-)

C/C++ apps (if we have any) should consider being run through Coverity's open source review. Coverity also accepts submissions for open source Java applications.

When approved to be Release Quality: Update the link to it on: the OWASP Project page and update its project quality tag on its project page to be Release Quality.

Recommendations:

Publicly accessible bug tracking system, ideally at the same place as the source code repository (e.g., at Google code, or Sourceforge)

Conference style Powerpoint presentation that describes the use and status of the tool. (This could be used by others to discuss the tool at OWASP Chapter meetings, serve as easy to review offline documentation, etc.)

UAT pass on functionality of the tool

Developer documents any limitations

Requirement: 2 Reviewers + 1 OWASP Board Member.

If possible, the project's lead should suggest two Project Reviewers. One of them should be an OWASP Project Leader.

If the project's lead can't find the Project Reviewers, the OWASP Board will identify them. The same will happen whenever the reviewers suggested do not have the required approval.

(which lists name of tool, author, e-mail address of author, current version number and/or release date)

Include documentation on how to build it from code, starting with getting it directly from the code repository. (Ideally, this would include easy to use build scripts, which is required for Release Quality)

When approved to be Beta Quality: Update the link to it on: the OWASP Project page and update its project quality tag on its project page to be Beta.

Requirement: 2 Reviewers.

If possible, the project's lead should suggest two Project Reviewers. One of them should be an OWASP Project Leader.

If the project's lead can't find the Project Reviewers, the OWASP Board will identify them. The same will happen whenever the reviewers suggested do not have the required approval.

FAQ

1. What is the purpose of the project ratings?

The rating system allows OWASP to monitor the quality of Projects in our subject areas, and to prioritize work on these projects. It is also utilized to prepare for static releases of Wikipedia content.

2. How do I add a project (tool or documentation) to the OWASP Projects?

Each category has a set of requirements/criteria to be met. Beta Quality implies that all of its requirements, as well as the Alpha Quality requirement have been met. Release Quality implies that all of the requirements, including Alpha and Beta, have been met.

Unfortunately, due to the volume of projects that need to be assessed, we are unable to leave detailed comments in most cases. If you have particular questions, you might ask the person who assessed the project; they will be happy to provide you with their rationale.

6. What if I don't agree with a rating?

You can list it in the section for assessment requests below, and someone will take a look at it. Alternatively, you can ask any member of the project to rate the project again.

7. Aren't the ratings subjective?

Yes, they are somewhat subjective, but it's the best system we've been able to devise. If you have a better idea, please don't hesitate to let us know!