Thank you to everyone who responded! I have placed them all below, including
my own:
================================
In a previous life I had initiated implementation of Usability Maturity
Model framework for refining the usability related processes. The
assumption was that with an overall mature (and continuously maturing)
process framework the usability organization would also be mature. The
framework relied on collecting metrics and refining the process and
practices so that the organization continually matured. From what I
remember the metrics that we collected fell in the following categories:
1. Problem recognition
2. Performed processes
3. Quality in use awareness
4. User focus
5. User involvement
6. HF technology
7. HF skills
8. Integration
9. Improvement
10. Iteration
11. Human-centered leadership
12. Organizational Human-centeredness
Though this may be more than what you need currently. The framework is
still interesting. If you are interested then I will be happy to share
more details.
Vikas
=======================================================================
A quick search on google for "usability metrics" returned the following:
http://www.useit.com/alertbox/20010121.htmlhttp://www.useit.com/alertbox/readingmetrics.htmlhttp://www.webword.com/moving/metrics.htmlhttp://www.tau-web.de/hci/space/i8.htmlhttp://www.usabilityfirst.com/auto-evaluation/
Lots of stuff there on how to measure metrics, who uses them and some of the
common metrics used.
Scott Berkun
Design & Usability Training Manager
Microsoft Corporation
===================================================================
I was curious...what is your Design Patterns service about? Sounds
interesting.
Jodi
*************************************
Jodi Bollaert
Usability Specialist/Information Architect
Compuware Professional Services
1-800-292-7432, ext. 10370
[log in to unmask]
=====================================================================
I would be very interested to hear a summary of replies to this post.
Thanks,
Julie Hillan
Web Content Developer
Marshfield Clinic
[log in to unmask]
(715) 389-4974
===========================================================================
It is funny, just yesterday I started working on the same thing for my job -
metrics for Human Factors. What we are measuring is the effectiveness of
our service, as well as looking for where upper management can help - the
metrics will show whether there is a problem of not enough people to perform
the tasks or if we are not getting what we need to complete a certain task
from other engineering groups. What I did is take the major items (similar
to what you listed below) and broke each down into discrete tasks. For
example, if I have a test plan to write prior to a test, I need to gather
source data, get equipment/system drawings from engineering, review the
information, write a draft, hold a peer review, incorporate comments, and
submit the report. I have then taken those tasks and measured them
according to planned start date, actual start date, planned completion data,
actual completion date. On a line or bar graph, you will be able to see
where things drop off or where you are not on target. If it is during the
"gather source data," this will permit management to see that either you
don't have enough time to do that and you need more bodies, or the people
you need to meet with are not available to you, thus management needs to
step in.
Overall, it will also permit you to see how successful you are at
maintaining your planned schedule or not.
I hope this helps - good luck!
Melissa Wills
Human Factors Engineer
General Dynamics C4 Systems
=================================
Here are a couple of suggestions.
1) For those activities in which you review/test products, you probably
(when appropriate) provide recommended solutions for your findings. We've
found tracking the adoption rate of those recommendations to be valuable.
It's not a direct measure of the effectiveness of your group role, but it
does speak to it's impact.
Not knowing your PLM's process this next one may or may not make sense.
2) Attempt to makes some measure of the efficiency, effectiveness of and/or
satisfaction with the end product your testing. I understand that the
Usability Group's efforts are one (sometimes small) piece of that puzzle.
But, if we can't draw some line between our efforts and an improvement in
those three measures then were at a huge disadvantage in the larger
development process.
good luck!
josh
Tape-Free and Scan Converter-Free Usability Labs
http://www.ovostudios.com
Josh Paluch
Usability Specialist
phone. 815.235.0080
mobile 815.670.5887
IM. jlpaluch @ Yahoo, MSN, & AIM
=======================================
The metrics I have drafted so far are:
=Number of products receiving usability support
#supported / #products = % of usability support
=UI Design support of products
User/Task Analysis
Guidelines and Design Pattern training
Conceptual Design
=Assessment support of products
Usability Testing
Heuristic Review
Defect and Enhancement creation and tracking
=Retention of key employees
=Survey Product Team members on our service delivery
Delight 'DeeDee' DeMulling Usability Manager
phone: 651.482.2705 fax: 651.482.4348
[log in to unmask]
EDS PLM Solutions
Collaborative PKM
4233 Lexington Avenue North Suite 3290
Arden Hills MN 55126-6198
--------------------------------------------------------------
Tip of the Day: Use the archives to research common questions
About CHI-WEB: http://www.sigchi.org/web/
Vacations, holidays and other subscription changes:
http://www.sigchi.org/web/faq.html#vacations
--------------------------------------------------------------