?

Log in

No account? Create an account
Proposal: distributed GNOME usability testing - Arvind Narayanan's journal [entries|archive|friends|userinfo]

Proposal: distributed GNOME usability testing [Jun. 28th, 2004|12:18 pm]
Arvind Narayanan
[Tags|, ]


Came across the paper The usability of open source software a few days back. After kicking myself for not having known about its existence so far, I devoured it in one sitting. Most of what it said was familiar to me from other sources, but one idea, towards the end, sounded very exciting:

Fragmenting usability analysis and design

We can envisage various new kinds of lightweight usability participation, that can be contrasted with the more substantial experimental and analysis contributions outlined above for academic or commercial involvement. An end user can volunteer a description of their own, perhaps idiosyncratic, experiences with the software. A person with some experience of usability can submit their analysis. Furthermore, such a contributor could run a user study with a sample size of one, and then report it. It is often surprising how much usability information can be extracted from a small number of studies (Nielsen, 1993).

In the same way that OSS development work succeeds by fragmenting the development task into manageable sub-units, so usability testing can be fragmented by involving many people worldwide each doing a single user study and then combining the overall results for analysis. Coordinating many parallel small studies would require tailored software support but it opens up a new way of undertaking usability work that maps well onto the distributed nature of OSS development. Work on remote usability (Hartson et al., 1996; Scholtz, 2001) strongly suggests that the necessary distribution of work is feasible; further work is needed in coordinating and interpreting the results.

This suggestion resonated with me since I'd been thinking about the importance of usability testing only a few days back.

So why not start a distributed usability testing project for GNOME! Given sufficiently many eyes, all usability bugs are shallow.

Recently a prominent GNOME hacker (perhaps HP, I'm not sure) mentioned that the enthusiasm among maintainers to tackle usability bugs that was there in the run up to 2.0 has diminished somewhat. In those days Sun's involvement was a major contributing factor, but currently in the absence of corporate usability testing it's time for the community to help.

At this point I realized someone had already implemented such a thing - and who else but Pamela Jones of groklaw :)

Welcome to GrokDoc's usability study of GNU/Linux newbies. Our goal is to create a useful manual on basic tasks that new users will find simple and clear and easy to follow, using what we learn from our study. GrokDoc is an offshoot of Groklaw.

Our idea is this: instead of technically proficient people explaining tasks and functions to newbies, we let newbies show us what is hard for them. Proprietary software companies do such usability studies, and they benefit from the knowledge gained. The Free/Open Source community has all that we need to do the same, using the many eyeballs approach, so to speak. Open source ideals applied to research can be very powerful.

While writing manuals based on usability studies is an admirable goal, IMHO it is even better to fix the software so that it is easy to use even without the manual. Of course, there is no tension between the two goals because they are complementary. So GrokDoc would be a perfect place to host a GNOME usability testing project. GrocDoc is a wiki, an excellent medium for carrying out the work of "coordinating and interpreting the results" which the paper mentions.

More concretely, I envisage a process like this:
  • GNOME enthusiasts find victims to do usability tests on, and post the results to GrokDoc. Tests can be for GNOME as a whole or for specific apps.
  • The results are analysed to find common issues, and to decide which ones are genuine usability bugs in the program, which ones can be solved with better documentation, etc.
  • Usability bugs are submitted to GNOME bugzilla; new documentation gets written or existing documenation is improved and submitted to the appropriate maintainer.

I just created a wiki page for GNOME at GrokDoc. Everyone is invited! Jump in and post your ideas. I suggest doing one test of say 1 hour duration each month. I plan to administer my first one some time this week.

It would be great if there were a way of rewarding participants of usability studies. Reporting a bug takes only a couple of minutes, but doing a test takes an hour at least, so its a significant investment of time both for the tester and the testee. Since a "30% discount on all GNOME software" doesn't sound too appealing, we'd have to think of something else. One nice gesture would be for each app to include a section in its Credits dialog listing the names of people who participated in usability studies of that app. Is there anyone selling GNOME merchandise? Maybe if they are feeling generous they could offer a discount for usability testers? Any other ideas are welcome.
LinkReply

Comments:
From: (Anonymous)
2004-07-19 03:47 am (UTC)

Gnome

Hallo!

"It would be great if there were a way of rewarding participants of usability studies. Reporting a bug takes only a couple of minutes, but doing a test takes an hour at least, so its a significant investment of time both for the tester and the testee."

"One nice gesture would be for each app to include a section in its Credits dialog listing the names of people who participated in usability studies of that app."

A formal usability test for each application to be released.



"Any other ideas are welcome."

What counts in our world are not nice ideas on what can be done. There is no lack of ideas.It is more important to change the usability community from overexited "idea flamers" to real substantial contributors.

KDE is my DE of choice.


(Reply) (Thread)