OpenHallway Blog

We’ve had a great response to OpenHallway since our launch three weeks ago!  Over the last few weeks, it’s been interesting to interact with with users and to find out that the concept of Hallway Usability Testing is not really well-known in the web development community.  So, I wanted to put together just a couple of references on Hallway testing and where we stumbled upon it.  We’re big fans of using Ruby on Rails and we stick to an agile development philosophy.  Being big believers in usability testing and fancying ourselves decent UX folks, we were surprised to see that there wasn’t much buzz at all in the Rails community for usability testing in general.  Just from a few discussions with some major players in the Rails world, we seriously wondered how many shops like ours were performing usability tests.  Before we built OpenHallway, we set out to do a bit of research on how we could fit usability testing into our own agile development process.  When we read about Hallway Testing, we loved it because it fit into our flow, and it came naturally to how we work.  Hallway testing is about testing the code you just wrote by “grabbing someone from the hall” and sitting them down to try out what you just built.

The Wikipedia entry for usability testing mentions Hallway Testing this way, “Hallway testing (or hallway usability testing) is a specific methodology of software usability testing. Rather than using an in-house, trained group of testers, just five to six random people, indicative of a cross-section of end users, are brought in to test the software (be it an application, web site, etc.); the name of the technique refers to the fact that the testers should be random people who pass by in the hallway. The theory, as adopted from Jakob Nielsen’s research, is that 95% of usability problems can be discovered using this technique.”  Since “bringing in” testers into our office was a hurdle for us, we decided to make a web application that would enable us to do usability testing remotely. We didn’t set out to be consultants or to claim we are usability experts, we just wanted a tool to give us quick usability feedback.

Joel Spolsky’s blog “Joel on Software” mentions it this way, “A hallway usability test is where you grab the next person that passes by in the hallway and force them to try to use the code you just wrote. If you do this to five people, you will learn 95% of what there is to learn about usability problems in your code.”

So there you go, a short primer on Hallway Usability Testing and how we got started.  It works for us, we hope it works for you too.

Last night we posted an update to OpenHallway. We made some UI changes based on some usability testing feedback we received (more to come). We also posted a quick tour video on the home page.  We’ve received some great feedback from our users and we look forward to getting it implemented.  Thanks to our new customer base for all the input.

Reid Hoffman, the founder of LinkedIn, once said, “If you’re not embarassed by the first version of your product, you’ve launched too late.”  (A good blog post on the quote)  I think we’d be lying if we didn’t say we had some small anxieties last night at 3am as we made the deployment and switched to production.  The good news is that we felt really confident about the app itself and the recording technology.  We’re confident that our customers can use the app for what it’s intended – recording and watching remote usability tests!

We’re getting close.  We’re planning to launch within the next few days, and we wanted to post up a couple of screenshots.  We’re looking forward to doing more usability testing on our own usability testing application! Thanks to my brother-in-laws and friends on Skype for putting up with me in the last few days. Moral of the story, good idea to pay those who test for you, or at least take them out to dinner.

- The test creator can create a new scenario for a project.  After this step, they get a link to send to people.

New Scenario - Test Creator

- When the tester receives the link from the test creator, here’s what they see…three simple steps to get them started.  Here’s step 1.  (We’re going to make Email optional)

Step 1 - Test Taker

- Some important info for testers to read in Step 2.

Step 2 - Test Taker

- In Step 3, the tester reads your instructions and can launch the test.

Step 3 - Test Taker

There you go…the sneak peak.  Time to court the mavens.  We’ll have some changes to this I’m sure, but hopefully this helps you see where we’re going with it.