Ronin Blog

The best way to improve an application’s usability is to run extensive studies - they allow you to get an idea of what it’s like to walk a mile in your users’ shoes. Sometimes, just the practice of sitting down with another human being and verbally walking through your application’s features brings out certain perspectives you may not have considered before. This is especially true if you wear many hats during the development cycle.

At Ronin, we run a usability study every few weeks, focusing on new features. We’re usually very considerate of our users throughout our application design process, but you’d be surprised how much slips through when you’ve looked at the source code of your application longer than you’ve looked at the actual interface. Running a usability study is an exercise that usually brings in a large list of low hanging fruit and a moderate list of long term application improvements.

That said, from our experience running these studies, there are a few things we realized you can never fully understand from just the studies themselves.

User patience

When a study participant sits down next to you to walk through your application, they’re in no hurry. They’ve committed the next 30 minutes to 1 hour of their lives to the purpose of completing the study. Compared to the average 9 second attention span you’re likely to get from a real world user, this is too distant from reality to simulate.

Unfortunately, there isn’t much you can do to recreate a frantic in-the-wild scenario. This negatively affects the ability to learn about edge-cases and error handling the most. A usability study user is likely to run into some errors that they are patient enough to figure out during a study, but would otherwise cause them to throw their hands up in disgust. To counteract this, its best to read between the lines when it comes to negative reactions. Multiply observed frustration during studies by many magnitudes to really understand how a user may react.

Also, try to develop an understanding of how user patience can be a big factor. Spend time testing your home, promo, and marketing pages with an emphasis on asking for a stream of consciousness verbal description of what they see with a time limit. After all, if they stick around long enough to try your application, you’ve won half the battle when it comes to user patience.

User attentiveness

Another problem is that study participants are usually more alert and attentive than their real-world counterparts. They’re willing to take their time and figure out a problem. They’re willing to devote their undivided attention to the task at hand. Compare this to the more likely scenario of an actual user trying out your application at 2 a.m. with the TV running in the background and after surfing to your promo page by coincidence. It’s night and day.

To counteract this, make sure you get immediate verbal feedback after each action they take, to get a sense for what they’re thinking before the interface has had time to soak in their consciousness.

User skepticism

Most deceiving of all is the mindset of a usability user. Depending on what your relationship is with the user (stranger, guest, friend, acquaintance, co-worker from another department, spouse, child) you need to realize they’re walking into the study with some intentions. Sometimes they’re participating because you’ve promised to pay them; other times they’re participating because they want to help you out. Whatever the case, nothing simulates user skepticism.

When a random user hits your site or application (say from Google), they’re coming with a guarded mindset. They’re cautious more than curious and they’ll immediately look for warning signs. A poorly designed interface with no obvious indications that there is active support or lack of no-obligation demos or screenshots are an immediate turn off. If they eventually try your application or dig deeper into your site, they’ll probably be less willing to try anything but the most obvious of features.

When a user hits your site or application from a referral or other more qualified method, they’re coming with a curious mindset. They want to dive right in and see what all the fuss is about. They’re also likely to be the ones who have heard of alternatives or are currently using an alternative and now they’re using your application to do some feature comparisons.

When a user participates in your user study, you get none of this. They’re neither overly guarded nor overly cautious. It is more likely that they feel you are working together in unison towards a common experimental cause. You really don’t get the stinging skepticism that a real world user carries.

It helps to keep these things in mind when conducting usability studies. Sometimes it’s too easy to pretend that you’ve aggregated all the feedback you can by running a well-designed series of studies. Even armed with that data, keep in mind that the best feedback is silent feedback. Make sure to analyze your logs. Don’t dismiss your analytics tools. Look over real user data from real users in the wild. Couple that data with usability studies and only then do you have a good macro- and micro- level understanding of your own application.

IE6 (also affectionately called Internet Exploder by some) has been a huge burden on the web as a whole for several years now. While the average web consumer might not notice it, the ubiquity of IE6 has forced web developers and designers to jump through many arbitrary hoops in the past to support the non-compliant browser.

This has meant that precious development and design resources over the last several years have been slaving away at making IE6 look good, instead of focusing on making web sites/apps/designs work better. This also has meant that one major strength of the web (portability) was hampered by the need to address a specific platform.

The good news is, several companies are beginning to drop support for IE6, in favor of more recent browsers, most recently Google. This is not surprising for Google as they have been pushing alternative browsers like Firefox and, more recently Chrome for a while now.

For web developers, this comes as great news - there is now a major web player moving the masses away from IE6, which hopefully accelerates the adoption of newer generation browsers.

For Ronin, we’ve never really supported IE6 to begin with and we’ve seen no reason to really bother with bending over backwards to begin supporting IE6 now that it’s hopefully on its way to the grave. Analysis of our traffic shows that only 16% of our traffic comes from Internet Explorer, and of that 16%, roughly 20% use IE6.

The chart on the left shows that the majority of our users prefer Firefox and the rest use Safari. Surprisingly, this doesn’t mean most of our users are on Macs - instead, Firefox/Windows is still the most popular combination.

This certainly has to do with our audience. We primarily attract high-tech professionals (whether its freelancers, design firms, small businesses) and with that tech-savvy crowd comes the preference for newer, more standards-compliant browsers. In fact, it would be a lie if I told you we didn’t have this fact in mind when we first started Ronin.

Here’s to a new year and leaving behind old worries!