Why it's important to assess the UX of your app
When you improve the design of your app's UX:
- It's easier for people to use your app.
- More people will find your app to be valuable.
- More people will like your app and the features it offers.
- More people will download and use your app.
- You will make more revenue from your app.
Assessing your app's design instills confidence that the product you're shipping is the product that you set out to ship, that the user experience is outstanding, and that users will find it useful, usable, and desirable.
Measuring your design's potential success
Start by defining the goals for your app. Goals help you streamline the app-creation process, and they can help you assess your app's success. For more information about setting goals, see Planning Windows Store apps.
There are several tools that you can use to evaluate the success of your app. How many of these tools you use depends on the amount of time and resources that you can dedicate to designing your app's UX. There are three stages of evaluation that you can use, depending on your time and resources.
- Minimal time and resources: Cognitive walkthrough
- More time and resources: User study
- After you ship: Iteration, telemetry, user ratings, and comments.
Stage 1: Self evaluation
| Overview | This is the first step in app UX evaluation and is based on the goals that you set previously. The intention of this evaluative method is to ensure that your design is on track with what you intended to do. This step focuses on the overall UX of your app. |
|---|---|
| Time | 15-30 minutes. The time is app-specific and depends on the number of key scenarios in your app. |
| When | You can use this evaluation during the conceptual phase of your app design, and you can use it at any point during development when you want to check progress against your original plans. |
| Who | This evaluation involves one or more of the app designers or developers. |
| How |
Document your experience ratings as follows:
|
App Evaluation Template: Self Evaluation
The following table is an app evaluation template that contains results from an example self evaluation.
| Success metric | Goals | Status (Date) | Comments | Issue | What's needed to reach goals? |
|---|---|---|---|---|---|
| Great At: What is your app great at? What should be the focal point of the visuals? | My app is great at providing people with a fun and entertaining experience where they can compete with friends at spelling words. | On track | n/a | n/a | |
| Usable: What should users be able to understand, know, or do more successfully because of your app? | People should be able to navigate the game, enter words, and submit them. | Concerns with plans | People are not able to navigate between friends when spelling. | Need to rethink UI layout. | |
| Useful: What about your app do you want your customers to value? What words would you like to hear and not hear? | People should value this app as fun and entertaining. | On track | |||
| Desirable: What are the parts of your app that you engineered to delight users or make them love it? | We expect to hear these terms when people describe our app: useful, welcoming, connects me to friends and family. | On track |
Stage 2: Cognitive Walkthrough
| Overview | A cognitive walkthrough is an evaluation method in which people perform defined tasks within your app and provide feedback as they do so. This method has a distinct advantage over a self evaluation, because the feedback that's provided is from actual users of your app. The most important aspect of this method is that users will talk aloud as they complete the key tasks that you've defined. |
|---|---|
| Time | ~30-60 minutes. The time is app-specific and depends on the number of key scenarios in your app. |
| When | You can use this evaluation during the conceptual phase of your app design, using wireframes, and you can use it at any point during development, when you want to check progress against your original plans. |
| Who | This evaluation involves one or more users of your app, who have been identified as your target audience.
Define your target audience by asking the questions like:
|
| How |
Document your experience ratings as follows:
|
App Evaluation Template: Cognitive Walkthrough
The following table is an app evaluation template that contains example results from a cognitive walkthrough.
| Success metric | Goals | Status (Date) | Comments | Issue | What's needed to reach goals? |
|---|---|---|---|---|---|
| Great At: What is your app great at? What should be the focal point of the visuals? | My app is great at providing people with a fun and entertaining experience in which they're able to compete with friends at spelling words. | On track | The 2 users evaluated were clearly able to see the intentions of the app. | n/a | n/a |
| Usable: What should users be able to understand, know, or do more successfully because of your app? | People should be able to navigate the game, enter words, and submit them. | Concerns with plans | Both users tested had issues with navigation and entering words. | Need to rethink UI layout. | |
| Useful: What about your app do you want your customers to value? What words would you like to hear and not hear? | People should value this app as fun and entertaining. | On track | Both had fun with the app. | ||
| Desirable: What are the parts of your app that you engineered to delight users or make them love it? | We expect to hear these terms when people describe our app: useful, welcoming, connects me to friends and family | On track | Both had words that directly matched our goals. |
Assessing desire and emotional connection
Windows 8 was designed by using a toolkit to understand the responses of research participants to experiences by using semantic differentials, or opposite words, like "clear" and "unclear". The toolkit rolls up all ratings for a particular study and enables comparison across multiple studies. This tool enables understanding the strength of feelings in response to experiences. Respondents are interviewed to understand more about what parts of the product, combined with their personal perspectives, contributed to the forming of these responses.
Here's a list of potential words that research participants can use to describe their experience. Pick three terms that you want people to say, and use these as a baseline for terms that you expect to hear when people describe their experience of your app.
- I'm excited to use this
- I'm confident I can achieve all my goals with this
- I want to incorporate this in important or common activities
- This makes me feel more satisfied or happier
- I'm proud to use this
- Useful
- Functional
- Fast
- Essential
- Welcoming
- Compatible
- Connects me to content and people
- Connects me across devices and environments
- Works with my individual needs
- Reflects me or my interests
- Visually appealing
- Premium
- Cohesive
- Comfortable
- Clean
- Natural
- Engaging
- I'm not excited to use this
- I'm uncertain I can achieve all my goals with this
- I don't want to incorporate this in important or common activities
- This makes me feel unsatisfied or frustrated
- I'm not proud to use this
- Not Useful
- Broken
- Slow
- Not essential
- Unwelcoming
- Incompatible
- Disconnects me from content and people
- Doesn't connect me across devices and environments
- Works against my individual needs
- Doesn't reflect me or my interests
- Not visually appealing
- Not cohesive
- Intimidating
- Chaotic
- Unnatural
- Not engaging
Assessing On Brand
When measuring On Brand, Microsoft uses sets of opposite pairs that make up our brand attributes. For example, research has found that the brand attribute "Connected" means many things, and defining it with four sets of words enables respondents to elaborate on their perceptions. These four axes help us to understand to what degree a particular experience is On Brand:
- Ready to go vs. Time consuming
- Connected vs. Lonely
- Safe vs.Vulnerable
- Plugged in vs. Disconnected
Sometimes when measuring the success of an experience, you target a few specific attributes to score highly on. Other times you focus on an experience that ranks high in terms of desire and across the four brand values.
If you're considering methods for understanding how desirable or On Brand your app's experience is, focus on the quotes that you hope new and experienced users would emote during a conversation with a trusted friend. Prioritizing the most important items is important. The entire experience should elicit positive emotions, but sometimes putting more energy toward one part of the experience means that the emotional response in another area is reduced. It's important to know what to expect and to listen for the things that are most important to you.
UX dashboard
Consider creating in internal team dashboard for tracking your confidence model. The confidence dashboard provides a unified portal for you to report on status against the goals and it reflects your priorities for what you're resourcing. Ensure that every scenario you build has clearly articulated goals. Ensure that these goals are used as a decision-making tool by the scenario team. Track and communicate progress against these goals—express your level of confidence. Evolve your understanding of what makes a user experience "successful and outstanding".
Confidence in the scenario is based on whether you deliver a good scenario for customers in all of these areas. Some metrics may be higher priority in some scenarios. For example, some scenarios may not require understanding of a model.
Your confidence will be highest when standard usability metrics, like success, discovery, and task completion correspond with other data channels, like feedback from the field, instrumentation, surveys, and newsgroups.
Related topics
Build date: 3/12/2013
