When conducting and analysing qualitative user research it is essential to be able to explore the research topic and identify relevant findings. Often we are in an environment where we are working to a tight time span, and don’t have the luxury to review research sessions numerous times, watching and analysing video recordings. In these scenarios I have found it essential to code and theme my research. This helps to identify trends in research and also makes your notes much easier to interpret, which is particularly useful when facilitating the sessions on your own. This article will explore these coding schemes and provide insight into how you might generate appropriate themes.
When referring to qualitative user research, I am primarily referring to user testing - one to one user testing sessions following a semi structured interview methodology.
The coding scheme used evolves over time and is tailored to each individual project, dependant on th goal of the research. Following a basic structure for the majority of research, coding may include the following: identifying and numbering tasks, classifying observed behaviours, participant quotes, non verbal language, experience issues etc. Example notation/output of a coding schema:
 - Struggling to find filters, takes time to find result [O] [I] [3:45]
This is made up of the following:
 - Number of the task the user was trying to complete (as per the interview script)
Struggling to find filters… - Description of user behaviour
[O] - Indicates this was an observed behaviour
[I] - Indicates this is an issue that is impacting task completion time/rate. [3:45] - The estimated time stamp of the behaviour
By applying this simple code to your research notes, it is now much easier to review and categorise your findings, rewatching the recording where appropriate.
Identifying themes throughout the research process is essential to the research achieving its goals. A theme can be described as an idea you want to validate or a repeating idea that emerges throughout the research process. Themes should help answer pertinent questions relating to the goal of the research. They can be formed before, during or after the research sessions. Typically you will start with an understanding of what you want to find out, from this you can identify some high level themes. Throughout the research process you would iterate, modify and discover further themes.
Here are some common examples of themes:
Identifying high level themes is relatively straight forward if the research you are performing has clear goals. Prior to performing the research you may want to explore a certain aspect of the project, for example: participants’ appetite for an online community. Any observations and discussions around this point can then be coded as part of the ‘Community’ theme. Other examples of high level themes might include the navigation or structure of the project.
Themes can also form during the testing process. For example, it may become evident that certain terminology is confusing to the participants. Creating a theme around these issues can assist in identifying these problems and lay the foundations to explore more suitable terminology.
When reviewing your findings you may identify further themes that were not evident when conducting the research. This is where the coding scheme developed earlier can add further value. Reviewing the codes and comments might uncover that there is a common response to a specific question, which didn’t match the observed behaviour. A theme could be created to group these together and this would potentially warrant further investigation.
Below is an example of a coding system I used for a recent user research project. It is essential that the coding system is clear to you and quick to apply during the note taking process.
# - Question number - refer to script
“Quote” - User quote
T1 - Theme 1 - refer to script
T2 - Theme 2 - refer to script
T3 - Theme 3 - refer to script
[O] - Observation
[U] - Usability issue
[Term] - Terminology
[I] - Idea
And here is a page of my research notes with the associated codes applied:
The next time you are conducting user research consider creating your own coding scheme and associated themes. Here are just some of the potential benefits: Memory - writing (particularly with pen and paper) and coding observations helps to embed the findings in your memory. By actively creating associations between notes we are creating extra memory cues. Interpretation - it provides you with structured notes that are much easier to interpret during analysis of the research. Time - saves time as you can minimise time spent re-watching the recordings of the research sessions. Findings - Provides the structure for presentation of the research findings. Additional research - Helps identify areas for further investigation.
NOTE: This article was originally published on January 18th 2012, but was lost during the site redesign. Re-publishing as the information should still be relevant.
WAI-ARIA is a technical specification published by W3C that specifies how to increase the accessibility of web pages, particularly those of a dynamic nature. It stands for Web Accessibility Initiative - Accessibility Rich Internet Applications
NOTE: There is some discussion about how HTML5 and WAI-ARIA sit together. This is a subject that I have not yet fully grasped. My current understanding is that they can and should be used together where appropriate.
People who will potentially find understanding and using WAI-ARIA useful include: content creators, web editors and front end developers. When referring to the different users who implement WAI-ARIA we will use the term developers from now on.
It provides a semantic structure for the different elements of the page; Enhances accessibility support particularly for dynamic content; Better support for keyboard accessibility, it allows elements that otherwise could not receive focus receive it. Assistive technologies (will be referred to as AT in future), such as screen readers along with most modern web browsers can read the information provided by WAI-ARIA and present it to the user.
WAI-ARIA is made up of Landmark roles, roles, states and properties. Roles define different areas of the page. States and properties provide specific information about objects/elements on the page. There are subtle differences between the two but W3C refers to both as attributes, so we will follow their lead.
WAI-ARIA provides a list of landmark roles that can be used. You can add a landmark role to your document by including: role=”banner” into your element. A list of landmark roles is provided below:
These are pretty self explanatory and it’s outside the scope of this article to go into further detail. Generally landmark roles would contain more than one element.
As well as defining landmark roles for the document, roles can also be used to define the function of individual elements. Within the context of e-learning most of the roles will be used at some time or other. Below are details on the most commonly used roles. A full list of roles is available.
I have tried to categorise the attributes into a list of groups and go in to a little more detail of how they can be used. There are many others not mentioned here, these are just a selection.
Attributes add details to elements, such as properties and relationships. So you can declare relationships between elements and also set their properties. For example you can set a text input to be related to a button and set it as a required element. Take a look at the further reading for supplementary information about attributes.
WAI-ARIA extends the tab index property. By allowing focus to be set on elements that traditionally could not receive focus. Setting the property tabindex=”0” on an element puts it into the tab order of the document. Setting a value of -1 removes it from the tab order but still allows access through scripted means.
You can set form elements to be optional or mandatory using the “aria-required” attribute. Another common issue with forms is relating the labels to their inputs, aria-labelledby and aria-describedby can be used to create these relationships.
HTML5 also provides a model for drag and drop interactions. This could be used in tandem with the WAI-ARIA properties. For example HTML5 allows you to set a property of an element to be draggable which would work nicely with the aria-grabbed property.
WAI-ARIA gives you the ability to identify regions of the page that change dynamically, this is done by adding the property “aria-live” to the desired element. There are various values for the property. These are: off, polite, assertive, or rude. These values control the characteristics of how the user is alerted of the change:
WAI-ARIA is only a small part of web accessibility and the article above describes only parts of it. I have included many useful links below which should hopefully fill in the areas I haven’t covered. As browsers and assistive technologies become better at considering accessibility it is important that we at least try to use the tools they provide us with.
Juicy studio accessibility toolbar
WAI-ARIA drag and drop
HTML5 drag and drop model
WAI-ARIA and HTML5 confusion
Intro to WAI-ARIA
WAI-ARIA List of roles
WAI-ARIA Supported states and properties
A brief intro to WAI-ARIA