Hartson, Rex, and Pardha Pyla. 2012. The UX Book. San Francisco, United States: Elsevier Science & Technology.
For a more detailed understanding, feel free to download the full survey report here.
Contextual Inquiry / Data Gathering
1. I gathered 15 different graphic representations normally attribute to the concept of progress and/or proficiency.
2. I asked the users how they perceive the respective graphic individually.
3. I gather all the graphic representation on a grid and ask which of them they thought to better represent the notion of progress and the notion of proficiency Goal Test for Distinction Bias.
Interesting Fact: Our users are all accountants. So, I found out with this study that they have an intrinsic need for every graphic representation to be followed by numbers, results or percentage description. They just love numbers :D
Contextual Analysis / Data Interpretation & Consolidation
The most voted graphic representation for Progress
20,69% of users voted with a significant expression for this representation as to the most recognizable for tracking progress.
The most voted graphic representation for Proficiency
30,43% of users voted with a significant expression for this representation as to the most recognizable for tracking proficiency.
Needs & Requirements / Extration
The two concepts are already complex to distinguish, so upfront, I exclude all the graphic representation that didn't have a significant result. I didn't want to fall in the mistake of creating an ambiguous representation. Accordantly, I immediately excluded the following representation:
Final Design Decision
Survey Study showed us that users associate proficiency more to starts representations and percentages and pie charts to the concept of progress. Accordantly, I followed the following graphic representation for the CPA exam review application. On the left for tracking proficiency and on the right for tracking progress.
05 Refine - My Approach & Survey Process
After finishing this first MVP, we conduct a Beta A/B Test with ~3 000 students. The AI algorithm needed to learn from individual users but also from the collective usage. To give time to test the algorithm, we launch the A/B test during three months and the end surveyed the students.
For a more detailed understanding, feel free to download the full outline survey here.
To begging analysis the exploratory research, I conduct aqualitative analysis to uncover themes in the data.
I start it by doing a thematic analysis to explore similarities and relationships between the different chunks of the survey data. The goal was gather more visual means of analysis to present to the stakeholders in the final session review of the design process.
Important Note: Visualization helps my team to understand better the similarities and relationships in the data more clearly to discloser better design and business decision.
Thematic Analysis Process
1. I like to print the results transcripts and start to highlighting important keywords and sections that are relevant to the study.
2. Next, I start cutting the paper texts sections that have been highlighted and start organizing them on an Affinity Diagram and making some side notes to them (descriptive labels).
Important Note: Ideally this process shouldn't be done alone. But I am a team of one and my stakeholders are on the other side of the ocean. I had to remedy the situation and conduct the analysis myself and be subject to my own bias.
3. I organize the data into blocks of themes based on their relationships, and start to figure out some categories. At the end I found 4 major themes:
User Interface (UI)
Note: This was a time consuming and difficult task because I had to analyze the feedback from 300 students. It was an interesting challenge to reduce to 4 categories.
4. Once I found my 4 categories, I start thinking about how these all relate to each other.
5. Since we are a remote team, I used Miro platform to build a visualized representation of the categories relationships to present to my stakeholders.
6. With the Product Owners we explore the results from the 4 points of view - Behavior, Algorithm, User Interface and Feature. We analyse the positive and negative feedback and come up with solutions and strategies to improve the feature efficiency and experience.
Important Note: For privacy and competitor reasons, I am not allowed to share this dashboard real content.
Case Study Design for Learning - AI Research Problem
The short turn strategy of Becker, is to move from a fill-to-all experience to a personalized one. With the support of Artificial Intelligence (AI), we are moving from the concept of Progress to Proficiency