In December 2016, the MCoE enthusiastically launched the MCoE website. In an ongoing effort to best optimize and better understand users’ experiences with the MCoE website, several user research studies have been conducted. Our primary goal is to create a website that is intuitive and user-friendly for all our current and prospective partners.
To date, we have conducted four studies utilizing three different methodologies: moderated usability test, unmoderated usability test, and a pulse survey. To learn more about the various UX Research methodologies visit the Methodology page.
If you would like to be involved in upcoming MCoE Website user research efforts, please contact Ryan Wise at Ryan.P.Wise@kp.org.
Homepage Redesign Study
In November 2018, we interviewed 13 KP employees asking them to evaluate a MCoE homepage redesign based on feedback analyzed from a previous MCoE website usability study. After perusing the homepage redesign, all 13 participants were asked what they liked, what they didn’t like, if anything was confusing or missing with the design, and what recommendations they may have for the design.
Identify participants’ overall thoughts and experiences with the MCoE website homepage redesign.
What We Learned:
- Almost all participants prefer the homepage redesign to the current homepage design – “bigger and easier to see” and “less clutter.” Although, a couple of participants mentioned liking the personalization of the current homepage.
- The most common word used to describe the homepage redesign was “clean” followed by “simple” and “easy.”
- Several participants did not like or see the purpose of the testimonials or thought they should be moved to the bottom of the homepage.
- With the homepage redesign, it would take one additional click from the current homepage design to get to the Engagement, Design, Development, and Certification sections.
- Some users may not know the three KP application categories.
On December 6, 2018, we enthusiastically launched our new MCoE homepage. Our new design addresses the key takeaways from our user research interviews.
- “Testimonials” have been replaced by “MCoE Spotlight.”
- Engagement, Design, Development, and Certification are now hyperlinked under the “Services” icon.
- The three classifications of KP apps (Consumer, Provider, and Workforce) are included and hyperlinked under the “App Portfolio” icon.
Unmoderated Usability Test
In June 2018, an unmoderated usability study with 13 participants was conducted using the UserZoom platform. Participants were asked to speak “aloud” as they were navigating the MCoE website. All tests were recorded and later viewed and analyzed by our user researcher.
Identify end-users’ pain points while navigating the MCoE website as well as gather insights on navigation fluidity.
What We Learned:
- Several participants had either forgotten that the MCoE existed and/or was uncertain what and who the MCoE is and does.
- Many participants felt overwhelmed by the homepage.
- Some participants would not think to go to the MCoE website to gain further information regarding KP mobile apps.
- All participants looked for a “Contact Us” section either on the global navigation bar, the footer, or both.
- Include an “About Us” section on the global navigation bar that conveys who and what the MCoE is.
- Include a “Contact Us” section on navigation bar or footer.
- Explore new designs of the homepage that capture the essence of the MCoE.
- Explore ways we could make the site more clean and simple.
“I think the homepage was a little busy and there were a lot of links to look through the entire site. Any way to simplify this would make the site cleaner.”
“That’s the problem with these mega menus – you scroll too far and then you lose everything.”
In April 2018, the MCoE emailed a pulse survey to 385 KP employees. Out of the 385 employees, 94 completed the survey, representing a 24% response rate.
Establish a baseline to help understand and better optimize end-users’ experiences with the MCoE website.
1. What section(s) of the website have you explored?
2. How helpful do you find the MCOE Website?
3. Would you be willing to participate in a MCoE website usability study?
What We Learned:
Most participants had visited several sections of the website. Certification was the most visited section.
Out of 77 participants, half (39) find the MCoE website “Very helpful,” followed by 14 participants finding the website “Extremely helpful” and 24 finding the website “Somewhat helpful.”
On a scale of 1-5 with 1 being “poor” and 5 being “excellent,” over half of the 77 participants rated their overall experience with the MCoE website at a 4, followed by 19 participants rating their overall experience at a 5, and 15 participants rating their overall experience at a 3.
Conduct an unmoderated usability test to explore users’ behaviors as they navigate the MCoE website to better understand users’ overall experiences and frustrations while completing specific tasks. 52% of participants said they would be willing to participate in a usability test.
“I’m very grateful for this resource and visit frequently. We were especially glad to use the Design Kits, and also the accessible design information.”
“I really like the MCOE website. That’s one of my quick tabs. It has complete updated information of latest tools being used by our devs. -things we [are] doing. Really great effort.”
Moderated Usability Study
In December 2016, prior to the launch of the website, a remote, moderated usability study with 14 participants was conducted on the UserZoom platform.
Understand the expectations of MCoE website users and to identify what elements of the MCoE website users would utilize the most and what elements users found confusing.
1. What is the minimum required font size for regular (non-legal) text? (80% success rate)
2. Is there a wheelchair icon? (80% success rate)
3. What could you use to test your new Android mobile application for accessibility? (40% success rate)
What We Learned:
- The minimum font sizes need to be clearer and easier to locate.
- Some participants had difficulty understanding what the page titles represented.
- Several participants had trouble locating the resources for developing a help guide of an iOS app.
- Include more imagery throughout the site. Several participants commented that the website was too text heavy.
- Implement a “Point of Contact” or “Help Tab” at the top right corner of the homepage or consider including a live chat during business hours.
- Include a “Certification Process” infographic to inform clients where their app is in the certification cycle.