Sahiyo

At a Glance:

• I conducted UX research for a gender-based violence organization looking to improve the accessibility of its website

• Through interviews, surveys, and heuristic analysis, I highlighted disparities between the experience of users with and without visual impairments and recommended steps to rectify them.

• The client resolved over half of its accessibility heuristics violations and redesigned one of its most popular pages based on feedback

Challenge

Sahiyo is an international organization dedicated to ending gender-based violence against women. It works to empower communities to end female genital cutting (FGC) and create social change through education and community involvement on their website, sahiyo.org. Sahiyo’s website is its primary hub for resources, but major accessibility issues inhibit its ability to reach its diverse population of community members. Sahiyo’s leadership has asked for a comprehensive needs assessment that highlights the steps needed to make their resources more accessible to all communities.

Research Question

What accessibility challenges do users encounter on Sahiyo’s website, how do these issues negatively impact user engagement across the site, and how can they be resolved?

User Interviews

To understand the goals, pain points, and needs of Sahiyo’s users, I conducted a series of virtual interviews. I led two of our six 60-minutes sessions with users, including two who self-reported as having a disability. Questions focused on general experience on the site, areas of the site they find most useful, and challenges they face finding what they need. Interview participants were recruited with the help of Sahiyo leadership and through online forums for people with disabilities.

Analysis:

Quotes and observations from these recorded sessions were captured by a notetaker. These notes were added to a Miro board and grouped by question. Common themes were extracted for each grouping and were used to determine the three most important findings.

Findings:

1. Users believe the Sahiyo’s website is generally effective at helping them find resources about FGC

2. Stories - particularly on the Sahiyo blog - are drivers to the site, but aren’t easy to find

3. User with and without disabilities mentioned color contrast, responsiveness, and navigation as key challenges for the site

SURVEY

Our interviews gave us a high-level overview of the positives and negatives of Sahiyo.org. To dig deeper into some of the accessibility challenges affecting people with varying abilities, we developed a survey using Qualtrics. Qualtrics was selected as it provides built-in features to grade and improve the accessibility of surveys.

The survey prompted responses similar to our interviews, but additionally asked participants to rate their experience locating and navigating to key resources on the site. This list of key resources was determined based on findings from our interviews as well as priorities highlighted by the organization.

Analysis:

I used spreadsheet crosstab functions to compare the average experience rating of users who reported disabilities and those who did not. I analyzed open-ended questions individually and used qualitative coding to form groups with insights gathered during interviews.

Findings:

People who reported disabilities had a less favorable experience finding resources than those who did not report a disability. These outcomes suggested that accessibility issues do in fact affect users’ ability to engage with the site, and therefore the organization as a whole.

Limitations:

• Despite our efforts, we could not recruit enough survey responses to elicit a statistically significant result

• The majority of our responses came from Sahiyo volunteers or others directly affiliated with the organization, which may have led to results not representative of typical users.

HEURISTIC EVALUATION

While the survey gave us an idea where to look for accessibility issues, performing a heuristic evaluation would allow us to pinpoint specific action items Sahiyo could take to remedy them.

Analysis:

Each team member analyzed the website according to Nielsen’s 10 Usability Heuristics and prioritized issues by severity and prevalence. We also included an 11th heuristic focused on accessibility. Scores were then aggregated and were used to rank problem areas.

Heuristic Severity Rating: 0 = no usability issues; 4 = severe usability issues requiring immediate resolution

Findings:

I identified 38 of our combined 130 unique heuristic issues. Key takeaways from this activity were:

1. Missing labels, nondescript alt-text on images, and poor color contrast were the most common accessibility challenges

2. The events and donation pages were particularly troublesome as they involved embedded content and forms which were not tab-accessible from the body of the page

USABILITY TEST

Our survey highlighted the negative experiences some users had navigating the site, but we lacked the specifics of what made those tasks difficult. I coordinated a usability test to 1) quantify some of the challenges highlighted during the heursitic evalution and 2) to establish a baseline for future usability tests conducted by Sahiyo.

We conducted five usability tests with people with disabilities. I led one of our five hour-long sessions over Zoom and took notes for two others. Due to time and recruiting constraints, we had to narrow our scope of disability and focus on visual impairments such as low vision and color blindness.

Analysis:

Based on findings from our survey and heuristic evaluation, we developed five tasks that reflected popular use cases of the site. For each task, we tracked the participant’s ability to complete it, the time they took, and their self-reported level of satisfaction they found completing the task. We also kept observations and quotes made during the session in a color-coded spreadsheet, supporting our quantitative measures with qualitative insights.

Findings:

Users generally had a high level of success completing the tasks, although some tasks required more than one attempt for some users. Users were, for the most part, satisfied with their experience, even if they remarked it took longer than they would have anticipated.

One problem area was Sahiyo’s events calendar page. Each of our five users struggled to obtain information for an upcoming event and many struggled to find the calendar altogether.

Insights

As research concluded, overarching themes and insights helped guide next steps. From my individual research findings and my widened knowledge of the organization over the course of this project, the following facts became evident:

• Neither we nor the organization had the resources to properly prioritize all forms of disability in the short-term, so based on Sahiyo’s user base and general ease of implementation, focusing on visual impairments would be an approachable first step

• Sahiyo, like many nonprofits, did not have an excess of time and resources to dedicate to web development changes. The idea of the “quick win” and a prioritized list longer-term fixes would be more likely to lead to change than a laundry list of accessibility errors.

• Due to the nature of the organization, many of Sahiyo’s visitors were willing to overlook or overcome usability challenges given the importance of the resources offered. Because we primarily had access to Sahiyo’s existing user base, there was a hesitancy to critique anything dealing with the organization, limiting the strength of our qualitative research.

Recommendations

As a summary and final deliverable, we proposed the following high-level recommendations to Sahiyo leadership:

1. Primary navigation reorganization – Many of the site’s most popular resources are buried in submenus making them particularly difficult to discover for people with disabilities

2. Prioritize fixes for visual impairments – Based on participation in our study and general consensus within the organization, visual impairments is the subset of disability that would be most impacted by improvements to accessibility. Results of our heuristic evaluation provide a starting point for earning quick wins in this area

3. Ensure content is up to date – Inaccurate page titles, outdated content, and broken links all contribute to a negative experience for people of all abilities.

For each of these recommendations, we provided specific instances of poor accessibility on the site and shared links to online resources covering the HTML/CSS code needed to fix them. This and other strategies and best practices for future research were included in a 6-page web accessibility improvement plan document.

Impact

Since this project, Sahiyo has already implemented some of our recommendations throughout the site. Of the 38 heuristic violations I highlighted, 21 have been resolved within six months of project completion. One great example of the site’s improvement is this accessibility-friendly redesign of its events page:

Notable changes:

• All calendar and search elements are tab indexed and keyboard accessible

• Relocation and default text in search bar distinguish it from the sitewide search option and improve findability

• Combination of color and text are used to differentiate event types

Next Steps

For Sahiyo to reach its desired level of accessibility, there are still steps that need to be taken going forward:

1. A recurring accessibility audit - As more content is added to the site, it will be critical to ensure proper systems are being followed. We recommended WAVE and axe DevTools as great options for quick accessibility checks.

2. Further testing with other forms of disability - Due to time constraints, we shifted our scope to focus on visual impairment issues, but the needs of others with disabilities should be considered as well.

3. Focused user testing on the site’s mobile version - Our client mentioned that most of their traffic came via desktop/laptops, but having a more mobile-friendly site will be important for the future of the organization.

Lessons Learned

1. Narrowing the scope of disability - No site is accessible to everyone. Focusing on one or two forms of disability while acknowledging others creates a more manageable set of tasks for both the client and the research team.

2. The value of the quick win – Improving accessibility can be a daunting task for a team with limited resources. When presenting the findings of our heuristic evaluation, our client appreciated our distinction of short- and long-term fixes.

3. Living with imperfection – We struggled to attain a diverse pool of interview participants and didn’t reach the quantity we wanted for our survey. Even with the limitations, we still learned enough with each method to inform our next steps. This allowed us to bring tangible value to our client while acknowledging there was still much more we did not know.