Stockgrok
STOCKGROK: UNDERSTANDING FINANCIAL DATA NONVISUALLY (Sponsored by Bloomberg L.P.)
People with visual impairments (PWVI) are at a professional disadvantage in finance because most data is presented in visual charts.
Bloomberg L.P. asked our team of five to help close the gap by making financial visualizations more accessible. We became fluent in accessibility design and conducted contextual inquiry interviews with PWVI and sighted finance experts to understand their workflows and pain points. We identified six research insights and four design opportunity spaces and rapidly created and tested twelve low-fidelity prototypes. Continually iterating on designs, we ran usability tests with PWVI on sonified price data and three navigation modalities. From Bloomberg chart specialists we discovered that the stock price's relative position to a moving average allows experts to predict future price movements and developed the user flow of our product to enable completion of an expert task.
Our solution, Stockgrok, uses pitch-mapping, timbre, and earcons to convey where a stock price and its moving average intersect and the distance and direction between the two values. Stockgrok provides context for understanding price data and enables observation of and navigation to areas of interest without relying on visuals. Because our solution has both auditory and visual components PWVI can collaborate with their sighted colleagues.
NOTE: This was a capstone project for Carnegie Mellon's Master's in HCI with Bloomberg as the client sponsor. Check out our website at stockgrok.github.io and our final product at stockgrok.github.io/prototype (Chrome desktop with sound on provides the best experience)
A screenshot of Stockgrok. Click through to interact (be sure to read the keyboard shortcuts when you do).
SCOPE
8 months
5 person team
MY ROLES
UX designer
UX researcher
Project manager
Client point of contact
SKILLS
Accessibility design, Agile project management, literature review, subject matter expert interviews, contextual inquiry with people with visual impairments and finance experts, affinity models, sequence models, empathy exercises, visioning, creative matrices, content strategy, visual and sound design, prototyping, user recruitment, usability testing
TOOLS
Keynote, Trello, Google Sheets, Google Forms, InDesign, Mural, Sketch, Illustrator, Streak CRM, Audacity, GarageBand
PROBLEM SPACE
Listen to this clip of stock data, and try to make a buy or sell decision.
This is the image of table you just heard.
And this is the same table data, represented by a data visualization.
The audio above, as read by a screen reader, is the best case scenario for people with visual impairments (PWVI). PWVI are at a huge disadvantage when it comes to making financial decisions because so frequently data is inaccessible: it's much easier to glance at a graph quickly to see trends in data.
Because of the expertise of Bloomberg's clients and employees, we set out to design a solution that would improve the ability to complete tasks for finance professionals with visual impairments & declining vision.
RESEARCH
Subject Matter Expert Interviews
To explore the complex domains of accessibility, data visualization, and finance, our team interviewed ten subject matter experts. Experts included academics and industry professionals in accessibility, emerging technology, and finance. We built our knowledge of assistive technology, the benefits and challenges of using different modalities for communicating data, and current efforts to make finance accessible.
Question-storming
I led a question-storming session, which allowed us to quickly generate a large number of research questions and align our team’s goals with our client's. I asked the client to generate short statements that represented the problem space, and after producing dozens of questions inspired by these prompts, we scoped to the most "up-voted" questions to guide our research:
How do the mental models and workflows of people with visual impairments differ from sighted people?
How might a non-visual tool benefit different audiences?
What financial data is essential for decision-making?
Empathy Exercise
We felt that it was necessary to put ourselves in the shoes of a person with visual impairments. It was important, however, to develop empathy in the context of our research, rather than just simulating blindness. After researching how to effectively design an empathy exercise, we wrote our own experiment protocol. In this exercise, we learned the basics of the Apple VoiceOver screen reader and used it to navigate our bank accounts on both laptops and iPhones. From this exercise we identified our own biases and expectations as sighted designers.
Literature Review
Our team collectively read over 100 research papers on topics including accessibility, cognitive science, and data visualization to develop a shared understanding of the research space. After individually recording notes for delegated papers, we shared the takeaways with the team and printed them out onto notecards. We then clustered the notes to see which themes emerged.
Digitized high-level themes from our literature review
Data in Finance
From reviewing instructional books, online financial resources, and talking to MBA students, I compiled a list of data formats frequently used to make investment decisions. To place them in context for our project, I coded them along quantitative-qualitative and textual-pictorial axes.
CONTEXTUAL INQUIRY
RECRUITING People with Visual impairments
We were initially unsure of how many PWVI worked with financial data, so I created an online screener survey through Google Forms and leveraged the networks of our client and SMEs. Our survey screened for people who used analogous tools to finance professionals. In total, we received 203 responses: 189 were legally blind, ranging from moderate to total vision loss. Encouragingly, over 40% of respondents had checked financial data in the last 30 days.
Contextual interviews
Our team needed to build a deep understanding of how PWVI use data to make decisions and the breakdowns in their processes. We scheduled remote and in-person interviews with 10 PWVI: five with congenital blindness, and five whose vision declined later in life. Our interviews were structured around observing how people accessed financial data. Some walked us through accessing their financial information in real time, while others were asked to recount recent interactions using financial data, such as using spreadsheets to analyze data or check a stock price.
One of our interviewees customized a bluetooth keyboard with tactile stickers to find his place while typing.
Our team also interviewed six finance experts about their use of data and visualizations for decision-making. Our goals were to learn more about the workflows of high performing experts and identify what data is and is not important for guiding their steps and decisions. We focused on interviewing people who actively managed investments and frequently made buy/sell decisions. Roles included stockbrokers, the president of a capital management firm, analysts, and MBA students.
Interpretations and Models
After each interview, three team members interpreted notes and recordings. To capture how each user interacted with data and made financial decisions, we created visual models of the workflows described in the interviews.
Affinity diagramming
Our team constructed an affinity diagram to organize the notes from our interpretation sessions into higher level insights. We color-coded notes based on whether they came from PWVI or finance experts. As insights began to crystallize, we were excited to see overlap between the two different domains, suggesting that both groups faced some similar challenges in their decision-making and workflows.
Walking the wall and Visioning
We spent several team sessions “walking the wall” and annotated the affinity diagram with questions, breakdowns, and design ideas. As we began ideating on possible solutions, we also created low-fidelity prototypes to explore the forms of potential solutions and tested them internally with tactile materials and audio cues. We considered the the input and output modalities of our top design ideas and the feasibility of each one. These methods for rapidly generating and evaluating a large number of prototypes continued to aid us in iterative design.
Design ideas plotted in terms of difficulty of implementation and solution impact.
Early rapid prototype for a tactile interface for navigating through prices of financial securities.
RESEARCH INSIGHTS
Book & Presentation
At the end of our research phase, we presented our findings to our client, advisors, and peers, as well as the Bloomberg accessibility, UX, and software teams. Our book and presentation, both titled "What Am I Missing?", offered six insights told from the perspective of a person with visual impairments:
We also identified four major opportunity spaces for designing solutions:
I led design, content strategy, and copy editing for the presentation. We selected colors for high contrast for visual accessibility. Additionally, we created a large print version of our book, along with screen reader-accessible versions of our book and presentation slides.
DESIGNING A SOLUTION
Creative Matrix
To quickly generate and record a number of design ideas, our team constructed a creative matrix using our opportunity spaces and the five senses.
Low-Fidelity Prototyping
From the creative matrix our team created twelve low-fidelity prototypes in parallel, including a conversational user interface, tactile charts, and auditory icons ("earcons") to represent data. Because we were not given access to Bloomberg clients, we piloted internally and with sighted Bloomberg employees to quickly uncover early usability issues.
Tactile graphs worked well for providing the gist, but were not technically feasible or affordable for replicating the complex charts used by professionals.
A user draws their mental model of a chart based on a prose audio description. We discovered that descriptions were insufficient for user confidence and would not suit every use case of a chart.
We played earcons (standardized audio alerts) with different timbres to represent different chart trends and asked users to draw the trends. We used both timbre and earcons in our final product.
Rose/bud/Thorn & 2x2 Matrix
To share successes, opportunities, and problems from early prototype usability testing, I led our team in a rose/bud/thorn exercise and mapping our ideas onto an impact-feasibility matrix.
Iterative Design
Our usability testing results and research insights led us to scope to sonifying stock price data for higher-fidelity prototyping and usability testing with PWVI. Sonification required no additional hardware and could easily fit into a finance professional's workflow. We used timbre and pitch to convey information rather than volume (which could change based on personal preference) or sound localization (which would require both headphones, isolating the user from their surroundings). Research with finance experts and competitive research led us to sonify open-high-low-close charts.
Our research also indicated that navigating through large sets of data is a major breakdown for PWVI. We tested sonified open-high-low-close charts and three navigational interfaces with a total of eight PWVI. Usability testing results, our earlier research insights, and fit with our client's culture led us to scope to keyboard controls.
We quickly abandoned the drawing tablet because it scored lowest on accuracy and lowest on user-reported confidence and control.
The knob provided similar results as the keyboard, and our research insight "changing tools is complex, so it better be worth it" directed us to avoid new hardware.
The keyboard requires no new hardware and many PWVI already make heavy use of keyboard commands with their screen readers. The keyboard also fits into the professional workflow, especially at Bloomberg where every client receives a Terminal keyboard (Image courtesy of Bloomberg).
A medium fidelity prototype using a dial to scrub through a graph
Validating our Expert SCENARIO
As we continued to iterate on our designs, I led our team in validating the expert task were we envisioning enabling for PWVI. We conducted short interviews and observations with former portfolio managers at Bloomberg as they viewed stock price charts and discussed buy/sell decisions. We realized we were providing the wrong context by focusing on open-high-low-close charts; whether an expert used OHLC or a line chart was a matter of personal preference. Regardless of chart type, we observed that experts used financial indicators, like the 50-day simple moving average, to contextualize price data. The relative position of a stock's price to its moving average and when the two lines intersect triggered buy/sell decisions, not the price values alone.
I fleshed out the sighted expert user scenario we wanted to enable for users with visual impairments. This informed the user experience of Stockgrok, as we designed for users to observe the distance between stock price and the moving average, the direction of the difference, and where the two values intersect.
High-Fidelity Prototyping
For our high-fidelity prototype we added more keyboard shortcuts and wayfinding cues for navigational control and improved the context for price data by focusing on its relation to the 50-day simple moving average (SMA). The distance between the two lines was pitch-mapped (i.e., the higher the difference, the higher the pitch), whether the price was above or below the SMA was indicated by two different timbres, and the day after the price intersected with the SMA was signified by an earcon.
We tested our prototype with two PWVI and two sighted finance experts. Because the PWVI we tested with were not investment professionals, it was important for us to test with domain experts to see if they could make decisions with Stockgrok based on audio alone. We asked each user: "Imagine you're trying to decide whether or not to buy or sell a stock. Use Stockgrok to explore the data and tell us what you observe."
Both visually impaired and sighted users were able to develop mental models of the data based on sound alone and make predictions about future price movements. Furthermore, because Stockgrok has a visual component, it also enables collaboration between PWVI and their sighted colleagues.
FUTURE OPPORTUNITIES
Our team is excited about the potential of Stockgrok to open up expert investment decisions to people with visual impairments, but we also recognize that there is more work to be done. With more time, we would have further improved the sound design. We also provided a large number of keyboard shortcuts for navigation control, but we recognize that many people prefer to customize hotkeys.
We used the 50-day SMA as the financial indicator for our proof-of-concept, but the technology behind Stockgrok could be further expanded to many more financial indicators, and could also be applied to any tabular data and any industry that requires the comparison of two lines.
