Salesforce
Researching and Alleviating Barriers to Generative AI Adoption in Businesses
A 3-month UX research client project. Through rigorous data collection and analysis, my team and I worked with Salesforce to explore and facilitate generative AI technology adoption in businesses and organizations, resulting in four comprehensive design recommendations.
UX Researcher
4 UX Researchers
Salesforce UX Team
12 weeks
Feb - May 2024
User interviews, surveys, competitive analysis
OVERVIEW —————————————————————————————————————————————–
MY ROLE
THE TEAM
METHODS
TIMELINE
Generative AI technology has taken the spotlight in consumer usage; however, its adoption in businesses and organizations has been notably lower, despite its potential to boost business operations by driving productivity, improving decision-making, and providing a competitive edge.
Understand the fundamental reasons for the slower adoption of generative AI in businesses and organizations.
Develop a set of concrete design recommendations for the UX team to help increase responsible generative AI integration into Salesforce teams and workflows.
Based on our research goals, I developed three proto-personas to identify the user base for future Salesforce generative AI products and experiences. We used these personas to help define target criteria to find representative participants for our primary research.
I began by reviewing existing research on generative AI adoption in business vs. consumer applications to understand the gaps in current knowledge, identify tenets of generative AI adoption (i.e. what factors influence whether a GenAI product is adopted or not), and help inform our survey and interview questions.
I wrote and conducted 10 semi-structured, 30-minute user interviews to develop a more comprehensive, in-depth understanding of the goals, pain points, motivations, and behaviors of each user group regarding generative AI usage and adoption within their organizations.
Out of the 10 interviewees, there were 8 ICs (3 from the UX industry, 1 from ed tech, and 4 graduate students), 1 decision-maker from the ed tech industry, and 1 marketer from the aviation industry.
I conducted a competitive analysis of 9 GenAI products commonly used for businesses and consumers across 8 dimensions: cost, learning curve, accuracy, data security, type of input, type of output, customization ability, and user interface. This allowed us to identify common weaknesses across products that may hinder business adoption rates and draw insights from their strengths.
We chose these products because they covered the highest variety of use cases - including using GenAI for content creation, for image generation, as a chatbot, for coding, etc.
I designed and conducted persona-specific surveys to gather quantitative data from a larger number of respondents in a relatively short amount of time. The goal of the surveys was to learn about users' experiences, concerns, and needs surrounding generative AI usage and adoption in their organization.
We recruited via LinkedIn, Slack, Discord, and in person at the UC Berkeley Haas School of Business and School of Information.
We received 58 responses: 23 individual contributors (ICs), 22 decision-makers, and 13 marketers.
Sample survey questions:
For ICs: How valuable have GenAI technologies been for achieving the following benefits? Please rank in the order of importance (1 = most important): time savings, increased creativity or inspiration, improved quality of output, learning/skill development
For decision-makers: Please rate the significance of each of the following concerns to adopting GenAI technologies in your organization (1 = not a concern, 5 = major concern): cost, complexity, data privacy and security, ethical considerations, reliability/accuracy, job displacement, other: ___.
THE PROBLEM ——————————————————————————————————————————–––––
RESEARCH GOALS
PROTO-PERSONA DEVELOPMENT
SECONDARY RESEARCH
SURVEYS
USER INTERVIEWS
COMPETITIVE ANALYSIS
RESEARCH METHODS ————————————————————————————————————————––––
PERCEIVED BENEFITS OF GENERATIVE AI IN BUSINESSES
PERCEIVED BENEFITS OF GENERATIVE AI IN BUSINESSES
CONCERNS ABOUT GENERATIVE AI USAGE IN BUSINESSES
USER NEEDS
RESEARCH FINDINGS ———————————————————————————————————————–––––
Brainstorming - Individual contributors and decision-makers find generative AI helpful for brainstorming, describing it as similar to talking to a peer, as it helps refine “muffled” ideas into well-thought-out concepts.
Efficiency and Speed - Generative AI accelerates tedious work processes by summarizing information, generating filler content, and debugging code.
Brainstorming - Individual contributors and decision-makers find generative AI helpful for brainstorming, describing it as similar to talking to a peer, as it helps refine “muffled” ideas into well-thought-out concepts.
Efficiency and Speed - Generative AI accelerates tedious work processes by summarizing information, generating filler content, and debugging code.
Transparency - 30.4% of individual contributors cite reliability and accuracy as a challenge, expressing a need for greater transparency around training data, answer sources, and how user data is used.
Increased Support from Company - 69.6% of individual contributors report low support from their company and want guidance on how AI can responsibly complement their work, not replace it.
Privacy and Security - Individual contributors rate privacy and security of generative AI at 3.7 out of 5, and highly regulated, more sensitive industries like healthcare and fintech are “very averse to adopting new technology…because the data they’re dealing with is often super confidential” (Participant 3).
Trust and Accuracy - Several interviewees reported encountering made-up information from AI, with 30% of sellers and marketers citing skepticism and fear as a major challenge.
Over-reliance - Several interviewees feared skill degradation due to over-reliance
DESIGN RECOMMENDATIONS ——————————————————————————————————————————–––––
Ethnographic Research: Observe how generative AI tools fit into everyday work practices, how they alter workflow dynamics, and their impact on organizational culture.
Usability Testing: Because we didn’t have an existing product to work off of, there was no usability testing to be done, but once a first prototype is developed, it will be essential to conduct usability testing.
Longitudinal Studies: Observe the long-term effects of generative AI integration on organizational performance; this is especially important because of the rapidly-evolving nature of generative AI.
GUIDANCE FOR FUTURE UX WORK
WHAT I LEARNED —————————————————————————————————————–––––————––––––
As I navigated the challenges of real-world UX research, I learned that good UX researchers must be adaptable, resourceful, and persistent. My greatest challenge during this project was participant recruitment, as our personas were quite niche and budget constraints meant that I was not provided with incentives to offer participants. This led to unexpected recruiting challenges, such as no-shows to scheduled interview sessions, and these factors making it tricky to find the “ideal” matches for participants.
To address these obstacles, I led the team in exploring more "out-of-the-box" recruitment strategies, seeking out proxy users, and leveraging existing networks and communities creatively. By continuously refining my research approach, I was able to collect meaningful data that informed our final design recommendations.