Comparing Quant UX Researchers w/ Data Scientists
It's a mess because everything's undefined
When it comes to being Quantitative UX Researcher, there are two questions that I hear fairly often: What’s the difference between a qUXR and a data scientist or a data analyst? I’m already a [some other job] how do I switch to qUXR? After pondering on the topic for a while, I’m starting to feel like the questions can be answered simultaneously. Understanding where qUXR sits in relation to other roles helps give people a roadmap to transition from any other role into qUXR, and I honestly can’t think of a clear direct definition so I only have comparative methods.
The base problem is that Quantitative UX Research as a field is still ill defined. A handful of large companies have the title, practitioners hail from a vast range of backgrounds, and there is a huge amount of variation in skills, actual job duties, and viewpoints. We know that DS has a wide variance with machine learning experts on one end and advanced product analyst folk on the other. I feel that qUXRs have an even wider variance with academic researcher types on one end and many flavors of data scientist types on another.
So I’ll preface this whole post by saying that anything I describe will necessarily be an over-simplification of reality. For at least some subset of real people, I’ll fail to even come close to their true experience. Hopefully, those people in the field will put their own thoughts out for everyone to consider. I know that there will be a couple of talks at Quant UX Con this coming June related to the topic.
If the (now somewhat tongue-in-cheek) definition of a Data Scientist is “Someone who is better at statistics than any software engineer and better at software engineering than any statistician”, then I’d say that the definition of a Quantitative UX Researcher is someone who thinks more about researching humans than a data scientist and understands more data science than a social science researcher.
Just like how data scientists essentially grew to fill a niche that came to exist between a data analyst and software engineering by analyzing data at massive scale to help decision-making, qUXR feels like it is stepping in to fill a niche between more traditional UX researchers and data scientists where understanding users started involving the use of massive amounts of data.
So let’s compare and contrast
Since there’s no formal definition of either title, I’m going to paint caricatures of qUXRs and data scientists in an attempt give a better understanding of how the roles are similar but different.
This method of drawing caricatures is almost guaranteed to offend people in both disciplines — I have a colleague who tried once and accidentally offended a bunch of data scientists with a bit of unfortunate word choice. The fact that attempting to paint either in broad strokes actually points to a very important aspect of both job titles though — both “data scientist” and “quantitative UX researcher” were titles created within various organizations to fill a relatively new niche of work that the existing org structure wasn’t equipped to fill with existing job functions.
What I mean by that statement is that what a data scientist or a qUXR actually does in a given organization depends primarily on the history of the organization. New hires come in due to there being a newly discovered gap in the existing employee talent pool. It’s like how the role of “social media manager” suddenly sprung into existence and used to skew pretty young.
Imagine there’s an insurance company in the late 2000s when data science was slowly taking off, I’d be willing to bet that they’re extremely unlikely to hire a data scientist to do financial and modeling of their customers and business, they have actuaries for that sort of work. Instead, since actuaries are less likely to know how to build recommendation models, or do research into the usability of their new shiny mobile app, that’s where they’d make their data science hires.
This gap-filling tendency means that, to a surprising extent, a data scientist might actually be doing the work of a qUXR already if they were focused on helping product/design folk do their work, understanding users, running experiments, collecting data. I’ve been in other organizations where that work was being done by a data analyst team of some sort. Sometimes this work is spread across multiple people (for example someone good at survey research and someone good at data analytics working together). In the end, the job title is a complete red herring.
Similarities between roles
As part of the basic requirements for the job, things tend to be pretty similar.
Statistics and Research Methods — Roughly what you’d expect of an experienced researcher. The good data collection, designing studies, uncovering causality, using different methods to meet the needs of different situations. Having the ability to explain statistical concepts and results clearly to non-researchers is also important.
Software Engineering — Working with large, noisy, changing data sets and making sense of them using code is critical to the role. While it’s not necessary to know how to manually write MapReduce jobs any more thanks to modern giant SQL datawarehousing tools, there’s still a strong need to write custom code to handle all sorts of situations and edge cases. Having these skills also helps with collecting data.
Domain Knowledge — Whether it’s healthcare, government, phones, video, hardware or software, domain knowledge is a necessary foundation for asking good questions and handling the data that comes out of those domains. Everyone needs to have a solid foundation in the domain they work in.
Machine Learning (or the lack thereof) — qUXRs are highly unlikely to be building ML models of things, because the primary goal of the job is to come to an understanding, not a model. At the most, we’d use ML models as tools on the path to doing research, but they’re rarely the end goal. That said, there are tons of data scientists out there who would say the exact same thing, so it mostly depends on where you are.
Focus on doing “research” — Obviously the role with “researcher” in the title is expected to be more focused on this. The problem comes from defining what research is. It’s entirely possible to run entire research programs under either job title and no one would really question it.
But speaking in general, it seems that data scientists can afford to be more atheoretic about things — “optimize this process and make it better by any means” is a perfectly valid project to hand over to DS. Meanwhile, qUXRs are more likely to be unhappy with using black-box methods because they don’t generate generalizable insight.
A bit more of the social sciences — I haven’t seen a giant survey of degrees that qUXRs across the industry have, but my sense is that there’s a bit of skew towards the social sciences (though there’s also a pretty healthy group of people from the neurosciences too). These folk are usually either highly technical out of personal interest, or did their work in areas like computational sociology, where Big Data techniques were part of the work. But ultimately it’s just a slight skew and much like data science, the academic backgrounds matter very little compared to practical work experience and skills.
Finding a way to transition to qUXR
Since there’s such a significant overlap between data roles and qUXR, there are many ways to jump over, but it all starts with this:
Care about understanding users!
Thinking about users, and looking at products from the perspective of users, is a muscle that takes practice to develop. It’s not just knowing that users can be have in unexpected ways, but building an understanding of why they may act in certain ways and how to anticipate those situations. It’s hard to get this outside of either concentrated study or experience.
You can practice this skill just as easily as a data scientist/analyst or a qualitative researcher with coding skills. There’s rarely a case where caring about the end customer experience will be detrimental to a business, so you can justify sliding that work in along with existing projects. Your work will also be better for it.
If you find out that a certain obscure feature is loved by users who use it, but very few people are using the feature, ask why. Try to figure out a way to figure out why that is, what’s special about those users, and most critically, what the problem is that’s causing this discrepancy. Is it that the current users have unique needs? Or is it a discovery issue? Perhaps cost? Or something else? It’s your job to dig deeper to understand what’s going on so that you can recommend solutions to the underlying problem.
Finally, pay attention to your stakeholders
As a rule, everyone you work with has their own jargon, priorities, and processes. The set of job functions you need to work with depend heavily upon how teams are structured. Data scientists and qUXRs generally have very wide groups of people they work with, across multiple functions and locations.
But in my experience, being a qUXR tends to set place a bigger demand on working with an even wider group of people than DS. It’s fairly rare that a data scientist needs to work with everyone within the organization, and then add in people outside the org too. Either way, learning the lingo for all these groups is important if you plan on becoming either.
The one thing that is perhaps a bit more unique to qUXR in this area is that they’re typically embedded within a UX team. UX as a discipline has its own unique set of jargon and processes that’s shared between UX designers, researchers, and engineers and is distinct from the broader engineering/research communities a typical data scientist will be exposed to. Luckily, UX folk tend to spend energy trying to make sure other people understand UX processes (for example, the now-overused term of “design thinking”) so we don’t have to figure it out ourselves.
Whatever the job title, there’s an interesting position out there for you
In the end, I think I want to emphasize two things:
Job titles bear little predictive power on what you actually do
There are roles where thinking about users is encouraged, find them
qUXR is a convenient title for finding a job that is interested in understanding users, but data science has plenty of positions that work too. You just need to get a good understanding of the exact problem space you’re supposed to be working in. After that, you can slip in as much user-centric thinking as you can.
Next week I’ll stop talking abut qUXR unless there’s a super interesting topic/question coming from you readers. Most likely we’ll go do something data-y nerdy instead.
Standing offer: If you created something and would like me to review or share it w/ the data community — my mailbox and Twitter DMs are open.
About this newsletter
I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With excursions into other fun topics.
Curated archive of evergreen posts can be found at randyau.com
All photos/drawings used are taken/created by Randy unless otherwise noted.
Supporting this newsletter:
This newsletter is free, share it with your friends without guilt! But if you like the content and want to send some love, here’s some options:
Tweet me - Comments and questions are always welcome, they often inspire new posts
A small one-time donation at Ko-fi - Thanks to everyone who’s sent a small donation! I read every single note!
If shirts and swag are more your style there’s some here - There’s a plane w/ dots shirt available!