If we think of management chains as these long games of telephone where goals, priorities, and incentives are passed up and down, specialized disciplines like data science, designers, and even customer support folk, are at a distinct disadvantage because they clearly don't have a direct representative on a typical executive suite.
Somewhere along the chain of command, these niche roles inevitably report into someone who very likely does not have any lived experience with that discipline. The mismatch can cause all sorts of trouble because not having an appreciation for the difficulty and investment needed in certain kinds of work leads to poor outcomes. One example being when a clueless executive asks a 2-person DS team why can't they just build a system similar to AlphaGo except for their widget selling recommendation system. There's so much explaining that needs go into convincing that executive that the question is nonsensical as-is and then redirecting them into articulating what they really want is a whole journey in and of itself.
So this week I wanted to just note down some thoughts and past experiences on living with a manager that isn't from a background that is quantitative/technical enough to really “get” data science. It's mostly aimed at lower level situations since these conversations get increasingly complex as they happen up the ladder. That is to say, it's one thing when a front line DS has a manager that's an engineering or product manager, and a different beast when it's a DS manager dealing with a eng/product director.
While we are finally in an age where it's not impossible to have a manager that has direct data science or similar quantitative experience, it is still a pretty rare thing. But even if a data science team with managers that have quantitative backgrounds exist, going up the chain eventually you'll find someone that is clearly not equipped to understand the deeper aspects of DS work.
In my experience, not having a DS manager isn’t the end of the world. It works just fine in many many situations. The real issue is that the less a manager understands what things go into making data science successful, the more the data scientists themselves have to step up to fill the knowledge gap to keep things running. This isn’t a problem with experienced folk who can articulate what they need, but can be a potential disaster for less experienced folk.
Some base assumptions about managers
Since managers are merely people, there’s huge amounts of variation in the group. That’s not to mention whether they’re themselves embedded in dysfunctional organizations or not. So to avoid constantly saying “it depends” all the time, I’m gong to lay out a few idealized background assumptions about managers to work with. The majority is just saying that they’re working in a relatively healthy environment where there’s trust and aligned goals within the whole organization.
Obviously, there are plenty of people that don’t fit some (or any) of these assumptions — some of them might even be good managers.
In general, a manager and their reports have aligned goals — that is incentives and goals are largely pointed in the same directions.
Managers are supposed to provide the support, resources, and direction for the team to best get work done. They’re responsible for delivery of the things the team is supposed to make.
If the team delivers results w/in expectations, the manager benefits. If they fail to deliver, the manager faces repercussions.
There’s no perverse situation that might encourage forms of sabotage in either direction.
What can go wrong
Expectation mismatch
The most common issue that comes up when data science gets a manager that isn’t from a related field is they have trouble understanding the work — both in terms of not being able to appreciate the extent of work that is needed to complete a given task, and also not being able to appreciate the difficulty of certain parts of the work. The ignorance by itself isn’t the problem, since such a knowledge gap will always exist when someone from one discipline is managed by someone from another.
Instead, the knowledge gap leads to issues when there isn’t enough communication and trust between the manager and the managed. If the manager is trying to figure out a timeline for a project and doesn’t have enough knowledge of the data science, there’s no way they can succeed in making a realistic project timeline. And if data science doesn’t find a way to provide that information, then they’ll find themselves under unreasonable deadlines and expectations and all the associated stress and dysfunction that comes from such a situation.
Now, a good manager would know that they are lacking in this knowledge and avoid making promises without actively discussing things with their team to come up with reasonable timelines. At the same time, I’ve also seen really bad managers who come from somewhat adjacent fields (like say, a former biostats professor or something) who might have a good grasp of the problem space surrounding the model-building process, but then overestimate their knowledge of the data engineering aspects. So they make promises to other stakeholders that can’t be kept, or start questioning the competency of their reports when t unforeseen (to them) issues come up.
This problem of insufficient communication is often made worse because, when considering how organizations hire and structure teams, the data scientists in this scenario are much more likely to be juniors with relatively little industry experience. Junior members might not know they have to be more assertive about the work that needs to be done, while at the same time they might not have enough experience themselves to make reasonable estimates. This is why junior data scientists are best off having a data scientists manager, or at least a very proactive and understanding non-quant manager.
On the flip side, people with a decent amount of experience, say, 5 years are less likely to have as many issues. They’ve likely seen enough things already to at least have a base intuition of where things are likely to go wrong. They also have more confidence to speak up when something isn’t going right.
Getting the wrong kind of work
A number of years ago, I was working with a project manager with a researcher and after spending a couple of months doing various quant/qual research studies for him. The researcher and I got a bit bored of having the same kind of work requests. So we took a couple of days off of the usual work to explore a question that had been nagging at us about the userbase. That line of research would go on to discover that 30% of users of the PM’s Product A also used another team’s Product B, but only 3% of users of Product B used Product A. And there were a LOT of Product B users and that represented a huge potential market. When we showed this result to the PM, their biggest reaction was “I didn’t even know you researchers did strategic work!”
It’s very easy to get pigeonholed. If you make one dashboard for a team, you can very quickly get thought of as “the dashboard guy” to the exclusion of all other work. If the researcher and I didn’t take it upon ourselves to do some strategic-level work based on our own intuition, we would’ve never been given the opportunity. Imagine if the person who pigeonholed you was your manager. How terrible and demoralizing would that be?
Since a non-DS manager is very unlikely to know the full palette of methods and projects that a DS team can handle, this situation is very easy to fall into.
Most dangerously, a manager can unthinkingly pigeonhole the DS team to a handful of methods or past projects and then keep creating or taking on more work along those lines. Very quickly the team becomes known throughout the organization as “the team that does experiments” or “the dashboard team”. This pattern can extend downwards, in the form of the work given to the DS team. It can also extend upwards, with how the manager represents the team to higher management.
While having a reputation for certain methods isn’t a bad thing per se, it’s a problem when that’s ALL that a team does. This places limits on all sorts of important things. There’s less access to high-impact problems because “that team doesn’t do that thing” even though they totally could. New opportunities, promotions, even budget to expand the team might all become out of reach if pigeonholing gets way out of hand and nothing is done to reverse it.
So again, having strong communication is key. A good manager would again not make assumptions about what a team can do and defer to the actual people who will be doing the work since they likely know best. (Remember how we’re gonna assume there’s no incentive for either side to lie or sabotage the other here. Because otherwise there’s probably a whole book of pathological cases just lurking under the surface.)
What a manager needs to do is bring down the broad questions and problems that upper levels of the organization are concerned about, and let the data scientists figure out is there anything that the team is capable of contributing to. If every executive is currently worried about customer retention, the solution isn’t to offer to build a retention dashboard so that we can all watch it sink together. There are plenty of analyses and studies and experiments that can be applied to understand and improve customer retention so long as the DS team knows that this is the important thing to work on.
One thing to do as a data scientist is to always make sure managers report back the big problems and general direction of interest that’s going on in upper level conversations. Very often, we’ll have tools that could be applied to a problem that no one has realized yet, and it’s incumbent upon us to pay attention and speak up when those opportunities arise.
Avoiding “invisible work”
Certain parts of data science are surprisingly high in toil. Probably the most well-known form of high-toil work is the maintenance of dashboards. Making a dashboard is fairly straightforward, but the true cost of one is in the constant maintenance needed to keep a critical dashboard functioning. The work involved is non-trivial, and can happen with no notice. But the worst part of it is that this sort of work is like keeping the railroads running on time — no one notices or cares until the trains stop running. Maintaining data pipelines, ML models, are other examples of this kind of work.
In general, regardless of managerial situation, it’s super important to make sure that this sort of work is properly budgeted for and talked about — that is, the work is deliberately made visible to all. But it takes on an entirely new dimension when there’s no manager on point to make sure those tasks are represented in larger time/budget discussions.
It seems repetitive, but communication is key
In order for a non-DS manager to be successful at managing a DS team, they need to defer at least some part of their managerial duties to the subject matter experts, the DS team itself. If the manager is aware of their gap in knowledge there and is proactive about it, this often isn’t a huge problem. Everyone can sit down and have regular conversations about what kinds of projects are available, which ones the team can pick up, and what the necessary timelines look like. When unexpected issues arise, that information can get escalated so that no one gets surprised.
But for managers who aren’t as self-aware, there’s more onus on the DS team to push for having these conversations, and that can be a significant struggle in some instances. While it’s not particularly a good thing to have any team struggle to educate their manager about the value they bring and hurdles they face, you gotta play with the managerial hand you’re dealt (or leave).
In such situations, you’ll have to take some initiative and actively help the manager with their managerial duties of finding problems to tackle, setting expectations on what can be done, defining timelines, and escalating issues when they arise. If the manager can’t see these things for whatever reason, they won’t be able to do the right thing on their own. The potential cost of doing these things poorly can be much more damaging, so it’s worth the effort.
That means having a lot more meetings, a lot more conversations, a lot more working to build and maintain a shared understanding of what’s going on, and assumes that both sides are at least somewhat willing to put in the effort. I’ve seen situations where this worked, and have also seen situations where it hasn’t.
It… depends.
But taking on semi-managerial duties isn’t all bad. It’s an important skill to have further down the line if you ever want to make the transition to being a manager. Otherwise, it’ll mean that you’ll be more well prepared for when you do come across a manager that you need to put the extra effort into keeping them up to speed.
Standing offer: If you created something and would like me to review or share it w/ the data community — my mailbox and Twitter DMs are open.
New thing: I’m also considering occasionally hosting guests posts written by other people. If you’re interested in writing something a data-related post to either show off work, share an experience, or need help coming up with a topic, please contact me.
About this newsletter
I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.
All photos/drawings used are taken/created by Randy unless otherwise credited.
randyau.com — Curated archive of evergreen posts.
Approaching Significance Discord —where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord.
Support the newsletter:
This newsletter is free and will continue to stay that way, share it with your friends without guilt! But if you like the content and want to send some love, here’s some options:
Share posts with other people
Consider a paid Substack subscription or a small one-time Ko-fi donation
Tweet me with comments and questions
Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!