You should pick your org chart when looking at a position

We rarely talk about this to newbies

Swag Phase 1 is here!

http://bit.ly/counting-swag-01

For now, while I’m behind making a storefront, I’ve slapped together a Redbubble store for some simple swag w/ the “Counting is Hard” logo. Magnets, notebooks, mugs, and tote for now. The prices are…. uneven and that fact annoys me to no end, but they’re just the default rates and markup.

No stickers on there because I already printed out 100 and need to unload them off my own storefront first.

Feel free to yell at me if stuff looks ugly, is priced stupidly, or any other comments you may have. I’m happy to make changes to things and do price corrections.

I promise to get my own storefront up by next week’s post. Seriously.


Recently a friend mentioned that they got an offer to a data science position, which is great for them because they’ve been wanting to switch into the field for a while now. They come from an electrical engineering background and was always quantitatively inclined, but the switch is still a ladder change from actual physical product engineering.

Then, very naturally, my wife (who is not in data, nor tech) asks me “oh wow, $Friend is in the same field as you now, what do you think of that DS job at $Company?” My obvious initial response was “Heck if I know. It depends.” Because, well, it does. $Company has been around for over a century and is a household name, but that almost means nothing since data science functions at any company run the gamut of “horrible” to “great” with very little warning.

You can potentially sniff red flags out by carefully reading job postings and during the interview1, but you can’t dodge every bullet out there. But when people ask me questions about “what should I, as a unique individual w/ this background, look for when I’m searching for a position”, I find myself giving variations of the same answer.

You’re matching your experience and ambitions to the company’s org chart.

What do I mean by this? There’s a concept known as “Conway’s Law”, which says that organizations will design things that ultimately reflect their internal communication structure.

In more practical terms, if your product was made by a distinct backend and frontend team, the friction of communication between the teams means that the product will wind up having a distinct frontend section that is loosely coupled to backend section. The two teams are highly unlikely to create a unified full-stack monolith product, because the teams organizationally aren’t collaborating at the level.

There’s apparently interesting organizational research literature that gives some evidence that some aspects of this is true, I haven’t had time to read through it tonight so I’ll save it for another day.

I didn’t even know about Conway’s Law until the past year or so when I came across it in some random bit of reading. But I suspect that people who have reached a certain level in their careers and have been exposed to enough organizational structures built an intuition about this. You see this somewhat in the fact that people interested in more junior positions are primarily concerned about the direct projects and things they’ll be working on day-to-day, while people deeper into their careers are often interested in who they’ll be working with, both inside and outside the teams.

Obviously both groups are still rooted in self-interest. Junior people tend to see their best value proposition being “I can do this cool stuff for you!”. More senior people often realize that their more ambiguous, complicated jobs requires working with outside teams and don’t want to fight an uphill battle.

Org structures can make things easier, or harder

Very often when I’m giving basic career advice to people, I try to judge how people along a few vectors so that I can personalize what I say to some degree.

One obvious vector is the skills vector. If you have lots of tech/stats/research skills, you have more options because you meet the hiring bars for various places. The specific balance of those skills tends to affect where a person can go.

Another is the general business acumen vector. Can you take vague business requirements and turn them into projects and priorities? Can you speak competently to other teams and make yourself understood? Can you understand what other teams want? This stuff isn’t really measurable in the same sense as skills, you can only guesstimate it by seeing how a person handles various situations.

But a less obvious is a “how much organizational bullshit can you handle?” vector. Do you know when and how to push back on certain requests? Can you weather your 50th team restructure? Can you overcome barriers between teams? Do you know how to help make data science (or whatever your team is) a respected/valued contributor when the perception is the opposite? Do the words ‘office politics’ scare you?

Having a good organization, which often implies having a good org structure, cuts down on a significant amount of the bullshit that needs to be handled, as well as making all the other things easier. But the higher up the ladders you go, the more you’ll have to deal with those issues. This stuff is extremely difficult to teach, direct experience and observation are the best teachers here.

Even if a company is perfectly cohesive and non-toxic, there still needs to be priorities set, debated, and balanced. “Organizational stuff” is a blanket summary of all those activities and more.

This last vector tips the scales in my brain for what places I can recommend. Someone who is fresh out of school and hasn’t been exposed to organizational challenges has very low chances of succeeding if they’re thrown into a solo data scientist position without someone to help them learn that stuff, like a skilled manager. Without support, they’ll make mistakes, be unable to find support and allies, focus on the wrong work, and generally get demoralized.

Meanwhile, someone who’s really good at handling all the organizational stuff can go almost anywhere they want and stand a fighting chance of doing well. Even if their technical skills might be weaker than average, they can “play the game” enough to avoid nasty pitfalls.

New people should look for solid teams

So my advice to people who don’t have confidence in their ability to juggle the ambiguity inherent in being the solo data person is that they should find a team to learn from.

It doesn’t have to be a big team, in fact having just a really awesome manager and no one else is enough. But people starting out need someone to provide the coverage and guidance to navigate those waters. This way they can learn those skills by watching things unfold.

So when searching for jobs and interviewing, put a bit of extra energy to probe who the manager is. Ask how much experience they have managing data scientists or similar quantitative roles. How do they help the team prioritize work? How does their team fit within the greater organization.

Next, applicants need to look at the teams they’re going to be working closest with. Taking Conway’s Law a step further, the problems you’re most likely going to face are going to be from the teams you speak to the most. If you’re working with the finance and legal teams the most, you’re naturally going to learn their needs and view of the world most.

Is that desirable to you? If you like to work on products, do you get to work directly with project managers and designers? Or are you throwing reports over the wall to them? How’s do teams engage with yours? How are responsibilities like implementing tracking get shared?

More experienced people can pick what challenges they want

Experienced folk have some advantages when it comes to picking out org structures to work under.

First, they have a sense of what works well for them already. If they have deep experience working with engineering, or customer support, whatever, they can choose positions where they can use that experience to bridge a gap that exists to those teams and thus take leadership of it. Or they don’t want to do that bridging work and want to leverage an existing relationship that’s already been built.

Second, they’ve likely seen examples of things that didn’t work and are aware of organizational failures in the making and can pick their battles. If the job involves fighting to make the company be data-driven but the other teams are resisting, an experienced person will know if that’s a fight they want to deal with. I’ve seen such efforts take multiple years, and you should go into those efforts with that in mind.

These problems aren’t things that a junior position is normally expected to have to deal with to begin with, though I’ve seen job postings that amounted to as much.

If you’re up to those sorts of challenges, and those challenges inevitably exist to some degree, the org chart is often a vivid reflection of those issues. Teams that have different reporting chains (and thus slightly different priorities) are very often a source of difficulty. Someone looking to throw themselves into the gap between the two teams (because that’s the job they’re applying for maybe) would do themselves a favor by asking about those pre-existing relationships beforehand.

Org is not destiny

Just because two teams don’t currently communicate with one another doesn’t mean that has to be true going into the future. Some of my best work came from talking to folk from faraway teams and helping them out.

Org charts are physical manifestations of lines of communication and priorities, but informal lines of communication also exist. Just like how many parks and campuses have well-worn desire paths that link places together despite the lack of pavement, it’s up to the individuals to shape their communication universe.

So yes, you can go into a situation fully intending to break through organizational barriers and bring about new and amazing things. There just has to be a lot of energy devoted to creating the connection and building it up.

Then, if you succeed, who knows what can happen. Maybe another re-org to either formalize the relation…. or throw a wrench into it.


Other random updates

It’s Monday, the day before this goes out, and my gem faceting machine arrived all the way from Sri Lanka! Expect a bunch of tweets and photos and comments about it as I slowly muddle way way into another skill.

I also really wanted to get back into piano after ~15 years and got a MIDI controller to tide me over until I have room for an actual piano. Audio tech has come such a long way since I was a kid in the 90s…

Anyways, I suspect that one, or both of these topics will weigh on my brain sufficiently that they’ll show up in my writing at some point in the future.

I promise to make it interesting. But you’ve been warned.


About this newsletter

I’m Randy Au, currently a quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. The Counting Stuff newsletter is a weekly data/tech blog about the less-than-sexy aspects about data science, UX research and tech. With occasional excursions into other fun topics.

Comments and questions are always welcome, they often give me inspiration for new posts. Tweet me. Always feel free to share these free newsletter posts with others.

All photos/drawings used are taken/created by Randy unless otherwise noted.

Share

1

Emily Robinson and Jacqueline Nolis devote a chapter to applying to and interviewing for DS positions in their great book Build a Career in Data Science. They cover spotting various red flags there.