Reflecting on our toolbox mastery

And is it "enough"?

Attention: As of January 2024, We have moved to counting-stuff.com. Subscribe there, not here on Substack, if you want to receive weekly posts.

This week, the whole house seems to have come down with some stomach plague brought home by the kiddo so things are a bit shorter and, dare I say, a bit more hallucinated than normal. Hoping things clear up later this week.

A lone little seed still attached to the tree

There’s some corkboard on the inside surface of one of my kitchen cabinet doors in a convenient spot for tacking on recipes and stuff while working. It’s probably 30 years old at this point, and one day the glue bond on the back just failed and the whole panel of cork and backing board fell off.

No big deal. Since I effectively live in a mini hardware store of collected tools at this point, I grab some wood glue and small clamps. Plan was super simple — dump some glue on the back, press it back into place, clamp it down until it dries. Except my clamps didn’t reach far enough from the door’s edge to get a good grip on the panel to keep it in place. Any pressure I could exert was uneven and made the whole board lever out of place. Getting some strips of wood to try to distribute the force better also didn’t work because I didn’t want to bother taking the door off to get clamps on both sides.

The solution? Grab a roll of blue painter’s tape and use that to hold the cork in place long enough for the glue to dry. “Tape clamps” is a common trick for holding awkwardly shaped wood bits in place long enough for glue to dry and do the real work of holding a piece together. The trick works great for many tough situations, but at the same time, it’s not the right tool for most jobs — you have to understand what you’re doing well enough to know when its appropriate.

Knowing “the right tools” and other tools

Tools in life are interesting things in that most problems allow for multiple methods for accomplishing something. My kitchen corkboard could have been attached to my cabinet door with small nails, or some screws. I could've even used different types of glue like hot melt, adhesive tape, or maybe cyanoacrylate super glue. I could've even used holes and hooks to make the setup removable instead of permanent.

But each method and tool for accomplishing the job involves making different tradeoffs about the effort I’d have to expend to get my end goal of “have a working cork board on my cabinet door” done. Some methods might be more susceptible to falling off soon. Different glues have different strengths and physical properties. Using nails would require hammering nails into a hanging, moving, barely supported cabinet door. Screws would need to be short enough to not punch through the door wood, while also being capable of gripping crumbly soft cork.

The lesson that you learn from working in other spaces is that there’s rarely one correct answer for “what tool do I use here”. There’s often a small set of choices that will get the best results possible, then another set that will get the job done to varying levels of acceptable quality, finally a bunch of choices that don’t work or will make a horrible mess of things. Part of mastering something is learning how to navigate those choices.

And so, I look at myself and my data science work and wonder to what extent have I learned to navigate the ever-changing solution space for various tasks and how’s that stack up to everything else.

When I started writing this post, I had originally started thinking about my skills and tool usage and wondering how that stacked up against the rubric of “do I use X well enough to know how to pick the right use for a situation”? But as I started thinking about the items individually, it was… really vague and hard to work with.

Essentially, I was poking at the edges of expressing a “measure” of skill or mastery over tools… and the more you think about it, the more impossible the task sounds.

Expressing depth of a skill is downright impossible

Take something that most of us data scientists are experts in (relative to people outside the field). We’re generally have to be very good at “cleaning data” because we’ve been forced to do it so often to get our work done. We can apply our experience and knowledge to massage data into a workable form. We know what hacks and corners can be cut to get our job done without doing it “properly”.

But think about how you would express that to anyone else. How would you talk about it on a resume, job application, or in a newsletter post that goes out to other data scientists. I don’t think I can even describe HOW I go about cleaning a pile of data, so I’d be equally unable to tell somehow how well I know any other tool or method. It’s all just a mishmash of examples and experiences in my head that are summoned when similar situations trigger neurons in my brain.

As another example, think of how you know whether a plumber or electrician is “good” or not. At most they have a special license that shows they meet certain qualifications set by the tradespeople and government. But even amongst the licensed tradespeople, there’s a huge gap between the really skilled ones who do quality work, and the ones who are willing to cut some corners for a quick profit. There’s little outward metric that captures how they the degree of skillfulness they express in their skills. At most you have to look at their work after the fact and make some judgement based on that — long after the initial hiring process.

Early on in my career, describing my skills was relatively easy… I didn’t have ANY skills of note, so I just threw all the tools I’ve touched onto my resume. I’m pretty sure “MS Office” was on there right next to Python (look, this was the mid-2000s). They were acting as checkboxes since everyone involved, including the interviewer, understood that any skill presented was largely a surface-level understanding.

Showing outcomes as a different measure

Nowadays, I have to do what a lot of mid-career people do to express how much expertise we have — we tell stories about outcomes. There comes a point when it doesn’t matter whether you’re using MS Word or a Jupyter notebook, you somehow need to use a bunch of your available skills and tools to make something happen. The implication is that I must have applied my skills and tools appropriately in order to achieve the outcome that actually occurred. Moreover, not all of involves just the “hard skills” of tools and methodology, but also all the other important skills of communication, organization, leadership, etc.

People treat outcomes as a proxy for skills. But just like anyone who’s hired a contractor with nice recommendations and pretty photos of past work can learn that said contractor can still be a nightmare to work with, the outcomes, tools, and skills are only vaguely correlated. Interviews that used to skew slightly towards people who can answer trivia about quirks in tool minutiae will, when applied to more senior people, skew towards people who can tell more compelling tales about their supposed impact.

So this whole line of reasoning makes taking stock of our own mastery of skills and tools pretty moot. There’s rarely a good measure, and the proxies that we use are far too crude, while no strong direct measure exists without forcing someone to just do actual work.

So, after all this spinning wheels around understanding to what degree I can talk about how skilled I am, I’ve just wound up exactly in the same position as a “how do you write your resume/do an interview” post. =\

But at least it’s somewhat more reasoned than prescriptive? I guess?

😕

---

Standing offer: If you created something and would like me to review or share it w/ the data community — just email me by replying to the newsletter emails.

Guest posts: If you’re interested in writing something a data-related post to either show off work, share an experience, or need help coming up with a topic, please contact me. You don’t need any special credentials or credibility to do so.

---

About this newsletter

I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.

All photos/drawings used are taken/created by Randy unless otherwise credited.

randyau.com — Curated archive of evergreen posts.
Approaching Significance Discord —where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord. We keep a chill vibe.

Support the newsletter:

This newsletter is free and will continue to stay that way every Tuesday, share it with your friends without guilt! But if you like the content and want to send some love, here’s some options:

• Share posts with other people

• Consider a paid Substack subscription or a small one-time Ko-fi donation

• Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!