Attention: As of January 2024, We have moved to counting-stuff.com. Subscribe there, not here on Substack, if you want to receive weekly posts.
A month ago, someone at work pinged me saying they thought it’d be a good idea for me to submit a talk to PyData NYC since I was a local and they thought I was a good fit. I took a brief look at what previous talks were like and they leaned towards the code-heavy technical side. Whelp, I figured that I might not be the kind of data nerd the audience is expecting, but I’m what they’re gonna get. So I submitted a proposal that’s probably very on-brand for me, given what I write about all the time here.
“Solving the problems in front of you”
I can’t spoil the talk since I’m flat in the middle of writing it this week, but since I use my writing as a way to think, it’s inevitable that the whole thing is dominating my thoughts. Thus, sadly, this week’s post is a short one due to lack of spare brain cycles. I’m also writing it super early because I have to submit it at work for review to make sure I don’t share something I’m not supposed to. Unlikely, but you never know.
The inspiration for my talk came from primarily looking at all the other talks and thinking how they’re all big, complicated, coding projects or large scale projects for business. For me at least, the biggest hurdle to finishing a “big” project is usually starting. Specifically, the step of going from a blank sheet of paper to finalizing a design to actually execute. I have a tendency overthink my plans and designs a ton before deciding to do something — endlessly spinning through what-if scenarios trying to make sure I can anticipate where things are likely to go wrong.
As an example, it usually takes over a year of slow immersion before I even take up a new hobby. Even something as “simple” as a vacation trip involves me overthinking the logistics of how much luggage the family will be bringing, what modes of transport work with that setup, what times of day work so we don’t get stuck in the middle of nowhere, etc.. This habit of overthinking tends to translate into me not willingly planning (nor going on) vacations much because it’s so much work.
This stands in contrast with the surprising amount of success that I have found when I effectively go “screw it, let’s just do this” either because of deadlines or frustration. It’s the equivalent of booking all my flights and hotels, but leaving the problem all the other logistics like getting supplies, local travel, and food, for my future self. It’s less planning than I’m comfortable with doing. It’s trusting that my future self can figure things out amongst the chaos that is traveling. I’m pretty sure it only works because all my prior research puts enough half-formed ideas into memory that it helps find solutions to issues faster.
Anyways, the talk riffs on this core pattern of dancing between over-thinking and not thinking “enough” (loaded term), and exploring things with examples from the various companies I’ve worked for. That includes both tiny (<25 employees) to gigantic (>150k) orgs, and across various project risk tolerances from toy throwaway models to “the critical data warehouse that all reporting everywhere will use”.
Decision problems are always fascinating, especially under imperfect information. This is just another one. There’s balances to strike. There’s trade-offs to make. Whatever solutions exist, there probably needs a healthy does of “screw it, let’s go.” involved. In my head at least, it sounds like a fun line of experience to explore. Ideally, if I play my narrative right, I can avoid shrugging and saying “It Depends”. Maybe.
As with all my writing, I’m sorta hacking the story together as I go and it takes me down paths I haven’t really considered. I’m about as curious where it’ll wind up as you might be.
Anyways, if you’re going to be at PyData NYC 2023, come find me and say hi! Seriously, I mean it. I’m the sort of person who can go through an entire conference, without saying a word unless spoken to. So meeting some friendly faces would be very enjoyable to me. And if you can’t attend, I think there are recordings available later. I’ll link to anything that gets shared when they’re made available.
Standing offer: If you created something and would like me to review or share it w/ the data community — just email me by replying to the newsletter emails.
Guest posts: If you’re interested in writing something a data-related post to either show off work, share an experience, or need help coming up with a topic, please contact me. You don’t need any special credentials or credibility to do so.
About this newsletter
I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.
All photos/drawings used are taken/created by Randy unless otherwise credited.
randyau.com — Curated archive of evergreen posts.
Approaching Significance Discord —where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord. We keep a chill vibe.
Support the newsletter:
This newsletter is free and will continue to stay that way every Tuesday, share it with your friends without guilt! But if you like the content and want to send some love, here’s some options:
Share posts with other people
Consider a paid Substack subscription or a small one-time Ko-fi donation
Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!
What comes to mind is Niklaus Wirth's programming language Modula 2. He first wrote it, threw it away, and then wrote it again - on purpose. I seem to recall he wrote something like he never got it right the first time so he just decided he would write it twice to begin with. I am also an OCD planner type, but have managed to "just do it" while also planning to likely toss whatever comes out and start over. It's always so much easier the second time.