Every org has a "literature"
Every org has a "literature"
And it's a giant mess
Last week, I was asked an interesting question — What do you do to retain and share knowledge and insights that your team’s learned?
I didn’t have a good answer to this, and it bothered me. You’d think that a group of researchers working at a megacorp whose main product to the world is providing information in an easily searchable and usable form would understand this problem more than just about anyone else… but it doesn’t appear to be true.
To provide some context, as a Quantitative UX Researcher working on a UX Research team, our findings are typically released in the form of slide decks or documents. We get a set of research questions, go forth and do our thing to answer the questions with Science, then come back filled with knowledge and deep insights to share with the people who originally requested such work. Since everything eventually needs to be presented, a lot of the insights wind up living in a slide deck. The question was, if everything lives in a slide, then how can other people find that knowledge and put it into their minds over the long term?
My unsatisfying answer is that… we don’t have a particularly good way. Sure, all our docs and slides are in “the cloud” so a randomly broken laptop can’t destroy expensively acquired knowledge, and those decks are often already shared out to a whole host of people who can find it using various search methods. We even have an internal site thingy that you can upload slides and apply metadata like tags or short descriptions. The site’s intent was to become the one-stop-shop of research insights where people dump their findings and other people in the future can locate them to use.
I think anyone who has used any such similar concept of a system can see where I’m going — yes, the system works and people are using it, but to what most people would say is to “limited success”. Not everything that should be in the tool makes it in there, and some people put more stuff in than others.
I mean no disrespect to the people who think hard and work on the tool, because in my personal experience, ALL such tools are essentially met with limited success. If any tool has ever solved the “knowledge collection/sharing” problem, we would’ve all heard about it and be raving about using it already. It’d revolutionize not just industry knowledge, but academia as well. That obviously hasn’t happened.
What’s currently broken
Everyone who has worked in any size organization has probably come across a facet of this knowledge problem before, but to articulate clearly for the purposes of this post, it’s a single problem with two sides.
First, there’s a knowledge storage/management problem. Things that people learn over the course of their work and studies first gets created within individual’s minds, then the knowledge needs to be materialized into the world and shared with others. There are a host of issues already, from convincing people to spend the effort to write down stuff they know, to organizing all that knowledge, and making sure it keeps up to date. It’s an expanded set of problems we see with code documentation, and we all are pretty horrible at even that limited scope task.
Second, there’s the knowledge retrieval problem. If the first problem is making sure people take knowledge from their brains and materialize it into the world for others, the reverse is also a problem. How does a person find the knowledge they need and ingest it into their brains, when they might not know where it is, or what it’s even called?
In fact, I experienced this exact issue when I was attempting to do research on the topic, I knew to my very bones that this whole topic was definitely a field of study and very smart people have thought for decades upon this problem. But I had couldn’t figure out what the field, or its constituent problems was even called, so I couldn’t search for papers on the topic. Ultimately I had to ask Twitter and someone knowledgeable suggested I check out terms like “Knowledge management”, “Succession planning”, as well as browse through parts of “Information science” and “Library science”.
Putting both sides of this knowledge seeking coin together, we start seeing the full scope of what this “knowledge collection/sharing tool” is supposed to help solve. No wonder it’s essentially doomed to “limited success”, because the scope essentially covers the entire human academic experience. We solve this and we “solve” academia too.
Our local “literature”
The reason I claim that solving the problem is effectively “solving” academia (which is a pretty meaningless statement since it won’t solve scientific inquiry), is because academia has figured out how it, as an institution, will solve this knowledge problem over the course of centuries of refinement. It’s likely not the ideal solution, but it is the one currently in production.
In academia, every field as a concept of “the literature”. It’s all the academic papers published within a certain area — a specific problem, or a whole field of study. Papers, a.k.a. knowledge, “enters” the literature by being published through peer review, and then other academics read it and maybe cite it in their own work. If the paper stands up to scrutiny, it spreads and becomes accepted and built upon by more and more other academics in the field. By then, the paper has become part of the whole conversation, “the literature” of that field of study.
Note that modern academia built itself around a system to make this work self-sustaining. People are incentivized to publish, to materialize their research out of their own minds, because there are highly valued rewards set up to encourage participation — jobs in and outside academia, tenure, social status, even prizes like the Nobel Prize. This takes care of the knowledge production and collection side of the problem.
On the other side of the coin, there’s an entire multi-year apprenticeship dedicated to training people to engage with the literature — we call it grad school. One of the primary functions of grad school is to teach students what the literature is and how to work with it. All the seminars reading the foundational papers of the field, the massive work needed to do the dreaded “lit review” for your PhD dissertation, MS thesis, or any other peer reviewed paper. It takes multiple years of study before someone has read and learned enough to even know where the current state of the conversation around an area of study even is. By the time you’re through the training process, you will have learned what the names for the interesting open problems are, what terms are used, what journals are worth reading and publishing in.
Now take a moment and look at your industry job.
Does ANY of that exist?
No? Doesn’t for me either.
Onboarding into a new job lasts somewhere from 6-12 months, not years of dedicated study time, and you’re working the whole time. There you’re faced with trying to find all the stuff that has been written down, while also figuring out how to access all the more important stuff that people haven’t bothered to write down — what some call “tacit knowledge”. All signs point to the tacit knowledge part vastly outsizing the explicitly written down knowledge.
Since all that knowledge that we want to access, effectively our “local literature”, lives primarily in people’s minds, there’s no strong incentive to documenting and materializing knowledge out. In fact there’s a disincentive because we all supposed to be working, not writing down stuff we “already know”. So the incentive structure is all broken. Hell, there are still tragic individuals out there that believe the whole “knowledge is power” thing and hoard knowledge jealously.
Plus, the little pools of local literature are very small and insular. It’s only accessible to employees, and current ones at that. If someone leaves, they take whatever knowledge is in their head with them unless there’s a dedicated, labor intensive effort to extract and record it (this particular problem is what succession planning attempts to solve).
In fact, in every place I’ve worked, the best way to get at knowledge is to find a large room of people, describe what I’m trying to do and see if anyone happens to have a bit of knowledge to share. It’s about as effective than just about any massive searchable database of papers simply because it works around the fact that I don’t know the keywords to search for.
With that kind of background, is there any surprise why everyone I’ve ever spoken to feels frustrated at the current state of the industry world? There’s rarely any incentives to put the effort into writing down knowledge and sharing it out, there’s barely a system to let people absorb info when they onboard, and the literature is often a tiny island that might have no relevance to other organizations. About the only time I’ve had to engage in heavy knowledge documentation was when I was either leaving a position and was trying to share what I know out, or I was preparing to onboard a new hire and needed to summarize the state of the world to them.
My brief scan of the knowledge management literature also found a number of mentions about how incentives are important to whether people will adopt such systems, which isn’t a surprise. We see the same discussion about how there’s no incentives to encourage good documentation of code either.
You sometimes hear stories of certain organizations that put a large amount of resources into succession planning and internal talent development, where there’s a chance this work is being done because gathering up that knowledge and making sure it is shared is a necessary step in people development. They’re notable because they’re rare, and that sort of resourcing only seems to be dedicated to the upper echelons of the hierarchy. It’s worth it to make sure the next executive has all the knowledge they need from the previous one, but it’d never happen for the new intern.
Enough talk, do you have a solution?
Not really, because if I did, do you think I’d be writing data newsletters for free instead of making money selling the solution to a centuries long problem that every organization on the planet faces? As far as I can tell, no one in the academic literature has found or observed a strong solution in practice either.
But I do think we should look to academia for some inspiration. We want to create a similar self-sustaining knowledge gathering and sharing system. We want incentives that reinforces the behavior.
Academia does it by tightly coupling the job with knowledge creation, sharing, and consumption — my gut feeling is trying to directly replicate this in industry is a bad idea since industry is already directed towards making money. Having two unrelated incentive structures compete will just cause conflict and confusion, so there’s going to be lots of delicate priority-setting involved.
There also needs to be incentives and time given to acquiring knowledge, which is can be difficult in the hyper-speed of modern work. This is largely a cultural issue in that this time is a necessary part of the work, but most employers don’t admit it up front. If it takes 5 days to do the background research to get up to speed, it needs to take that long without penalty.
Tech isn’t the full solution
I literally work at a search engine company and this problem of searching out research findings hasn’t been fully solved. I’m not holding my breath for big innovation to fix the issues.
Instead, to my dismay, I suspect that the current mishmash of using tech to provide search and storage (effectively, a long memory) and leveraging humans to help other humans navigate that memory is the best we have thus far. It’s highly unsatisfying because the “best” still represents a very crappy user experience. It’s slow, inexact, lossy, and requires time and energy from everyone involved and we wish the machines can do it for us.
Perhaps one day, but not today.
About this newsletter
I’m Randy Au, currently a Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. The Counting Stuff newsletter is a weekly data/tech blog about the less-than-sexy aspects about data science, UX research and tech. With occasional excursions into other fun topics.
All photos/drawings used are taken/created by Randy unless otherwise noted.
Supporting this newsletter:
This newsletter is free, share it with your friends without guilt! But if you like the content and want to send some love, here’s some options:
Tweet me - Comments and questions are always welcome, they often inspire new posts
A small one-time donation at Ko-fi - Thanks to the folks who occasionally send a donation! I see the comments and read each one. I haven’t figured out a polite way of responding yet because distributed systems are hard. But it’s very appreciated!!!
I used to work at the research lab for Shell Oil in the US. We used the academic model - though it was beginning to break down as the dreaded PowerPoint began to dominate. I also published in an internal technical magazine distributed company-wide that was less formal but had wider reach. But all that takes time and dedication, and management support. Internal conferences drove much of the creation of documents - you need to give a paper at the conference, a paper in addition to the presentation is required, so a paper gets generated.