As usual, this is a roughly bi-weekly paid post full of half-finished musings of whatās on my mind.
Iām in the last generation that will have memories of navigating in a car using paper maps. In my childhood, since I could read English faster on small road signs than my immigrant parents can at >50mph, Iād be sitting in the passenger seat, map unfolded, pointing out what street to turn on, what exit to take. Later the paper map would give way to printouts of MapQuest directions. Many years later, weād finally get an in-car GPS unit. Now, everyone just uses their phone.
I occasionally think about that not-so-slow transition of society whenever Iām watching new paradigm-changing technology pop up. The release of GPT-4 this week and the ridiculous hype around large language models in general the past few years, keeps bringing it back to mind.
The reason I keep thinking about navigation apps is because it massively changed how people go from place to place, while very few of us even realize how much of a change it was. Itās convenience and overall harmlessness meant we just accepted it. Itās effect is simultaneously pervasive, but also invisible.
But even āharmlessā navigation systems have downsides to them. Now that theyāre ubiquitous and will route around traffic, people have noticed that traffic is being routed to side streets and causing traffic issues because those roads werenāt designed with that kind of traffic flow in mind. Thereās also endless stories about people trusting the navigation system too much and driving into lakes, construction areas, or undeveloped wilderness. These issues arenāt within the technology itself, but issues in how we humans apply the technology to our lives.
Which is why I look at LLMs and worry. Iām no futurist nor visionary, but itās obvious that the technology isnāt going to go away ā weāre all going to be using a descendant of these things in some form in the future. Itās going to take society somewhere and we wonāt know if itās a net good or bad place until we get there. The lasting effect of a technology isnāt usually what it enables individuals to do, but the emergent effects that manifest when huge amounts of individuals use the technology in their lives.
For now, Iām going to put aside the common criticisms of LLMs. That LLMs āhallucinateā and generate plausible bullshit, that they cost a ridiculous amount of energy to train and use, how have soaked up all the biases baked into much of the text available in the internet, and how theyāre are effectively vehicles for laundering plagiarism and exploitation at a scale so large you can forget it happened. Any of those issues alone puts me in the camp of āburn it all to the groundā, but I donāt get to make these decisions.
Instead, Iām just shaking my head at how much the enthusiastic hype for LLM-derived tools is centered around giving people an edge in productivity. Use this tool to do your homework faster! Use it to write those emails you hate writing. Make art or use it to code whole programs like magic! Watch how it can pass exams!
People quickly notice how these things will help them do certain tasks faster. They can see how it gives them a ācompetitive edgeā. But an āedgeā given to a large enough chunk of the population isnāt an edge at all. Past a point it simply becomes a shared burden, like how credit card reward points are a tax upon those who use cash.
Maybe some of you remember when navigation systems only had maps and did not incorporate traffic information. Back then, it was still possible to ābeat everyone elseā if you knew a secret back road. Then, the first adopters of the traffic routing feature would suddenly have an edge ā theyād be routed to the back roads too despite not being locals! But nowadays, since everyone is using the exact same set of systems and data, no one has an edge. Traffic load has been balanced by āThe Algorithmā such that everyoneās trip times are minimized by balancing all the cars across all the available roads. Thereās no real shortcuts anymore. The only people taking slower routes are the ones who donāt want to, or canāt afford to use the tech now.
I quite convinced that LLM-based systems, or their descendants, will follow this general story arc of adoption since the tech is long out of the bag. You can download versions of similar models to run on your local computer if you want. The question is mostly what the world will look like on the other side when the āedgeā is gone. Will there be new and super convenient natural language interfaces to things? Are we witnessing the creation of a natural language compiler? Will society perceive and value ācreativityā differently than we do now? Will we find a way to mitigate whatever downsides we discover?
What I hope is that it doesnāt wind up like performance enhancing drug use in sports, where the logic behind theyāre banned is because if everyone took the drugs, then the playing field is identical as before except everyone now has the added negative side effects of the drugs. Weāre already seeing jokes about the very real possibility of people using AI chatbots to talk to other AI chatbots in an effort to bypass useless machine interactions that stop you from talking to a humanā¦ if we get to that point weāre literally just having conversations with extra energy waste on top. I have no idea how to stop such a tide short of heavy-handed regulation.
In the early 2010s, as a joke project at work, I took 3 years of IRC logs that I had retained from the office IRC server and fed a simplistic Markov chain based language model to imitate how various coworkers āspokeā. It was hilarious incomprehensible most times, but occasionally would create something thatād make us all laugh. I find it funny that a similar concept scaled a gazillion times up is the cause of all this upcoming societal turmoil.
I know that humanity will adapt to things soon enough ā itās not like we have any choice ā but I really hope we get through this current awkward phase of frantic experimentation and figure out what we actually want to keep and what we want to discard.
For now, Iām just going to continue sitting in my little cave here, handcrafting these artisanal blog posts and unhappy that āhandcrafted blog postā is most definitely going to be a marketing term in the near future.