Around the time that I turned twenty I packed my bags and moved across the country. For some odd reason I had decided my PC couldn't make the journey. With the little bit of cash I had left I built another from old parts found at local thrift stores. It was at this moment that I decided that I would no longer use Windows. Not just as my primary operating system; I didn't want it installed at all. I was tired of Microsoft's ever declining quality and customer hostility. I had been dual booting Ubuntu on my old rig and figured I could handle the transition. Why not? It was, of course, the year of Linux desktop. Every year is the year of the Linux desktop. I had been reading countless praise on forums about how easy it had been for my fellow nerds to make the permanent switch. I made the commitment, installed Ubuntu, and have been happily using that same install ever since.
Except that's not at all how it worked out. Several devices, several distros, several years, and several crash outs later I have given up. I'm back on Windows despite the fact that it has seemingly never been worse and Linux has seemingly never been better. Why? It's simple, there has always been issues and I just want my computer to work. I was tired of tinkering and I think that says a lot because I'm a tinkerer by nature. I want to tinker by choice not because I need to figure out why my screen share is completely non-functional before an important meeting. I could spend years growing to love my desktop only for it to brick itself because of one bad update. Should I have spent more time trying to learn the arcane art of Linux system recovery? Sometimes I would try, but the complicated tangle of system and user configuration always became an unmanageable nightmare. It was easier to kiss my beloved goodbye and try another distro. The feeling was very bitter sweet. A new toy to tinker with, but I'm tired of tinkering boss.
At this point, if you're a mega nerd like me, you're probably having one of two reactions. You either understand where I'm coming from or you're getting ready to send me an angry email explaining how I'm obviously the problem. I promise you that I've considered that. I spent many a sleepless night wondering why it's been so hard for me and seemingly so easy for those on the forums. Am I a fool? Are they lying? Even now I'll admit that I don't know, but I am ready to make some confident statements about the truth regardless.
I think that there is an unfortunate truth in our software. Software has long been beyond our understanding. Linux makes a particularly good example. It's comprised of code written by tens (hundreds?) of thousands of individual people. Each module is complex and both tightly and loosely coupled to all of the others. The proliferation of distros and their endless customization makes us users feel good, but it is a fragmentation nightmare. I would wager that the typical Linux desktop drifts into completely unique software stack territory a couple of years after it is created. If your Linux desktop is completely unique, how am I supposed to understand it? How can I possibly write code that lives in harmony with it? Standards are great but they only go so far.
I'm not exactly breaking new ground here. In the late 1960s there was talk among programmers of a "software crisis". Computers had been growing, as they tend to, exponentially more capable. It should come as no surprise that as we are able to do more we seek to do so. Project scope grew at the same exponential pace, completely disregarding the fact that the programmers could no longer understand the systems they were building. The problem got so out of hand that NATO, yes really, coined the term software engineering and held two conferences on the subject. While the collective efforts of the world helped put the cat back into the bag, it would only remain inside for so long. Computers would not stop their exponential progression any time soon. Dijkstra, while receiving his 1972 Turing award, gave his The Humble Programmer lecture and captured the essence perfectly by saying: "To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem. [...] as the power of available machines grew by a factor of more than a thousand, society's ambition to apply these machines grew in proportion, and it was the poor programmer who found his job in this exploded field of tension between ends and means. The increased power of the hardware, together with the perhaps even more dramatic increase in its reliability, made solutions feasible that the programmer had not dared to dream about a few years before. And now, a few years later, he *had* to dream about them and, even worse, he had to transform such into reality! Is it a wonder that we found ourselves in a software crisis?"
I would highly recommend that you read the full lecture. It is incredible just how clearly he could see the patterns of software and computers. I suspect that you too can see that this pattern has played out over and over again. While I have argued that Linux is experiencing something of a software crisis, I think I should address the elephant in the room. You, me, and even my grandma have heard that AI is beyond our understanding. It's true that we don't know how exactly an extremely long series of mathematical operations can hold a sensible conversation. We do have high level understandings. We can point at some vectors and say "this here is its representation of a dog, we got it by subtracting dog thought from not dog thought." How meaningful is that in the face of fantastic emergent capability though? This is a new type of software that can grow its own complexity just as fast as we can grow compute. It is far beyond our understanding and I'm not going to tell you that isn't worrying. I just want you to know that we've dealt with these software crises before and things turned out well. It's not going to be easy. Not all software crises have the same shape. The issues I've had with Linux may be solved with a little more coordination and rigor just like the 1960s. With AI we're going to need to develop new tools and techniques. As Dijkstra said, we must both dream of things we never have and make them reality.