A Non-Obvious Reading List for Thinking About AI
My recent writing about the risks of AI might seem pessimistic. That couldn’t be further from the truth. I’m very excited about AI technology. I’ve used it daily for two years now. My real concern is that we are looking in the wrong places for information to help guide how we think about AI development. This applies to both developers and users, we need to actively think about how to use AI rather than passively accepting its outputs when it works well for us.
These concerns aren’t pulled out of thin air, I’ve compiled them through much of my recent reading. So I figured I’d share some of that reading with you. I’ve cited nearly all of these in recent essays, but wanted to put them all in one place so others might piece together the same puzzle I’ve been mulling for months. I’m grouping them into disciplines. If you are a STEM major, some of the technical writing will be well-known to you. If you majored in philosophy, the philosophical works I’m citing are mostly canonical (though maybe worth revisiting in moments like these!). I’m not on the cutting edge of any of this, but hopefully the cross-disciplinary recommendations will be useful and thought-provoking for you.
Philosophy of the Mind
What Is It Like To Be A Bat? by Thomas Nagel
If you’ve got a public library subscription you can probably access this through JSTOR. It’s a profound 16-page essay that elegantly presents why we can never know what an experience feels like to somebody else. It helped me understand an idea that I think we all recognize but struggle to put into words.
Philosophical Investigations by Ludwig Wittgenstein
I’m linking the book, but you can find copies of this on the internet archive to read for free. This is not easy reading, but it helped me better understand, on a foundational level, how we use language. It also drove me to contemplate the deeper nature of our communication failures. With respect to AI, it helped me better grasp why AI is incapable of truly understanding our words.
The Conscious Mind by David Chalmers
Another relatively dense read, but important nonetheless. I’ve been aware of the ‘hard problem of consciousness’ for some time, but decided to read the seminal text. I’m happy I did. There’s a lot of value in examining the things we do not know, and can’t explain. It illuminates the fine line between knowledge and faith. Most of us believe humans are conscious beings and are fundamentally different from the other beings in the world. It's worth knowing that this belief is grounded in faith, not provable fact. And knowing what we don’t know helps us understand practical problems more completely.
Epiphenomenal Qualia by Frank Jackson
Yet another one full of big ideas. Jackson helped me more deeply understand subjective experience. The famous ‘redness of red’ thought experiment is in this essay. I had not read this, and was not aware of it before I started exploring this space, and I find the ideas fascinating. Once you start going down the rabbit hole of exploring subjective phenomenal experience, it sometimes makes human cooperation seem like a bit of a miracle.
Political Philosophy
After Virtue by Alasdair MacIntyre
This is the first book I turned to in order to better understand how AI could change us. MacIntyre’s view of the future is decidedly dystopian (well, unless we change some things). But his read of human nature and what’s going wrong in modern society will ring true for many. I see AI driving us deeper into this crisis of modernity. We can choose a different path, but it requires commitment to very human, virtuous pursuits.
Eichmann in Jerusalem by Hannah Arendt
Like the Chalmers book, even if you haven’t read this one you might be familiar with Arendt’s concept of ‘the banality of evil’. I found this jarring to read, even if I already had a general sense of it. At pivotal moments, like the one we are in, it is important to remember that true evil is not often committed by cartoon villains. It is the result of cumulative bits of brainless compliance from outwardly normal people. Relatedly, as the digital world advances, each technological advance brings with it the potential for evil. Social media is a scourge that we can’t seem to quit. Targeted ad revenues birth conspiratorial thinking because it keeps the eyeballs on the page. AI has the potential to supercharge all of the bad, and most people are simply unaware of the threat or feel powerless against the inertia of it all. At times like this, we should strive to be less ordinary. We should take a stand and demand thoughtful and human-centered development of what might be the most world-changing technology we’ve ever seen.
AI Specifically
Superintelligence: Paths, Dangers, Strategies by Nick Bostrom
If you only read one book about AI, make it this one. The danger of orthogonality, the idea that regardless of AI’s intelligence it could get fixated on any arbitrary goal, is critically important for both developers and users. If you tell AI to maximize profitability and it fixates on that to the exclusion of everything else, every action it takes and bit of advice it gives will be in furtherance of that goal, for good or for ill. Many of you may have heard of the illustrative hypothetical about an AI optimized on making paperclips. There’s even a fun game related to the idea.
The Best of the LessWrong Forum by various writers
Like Bostrom, the LessWrong community is very concerned about AI safety. Eliezer Yudkowski is probably the most famous writer from the forum, and has been one of the most important voices in the AI safety discourse. He also thinks that ASI (Artificial Superintelligence) will, with a fairly high degree of certainty, cause human extinction. I don’t agree with the LessWrong folks on much of anything when it comes to forecasting our future with AI, but I appreciate their insistence on rigorous argument. Even if you think the doomsday fears are overstated, it’s worth reading their ideas, because it is quite difficult to dismiss the possibility that they are right.
Most of these works are not “about” AI in a narrow sense, but all of them have helped me think more clearly about what’s at stake. If we want to build machines that serve human ends, we need to orient ourselves on what it means to be human. There’s more beyond this list, but these are the ones that have captured my thinking the most over the last half a year. Feel free to message me if you’d like more recommendations.