Some interesting things to read the last weekend in June

Dear Friends,

The most interesting tech essay I’ve read this month is Leopold Aschenbrenner’s “Situational Awareness,” a many-part manifesto about what he sees as the future of AI. Aschenbrenner is a former researcher at OpenAI, and the 165-page document is a primer on what will happen to the world if current trend lines continue: we’ll soon have Artificial General Intelligence; that will put into a motion a process whereby AI is actually able to do AI research, and at a much faster rate than humans; then we’ll have super intelligence; then we’ll have some serious problems. “The small civilization of superintelligences would be able to hack any undefended military, election, television, etc. system, cunningly persuade generals and electorates, economically outcompete nation-states, design new synthetic bioweapons and then pay a human in bitcoin to synthesize it,” he writes.

It’s an easy essay to dismiss, and you might be tempted to dismiss it on page one when he smugly declares, “There are perhaps a few hundred people, most of them in San Francisco and the AI labs, that have situational awareness. Through whatever peculiar forces of fate, I have found myself amongst them.”

But push through that and read his analysis. It’s consistently thoughtful and well-structured, with each section taking another step from the known to the unknown. I was particularly interested in the section on superalignment: the thorny problem of how you make machines that are more powerful and intelligent than humans conform to human values and needs. The systems of control that we have now, namely RLHF, won’t scale. Perhaps, he argues, the way to do this will partly be through better interpretability (i.e. the depth of our knowledge of how these AIs make decisions), as I wrote about in the last newsletter; perhaps it will come through having advanced AI models consistently testing their successors and telling humans where the new AI has advanced and where it still has blindspots. A human working in concert with a model may be able to align the next-generation, more powerful model. Aschenbrenner is also not wrong that vastly more attention needs to be paid to securing the secrets of the most advanced AI labs. 

The principle counterargument is that the trend lines of bigger and more powerful AIs are relatively unlikely to continue. Just because GPT-4 vastly outperforms GPT-3 doesn’t mean that GPT-6 will be that much better than 5—or that we’ll have the energy, processing power, or capital investment to support it. His analysis of geopolitics and how AI might assist and encourage authoritarian governments is extremely reductive. But you don’t have to agree with something to be intrigued by something. And if you don’t want to read, you can also listen.

I also enjoyed reading the absolutely bananas essay in the journal Alta about the cult, and seeming concubines, of the New Age icon Carlos Castaneda. I had a girlfriend in college who read his books, but I had lost track of his work after many scholars came to doubt the veracity of the shamanistic experiences that he describes. But there’s now a tangled mystery of what happened to six women who were following him when he died, and who may have ended up with a large part of his inheritance.    

In lighter news, Time has a profile of my favorite male sprinter, Noah Lyles. How can you not like this guy? “His house doubles as a dork shrine. Games are stacked near the entryway: Catan, Magic: The Gathering, the Chameleon (Lyles hosts weekly game nights). Upstairs, short, stout figurines-—called Funko Pops—of characters from The Office line one shelf. Lego Bowser, playing a piano, sits below them. Lyles hasn’t opened his Princess Leia Lego set, but a Lego Star Wars Star Destroyer takes up prominent coffee-table space in the upstairs TV room.” Sticking with track stars, I recently had the pleasure of interviewing the celebrity Gabby Thomas, my favorite female sprinter. Among other things, she told me that her mother would rather Gabby earn a PhD than win a gold medal in Paris, and that she has never run a full mile in her life.

Sprinting is a young person’s game, but it’s important to remember that you can thrive at things you start late in life. I was impressed, too, at the reporter Jamaal Abdul-Alim’s efforts to track the gun used in a tragic shooting of a thirteen year-old girl in 2018, and Brian Stelter’s deep investigation for The Atlantic  into the biggest story in journalism right now. 

This section is sponsored by Elastic, the Search AI Company. Elastic enables everyone to find the answers they need in real-time, using all their data, at scale. Its solutions for search, observability, and security are built on the Elastic Search AI platform—the development platform used by thousands of companies, including more than 50% of the Fortune 500. Visit here to learn more.

The legendary editor of The New Yorker, William Shawn, was a quiet, soft-spoken man who nevertheless imposed his will on the magazine. There were certain words that he hated, and woe to any writer who included them in a manuscript. Some of the banned words could seem quite innocuous, such as “massive.” One day, two wits at the magazine, Nancy Franklin and John Bennet, devised a sentence to remember all of the outlaws: “Intrigued by the massive smarts of the bright, balding, feisty, prestigious workaholic, Tom Wolfe promptly spat on the quality photo located above the urinal.” (Wolfe features in the sentence because he had written a takedown of Shawn’s New Yorker titled: “Tiny Mummies! The True Story of the Ruler of 43rd Street’s Land of the Walking Dead!”)

I bring up this Shawn story to illustrate a knowledge-work problem: the bewildering thicket of lingo and terminology that greets anyone on their first day at a new job. During my own early days at The New Yorker, I had to learn what “A-issue” meant and what went into “The Well,” not to mention to coöperate and always add a hyphen to teen-ager. How nice it would have been if there had been a William Shawn AI that I could query and have my questions answered? The preservation and transmission of institutional knowledge is a knot that AI search is working hard to untangle. Just take a peak at any company’s shared Google Drive and you can grasp the absurdity of the situation.  (While we’re at it, please read the memoir by Mary Norris, the beloved New Yorker copy editor who taught me many of the rules over the years.)

There are prosaic orientation problems that AI has already solved: a chatbot that can inform you of work holidays or the company’s policy on parental leave. Things get interesting when you start to imagine an onboarding AI that can explain strategic initiatives or that can tell you where the business has thrived and where it has faltered. Let’s say you joined a car company as a new designer: the AI might have sifted through all of the customer feedback on a car model and presented you with a list of issues that need improvement. And then, perhaps, there is a deeper layer: an AI that has been observing the ups and downs of a company for years. What has it noticed, what insights has it gathered? And who will ask it the right questions? This isn’t quite as dramatic as the problems of superalignment, but it’s one that you are perhaps more likely to face in the next year or two, whether you live in San Francisco or not.

Cheers * N

I hope you enjoy this newsletter, which, thanks to your support, now has more than 400,000 readers. Please continue to forward it to anyone else who might enjoy it. They can sign up here.