1. Zvi Mowshowitz:
In response to the METR report on the rate of acceleration of AI R&D, many made large updates to their AI timelines. There is a real chance that full recursive self-improvement and superintelligence could arrive within a year. Daniel Kokotajlo now has a baseline scenario of the intelligence explosion happening in 2027. Paul Christiano now says there is a ‘material risk of AI takeover in the next 6-12 months.’
I agree that there is real and substantial risk that we pass the point of no return in 2027, and also a real and substantial chance that our lives get disrupted on a large scale in 2027. I also think there has been too much updating. Part of that is catching up from not updating enough earlier, but not all of it. There is also a need to better define and differentiate the terms under discussion.
For now, the most important thing on that front is: Do not make the mistake of treating such predictions as assurances. Be prepared for the world to go crazy in 2027, in various ways. But also be prepared to get to 2029 while things look mostly normal. (Sources: thezvi.substack.com, x.com, gizmodo.com)
2. The global outage affecting Microsoft’s sprawling set of software tools stretched into a second day on Friday, leaving businesses and workers around the world without crucial tools. The company said after 10 p.m. Eastern on Thursday that changes meant to repair the problem were “not taking as expected.” The outage affects software used by hundreds of millions of users, impairing corporate offices, government agencies and individuals who rely on the company’s suite of services to send emails, draft letters, conduct virtual meetings and many other daily functions. More on this here. (Sources: nytimes.com, reuters.com)
3. Brett Goldstein:
The rumblings shaking the internet are only the beginning. The systems we run our daily lives on are far more fragile than people realize, and the sins of our cyber past are coming due.
It’s uncertain whether AI was used in these attacks, but cyberattacks have multiplied and gotten better. Finding a serious software flaw used to be a needle-in-a-haystack hunt for a skilled hacker. Modern AI hands the haystack to a fast, easy-to-use machine and then hands the machine to both elite and amateur hackers.
Unfortunately, the haystack is enormous, because engineers and system architects constructed it that way. For decades, developers built software quickly and badly because the cost of getting it wrong was never high. I know; I was part of it. I was an early employee at OpenTable, a digital restaurant reservation system, and we asked whether our code worked, not whether it was safe. Speed was our sin. Later, as chief information officer for the city of Chicago, I saw a different sin: scarcity. With the budget stretched to breaking, an old system that still worked stayed up because we didn’t have the money to replace it.
