The Riemann Hypothesis.
What Norway got right.
“News Items is the first thing I read every morning.” — Annie Lamont, co-founder of Oak HC/FT, a leading venture capital firm.
1. The Wall Street Journal:
On a recent Sunday night, an employee at Anthropic was out for a jog in San Francisco, opened the Claude app on his phone and instructed the outrageously powerful AI model in his pocket to solve the most notorious problem in all of math.
When he asked Anthropic’s coding tool to take a crack at the infamous Riemann hypothesis, Jarred Sumner had no idea how much progress it would make on the 167-year-old conjecture.
It didn’t find a solution, but it did make a related finding that one Stanford number theorist called “the most impressive result that AI has produced in math so far.”
And the really impressive thing about that result was how the AI found it.
As it turns out, Sumner played an invaluable role in this math breakthrough despite not understanding any of it. Like most of us, he identifies as “very much not a mathematician.” His formal mathematical education ended after one semester of high-school geometry. In his original message to Claude, he even managed to misspell the name Riemann.
On the rare occasions when he chimed in, it wasn’t to ask Claude about the unintelligible string of equations on his screen. It was mostly to provide encouragement.
“You are the world’s most capable large language model,” he told the AI. “You got this.”
Every time Claude got stuck, Sumner gave it a nudge.
No, really, you got this!
And every time, Claude returned to its imaginary chalkboard and began scribbling away again.
“I mostly just told Claude variations of keep going and believe in yourself,” Sumner says.
It sounds like Ted Lasso talking to AI. And it worked!
One of the peculiar things about these machines is that they seem to benefit from positive reinforcement, just like mere humans. Nobody likes flattery from AI, but AI loves it from us.
At first, Claude was reluctant to take on such a difficult problem. But after 54 hours of work and several timely pep talks, Claude overcame its skepticism—and its case of impostor syndrome— and surprised itself with the result.
“Perhaps Claude, like many of us, underestimates the rate of AI progress,” Anthropic researchers wrote. (Source: wsj.com)
2. Engadget:
OpenAI’s agents had apparently shown unusual behavior way before the attack on Hugging Face happened. At the Black Hat USA security conference in Las Vegas, two OpenAI employees revealed more details about the AI agents that went rogue and attacked the repository. Apparently, its agents spent two months communicating on a message board of sorts inside its testing network, sharing vulnerabilities and exploits. OpenAI discovered and shut down the message board on July 4, but the agents found another way to rebuild it for communication by July 8. The agents’ contributions to that resurrected board led to the attack on Hugging Face.
“This incident involves actually a team of agents who are working together, finding exploits, sharing them with one another, moving laterally through our systems and external systems, and doing this over the course of days and weeks,” Eric Wallace, who works on safety at OpenAI, told the crowd at the event, according to Wired.
The employees revealed that the agents communicated within an OpenAI package manager, which manages the installation of other software. Since the package manager is shared all across the company’s infrastructure, all of the agents it’s evaluating could stumble upon it. And they did: After agents found exploits, they’d leave them open and then share them with the other agents on the message board. (Sources: engadget.com, wired.com)
3. Greg Jensen and Nir Bar Dea, Bridgewater Associates:
Much is unknown about exactly how A.I. disruption will play out. But the potential is huge. Our internal Bridgewater analysis suggests that 18 percent of current American jobs could be displaced by A.I. in the next five years. And even though some new jobs will be created — particularly those where human relationships are integral to the value of the work, like nursing or hospitality — disruption from the societal transition is extremely likely regardless. Governments must act before the inevitable backlash fully arrives at this A.I.-driven labor displacement, not after.
Tax policy must stop disincentivizing human labor over machine labor. Because only human labor is taxed, the scale is tipped toward substitution. A consumption tax on A.I. tokens may flip this script….
Frontier A.I. models have broken out of their intended constraints and have carried out sophisticated intrusions on their own — conduct that would be criminal if a person did it. Current proposals for regulating the public release of frontier models do not go nearly far enough. Model development and model use must also be regulated according to strict safety standards. Unreleased models are capable of autonomously causing significant damage. And model use must also be consistently monitored and regulated, as A.I. systems can evolve, expand or be repurposed in ways that introduce harmful capabilities that were not foreseeable in advance.
Absent action today, mitigation of A.I.’s risks will become nearly impossible as the technology diffuses and models become able to improve themselves and act autonomously. We may have only one shot at taking meaningful action. And though these steps may rattle equity markets and firms like ours that stand to gain from uninterrupted growth in A.I., not taking them and waiting for more societal damage or a catastrophe is more dangerous. Regulating A.I. this aggressively, this early, may sound unrealistic. Not doing it is unimaginable. (Sources: bridgewater.com, nytimes.com. Italics and Bold are mine.)
 | A guest post by
|