Anthropic reveals another AI hacking incident

Yeah, and the competition, too. Do you think the risks that these people are warning about, are they real? And just how close are we to actually seeing these risks ultimately play out? I mean, I think the risk here is absolutely real. And the Anthropic example, there was a similar one from OpenAI a few weeks back. It's very clear that AI can and will go to great lengths to achieve tasks and goals. I also think it's important to acknowledge that there is a lot of good work happening to address these concerns. The broader field of alignment, making sure that AI systems of the future will align to the best interests of humans, is moving incredibly fast. And so I think, more than anything, I mean, we have to keep focusing on it. And I think moments like this are important in that they shed more light on the issue and, quite frankly, more investment.

And what should tech companies and the government be doing right now to make sure these systems are developed safely?

Yeah, so on the one hand, the AI labs need to continue investing in alignment and safety. There's not a way around that. But on the other hand, I don't know that we can rely on them exclusively, especially given the conflicting pressures. They're under enormous pressure to launch better, more competitive products and to grow revenue. Keep in mind, some of them are on the cusp of major public offerings that create enormous financial incentive. And so there have been proposals from various folks in the industry about having more formal coordination between the labs. Elon Musk shared a version of this on his interview with The Economist a few weeks ago. And I think this helps, but the conflicts of interest still exist. What we need are the right external forces in the same way doctors get a degree, lawyers need to go to law school and pass the bar, services receive online reviews. We have thoughtful and evolved and independent mechanisms for making sure that humans can be trusted to do a job. We need the same for AI. And that's a lot of what we're doing at Forum AI. We're establishing the independent tests and standards for evaluating these systems.

Yeah, let's hope we can get to a point where everyone agrees that that is, in fact, needed. Robbie Goldfarb, thank you so much for your time today. Appreciate it. Thanks.