Last month, I reported on the widening web of connections between the effective altruism (EA) movement and AI security policy circles — from top AI startups like Anthropic to DC think tanks like RAND Corporation. These are linking EA, with its laser-focus on preventing what its adherents say are catastrophic risks to humanity from future AGI, to a wide swath of DC think tanks, government agencies and congressional staff. Critics of the EA focus on this existential risk, or ‘x-risk,’ say it is happening to the detriment of a necessary focus on current, measurable AI risks — including bias, misinformation, high-risk applications and traditional cybersecurity. Since then, I’ve been curious about what other AI and policy leaders outside the effective altruism movement — but who are also not aligned with the polar opposite belief system, effective accelerationism (e/acc) — really think about this. Do other LLM companies feel equally concerned about the risk of LLM model weights getting into the wrong hands, for example? Do DC policy makers and watchers fully understand EA influence on AI security efforts? At a moment when Anthropic, well known for its wide range of EA ties, is publishing new research about “sleeper agent” AI models that dupe safety checks meant to catch harmful behavior, and even Congress has expressed concerns about a potential AI research partnership between the National Institute of Standards and Safety (NIST) and RAND, this seems to me to be an important question. In addition, EA made worldwide headlines most recently in connection with the firing of OpenAI CEO Sam Altman, as its non-employee nonprofit board members all had EA connections. What I discovered in my latest interviews is an interesting mix of deep concern about EA’s billionaire-funded ideological bent and its growing reach and influence over the AI security debate in Washington DC, as well as an acknowledgement by some that AI risks that go beyond the short-term are an important part of the DC policy discussion. The EA movement, which began as an effort to ‘do good better,’ is now heavily-funded by tech billionaires who consider preventing an AI-related catastrophe its number one priority, particularly through funding AI security (which is also described as AI ‘safety’) efforts — especially in the biosecurity space.  In my December piece, I detailed the concerns of Anthropic CISO Jason Clinton and two researchers from RAND Corporation about the security of LLM model weights in the face of threats from opportunistic criminals, terrorist groups or highly-resourced nation-state operations. Clinton told me that securing the model weights for Claude, Anthropic’s LLM, is his number one priority. The threat of opportunistic criminals, terrorist groups or highly-resourced nation-state operations accessing the weights of the most sophisticated and powerful LLMs is alarming, he explained, because “if an attacker got access to the entire file, that’s the entire neural network.”RAND researcher Sella Nevo told me that within two years it was plausible AI models will have significant national security importance, such as the possibility that malicious actors could misuse them for biological weapon development. All three, I discovered, have close ties to the EA community and the two companies are also interconnected thanks to EA — for example, Jason Matheny, RAND’s CEO, is also a member of Anthropic’s Long-Term Benefit Trust and has longtime ties to the EA movement. My coverage was prompted by Brendan Bordelon’s ongoing Politico reporting on this issue, including a recent article which quoted an anonymous biosecurity researcher in Washington calling EA-linked funders “an epic infiltration” in policy circles. As Washington grapples with the rise of AI, Bordelon wrote, “a small army of adherents to ‘effective altruism’ has descended on the nation’s capital and is dominating how the White House, Congress and think tanks approach the technology.” Cohere pushes back on EA fears about LLM model weightsFirst, I turned to Nick Frosst, co-founder of Cohere, an OpenAI and Anthropic competitor which focuses on developing LLMs for the enterprise, for his take on these issues. He told me in a recent interview that he does not think large language models pose an existential threat, and that while Cohere protects its model weights, the company’s concern is the business risk associated with others getting access to the weights, not an existential one. “I do want to make the distinction…I’m talking about large language models,” he said

NEW YORK, NEW YORK - AUGUST 11: Former FTX CEO Sam Bankman-Fried arrives for a bail hearing at . [+] Manhattan Federal Court on August 11, 2023 in New York City. Federal prosecutors are asking U.S. District Court Judge Lewis Kaplan to revoke Bankman-Fried's bail and to be jailed until his October criminal trial. Bankman-Fried who has pleaded not guilty to multiple conspiracy and fraud charges was accused of witness tampering after the New York Times published a story featuring personal documents of Caroline Ellison, former Alameda Research CEO. Judge Kaplan will also hear arguments on the gag order placed on Bankman-Fried that was placed as part of his bail agreement for the alleged witness tampering

NEW YORK, NEW YORK - AUGUST 11: Former FTX CEO Sam Bankman-Fried arrives for a bail hearing at . [+] Manhattan Federal Court on August 11, 2023 in New York City. Federal prosecutors are asking U.S. District Court Judge Lewis Kaplan to revoke Bankman-Fried's bail and to be jailed until his October criminal trial. Bankman-Fried who has pleaded not guilty to multiple conspiracy and fraud charges was accused of witness tampering after the New York Times published a story featuring personal documents of Caroline Ellison, former Alameda Research CEO. Judge Kaplan will also hear arguments on the gag order placed on Bankman-Fried that was placed as part of his bail agreement for the alleged witness tampering