The Biggest Key to Open Source AI Safety is Also the Biggest Gap in Security
As in experienced security professionals, who play a critical role in ensuring that open source models are safe to use.
New data from Ramp’s Ara Kharazian offers concrete evidence of something much discussed in the media lately: Open source AI models, like the one NVIDIA just released, really are making headway with corporate AI buyers. 6.1% of them were using one as of July, up from 5.9% in June and 4.5% in January.
Modest numbers to be sure, but rising, which is not true of OpenAI and Anthropic, Kharazian says:
“That’s not because new AI spenders are switching to open source / Chinese models (they most definitely are not doing that — first-time buyers on AI are still using the American model companies). But it means more of their growth will have to come from existing businesses spending on AI, particularly the advanced spenders, and those businesses are increasingly spending on open source.”
Cost is an important reason why, the post continues, noting that Anthropic’s performant but pricey Fable 5 model accounted for only 6% of tokens purchased from Anthropic last month. But ISC2’s France, perhaps with Fable 5 in mind, suggests that a second issue he calls the “AI kill switch” might have been a contributing factor too.
“If you’re relying on one of the big vendors for their hosted model, they can turn it off,” he observes. “They can be instructed to turn it off as well, and that may be a big problem for your business.”
Installing an open source model locally is an effective way to mitigate that risk, but it poses risks of its own if you don’t know where that model came from and whether or not it’s safe. No one would start uploading IP to a conventional business application without vetting it first, France notes.
“You should do the same for models and look at their lineage,” he says.
Sounds easy enough, but there’s a caveat. “Skilling to do that might be a bit of a challenge,” France says. Meaning that just as cyber skills generally are in short supply, so too are the skills required to confirm the safety of the same open source AI models rapidly gaining popularity in IT environments.