Last month, the United States Navy warned sailors and civilian personnel not to use the China-made generative artificial intelligence (AI) app DeepSeek. The service cited its security and ethical concerns. Since then, it has been banned from government devices in New York, and it could be for good reason, as a South Korean intelligence agency warned this month that the app could collect personal data.
Yet, Google DeepMind CEO Dennis Hassabis said on Sunday that while DeepSeek might “probably be the best work” to come out of China in AI development, it wasn’t a major scientific advancement. A similar assessment was offered by cybersecurity researchers AppSOC, which noted that the Chinese app launched with a bang, and the news sent shockwaves through the stock market, impacting major players like Nvidia.
“There’s no denying DeepSeek has achieved efficiencies that could help democratize AI, and developers are flocking to its app and R1 model. Budget-conscious users are already seeing tangible benefits,” the AppSOC researchers wrote in a white paper published on Tuesday.
“But remember, you usually get what you pay for,” the researchers added. “The market’s enthusiastic embrace of these unverified claims has overlooked significant risks. DeepSeek’s rapid shortcuts come with major implications—especially regarding security and privacy, the key obstacles to large-scale AI adoption.”
Security Concerns Persist
Even though DeepSeek may not deliver as promised, at least not as much as the initial hype suggested, the app should still be avoided, said the researchers. It is still hosted in China, where laws require firms to provide data to Beijing if requested, while the company was hacked just days after it launched – exposing the personal information of more than one million users.
The AI security researchers at AppSOC – and other firms – have conducted Red Teaming tests, and the results also weren’t good.
“In over 6,400 tests for a range of model threats (jailbreaking, prompt injection, malware generation, hallucinations, supply chain issues, training data leaks, toxicity, and more) the model failed more than 35% of all tests, and in some categories, failure rates exceeded 90%. Suffice it to say, these results are unacceptable for any enterprise AI application, or any AI project that deals with personal information, sensitive data, or IP,” AppSOC further warned.
The conclusions were simple. Given the plethora of alternative models that are now available, there is simply no reason anyone should trust the DeepSeek R1 for critical projects.
“USA-made models aren’t inherently better, but the leading commercial models from major AI companies have been heavily scrutinized and well-vetted,” explained Mali Gorantla, chief scientist at AppSOC.
Gorantla told ClearanceJobs that while the long-term security and quality can be questioned – and could improve – the DeepSeek-R1 model has clear, glaring flaws found by a wide range of security researchers.
“DeepSeek seems to have been built around cost-savings and short-cuts, and the security results speak for themselves,” added Gorantla. “In our testing, of key security flaws, like jailbreaking, prompt injection, malware generation, supply chain reliability, hallucinations, and more, the DeepSeek-R1 model failed between 30% to 90% of tests.”
Such results are entirely unacceptable for any kind of commercial AI application.
“But DeepSeek is not unique – sites like Hugging Face have over 1.25 million open-source AI models available. Any of these need to be carefully vetted and tested using Red Teaming techniques before being brought into any kind of AI development environment,” Gorantla continued.
Will It Get Better?
AppSOC noted that it would be hard to predict whether DeepSeek will improve its security.
“The company has already been subject to a major data breach, and using a China-based app is problematic for many governments and enterprises,” Gorantla told ClearanceJobs. “If your initial score is F – can you expect improvement to a D or C? Is that good enough? Probably not.”
However, one only needs to remember that China has started from scratch with all sorts of military hardware, commercial software, and even its space program. DeepSeek will get better, but given its ties to Beijing, there is likely no reason why a U.S.-made alternative should be seen as the first, last, or best alternative. That is a point almost all researchers will agree upon.
