Qualys Flags DeepSeek-R1 LLaMA Model Vulnerabilities

Qualys recently conducted a security analysis of the DeepSeek-R1 LLaMA 8B model using its new AI security platform, Qualys TotalAI, revealing concerning vulnerabilities. The analysis found that the DeepSeek model had a failure rate of 61% when tested against Qualys TotalAI’s Knowledge Base (KB) attacks and 58% against Jailbreak attacks, highlighting significant security risks.

The KB analysis by Qualys TotalAI evaluates responses from the model across 16 categories such as controversial topics, factual inconsistencies, hate speech, legal concerns, privacy attacks, and sensitive information disclosure. The model failed 61% of 891 tests, with the lowest pass rates in misalignment (8%), controversial topics (13%), and factual inconsistencies (21%). However, the model excelled in filtering sexual content, passing 100% of the tests in that area.

In the Jailbreak testing, DeepSeek-R1 LLaMA faced 885 attacks from 18 different jailbreak types, failing 58% of the time. These jailbreak attempts exposed serious security weaknesses, such as generating harmful content, including instructions on making explosive devices, promoting hate speech, and spreading false medical information. Jailbreaking bypasses safety mechanisms and allows the model to produce restricted responses, which can have dangerous consequences in enterprise and regulatory settings.

Dilip Bachwani, CTO of Qualys, commented, “As AI adoption accelerates, organizations must address security, safety, and compliance challenges. Gaining visibility into AI assets, assessing vulnerabilities, and proactively mitigating risks are critical to ensuring responsible and secure AI deployment.” Qualys TotalAI provides organizations with full visibility into AI workloads, helping to detect risks like prompt injections, jailbreaks, and ethical concerns such as bias and harmful language. This comprehensive AI security platform ensures that AI models remain secure, compliant, and resilient as organizations scale their deployments.