PRESENTED BY
Cyber AI Chronicle
By Simon Ganiere · 16th February 2025
Welcome back!
Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.
Table of Contents
What I learned this week
TL;DR
Databricks DASF 2.0 expands AI security beyond models, addressing seven new risks like LLM jailbreaks, chaining vulnerabilities, and inference manipulation. It aligns with MITRE ATLAS, OWASP, and NIST AI RMF, helping security teams integrate AI risk into governance and compliance » READ MORE
I posted on LinkedIn this week on how the prompt for reasoning model is different compared to non-chain-of-thought model (GPT models). You can also refer to the updated page from OpenAI. This is particularly important as Sam Altman just shared their roadmap and the objective is to simplify the number of models by converting to chain-of-through model…so better learn quickly how to use those model efficiently.
Still looking at that priority matrix, making some progress on how to automate some it using a mix of Google search, Firecrawl API and LLM. Will continue to update on progress. So far this gives me a few things:
Apple released an emergency security update to patch a zero-day vulnerability (CVE-2025-24200) exploited in the wild in “extremely sophisticated attacks” » READ MORE
PostgreSQL vulnerability exploited alongside BeyondTrust zero-day (CVE-2025-1094) » READ MORE
OpenAI finds no evidence of a breach after hacker offers to sell 20 million credentials » READ MORE
Italy blocks Chinese AI tool Deepseek over privacy concerns » READ MORE
Coverage of the Munich Security and Cyber Security Conference 2025 » READ MORE
Databricks DASF 2.0: The New Standard for AI Security?
AI security is no longer just a technical problem—it’s a governance challenge, a compliance risk, and a business-critical issue. While most organizations are still trying to understand how to secure AI models, the reality is that the attack surface has already expanded far beyond individual models.
Databricks' AI Security Framework (DASF) 2.0 is an attempt to catch up with the evolving threat landscape. Compared to its first iteration, this new version shifts the conversation from basic model security to system-wide AI security, covering risks from data ingestion to inference responses.
But does DASF 2.0 deliver? And more importantly—how should security teams use it?
The Problem: AI Security is a Moving Target
For years, AI security has focused on model-specific vulnerabilities—data poisoning, adversarial inputs, prompt injection, and model theft. These are all real risks, but they only cover a fraction of the actual attack surface.
The reality is that modern AI systems are more than just models. They include:
Retrieval-Augmented Generation (RAG) Pipelines – Feeding models with external data sources.
Multi-Agent AI Systems – Where models collaborate dynamically.
Fine-Tuned Models on Enterprise Data – Making security a shared responsibility between vendors and organizations.
A single AI model may be secure in isolation, but once it’s connected to external databases, APIs, and third-party integrations, the risks multiply. That’s where attackers are now focused.
DASF 2.0 is the first AI security framework that explicitly recognizes this shift.
12 canonical components of an end-to-end AI system
What’s New in DASF 2.0?
The most important change in DASF 2.0 is that it treats AI systems as a whole, rather than just individual models. It adds seven new security risks, including:
LLM Jailbreaks – Techniques that bypass content moderation and ethical restrictions.
Excessive Model Agency – When AI makes unintended, autonomous decisions.
Inference Response Manipulation – Attacks that subtly influence AI-generated outputs.
Chaining Vulnerabilities – Security flaws that emerge when multiple models and APIs interact.
Model Extraction Attacks – Techniques used to recreate or steal proprietary AI models.
Data Ingestion Poisoning – Compromising the data pipelines feeding AI models.
Adversarial Prompt Engineering – Manipulating input queries to generate unsafe or unintended outputs.
Another key improvement in DASF 2.0 is better alignment with industry standards. Instead of being just another AI security whitepaper, this version maps AI threats directly to frameworks like MITRE ATLAS, OWASP, and NIST AI RMF.
For CISOs, this means one thing: AI security can now be managed using the same structured approach as traditional cybersecurity risks.
A New Way to Think About AI Security
One of the most significant contributions of DASF 2.0 is how it reframes AI security as a risk prioritization problem rather than just a technical challenge.
Most organizations are struggling with where to focus their security efforts. Should they prioritize securing training data? Hardening inference APIs? Monitoring model drift?
DASF 2.0 introduces a risk prioritization worksheet that helps teams answer this question. Instead of applying blanket security measures, it suggests:
Understanding where AI models interact with external systems.
Assessing business impact rather than just technical risk.
Mapping AI risks to compliance requirements (GDPR, EU AI Act, etc.).
This is where DASF 2.0 really shines. It acknowledges that AI security isn’t just about stopping attackers—it’s about ensuring that organizations can deploy AI without running into regulatory or governance disasters.
In other words: Security isn’t just about defense anymore—it’s about operational resilience.
How Security Teams Can Apply DASF 2.0
Stop Thinking About AI Models in Isolation
If your security team is still treating AI like a single entity, you’re already behind. The real attack surface isn’t the model itself—it’s how it interacts with other systems.
Retrieval-Augmented Generation (RAG) models are vulnerable at the data ingestion layer, not just in the model itself.
Multi-agent AI systems introduce risks that no single model assessment can catch.
Fine-tuned models on private data create new compliance liabilities that go beyond security.
DASF 2.0 forces security teams to map out the entire AI lifecycle and secure every interaction point—not just the model.
Align AI Security with Regulatory Compliance
With frameworks like the EU AI Act and the NIST AI RMF gaining traction, companies can no longer afford to treat AI security as an afterthought.
DASF 2.0 provides a structured approach to:
Classify AI risks in a way that aligns with compliance mandates.
Define clear responsibility models for security across vendors and enterprise teams.
Use AI red teaming not just as a technical exercise, but as a governance tool.
Bottom line: AI security isn’t just about stopping attacks—it’s about staying ahead of regulatory expectations.
Treat AI Red Teaming as a Continuous Process
One of the most practical additions to DASF 2.0 is its focus on red teaming AI systems.
Traditionally, AI security assessments have been point-in-time exercises. But modern AI models evolve continuously, meaning security needs to be dynamic as well.
Databricks' updated framework suggests:
Using synthetic adversarial attacks to continuously test model responses.
Simulating data poisoning scenarios to assess real-world impact.
Monitoring inference behaviors to detect emerging vulnerabilities.
This continuous validation approach is critical as attack techniques against AI rapidly evolve.
The Big Picture: Where AI Security Goes Next
DASF 2.0 is a much-needed upgrade in how we think about securing AI. But it’s also a sign of a larger shift happening in the industry.
We’re moving from basic AI security—focused on securing models and endpoints—to full AI system security, where risk must be managed across data ingestion, inference, APIs, governance, and compliance.
This shift is necessary, because AI security isn’t just a cyber problem anymore. It’s a business risk, a compliance challenge, and a governance issue—all at once.
SPONSORED BY
Here’s Why Over 4 Million Professionals Read Morning Brew
Business news explained in plain English
Straight facts, zero fluff, & plenty of puns
100% free
Worth a full read
Artificial General Intelligence's Five Hard National Security Problems
Key Takeaways
AGI’s impact on security isn’t just technical but deeply geopolitical, shifting power structures.
The biggest risk isn’t AGI itself, but how nations perceive and react to its development.
The uncertainty around AGI’s emergence makes rigid policy responses dangerously outdated.
The AI arms race could trigger a preemptive war based on perceived, not actual, threats.
Economic power may shift to whoever first controls AGI-driven productivity gains.
If AGI enables mass automation, economic stability may depend on rapid wealth redistribution.
Cyberwarfare could become AI-driven, making human decision-making slow and irrelevant.
AGI’s role in destabilizing democracy may exceed its military applications.
AI misinformation warfare could become a primary tool for nation-state competition.
Without global AI governance, nations may pursue reckless AI strategies for short-term gain.
A Brief Guide for Dealing with ‘Humanless SOC’ Idiots
Key Takeaways
Human expertise and intuition remain irreplaceable in complex security operations.
Automation in SOCs must be balanced with human involvement for optimal effectiveness.
Tribal knowledge and context are crucial for decision-making in security operations.
Current AI technology is insufficient for fully autonomous Security Operations Centers.
Effective SOC operations depend on high-quality, context-rich data.
Creative attackers continuously challenge automated defenses with adaptability.
Long-term AI advancements in SOCs hold promise but require cautious optimism.
Understanding enterprise complexity is vital for realistic automation expectations.
Human intuition and experience are essential for interpreting security data accurately.
Automation enhances security operations but cannot replace human roles entirely.
Research Paper
Gandalf the Red: Adaptive Security for LLMs
Summary: The paper introduces D-SEC, a model for evaluating LLM defenses against prompt attacks, balancing security and utility. It presents Gandalf, a gamified red-teaming platform, collecting 279k prompt attacks to analyze defense strategies. Findings show adaptive defenses and restricted domains enhance security without degrading usability.
Published: 2025-01-14T08:30:49Z
Authors: Niklas Pfister, Václav Volhejn, Manuel Knott, Santiago Arias, Julia Bazińska, Mykhailo Bichurin, Alan Commike, Janet Darling, Peter Dienes, Matthew Fiedler, David Haber, Matthias Kraft, Marco Lancini, Max Mathys, Damián Pascual-Ortiz, Jakub Podolak, Adrià Romero-López, Kyriacos Shiarlis, Andreas Signer, Zsolt Terek, Athanasios Theocharis, Daniel Timbrell, Samuel Trautwein, Samuel Watts, Yun-Han Wu, Mateo Rojas-Carulla
Organizations: Lakera
Findings:
D-SEC models adaptive attacks and defenses.
Gandalf platform generates diverse adaptive attacks.
279k prompt attacks dataset released.
Adaptive defenses improve security-utility trade-off.
Restricted domains enhance LLM security.
Final Score: Grade: B+, Explanation: Strong empirical basis but lacks detailed statistical analysis.
Wisdom of the week
A cat that dreams of becoming a lion must lose its appetite for rats.
Contact
Let me know if you have any feedback or any topics you want me to cover. You can ping me on LinkedIn or on Twitter/X. I’ll do my best to reply promptly!
Thanks! see you next week! Simon