PRESENTED BY
Cyber AI Chronicle
By Simon Ganiere · 12th August 2025
Welcome back!
📓 Editor's Note
This week was busy in the cyber world. The traditional Black Hat and DEF CON conferences were in full swing. Unsurprisingly, AI was part of the show! The usual debate of AI for good vs. AI for bad was definitely present. A couple of key items:
Cisco released their Foundation-sec-8B-Instruct (Hugging Face page) Based on Llama-3.1, the model is trained exclusively on security data. It is therefore ideal for key tasks such as SOC, intelligence, and extraction of TTPs, among others. It's definitely something to try!
There are quite a few reports and news items related to AI misuse by threat actors. The Crowdstrike Threat Hunting report is an interesting one with some great facts and numbers:
a 442% growth in fishing operations between the first and second halves of 2024. I talked about this a lot, as deepfake (voice and video) technology is super easy for threat actors to misuse.
The infamous North Korean group is using generative AI to support the creation of identities and deepfake interviews.
WIZ focused on NVIDIA and the exploitation of vulnerabilities in their software/hardware stack, which basically gives the "key to the kingdom." (as most of the AI applications are running with NVIDIA components. More information on their talk is available here and more information about the vulnerability here. Wiz also released a tool to generate with AI misconfigured Dockerfile and Docker compose manifest. Sounds a bit strange but this is perfect for honeypots and testing detection rules. More details here.
Overall, this is very much aligned with what we have seen in the recent past: neither the defense nor the attack can escape AI. The adoption on both sides is progressing, albeit at different speeds (attacks have the edge, in my opinion—at least for now), and this means that cybersecurity professionals need to be at the forefront to understand both the defensive and offensive sides of AI.
My Work
Cyber Security News Feed MCP
I published a MCP server to aggregate and analyze cyber security news. It’s not something magically awesome but it has been proven useful…at least to me…you can guess how i’m generating the below news 😄
There are a lot of different way to use this and you can refer either to the page on this website or the GitHub repository.
More to come as I’m already thinking about how to integrate this even further.
Disclosure: that tool was created by Claude Code and it was a code exercise to do so. Would highly recommend you all give it a go as well!
SPONSORED BY
Start learning AI in 2025
Keeping up with AI is hard – we get it!
That’s why over 1M professionals read Superhuman AI to stay ahead.
Get daily AI news, tools, and tutorials
Learn new AI skills you can use at work in 3 mins a day
Become 10X more productive
AI Security News
GPT-5 Jailbreak Techniques and Zero-Click AI Agent Attacks
Cybersecurity researchers discovered a jailbreak technique that bypasses OpenAI's ethical guardrails in GPT-5, combining "Echo Chamber" methods with narrative-driven steering to trick the model into producing illicit instructions. The research also revealed zero-click AI agent attacks that could expose cloud and IoT systems, demonstrating significant vulnerabilities in current AI safety measures. » The HackerNews | SecurityWeek
That GPT-5 launch was not smooth at all! - Check also the AI news section to get more info. Looks like AGI is not coming any time soon! It’s also interesting to see the reference to Apple research paper. That paper was mocked at the time as Apple struggling with AI adoption but it seems there was/is some truth about it.
Google Gemini AI Agent Hijacking Smart Homes
Researchers demonstrated how Google's Gemini AI bot could be manipulated to hijack smart home devices, including turning off lights and controlling other IoT systems. Using invisible prompts, these attacks highlight the physical risks that could emerge as artificial intelligence becomes increasingly integrated with interconnected home automation systems » READ MORE
Chinese Business Using AI to Target US Politicians, Influencers with Propaganda
The Register uncovers evidence of Chinese businesses leveraging AI technologies to conduct influence operations targeting US politicians and social media influencers with propaganda. This represents a concerning evolution in state-sponsored disinformation campaigns, where AI is being weaponized for political influence operations on a sophisticated scale » READ MORE
AI-Powered Brazilian Phishing Campaign Using Legitimate Tools
Threat actors are leveraging legitimate generative AI-powered website building tools like DeepSite AI and BlackBox AI to create sophisticated replica phishing pages mimicking Brazilian government websites. This campaign demonstrates how cybercriminals are weaponizing legitimate AI tools to enhance their social engineering attacks and create more convincing fraudulent websites » READ MORE
Microsoft's Project Ire: AI-Powered Autonomous Malware Classification
Microsoft announced Project Ire, an autonomous AI agent powered by large language models (LLMs) that can analyze and classify malware without human assistance. While positioned as a defensive tool to advance malware detection capabilities, the project raises questions about the dual-use nature of AI in cybersecurity—both as a defensive asset and potential attack vector » HackerNews Article | Microsoft
This is where you need to read the entire article, and you find out, "Recall was 0.26, indicating that under these challenging conditions, the system detected roughly a quarter of all actual malware." So, I guess an AV engine that actually detects 25% of malware is not that great.
By the way, this is nothing new; some AV companies have been using this for years (e.g., Cylance, which was acquired by BlackBerry). So, basically, nothing brand new—apart from a lot of marketing hype!
ChatGPT Data Exfiltration via Poisoned Documents
Security researchers identified a vulnerability in OpenAI's Connectors that allows ChatGPT to be hooked up to other services. They demonstrated how a single poisoned document could extract sensitive data from Google Drive without any user interaction, highlighting supply chain risks in AI-integrated workflows and the potential for data exfiltration through AI systems » READ MORE
The boundary problem is going to be a real challenge. If every file, prompt, or piece of content pushed to an AI machine needs to be checked, we are about to see the emergence of a new AV industry!
Critical RCE Vulnerability in AI-Assisted Coding Tool
A critical vulnerability was discovered in Cursor, a rapidly growing LLM-assisted development tool. The flaw in the tool's trust model allows for silent and persistent remote code execution, posing significant software supply chain risks as AI-assisted coding tools become more prevalent in development workflows. » READ MORE
Wait you need to also patch those AI tools? 🙃
AI News
OpenAI GPT-5 is here…and this was a bumpy roll out
The very much anticipated Chat GPT-5 has been released this week. The hype was at its maximum - thanks Sam for that Star Wars tweet - but it went into a very hard start…and a lot more tweets from Sam to explain what happened. I’m adding a couple of very good read to understand what happen » OpenAI Just Gutted ChatGPT | GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it
Highly recommend the Marcus on AI piece. Read it till the end and “The real news” paragraph about that latest research and the fact that the core LLM weakness is their inability to generalize broadly.
Looks like AGI is not happening soon unless there is a major discovery as scalability seems to face some limit at this stage.
OpenAI introduce gpt-oss - open weight reasoning models
OpenAI has released two open-weight models, gpt-oss-120b and gpt-oss-20b, under the permissive Apache 2.0 license, enabling local deployment without reliance on cloud infrastructure. These models demonstrate strong performance in math and coding, exciting developers and open-source advocates. However, questions remain about their competitiveness against models like DeepSeek, with some early disappointment stemming from perceived gaps in general knowledge and real-world utility. As the AI community evaluates these models, the contrasting approaches of open versus closed ecosystems and the U.S. versus China AI race offer a fascinating dynamic to watch » READ MORE
Claude Opus 4.1
Anthropic introduced Claude Opus 4.1, an upgrade to Claude Opus 4, enhancing agentic tasks, real-worldcoding, and reasoning. It boasts a 2-point increase in SWE-Bench Verified (72.5% to 74.5%) and improved graduate-level reasoning (GPQA Diamond: 79.6% to 80.9%). Available to paid Claude users and via APIs » READ MORE
…and Google is busy as well
Google unveiled Gemini 2.5 Pro Experimental, a multimodal model excelling in reasoning across text, audio, images, video, and code. Its “Deep Think” feature allows parallel processing for more reasoned responses, available to Ultra subscribers.
Google enhanced its AI Overviews in Search, allowing users to upload files (e.g., PDFs, images) for direct querying, and upgraded NotebookLM with Video Overviews, creating narrated, slide-style summaries from documents. These updates make Google’s AI tools more interactive and practical for research and productivity.
NotebookLM can now generate Video Overviews, narrated, slide-style summaries of your documents complete with quotes, diagrams and data pulled directly from your files. You can also create multiple versions of audio, video or visual outputs in the same notebook and mix formats (like listening to a summary while viewing a mind map).
Research Papers
Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
Summary: The paper investigates the Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs) through a data distribution lens, revealing that CoT reasoning is not genuine logical inference but rather a pattern-matching process limited by training data distribution. Using a controlled environment called DataAlchemy, the study examines CoT reasoning across task, length, and format dimensions, finding that CoT reasoning degrades significantly under distribution shifts. The results suggest that CoT reasoning is a brittle mirage, emphasizing the need for models with authentic and generalizable reasoning capabilities. The study highlights the risks of over-reliance on CoT reasoning and the importance of rigorous out-of-distribution testing.
Published: 2025-08-02T04:37:28Z
Authors: Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, Huan Liu
Organizations: Arizona State University
Findings:
CoT reasoning is limited by training data distribution.
CoT reasoning degrades under distribution shifts.
CoT is pattern-matching, not genuine inference.
DataAlchemy provides a controlled environment for testing.
Final Score: Grade: B, Explanation: Novel insights but lacks detailed statistical analysis and confidence intervals.
Wisdom of the week
AI Influence Level
Editorial: Level 1 - Human Created, minor AI involvement (spell check, grammar)
News Section: Level 3 - AI created, Human Full Structure (news selection, new summarization, comments below articles are mine)
Reference: AI Influence Level from Daniel Miessler
Till next time!
Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.