PRESENTED BY

Cyber AI Chronicle
By Simon Ganiere · 23rd June 2024
Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.
Table of Contents
What I learned this week
TL;DR
First, welcome to all the new subscribers! I was looking at the stats the other day and saw a nice increase in the last few weeks so appreciate it! I started this newsletter as a way to track my learning and it seems that other people are benefiting as well which is all I’m asking for!
So Apple had its big day with the WWDC 2024! I’m an Apple fan boy since the beginning of time. No big surprises in the approach Apple is taking - model on the device and a big focus on privacy. In the past, Apple has demonstrated its ability to integrate technology and provide an end-to-end experience which was a key differentiator. Let’s see if they can make it happen again. Will definitely spend some time learning more about that split between on device and OpenAI and obviously the Private Cloud Compute setup as well. Also, the play on the name is just awesome: Apple Intelligence = AI! Love it!
A huge piece on AI and the future by Leopold Aschenbrenner. Have not finished it yet, as it’s a 165 pages document! For those who don’t know him, he worked at OpenAI in the superalignment team at OpenAI and has a pretty impressive track record(to say the least). You can also find a YouTube interview here.
An interesting read on start-up hiring, where Ross Haleliuk is explaining that hiring top performers from larger cyber company is not necessarily a good thing for a start-up. You can be a start-up or a big company, you need to hire the talent that you actually need and not just jump for the top performer. This is going back to the usual story of talent shortage in cyber. I always had my opinion on this, which was more the fact that the industry is bad at identifying the skill sets it actually needs. Hint: it’s not someone with a collection of certification and 25 years of experience with a technology that is only 10 years old. This topic require a longer write up but that article from Ross should not be just read from a start-up point of view. What matters is the thought process and approach to identify what skill sets you actually need. This matter even more at the moment based on the state of the market » MORE
I’m back with some Cyber AI magic. I wanted to move to an agentic workflow to help understand incident or attack. The workflow is (still) basic but powerful:
Start with a basic request such as a threat actor name or an incident.
Identify the key elements: attack description, TTPs, actors, victims, timeline, impact and recommendations.
Write a summary with key sections including a timeline.
I’m sharing the full script and a couple of example output. Hopefully that’s helpful for someone else » MORE
Agentic Workflow to Track Cyber Threats
As mention previously, agentic workflow can be very powerful. Most importantly they can solve more complex situation and string together a few prompts so you can pass the input/output to the next steps. With that in mind, I decided to take my previous prompts and put them in a basic agentic workflow.
I’m using CrewAI for this. For those not familiar with it, you can find all the documentation here and below is a picture they use to describe their framework:
CrewAI is just one of the options, the other obvious one is AutoGen from Microsoft. Enough on tooling, let’s go back to our setup!
3 Agents, 3 Tasks and a couple of tools
Our workflow is (very) simple and basic. I have defined 3 agents and 3 tasks:
A researcher Agent who is in charge to “uncover the latest development in the cyber security domain”
A writer Agent who is in charge of “crafting compelling content on cyber security”
A timeline Agent who is in charge of “creating a timeline of security incident”
A research task which is basically the start of workflow. This is where i’m asking about a specific threat actor or an incident.
A timeline task to basically get the content from the researcher agent and create a timeline in the mermaid format.
A writer task which is basically putting it together.
In terms of tools, this is where I believe there is a lot of potential for improvement. Currently i’m leveraging a couple of built-in tools from CrewAI such as SerperDevTool, WebsiteSearchTool and CSVSearchTool.
Serper is basically a Google Search API. Super useful to search the internet and get some of the news.
The website search tool, is a RAG tool that search a specific websites.
The CSV search tool, is a RAG tool that search a specific CSV file.
Now you might be asking, why a CSV file? Well, I can’t really add all of the key cyber/security websites under the web search tool. So what I’m using a trick that leverages Feedly and Zapier. Basically putting a selected list of feeds in Feedly in a specific folder and using Zapier to extract the content and save it to a CSV file on Google Docs. The only trick is that the feeds trigger too many tasks for the free tier of Zapier…so I have a backlog or I need to pay to get more tasks from Zapier. Not ideal but it is what it is. If you have a better idea on how to do that please let me know!
The Output
I have uploaded the full script i’m using here. I have also added a couple of the different run I have done:
Run 1: first run, the Google Threat Intelligence Blog was not included.
Run 2: have added the Google Threat Intelligence Blog in the website tool, hoping for a better result knowing Mendicant has published some very details blog post.
Run 3: have disabled the delegation of tasks.
Run 4: have enabled delegation of tasks for the writer and the timeline. Have switch to hierarchical rather than sequential. However the timeline is not in the right format.
Here is a copy of the output of Run 4 in PDF format. Let me know what you think but this look pretty good to me. The obvious issue is that the timeline is not in the right format.
A few things to note
Stating the obvious here, but garbage in / garbage out. If your input is bad or the tools you used are not the right one, the outcome is obviously not going to match. Highlight again the importance of data in anything AI workflow.
I had a couple of runs where the whole workflow seems to get lost. What I mean is that if you look at the debug info, it looks all good until suddenly it takes a wrong turn. I had a very early run where the output ended up being a list of cyber incidents in 2023 for example.
This means you need to be precise in the different prompt you are using. Even if you have to state the obvious like don’t do X or focus only on X.
The biggest challenge i’m seeing at the moment is consistency. Knowing this is GenAI I don’t mind some changes in terms of wording but the fact that the timeline section for example is changing significantly from one run to the other is a problem.
If you have experienced similar challenges, please reach out and let me know if you found a solution. I’ll continue to tweak and test to see if I can achieve that consistency.
Worth a full read
Predicting AI’s Impact on Security
Key Takeaway
AI excels in creativity, problem-solving, and information synthesis, areas previously thought difficult to automate.
AI learning is similar to human learning but lacks automatic knowledge transfer between phases.
Future AI advancements include expanded context awareness, continuous self-improvement, localized intelligence, and decision-making.
AI will impact enterprises through meeting transcription, self-updating wikis, automated reports, and specialized oracles.
Engineering will see self-documenting code, requirements as code, automatic integrations, and localized models for operations.
CISOs face persistent challenges in vulnerability management, detection, compliance, measurement, third-party incidents, and least privilege.
The underlying issues inhibiting solutions are coverage (width and depth), context (who, what, where, why, how), and communication (translation).
AI can address coverage by scaling triage, context by synthesizing information, and communication by tailoring data for audiences.
AI can help with communication by synthesizing data and translating it for the intended recipients.
Hiring top performers from large cybersecurity enterprises looks like a good idea
Key Takeaway
Hiring top performers from large cybersecurity vendors can ruin early-stage startups.
Sales at a startup differ significantly from sales at a large vendor.
Startup sales require educating prospects about new problems and solutions.
Startup salespeople must be agile educators, ready to hear "No" frequently.
Revenue leaders at startups must learn who their customers could be.
Hire entrepreneurial, results-driven people with high ownership for early-stage startups.
Move fast, learn by doing, and iterate as needed in startup roles.
Focus on hiring in-network referrals for better matches and retention.
Ensure executives from large vendors are willing to work as individual contributors.
Prioritize learning, unlearning, and relearning in new environments.
Some more reading
Consistently Prepared: Year-round strategies for career growth » READ
Attackers deploying new tactics in campaign targeting exposed Docker APIs » READ
Introducing YetiHunter: An open-source tool to detect and hunt for suspicious activities in Snowflake » READ
Disgruntled ex-employee costs company over $600,000 after he deletes all 180 of its test servers — found server deletion scripts on Google » READ
How AI and LLMs are revolutionizing cyber insurance » READ
CISA leads tabletop exercise focusing on hacked AI » READ
New hacker group targets Chinese users with compromised deepfake porn software, malicious VPN installers » READ
OpenAI co-founder Ilya Sutskever just launched a new AI venture focused solely on developing safe and powerful superintelligence, coming just a month after his official departure from OpenAI. » READ
Wisdom of the week
I’ve never scored a goal in my life without getting a pass from someone else.
Contact
Let me know if you have any feedback or any topics you want me to cover. You can ping me on LinkedIn or on Twitter/X. I’ll do my best to reply promptly!
Thanks! see you next week! Simon



