PRESENTED BY
Cyber AI Chronicle
By Simon Ganiere · 21th July 2024
Welcome back!
Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.
Table of Contents
What I learned this week
TL;DR
I’m not going to lie, since the creation of this newsletter there was a few instances where I was like what am I going to write?…same this week. Combined with a busy week at work and a lot of things happening on the personal side and I had nearly zero time to think about this newsletter…then Friday happened 🙃 That CrowdStrike update will for sure stay as an epic failure. I’ll share my thoughts on what it means from a resiliency perspective. Also massive shoot out to all people who had to work to restore their system so their company can be back online. Also, to my friends at Crowdstrike as this is not an easy time for them either. The cyber industry is about people and collaboration especially and the fact that a security vendor had a massive issue should not change that » MORE
Before that happen there were quite a few other news worthy items during the week. Here are a couple of key items that attracted my attention:
The Coalition for Secure AI (CoSAI) was announced on the 18th July. It’s an open-source initiative designed to give all practitioners and developers the guidance and tools they need to create Secure-by Design AI Systems. OpenAI has released a new model GPT-4o mini the most coefficient small model. The model score 82% on MMLU. The model is also 60% cheaper than GPT-3.5 Turbo. Some significant investment in deepfake defense company Pindrop landed a $100M loan to grow its offering.
In the cyber world, law enforcement arrested a 17-years-old (!) boy suspected of being a member of the notorious Scattered Spider cybercrime syndicate. This arrest seems to be linked to the ransomware attack against MGM Resorts Casino. It seems that Google is about to spend $23 billion to acquire Wiz. A US District Court judge has upheld a securities fraud claim brought by the SEC against SolarWinds and its CISO Tim Brown. However, the judge also rejected the SEC bid to oversee corporate cyber security controls. Pretty curious about the response from the SEC.
On the geopolitics side, some interesting stats on the fact that since the October 7 the number of attacks against Israel organisation has more than doubled.
How a (simple) Update Broke (most) of the World
Let’s start with what happened. Here is a short visualisation (generated by Anthropics) that summarize this article from Crowdstrike.
Summary of what happen
Whilst this is a good summary, I wanted to add a couple of important things as well:
This was a “channel file” update, not a full agent update. “Channel files are basically the content file with the detection logic. Those are pushed by CrowdStrike on a very regular basis to ensure the agent has the latest detection logic for the current threats.
The trade off here is between the speed at which you want new detection logic and your change management process.
The channel files are not kernel driver, despite having a .sys extension.
Full agent updates are not being pushed automatically. The customer receives the package and can decide to apply the update or not and obviously do all of the necessary change management steps.
Most of the people do not necessarily understand how complex and interconnected the IT world is. Neither do they understand how important all those IT systems are and that a disruption in that ecosystem has a very direct impact on the life of more or less everybody. After last Friday, I’m sure people will have a better appreciation of the dependency on IT system and software:
Transportation industry was heavily hit, with multiple airports experiencing issue, airlines grounded their planes or provided paper written boarding pass, disrupted train services
Multiple media companies, such as News and TV outlets were not able to broadcast
Healthcare was also impacted where NHS reported general practitioners experiencing disruption in the medical record system and appointment systems. 911 service was disrupted in a couple of US states.
Microsoft confirmed that they estimate that 8.5 million computers were impacted compared to 300k computers impacted by WannaCry. Obviously, the comparison is difficult and very different (intend was not malicious and there was no encryption) but in a way it was a ransomware simulation done at scale.
Checking my social media feeds this morning, there is a lot of fake news flying around. Some of those are supposed to be meme but as always people tend to believe them. For example, there is no confirmation that this was due to an “intern” having access to production release. Please be careful and double check the information.
Don’t Blame, Learn from it!
It would be too easy to just blame CrowdStrike. Don’t get me wrong, they obviously messed this one up really badly and I hope we will very soon see full transparency on what happened. Company are judge on their response so we will see if CrowdStrike is up to the task soon enough.
I suspect a lot of CrowdStrike customers will be having internal discussions on Monday to decide if they cut their contracts or not. Legitimate discussion if you ask me, that being said, I think there are a couple of bigger discussions that are equally, if not more, important:
Resiliency:
If anyone is surprised, you need to come out from under that rock you have been hiding under! It happened to CrowdStrike on Friday but could and has happened to other BigTech vendors before (well Microsoft had an outage (see for 18th July)) just before Crowdstrike. I talked at length about the fact that cyber needs to move towards resiliency as in the end what matter is the ability of a company to sustain disruption (may it come from cyber, a software bug or something else). I see two key topics here:
Regulation of BigTech: There is a question about BigTech (e.g., the Google, Microsoft, Amazon, etc. of the world) and how to regulate them. Some industries are rightfully regulated with strict rules, but BigTech is not one of them. There is a real question about whether these companies that provide ubiquitous services should be forced to provide transparency and more guarantees about the software they provide.
Resiliency in Cybersecurity: Cybersecurity must be all about resiliency. Although Friday's incident was not a cyberattack, the end result was the same. Organizations need to be able to deal with a significant outage and recover as quickly as possible. Building resilient systems means having contingency plans, redundant systems, and a culture of continuous improvement and testing.
Threat Modelling:
It might sound unusual, but yes, you need to threat model your security products as well:
Anticipate Potential Failures: Just as you would model threats to understand potential security risks, you should also model potential points of failure within your security tools themselves. What happens if an update goes wrong? How will it affect your systems? How can you mitigate those risks?
STRIDE Methodology: A useful approach here is the STRIDE methodology, which stands for Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege. By applying STRIDE to your security tools, you can systematically identify and address potential threats:
Spoofing: Could an attacker impersonate a component of your security system?
Tampering: Can the data processed by your security tool be maliciously altered?
Repudiation: Are there actions that could be performed without traceability?
Information Disclosure: Could sensitive information be leaked by the security tool?
Denial of Service: How might the security tool be used to deny service to your users?
Elevation of Privilege: Could the security tool be exploited to gain unauthorized privileges?
Assessing Security Tools: Regularly assess your security tools using threat modelling. This involves:
Identifying Assets and Entry Points: Determine what needs to be protected and where vulnerabilities might exist.
Identifying Threats: Use STRIDE to uncover potential threats to each asset and entry point.
Developing Mitigations: Create strategies to mitigate identified threats. This could include additional safeguards, monitoring, or changes to processes.
Testing and Validation: Regularly test updates in controlled environments before full deployment. Simulate potential failures and develop robust recovery plans.
I have asked Claude to do a quick threat model based on this prompt. Attached is the output. Obviously this is far from complete but as with any of the AI tools it’s a solid start and improving this should not be too difficult.
Impact on AI technology
The implications of such a massive outage extend to AI technology as well. AI systems are increasingly becoming integral part of our life and with all the investment and new product it will soon be ubiquitous. Having a similar outage on AI technology would also have significant impact:
Data Integrity and Availability:
AI systems rely heavily on data. An outage affecting data availability or integrity can disrupt AI training and operations. Ensuring data redundancy and real-time backups is crucial.
Model Reliability:
An interruption in services can lead to AI models becoming outdated or unreliable. Continuous model monitoring and automated retraining pipelines can mitigate this risk.
Operational Continuity:
AI-driven applications in critical sectors, like autonomous vehicles or medical diagnostics, need to maintain operational continuity. Implementing failover mechanisms and redundant AI systems can help ensure uninterrupted service.
Security Vulnerabilities:
AI systems themselves can become targets during outages. Implementing robust security measures and regularly updating threat models to include AI-specific risks is essential.
Ethical Considerations:
Outages highlight the need for ethical considerations in AI deployment. Ensuring transparency, accountability, and fairness in AI systems can build trust and resilience.
Concrete Actions and Solutions:
Ensure you understand which tools can have a wide impact in your environment. Security tools are an obvious candidate but they are not the only one (e.g. performance monitoring, log management, any agent that run on your environment, etc.).
No matter what, threat model your security tools. Use STRIDE or other methodology to help you.
Ensure you understand how your security product fail (e.g. close or open). Yes an EDR update can crash your box, but a failed update on your DLP can prevent email going out (which might be equally bad).
Ensure your resiliency plans are up to date (business continuity, recovery plan, etc.). Including worst case scenario where a manual recovery is required.
Ensure you have contact details for your vendors and the right level of support defined in your contract.
Ensure your third-party management process are updated and you might want to add a couple of questions based on this event as well.
Friday was a bad day, but it is crucial that we use such events as wake-up calls to ensure that the focus is on resiliency. By learning from these incidents and incorporating these lessons into our threat models and resiliency planning, we can build stronger, more robust systems that can withstand even the most unexpected disruptions.
Important: threat actors are always trying to exploit those situation. Crowdstrike published a blog post providing information about phishing attempt. Please be careful and crosscheck any Crowdstrike communication is coming from them. if you are a customer you can always check with your assigned representative.
Worth a full read
SAPwned: SAP AI vulnerabilities exposed customers’ cloud environment and private AI artefacts
Key Takeaway
AI training platforms' access to sensitive data inherently increases their vulnerability to attacks.
Effective tenant isolation in cloud services is crucial to prevent unauthorized access and data breaches.
The blend of control and data planes in Kubernetes architecture poses significant security risks.
Arbitrary code execution in AI R&D necessitates robust security guardrails for platform integrity.
Internal network trust assumptions can lead to significant security oversights in cloud environments.
Hardening internal services is essential for reducing the severity of potential security incidents.
The discovery of vulnerabilities through legitimate use cases highlights the complexity of cloud security.
Collaboration between security researchers and cloud service providers is key to enhancing cloud security.
The rapid response and fix of vulnerabilities by SAP demonstrate the importance of responsible disclosure.
Security challenges in AI platforms underscore the need for continuous improvement in cloud security practices.
SolarWinds and its CISO not off the hook over “materially misleading” security statement
Key Takeaway
Misleading security statements by companies can lead to significant legal consequences.
Accurate cybersecurity disclosures are crucial for corporate accountability and trust.
CISOs play a critical role in ensuring truthful cybersecurity practices and disclosures.
Legal scrutiny of cybersecurity disclosures is increasing, setting new precedents.
Companies must rigorously assess and disclose cybersecurity risks to avoid legal pitfalls.
Cybersecurity negligence not only poses technical risks but also legal and financial ones.
The intersection of cybersecurity and legal obligations is becoming more defined.
Corporate leaders are held accountable for the accuracy of cybersecurity statements.
Research Paper
Codexity: Secure AI-assisted Code Generation
Summary: Codexity is a security-focused code generation framework that integrates five LLMs with static analysis tools to mitigate vulnerabilities in AI-generated code. It demonstrates a 60% reduction in vulnerabilities compared to ChatGPT, using two repair strategies: Iteration Repair and Preshot Repair. The study evaluates Codexity's effectiveness using real-world benchmarks and compares it with FootPatch and GitHub Copilot, showing significant improvements in secure code generation.
Published: 2024-05-07T01:11:14Z
Authors: Sung Yong Kim, Zhiyu Fan, Yannic Noller, Abhik Roychoudhury
Organizations: National University of Singapore, Singapore University of Technology and Design
Findings:
Codexity prevents 60% of vulnerabilities in LLM-generated code.
Iteration Repair strategy reduces vulnerable programs by 60%.
Preshot Repair strategy reduces vulnerable programs by 36.6% (StarCoder) and 29.5% (SantaCoder).
FootPatch failed to patch any vulnerabilities in the benchmark.
GitHub Copilot generated 84.44% vulnerable programs in the benchmark.
Final Score: Grade: A-, Explanation: Novel approach with strong empirical evaluation but lacks some statistical details.
Wisdom of the week
If the culture is strong, new people will become like the culture.
If the culture is weak, the culture will become like the new people.
Contact
Let me know if you have any feedback or any topics you want me to cover. You can ping me on LinkedIn or on Twitter/X. I’ll do my best to reply promptly!
Thanks! see you next week! Simon