PRESENTED BY
Cyber AI Chronicle
By Simon Ganiere · 21st April 2024
Welcome back!
Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.
Table of Contents
What I learned this week
TL;DR
The total cost of a successful ransomware attack is often underestimated (and under-reported). UnitedHealth reported this week an initial financial loss of $872 millions and a total direct cost estimated at more than $1 billion. This is not even counting the long-cost tail: legal, possible fines, loss of revenue, etc. All of this despite them allegedly paying the ransom…and the data still being available on ransomware platform…so they might have to deal with a second wave of problems. Playing out those scenarios fully is complicated but it's worth planning for the worst case scenarios.
In the Adversarial Machine Learning space, an interesting research paper about adaptive attack against LLMs. By combining manual prompt engineering, automated optimisation technic and leveraging any available information or capabilities an attacker is able to achieve near 100% success rates in jailbreaking even the most safety-aligned LLMS. As usual with cyber security, the ability to keep pace with the progress of the technology and the progress of the hacking techniques is a key success factor.
This week we are still learning on AI Governance. I have reviewed and summarise a list of 10 key AI principles. The objective is to understand the underlying principles that define the minimum baseline for AI system.
Key AI Governance Principles
This week will keep it simple and we will focus on a simple list of 10 key AI Governance Principles. The objective here is to establish a list of core principles that should drive all of your AI projects. Said otherwise this is basically your minimum bar requirements.
1. Ethical Principles and Standards
These guidelines ensure that AI systems prioritize fairness, non-discrimination, and societal well-being. By adhering to ethical standards, developers and stakeholders mitigate the risks of biased outcomes or harm to individuals and communities. Ethical considerations are integrated throughout the AI lifecycle, from design to deployment, ensuring sensitivity to diverse perspectives and needs. Moreover, ethical AI fosters trust and transparency, essential for widespread acceptance and adoption.
2. Regulatory Compliance
Regulatory compliance frameworks provide a necessary foundation for addressing legal requirements and industry standards, guiding developers and organizations in navigating complex legal landscapes. By complying with regulations, stakeholders demonstrate a commitment to transparency, accountability, and respect for user rights. Moreover, regulatory compliance helps mitigate legal risks and potential liabilities associated with AI applications, fostering trust among users and stakeholders.
3. Risk Assessment
Risk assessment serves as a foundational principle in AI governance, ensuring the responsible development and deployment of AI systems. By systematically evaluating potential risks associated with AI applications, such as privacy breaches, security vulnerabilities organizations can proactively address concerns and mitigate harm. Stakeholders can identify and prioritize risks, enabling informed decision-making and risk mitigation strategies. Moreover, ongoing risk monitoring and reassessment are essential for adapting to evolving threats and changing circumstances, fostering resilience and trust in AI technologies.
4. Data Governance
Data governance is a cornerstone principle in AI, ensuring the reliability and ethical use of data. Robust governance policies guarantee data quality, accuracy, and integrity, crucial for effective AI outcomes. Moreover, such policies prioritize data privacy, ensuring compliance with regulations and safeguarding individuals' rights. By implementing comprehensive data governance frameworks, organizations foster trust in AI systems and mitigate risks associated with data misuse or bias. Effective data governance also promotes transparency and accountability, enabling stakeholders to understand and oversee the data processes underpinning AI applications.
5. Transparency and Explainability
Transparency and explainability are fundamental AI principles as they drive promoting trust and accountability. Transparent AI operations and decision-making processes enable stakeholders to understand how AI systems function and the rationale behind their outputs. Explainability ensures that AI algorithms are interpretable, allowing users, developers, and regulators to comprehend why certain decisions are made. By prioritizing transparency and explainability, organizations enhance transparency, mitigate risks associated with opaque AI systems, and foster confidence among stakeholders. Ultimately, transparent and explainable AI contributes to responsible deployment, regulatory compliance, and ethical practice in AI development and use.
6. Accountability and Oversight
Clear accountability structures delineate roles and responsibilities for AI systems' outcomes and decisions, holding individuals and organizations accountable for their actions. Oversight mechanisms, including audits and reviews, monitor AI performance and compliance with governance principles, enabling proactive identification and mitigation of risks. By prioritizing accountability and oversight, organizations demonstrate commitment to transparency, fairness, and adherence to ethical standards in AI development and deployment. These principles also promote trust among stakeholders and support continuous improvement of AI systems.
7. Security and Privacy
Security and privacy are paramount AI governance principles, safeguarding systems and user data. Robust security measures shield AI systems from cyber threats and unauthorized access, mitigating risks of data breaches and manipulation. Additionally, privacy considerations must be embedded throughout AI system design and operation, ensuring compliance with regulations and protecting user privacy rights. By prioritizing security and privacy, organizations uphold trust, integrity, and ethical practice in AI development and deployment. These principles are essential for maintaining confidentiality, integrity, and availability of data, fostering user confidence and mitigating potential risks associated with AI technologies.
8. Fairness and Non-discrimination
Fairness and non-discrimination are paramount in AI governance, ensuring equitable treatment for all individuals and groups. Organizations must actively combat biases within AI systems to prevent discriminatory outcomes. By promoting equity and inclusivity, AI governance principles prioritize fairness in decision-making processes and algorithmic outputs. Proactive measures to identify and mitigate biases contribute to building trust and credibility in AI technologies. Upholding fairness and non-discrimination principles safeguards against the perpetuation of societal inequalities and fosters an environment of equal opportunity and respect within AI ecosystems.
9. Human-Centric Design
Human-centric design in AI governance prioritizes human values and well-being throughout the development and deployment process. By placing humans at the center, AI systems are designed to enhance human capabilities, respect autonomy, and uphold dignity. This principle ensures that AI technologies serve the best interests of individuals and society, aligning with ethical standards and regulatory requirements. Human-centric design fosters inclusivity and accessibility, considering diverse perspectives and needs. Ultimately, prioritizing human values in AI governance promotes trust, acceptance, and positive societal impact, reinforcing the notion that AI should serve humanity's collective interests and aspirations.
10. Continuous Monitoring and Improvement
Regularly assessing the performance and impact of AI systems enables organizations to identify potential risks or shortcomings promptly. By incorporating feedback, technological advancements, and evolving societal expectations into governance practices, stakeholders can adapt and enhance AI systems over time. This iterative process fosters responsiveness to changing circumstances, promotes innovation, and strengthens trust in AI technologies. Continuous monitoring and improvement reinforce accountability, transparency, and alignment with ethical standards, ultimately contributing to the responsible development and deployment of AI.
Conclusion
By adhering to principles such as Ethical Standards, Regulatory Compliance, Risk Assessment, and Data Governance, organizations can foster trust and accountability in their AI applications. Transparency and Explainability are crucial for user understanding and acceptance, while Accountability and Oversight ensure that there are clear lines of responsibility for AI-driven outcomes. Security and Privacy protect against misuse and breaches, and Fairness and Non-discrimination ensure that AI systems treat all users equitably. Human-Centric Design places human welfare at the forefront of AI development, and Continuous Monitoring and Improvement guarantee that AI systems evolve responsibly over time. Collectively, these principles form a comprehensive framework that guides organizations towards the responsible stewardship of AI technologies, balancing innovation with the imperative to protect and enhance human interests.
Worth a full read
Simple Adaptive Attacks
Key Takeaway
1. Demonstrated nearly 100% success in jailbreaking GPT-3.5/4, Llama-2-Chat, Gemma-7B, R2D2, and all Claude models using adaptive attacks.
Utilized adversarial prompt templates and random search to exploit logprob vulnerabilities in models.
Highlighted the importance of adaptivity in attacks, exploiting unique model vulnerabilities and API features.
Provided code, prompts, and logs for the attacks on GitHub for further research and analysis.
Showed that even adversarially trained models like R2D2 are vulnerable to tailored in-context learning prompts and random search.
Demonstrated that Claude models could be jailbroken via transfer or prefilling attacks despite not exposing logprobs.
Found that manual adaptation and random search on a restricted token set are effective for trojan detection in poisoned models.
Revealed that current open-weight and proprietary LLMs are non-robust to simple adaptive adversarial attacks.
Is it safe to let my employees login with Google?
Key Takeaway
Social logins use OAuth 2.0, allowing secure third-party access without sharing passwords.
Employing social logins inherits multi-factor authentication (MFA) from Google accounts, enhancing security.
Simplifies password resets, reducing the impact of phishing or malware attacks.
Reduces complexity and security risks associated with managing multiple passwords.
Risks include potential over-sharing of sensitive data through extensive permissions.
Privacy concerns arise as social logins reveal basic personal information to SaaS platforms and Google.
Despite drawbacks, the benefits of using social logins for business outweigh the risks for most companies.
Some more reading
Meta’s Oversight Board probes explicit AI-generated images posted on Instagram and Facebook » READ
Meta released Llama 3, which was trained on 7x the data set of Llama 2 and has a context window of 8k (double from Llama 2 » READ
How Microsoft discovers and mitigates evolving attacks against AI guardrails » READ
Ivanti warns of new critical flaws in its Avalanche MDM solution. Knowing they have made a public commitment on security, we should expect more CVE and patches » READ
VASA-1 Lifelike Audio-Driven Talking Faces Generated in Real Time from Microsoft. I keep seeing this and I can only wonder how this will end up being misused at some point » READ
Attempted Audio Deepfake Call Targets LastPass Employee » READ
The cyber market is in consolidation mode (not a surprise), Wiz is an advance negotiation to acquire Lacework » READ
Ransomware attack has cost UnitedHealth $872 million; total expected to surpass $1 billion » READ
LLM Agents can Autonomously Exploit One-day Vulnerabilities » READ
Wisdom of the week
92% of enterprise PCs are unprepared for AI integration
Contact
Let me know if you have any feedback or any topics you want me to cover. You can ping me on LinkedIn or on Twitter/X. I’ll do my best to reply promptly!
Thanks! see you next week! Simon