PRESENTED BY

Cyber AI Chronicle

By Simon Ganiere · 25^th August 2024

Welcome back!

Project Overwatch is a cutting-edge newsletter at the intersection of cybersecurity, AI, technology, and resilience, designed to navigate the complexities of our rapidly evolving digital landscape. It delivers insightful analysis and actionable intelligence, empowering you to stay ahead in a world where staying informed is not just an option, but a necessity.

Microsoft Copilot is deeply integrated into the Microsoft 365 ecosystem. It comes with a range of built-in security features designed to protect organizational data. Let’s explore these key security features, highlighting how they (are trying to) address real-world cybersecurity challenges and considering how they may evolve to meet future AI-related threats. The below is based on the Microsoft documentation, whilst not aiming to be a sales pitch it gives you the view from Microsoft about their product. Next week i’ll cover more in depth the potential gaps and the threat scenarios.

One of the foundational security features of Microsoft Copilot is its strict adherence to data access and permission management. Operating within the Microsoft Graph, an API-driven platform, Copilot connects to various data sources within the Microsoft 365 environment while respecting all existing access controls. This ensures that Copilot only accesses data that users have permission to view, aligning with the organization’s security protocols.

Assuming your access management practices are robust and well implemented, this means that Copilot can help prevent unauthorized data access by ensuring that its outputs are informed solely by data the user is authorized to access. For example, if an employee inadvertently gains access to sensitive files, Copilot's strict adherence to permission can prevent those files from being included in AI-generated outputs.

Data privacy is a critical concern for any organization, especially when implementing AI tools that interact with sensitive information. Microsoft Copilot ensures that customer data is never used to train the underlying Large Language Models (LLMs). Unlike some AI systems that continuously learn from user interactions, Copilot’s LLMs are pre-trained on large datasets and do not use customer prompts, responses, or data for further training. This approach safeguards organizational data, ensuring that it remains private and contained within the enterprise’s secure environment.

This means that in principle even as Copilot processes and analyzes large volumes of data, the risk of data leakage or inadvertent exposure outside the organization is minimized. For industries with stringent regulatory requirements, such as finance or healthcare, Copilot’s compliance with standards like GDPR offers an added layer of reassurance.

Encryption is a cornerstone of data security, and Microsoft Copilot leverages robust encryption protocols to protect data both in transit and at rest. All data processed by Copilot, including user prompts and AI-generated responses, is encrypted using industry-standard methods. This encryption ensures that even if data were intercepted during transmission, it would be unreadable to unauthorized parties.

If an adversary attempts to intercept data between Copilot and the Microsoft Graph, encryption protect the integrity and confidentiality of that data. Future advancements might include more sophisticated encryption methods tailored specifically to AI-generated data and interactions.

Beyond encryption and access controls, Microsoft Copilot incorporates additional security, compliance, and privacy reviews during its post-processing phase. After the LLM generates a response, Copilot performs a series of checks to ensure that the output adheres to the organization’s security policies and ethical guidelines.

These post-processing safeguards are crucial in real-world settings where the potential for generating harmful, biased, or inappropriate content exists. For instance, in a scenario where Copilot generates a summary from sensitive documents, these security checks help ensure that the output does not inadvertently include confidential or ethically sensitive information.

Microsoft has committed to developing AI that operates within ethical boundaries, and Copilot is no exception. The system is built with responsible AI principles in mind, incorporating measures to prevent the generation of biased or harmful content. These principles are embedded into Copilot’s architecture, ensuring that the AI supports positive and ethical outcomes in its interactions with users.

In practice, this means that Copilot is designed to avoid amplifying biases or producing content that could lead to ethical or legal concerns. For example, in generating responses that involve sensitive topics, Copilot’s responsible AI safeguards work to mitigate the risk of producing biased outputs.

Finally, Microsoft Copilot includes real-time abuse monitoring features designed to detect and prevent malicious activity. This includes protection against prompt injections and other attempts to exploit the AI system. While Copilot itself does not allow standing access to customer data for monitoring purposes, it is equipped with safeguards to mitigate potential threats as they occur.

Real-time monitoring is essential for maintaining the security of Copilot’s operations. For instance, if an attacker attempts to manipulate the AI through prompt injection, Copilot’s abuse monitoring features can detect and neutralize the threat before it causes harm. As AI-related threats continue to evolve, the real-time monitoring capabilities of systems like Copilot will need to become increasingly sophisticated, potentially incorporating AI-driven anomaly detection and adaptive security responses.

While Microsoft Copilot introduces powerful capabilities to enhance productivity and streamline workflows, it also amplifies existing security challenges within an organization. In many enterprises, there are often gaps in access management, data classification, and awareness of who has access to sensitive information. Also Copilot operates at scale, quickly processing vast amounts of organizational data to generate insights and responses.

This means that any existing misconfigurations in access permissions or lapses in data classification can lead to unintended consequences. For instance, an employee may inadvertently have broader access to sensitive information than they realize (e.g. they changed role) and Copilot could use this data to generate content that might be shared more widely than intended. This scenario creates new layers of insider threat risks, where data could be exposed or misused, either unintentionally or maliciously.

To mitigate these risks, organizations must ensure that their access management practices are robust, data is properly classified, and employees are fully aware of the sensitivity of the information they interact with. By addressing these foundational security issues, companies can better control the potential risks introduced by Copilot and fully leverage its capabilities in a secure and compliant manner. As AI continues to evolve, so too must the security measures that protect it, ensuring that tools like Copilot remain beneficial without becoming liabilities.

Leaked environment variables allow large-scale extortion operation of cloud environments » READ

Post quantum cryptography » READ

FIN7: The Truth Doesn’t Need to be so STARK » READ

Cybersecurity should return to reality and ditch the hype » READ

How Phishing Attacks Adapt Quickly to Capitalise on Current Events » READ

Let me know if you have any feedback or any topics you want me to cover. You can ping me on LinkedIn or on Twitter/X. I’ll do my best to reply promptly!

Thanks! see you next week! Simon