Building unshakeable trust: the bedrock of cybersecurity in generative AI


Galeboe Mogotsi, chief information and security officer (CISO) at the University of the Witwatersrand, discusses the imperative to fortify the cybersecurity foundations of generative AI. 

Generative AI (GenAI) has revolutionised industries and transformed lives with its unparalleled ability to generate, create, and innovate. However, this rapid adoption has also introduced a plethora of vulnerabilities and attack surfaces, making it a tantalising target for cybercriminals. 

As AI systems become increasingly sophisticated, so too do the threats they face, compromising sensitive data and eroding trust in these systems.

A robust cybersecurity framework for GenAI must be built on the following unshakeable pillars:

  1. Data security: the Fort Knox of GenAI

Protecting the integrity and confidentiality of data used to train and fuel GenAI systems is paramount. This demands the implementation of robust access controls, encryption and data anonymisation techniques to prevent data breaches and unauthorised access.

A case study in healthcare – a leading hospital chain utilised GenAI to develop personalised treatment plans for patients. However, they failed to implement adequate data security measures, resulting in a devastating data breach that compromised sensitive patient information.

  1. Model security: the lock and key of GenAI

Ensuring the security and integrity of GenAI models themselves is vital. This involves implementing secure coding practices, vulnerability management and continuous monitoring for potential backdoors or exploits, safeguarding against model manipulation and data tampering.

A case study in finance – a prominent bank developed a GenAI-powered fraud detection system, but neglected to secure the model itself. Cybercriminals exploited this vulnerability, manipulating the model to facilitate fraudulent transactions.

  1. Infrastructure security: the bedrock of GenAI

The underlying infrastructure supporting GenAI systems must be secure, with robust network security, secure cloud configurations, and regular software updates and patching, preventing vulnerabilities and weaknesses that could compromise the entire system.

A case study in retail – an e-commerce giant deployed GenAI to enhance customer experience, but overlooked infrastructure security. This led to a massive data breach, compromising customer information and damaging the company’s reputation.

  1. Human-centric security: the weakest Link

Educating and awareness programmes for developers, users and stakeholders are essential to prevent human-error-induced security breaches, ensuring that the human element is not the weakest link in the cybersecurity chain.

A case study in education – a leading university implemented GenAI-powered adaptive learning systems, but failed to educate users about security best practices. This led to a phishing attack that compromised the entire system.

To build trust in GenAI systems, we must prioritise transparency, explainability, and accountability, fostering a culture of security and responsibility. Understanding how GenAI models make decisions and take actions is crucial for identifying potential security risks and building trust in these systems.

Continuously evaluating and testing GenAI systems for vulnerabilities and weaknesses ensures swift action in the event of a security breach, preventing devastating consequences. In addition,establishing clear incident response plans and procedures ensures swift action in the event of a security breach, minimising damage and restoring trust.

The future of AI depends on our ability to prioritise cybersecurity and build trust in AI-driven systems. By establishing a robust underlying foundation of cybersecurity and promoting transparency, explainability, and accountability, we can harness the power of GenAI while minimising the risks, and creating a secure and trustworthy GenAI ecosystem for generations to come.

Related articles

The role of CISOs in shaping robust cybersecurity laws

Newly appointed chief information and security officer (CISO) at the University of the Witwatersrand Galeboe Mogotsi discusses the important role security officers must play in shaping the country’s cybersecurity laws.

Navigating the complex terrain of IT adoption

In the dynamic and ever-changing world of business, maintaining a competitive edge and maximising efficiency requires the integration of innovative IT projects that hold the promise of transformation. However, the path to successful adoption of these projects is complex and riddled with obstacles.

The age of AI has arrived

Yasvanth Singh and Kurt Kruger explain why it’s time South African CIOs get on board – or get left behind.