In the race to adopt AI leaders need to see the bigger picture

The guide, Governance in the Age of AI: A Practical Framework for Responsible Leadership, emphasizes that the core fiduciary duties of trustees remain unchanged but urges leaders of pension schemes to adapt existing governance, data security and risk frameworks to oversee the technology’s growing influence on scheme administration, investment strategies and communication with members. To assist with this, it outlines a set of principles that essentially amount to assessing the level of risk associated with different AI uses, ensuring that there is human involvement in decision-making, guarding against AI-enabled cyber attacks and seeking to prevent the spread of messages based on inaccurate or “hallucinated” guidance from public AI tools.

As such, the society – which includes pension scheme trustees, advisers and administrators among its members – is largely falling into line with other attempts to harness the power of the fast-developing technology while ensuring that its advance does not create more problems than it solves. For example, in the wake of a spate of lawyers using fake or hallucinated cases generated by AI tools in court documents, the Law Society, the professional body for solicitors in England and Wales, has this month published an article warning about the use of the technology. Pointing out that the issues emerge when there aren’t the necessary safeguards and governance processes in place to manage this technology, it says that firms need to ask questions about how the tools are embedded in their organizations, the structures around them and the support, including training, that individual lawyers have to verify what is produced by AI. The article adds: “AI tools have the potential to make legal work much more efficient. But because legal authorities carry so much weight, you’ll always need to verify your citations and reasoning. Your regulatory framework doesn’t change if you’ve used AI – you will always be held accountable for the work you produce.”

Cate Pye is Global Digital Trust and Cyber security Lead at PA Consulting. In a recent telephone interview with me, she summed up some of the pros and cons of the technology, particularly with regard to cyber attacks.

She added that it was able to filter out noise around data sets and spot anomalies and even shifts in behavior. But she also insisted that it was vital that there was “a human in the loop” and “sufficient guardrails” to limit the AI’s autonomy. Otherwise, she added, “agentic AI can quickly get away from you.”

Even away from regulated activities, there are concerns about how AI is being adopted. At Studio Graphene, a London-based AI-native digital product studio, founder Ritam Gandhi told me in an interview earlier this year that many adopters of the technology were not seeing the returns on investment that they expected. “A lot of people don’t have governance around the tools,” he said, adding that some were using the tools without having “the necessary structure” within their organizations.

Clearly, AI can bring real gains in certain areas. For example, PA’s Pye points out that the sheer computing power of AI enables leaders to use it to look at their defences and predict where cyber attacks might come from, and so proactively make it more difficult for cyber assailants to succeed.

But it appears that more generally the big gains will not come until leaders grasp the nettle and realise that AI is not like another software development falling under the IT budget. It is far bigger than that and goes to the heart of how an enterprise is set up. As Studio Graphene’s Gandhi said, there is a need to reinvent business in the AI era. But that will be too disruptive for some.

This article was first published in Forbes.