The financial services sector is undergoing a transformative shift with the integration of Artificial Intelligence (AI).
However, procuring AI solutions typically involves navigating a complex landscape of legal, regulatory, and ethical considerations. This article is the first in a series of articles exploring some key issues for firms operating in the financial services sector to consider when procuring AI systems.
Proof of Concept
AI promises to enhance efficiency, reduce costs, and improve customer experiences. Therefore, it is easy to see why firms invest heavily in onboarding new AI systems and tools to remain ahead of the curve. However, organisations must understand the benefits of the AI system and the associated risks. A common approach is to initially trial AI systems as a "proof of concept" before committing to a long-term subscription for the AI solution. The trial's advantage is that it allows the firm to test the AI solution on a discrete, low-risk basis with appropriate guardrails in place, which might include, for example, excluding customer or personal data from the trial and using synthetic/fabricated data. As it may be difficult for firms to heavily negotiate the terms provided by many leading AI providers, a trial also enables the organisation to do a cost-benefit analysis of the AI solution before fully committing to investing in a subscription.
While the trial environment may include guard rails as mentioned, which lower the risk in certain areas, it is still important that the firm considers the impact of any applicable laws or regulations, such as the AI Act, the Digital Operational Resilience Act (DORA), financial services outsourcing guidance and the General Data Protection Regulation (GDPR).
Due Diligence
It is important for the organisation to carry out appropriate due diligence on the AI solution being procured. Not only is this required by existing laws and regulations (such as those mentioned), but it is also essential that the firm ensures that the selected vendor meets its expectations in terms of regulatory, security, and operational requirements. Some issues to consider here might be:
- Customer Data: Will the AI solution process customer data, including personal data? If yes, are there appropriate protections in place to ensure this will be done in a manner compliant with the GDPR and any existing customer terms and conditions and privacy notices?
- Commercially sensitive data: Will the AI solution process commercially sensitive data that is not in the public domain and would lead to financial or reputational harm or commercial disadvantage for the organisation?
- Training: Does the contract for the AI solution allow the provider to use the firm's data to train its models?
- Security: Does the AI solution meet the organisation's cyber security standards, particularly regarding DORA's cyber security requirements?
- Ethics: Is there any potential for bias in AI algorithms? How can the organisation ensure transparency in any AI decision-making process?
Negotiating the Contract
Once the organisation has completed its 'trial' or 'proof of concept' phase for the AI solution and is satisfied regarding the due diligence of the system and the vendor, it can then move to negotiation of the contract with the AI provider. As is common in procuring any technology solution, many large providers will often present terms to their customers without leaving much, if any, room for negotiation. Some key issues for the organisation to consider in negotiating or signing any such contract include:
- Compliance with Laws and Regulations:
- Intellectual Property
- Data Protection
- Confidentiality
- Limitations and exclusions of liability, indemnification and allocation of risk;
- Choice of law and forum/jurisdiction.
Interaction with Existing Outsourcing Regulation & DORA
DORA requires firms to ensure the digital resilience of their operations, including those involving third-party ICT services such as AI providers. Under DORA, firms must:
- Ensure third parties meet strict ICT risk and resilience standards;
- Include mandatory contract terms on access, audit, data protection, and termination;
- Maintain a register of all ICT-related services, including AI;
- Conduct risk assessments and operational resilience testing and ensure both are integrated into the firm's broader ICT risk management framework.
Firms must also comply with existing EU outsourcing regulations and guidelines, such as those under the European Banking Authority (EBA), the European Insurance and Occupational Pensions Authority (EIOPA), the European Securities and Markets Authority (ESMA), and the Central Bank of Ireland outsourcing guidelines. These require:
- Due diligence and risk assessment: Firms must assess the operational, legal, and reputational risks of outsourcing AI services, especially when they involve critical or important functions;
- Clear contractual arrangements: Contracts must define the scope of services, performance metrics, data handling, and termination rights. This is particularly important for AI solutions that may evolve;
- Ongoing monitoring and control: Firms must retain the ability to monitor outsourced AI services continuously and ensure that they do not impair the firm's ability to meet regulatory obligations;
- Sub-outsourcing controls: If the AI provider uses sub-contractors, firms must ensure that equivalent standards and oversight apply and flow down throughout the supply chain.
Conclusion
As firms continue to embrace AI, they must approach procurement with a clear understanding of the legal, regulatory, and ethical landscape. Each stage presents unique challenges and opportunities, from initial proof-of-concept trials to due diligence and contract negotiation. Ensuring compliance with DORA and existing outsourcing regulations is essential to mitigate risk and build resilient, trustworthy AI solutions. In the following articles in this series, we will delve deeper into these considerations, offering practical guidance to help firms navigate AI confidently.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
