5 Lessons Learned From 3 Years of Open Banking

Introduction

Open Banking has come a long way in the last three years.

With our partner tomato pay, APImetrics has been monitoring Open Banking APIs since the earliest days of UK Open Banking. Monitoring production endpoints and FAPI consent flows has taught us a lot about how the Open Banking rollout has evolved.

This paper outlines some of the lessons we've learned monitoring Open Banking stacks over the last three years, and what it has taught us about DevOps in regulated sectors where it's entirely possible for two teams of operations engineers to face off in front of a regulator and be able to prove that the other is in the wrong.

A Brief History of Open Banking in The United Kingdom (…and Why it Matters)

The UK created one of the earliest Open Banking environments, which was built on a couple of key concepts:

• Data belongs to the consumer, not to the bank or institution
• Integration to banking and other highly sensitive systems needs to be more secure (see screen scraping)
• Consent and permission must be baked in from the start
• Competition is essential for future economic growth

Open Banking Timeline

Methodology

Since 2019, APImetrics has made more than 30 million API calls to the production environments of more than 25 major UK providers of open banking services. Most of the calls are based on the FAPI consent flow, which is the basic process for getting access to a bank account. This confirms that the connecting institution has permission to access banking data.

We've considered FAPI consent flows only because if you can't get consent, you can't access anything. They're also all functionally identical, which makes this comparison cleaner.

We grouped providers according to institution type:

1. 01

   CMA9 banks

   the largest incumbent banks

2. 02

   Traditional banks

   established smaller banks and building societies

3. 03

   Neobanks

   innovative entrants to the UK banking services market

Lesson One: “Quicksand”

The UK regulator insisted that banks provide a sandbox environment against which developers and TPPs (Trusted Third Parties) could test their integrations. They also insisted that these sandboxes be up and running before the production OB systems came on stream.

This presented a few persistent problems for all parties in the UK Open Banking ecosystem.

1. 01

   The sandbox looked nothing like the final Open Banking solution

2. 02

   The banks didn't update the sandboxes to reflect production systems

3. 03

   The banks ignored the sandboxes

   (it's still not uncommon for sandboxes to be down for weeks at a time!)

Sandboxes are a good idea.

Developers need something they can build to and test against. But if the developers and Third-Party apps must be re-implemented between the sandbox and production, the whole system is a colossal waste of effort.

One of the lessons the neobanks learned from best-in-breed enterprise IT API providers is that by having a seamless transition from sandbox to production, simply changing the access permissions and root URI in the API call makes it easier to streamlines adoption.

This is Modern Cloud/API Architecture 101 and should be something ALL providers consider.

Have a sandbox that matches your production environment. Have a smooth rollover system from sandbox to production.

Sandboxes being part of a separate directory with different access controls makes it hard to keep solutions in sync. However, many neobanks have been able to create essentially the same architecture and approach in both stacks to make migration easy. A notable mention for CMA9 banks would be Barclays Sandbox, which cleanly simulates the full security process, enabling meaningful setup testing.

Lesson Two: “Documentation Blues”

An API is only as good as its documentation, and Open Banking is well defined with a clear Open API Specification. But the inherent weakness in the Open API Specification is that it currently has no mechanism to handle sequences or workflows, especially if those flows involve calls outside the spec itself.

FAPI authentication defines a complex, secure sequence to sign requests digitally, creating a JSON web token (JWT) as part of OAuth user authentication.

As this is outside of what can be defined in an Open API spec and can't be documented by the tools typically used for the process of documenting API calls, it created a bit of a gap in the onboarding process.

When APImetrics onboarded the first 30 banks, the first 50% took a few days and had well-documented processes that had been thought out from the point of view of the developer. The next 35% took over a week as we found that bits were missing. The last 15% took weeks, literally – and some were never completed.

Some of the issues were:

1. 01

   Mistakes in the documentation

   concerning specifying scopes and other variables – by the time we'd done a dozen, we got good at guessing the mistakes

2. 02

   The required strings, published in PDFs, were complex and almost impossible to copy from in a consistent manner

3. 03

   The back-end service we were connecting to was down

   but in a creative, cunning way that appeared to be working, but wouldn't work no matter what you did

DOCUMENT, DOCUMENT, DOCUMENT!

And have someone who isn't familiar with your solution verify that they can onboard using your documents.

APImetrics can help with this if you would like a review – it's part of the onboarding process for our own customers.

Lesson Three: “There and Back Again”

Nothing beats monitoring end-to-end with a synthetic call that attaches to the production endpoint. Nothing.

Even if you have all the call logs, there are issues:

• You aren't checking every single call because it would slow the production system to a crawl
• HTTP codes are only part of the story
• You won't see traffic that doesn't reach your edge/gateway if the ingress point of external calls do not match the real-world customer experience

In a multi-cloud, complex world with highly secure servers all running their own Infosec rules, you can't assume what calls look like externally. What works from one AWS location might not work the same way from another – and might not work at all from a certain Google one. In fact, in production we saw multi-second differences in some circumstances between different data centers.

Lastly, it's never DNS – until it's DNS.

Don't assume your CDN or Cloud provider is infallible. Equally, don't assume doing it yourself will yield better results than industry leaders.

And review your setup. For example, consider what an appropriate Time-To-Live value should be. Too short reduces the advantages of caching.

Be wary of having too many CNAMEs chained together. We found this can make a significant difference to the speed and effectiveness of DNS lookups from certain cloud data centers.

Lesson Four: “Everybody Lies. HTTP Codes Lie More Than Others”

If you rely on your internal APM stack and/or gateway logs to monitor your stack, you are almost certainly guilty of triaging out HTTP 4XX errors.

They're a client error. Right? Nothing to do with the platform.

Well, they might be – but do you know for sure? The simple fact is that you don't.

They might be a symptom of a more serious problem with your consent flow or authentication server. It could be that a legacy system you were unaware of is down – something we have seen older banks struggle with. Their dated architectures include databases that pre-date modern DevOps and fail in a way that is only apparent to end users and TPPs.

With Open Banking, we have also seen the problem of ‘200 OK’ actually not being OK. A common scenario is receiving a valid response with a 200 OK and requesting a list of transactions – but the list is empty.

This is where relying on your internal server and gateway tools can leave you unprepared, and why it is essential to have a valid synthetic sample that specifically checks payloads.

Lesson Five: “Leave That Frog Alone!”

Our final lesson is one we've seen multiple times in multiple industries, and it's a factor in how modern DevOps APM tools monitor environments. We call it the “boiling the frog” problem.

The old story goes that you can place a frog in a pot of water, heat it, and the frog won't notice it's being cooked until it's too late.

We see a variant of this with latency analysis, and it manifests in two ways.

Type 1 – Slow & Low

We've seen a few variations on this problem, but they largely relate to how trend analyses or thresholds are managed.

The first issue is that a threshold based on a rolling average is set MUCH higher than the average latency performance. You can be slow for large chunks of a day and even down entirely for several users. However, the day's average never reaches the point where an alarm would trigger.

The second is that things slow down every day, but you check for a moving trend. If you don't occasionally check the ABSOLUTE values of your latency components, you might miss that you've gone from 700ms to 17,000ms over the course of a few weeks!

Type 2 – Ship That Code!

You push code that passed all the build and integration tests, but no one checked whether there is a consequential change in performance. This can be tricky to spot, especially in conjunction with a Type 1 slowdown. If it takes place over the course of weeks or months, you might fall off the number of logs you store over time in something like Splunk.

The solution to this is to use something which stores logs that might be useful for this scenario for much longer periods we've been able to trace an event back to a specific CI/CD push that blew up the performance.

About APImetrics

APImetrics is used by many of the world's largest banking groups and many financial services providers to ensure their Open Banking solutions are functional and compliant to local regulatory frameworks.

Our patented technology automates the process of producing regulator-ready reports for financial services providers around the world including the UK, Europe with PSD2 and beyond.

Our active monitoring API run-time governance integrates to many of the leading developer operations suites and provides API centric views of:

• Real-time API performance from more than 80 locations worldwide on four clouds and six continents
• Fully integrated security monitoring designed and built for the needs of the financial services industry
• Machine learning based analysis driven by a database of more than a billion, real API calls
• Integrated reporting, analysis, and alerting
• 360-degree visibility with CASC the (Cloud API Service Consistency) scoring (CASC), allowing for at-a-glance service and competitor comparisons

Glossary

Terms used throughout this white paper, defined as APImetrics applies them when monitoring regulated API estates.

CMA9

The nine largest UK banks that the Competition and Markets Authority required to provide API access to bank and customer data.

FAPI

Financial-grade API — the OpenID Foundation security profile used by UK Open Banking for the digitally signed, JWT-based consent journey. If the FAPI consent journey fails, nothing downstream is reachable, which is why APImetrics monitors it as the canonical Open Banking transaction.

OBIE

The Open Banking Implementation Entity, the body created by the CMA to deliver and oversee UK Open Banking standards and adoption reporting.

PSD2

Payment Services Directive 2, the pan-European agreement on payment services applicable to financial services providers doing business in the EU and United Kingdom.

Sandbox

A non-production environment provided so developers and TPPs can build and test integrations before connecting to live systems. A sandbox only pays for itself when moving to production requires nothing more than new credentials and a new root URI.

Synthetic monitoring

Making scheduled, real API calls from external cloud locations to a production endpoint, and validating the full response including payload contents. It is the only way to see calls that never reach your gateway, and the only way to catch a 200 OK carrying an empty payload.

TPP

Third-party provider (also Trusted Third Party) — a regulated organisation that accesses bank data or initiates payments on a customer's behalf under Open Banking.

TTL

Time-to-live — how long a DNS record may be cached. Values that are too short give up the benefit of caching; long CNAME chains compound the cost of every lookup.

About APIContext

APIContext eliminates blind spots for enterprises across the digital delivery chain with proactive synthetic monitoring, performance analytics, and automated conformance validation. Our platform delivers actionable insights so connected systems perform and conform—ensuring every interaction is trusted, secure, and compliant.

What are your APIs saying to AI? Contact us to find out.