5 Lessons Learned From 3 Years of Open Banking

Introduction

Open Banking has come a long way in the last three years.

With our partner tomato pay, APImetrics has been monitoring Open Banking APIs since the earliest days of UK Open Banking. Monitoring production endpoints and FAPI consent flows has taught us a lot about how the Open Banking rollout has evolved.

This paper outlines some of the lessons we've learned monitoring Open Banking stacks over the last three years, and what it has taught us about DevOps in regulated sectors — where it's entirely possible for two teams of operations engineers to face off in front of a regulator and each be able to prove that the other is in the wrong.

A Brief History of Open Banking in the United Kingdom (…and Why it Matters)

The UK created one of the earliest Open Banking environments, which was built on a couple of key concepts:

• Data belongs to the consumer, not to the bank or institution.
• Integration to banking and other highly sensitive systems needs to be more secure (see: screen scraping).
• Consent and permission must be baked in from the start.
• Competition is essential for future economic growth.

Open Banking timeline

Methodology

Since 2019, APImetrics has made more than 30 million API calls to the production environments of more than 25 major UK providers of Open Banking services. Most of the calls are based on the FAPI consent flow, which is the basic process for getting access to a bank account. This confirms that the connecting institution has permission to access banking data.

We've considered FAPI consent flows only because if you can't get consent, you can't access anything. They're also all functionally identical, which makes this comparison cleaner.

We grouped providers according to institution type:

1. 01

   CMA9 banks

   The largest incumbent banks.

2. 02

   Traditional banks

   Established smaller banks and building societies.

3. 03

   Neobanks

   Innovative entrants to the UK banking services market.

Lesson One: “Quicksand”

The UK regulator insisted that banks provide a sandbox environment against which developers and TPPs (Trusted Third Parties) could test their integrations. They also insisted that these sandboxes be up and running before the production Open Banking systems came on stream.

This presented a few persistent problems for all parties in the UK Open Banking ecosystem.

1. 01

   The sandbox looked nothing like the final Open Banking solution.

2. 02

   The banks didn't update the sandboxes to reflect production systems.

3. 03

   The banks ignored the sandboxes.

   It's still not uncommon for sandboxes to be down for weeks at a time.

Sandboxes are a good idea

Developers need something they can build to and test against. But if the developers and third-party apps must be re-implemented between the sandbox and production, the whole system is a colossal waste of effort.

One of the lessons the neobanks learned from best-in-breed enterprise IT API providers is that a seamless transition from sandbox to production — simply changing the access permissions and root URI in the API call — streamlines adoption. This is Modern Cloud/API Architecture 101 and should be something all providers consider.

Lesson Two: “Documentation Blues”

An API is only as good as its documentation, and Open Banking is well defined with a clear OpenAPI specification. But the inherent weakness in the OpenAPI specification is that it currently has no mechanism to handle sequences or workflows — especially if those flows involve calls outside the spec itself.

FAPI authentication defines a complex, secure sequence to sign requests digitally, creating a JSON web token (JWT) as part of OAuth user authentication.

As this is outside of what can be defined in an OpenAPI spec, and can't be documented by the tools typically used for documenting API calls, it created a gap in the onboarding process.

What onboarding 30 banks looked like

When APImetrics onboarded the first 30 banks, the first 50% took a few days and had well-documented processes that had been thought out from the point of view of the developer. The next 35% took over a week as we found that bits were missing. The last 15% took weeks, literally — and some were never completed.

Some of the issues were:

1. 01

   Mistakes in the documentation

   Errors concerning specifying scopes and other variables — by the time we'd done a dozen, we got good at guessing the mistakes.

2. 02

   Unusable published strings

   The required strings, published in PDFs, were complex and almost impossible to copy from in a consistent manner.

3. 03

   A back end that was down, creatively

   The service we were connecting to was down — but in a cunning way that appeared to be working, and wouldn't work no matter what you did.

Lesson Three: “There and Back Again”

Nothing beats monitoring end-to-end with a synthetic call that attaches to the production endpoint. Nothing.

Even if you have all the call logs, there are issues:

• You aren't checking every single call, because it would slow the production system to a crawl.
• HTTP codes are only part of the story.
• You won't see traffic that doesn't reach your edge or gateway, and the ingress point of external calls may not match the real-world customer experience.

In a multi-cloud, complex world with highly secure servers all running their own infosec rules, you can't assume what calls look like externally. What works from one AWS location might not work the same way from another — and might not work at all from a certain Google one. In production, we saw multi-second differences in some circumstances between different data centers.

Lastly, it's never DNS — until it's DNS

Don't assume your CDN or cloud provider is infallible. Equally, don't assume doing it yourself will yield better results than industry leaders.

And review your setup. For example, consider what an appropriate time-to-live value should be: too short reduces the advantages of caching.

Be wary of having too many CNAMEs chained together. We found this can make a significant difference to the speed and effectiveness of DNS lookups from certain cloud data centers.

Lesson Four: “Everybody Lies. HTTP Codes Lie More Than Others”

If you rely on your internal APM stack and/or gateway logs to monitor your stack, you are almost certainly guilty of triaging out HTTP 4XX errors.

They're a client error. Right? Nothing to do with the platform.

Well, they might be — but do you know for sure? The simple fact is that you don't.

A 4XX might be a symptom of a more serious problem with your consent flow or authentication server. It could be that a legacy system you were unaware of is down — something we have seen older banks struggle with. Their dated architectures include databases that pre-date modern DevOps and fail in a way that is only apparent to end users and TPPs.

Lesson Five: “Leave That Frog Alone!”

Our final lesson is one we've seen multiple times in multiple industries, and it's a factor in how modern DevOps APM tools monitor environments. We call it the “boiling the frog” problem.

The old story goes that you can place a frog in a pot of water, heat it, and the frog won't notice it's being cooked until it's too late.

Type 1 — Slow & Low

We've seen a few variations on this problem, but they largely relate to how trend analyses or thresholds are managed.

The first issue is that a threshold based on a rolling average is set MUCH higher than the average latency performance. You can be slow for large chunks of a day, and even down entirely for several users — yet the day's average never reaches the point where an alarm would trigger.

The second is that things slow down every day, but you check for a moving trend. If you don't occasionally check the ABSOLUTE values of your latency components, you might miss that you've gone from 700 ms to 17,000 ms over the course of a few weeks.

Type 2 — Ship That Code!

You push code that passed all the build and integration tests, but no one checked whether there is a consequential change in performance. This can be tricky to spot, especially in conjunction with a Type 1 slowdown. If it takes place over the course of weeks or months, you might fall off the number of logs you store over time in something like Splunk.

About APIContext

APIContext (formerly APImetrics) is used by many of the world's largest banking groups and financial services providers to ensure their Open Banking solutions are functional and compliant with local regulatory frameworks.

Our patented technology automates the process of producing regulator-ready reports for financial services providers around the world, including the UK and Europe with PSD2, and beyond.

Our active monitoring and API run-time governance integrates with many of the leading developer operations suites, and provides API-centric views of:

• Real-time API performance from more than 80 locations worldwide, on four clouds and six continents.
• Fully integrated security monitoring designed and built for the needs of the financial services industry.
• Machine-learning-based analysis driven by a database of more than a billion real API calls.
• Integrated reporting, analysis and alerting.
• 360-degree visibility with CASC (Cloud API Service Consistency) scoring, allowing for at-a-glance service and competitor comparisons.

Glossary

Terms used throughout this white paper, defined as APIContext applies them when monitoring regulated API estates.

CMA9

The nine largest UK banks that the Competition and Markets Authority required to provide API access to bank and customer data.

FAPI

Financial-grade API — the OpenID Foundation security profile used by UK Open Banking for the digitally signed, JWT-based consent journey. If the FAPI consent journey fails, nothing downstream is reachable, which is why APIContext monitors it as the canonical Open Banking transaction.

OBIE

The Open Banking Implementation Entity, the body created by the CMA to deliver and oversee UK Open Banking standards and adoption reporting.

PSD2

Payment Services Directive 2, the pan-European agreement on payment services applicable to financial services providers doing business in the EU and United Kingdom.

Sandbox

A non-production environment provided so developers and TPPs can build and test integrations before connecting to live systems. A sandbox only pays for itself when moving to production requires nothing more than new credentials and a new root URI.

Synthetic monitoring

Making scheduled, real API calls from external cloud locations to a production endpoint, and validating the full response including payload contents. It is the only way to see calls that never reach your gateway, and the only way to catch a 200 OK carrying an empty payload.

TPP

Third-party provider (also Trusted Third Party) — a regulated organisation that accesses bank data or initiates payments on a customer's behalf under Open Banking.

TTL

Time-to-live — how long a DNS record may be cached. Values that are too short give up the benefit of caching; long CNAME chains compound the cost of every lookup.

About APIContext

APIContext eliminates blind spots for enterprises across the digital delivery chain with proactive synthetic monitoring, performance analytics, and automated conformance validation. Our platform delivers actionable insights so connected systems perform and conform—ensuring every interaction is trusted, secure, and compliant.

What are your APIs saying to AI? Contact us to find out.