We have entered March like a lion and point of the week when it is time to talk of the State of the APIs has rolled into view once more, not just over the last seven days, but the past month. We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. The basic service is free and will remain free, but other, related services will be built on top of it, including your very own API.expert dashboards if you want them! Each week, month and year we’ll look at the APIs we track in a variety of sectors for their quality (using our patent pending CASC score) and other factors like uptime and availability. If something is missing that you think should be here, don’t hesitate to drop us a line! Now, onto what the last seven days and last month can tells us about where the world of APIs is in early 2020.
API Performance Headlines
We look at over 200 APIs and pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category. We do see that certain providers consistently vie for top spot in their category including Twitter, Slack and Github with others doing less well such as Cisco’s Spark services. Top Performers based on overall quality in each category were:
CASC score
Week ending March 2, 2020
Category
Organization
CASC score
Corporate Infrastructure
GitHub
963
Cryptocurrency Exchanges
LATOKEN
943
PSD2 Banks
ABN AMRO Bank
955
Search
956
Social Networks
968
UK Government
Gov.UK
973
UK Open Banking (Open Data)
Bank of Ireland
973
US Government
Department of Justice
983
A fair amount of movement over the last month with changes in five of the eight categories (GitHub replacing Slack, BNP Paribas Railsbank, Twitter Google (in Social Networks), GOV.UK Police.UK and the Department of Justice the General Service Administration. All of these APIs have maintained an excellent level of performance over the month, but the amount of turnover indicates that it is tough at the top. It will be interesting to see how many APIs can maintain their position in March.
Top performers by latency
Week Ending March 2 2020
Category
Organization
Median latency
Corporate Infrastruture
GitHub
176 ms
Cryptocurrency Exchanges
FTX
233 ms
PSD2 banks
Open Bank Project
208 ms
Search
391 ms
Social Networks
121 ms
UK Government
Police.UK
75 ms
UK Open Banking (Open Data)
HSBC
86 ms
US Government
Department of Justice
73 ms
No changes at all again this week, which shows that APIs are more consistent in latency than they are in overall performance. An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score! As so often, FTX might have a low enough latency to top the Cryptocurrency Exchange category, but they also have a lot of outliers and a high variance on their latency, which is why they come out 9th out of 18 in terms of the CASC score this week, up from 15th last week. They do have a CASC score of 845, which is a reasonable CASC score. It’s no good being fast if you are flaky, although being fast does help.
Top performers by latency
Month of February 2020
Category
Organization
Median latency
Corporate Infrastructure
Slack
180 ms
Cryptocurrency Exchanges
FTX
243 ms
PSD2 Banks
BNP Paribas
136 ms
Search
390 ms
Social Networks
121 ms
UK Government
Police.UK
78 ms
UK Open Banking (Open Data)
HSBC
83 ms
US Government
Department of Justice
73 ms
Two changes since January with Slack replacing GitHub and Google Foursquare. It is easier to be fast than to be good! FTX are first again for latency, but 11th out of 18 overall with a respectable CASC score of 812.
Worst quality across all categories
Week ending March 2, 2020
Category
Organization
CASC Score
Corporate Infrastructure
Cisco Spark
779
Cryptocurrency Exchanges
Oasis Dex
624
PSD2 Banks
Visa
657
Search
Nobody below 900!
Social Networks
Nobody below 800!
UK Government
NHS
419
UK Open Banking (Open Data)
Halifax
764
US Government
Department of Commerce
677
Two changes this week with Halifax dropping to the bottom of UK Open Banking (Open Data) replacing Santander UK and the Department of Commerce replacing the Federal Communications Commission. The perennial under-performer NHS retains the overall Wooden Spoon for the week, in the Red Zone again and definitely in need of some attention.
Month of February 2020
Category
Organization
CASC Score
Corporate Infrastructure
Cisco Spark
717
Cryptocurrency Exchanges
Oasis Dex
573
PSD2 Banks
Visa
631
Search
Nobody below 900!
Social Networks
Nobody below 800!
UK Government
NHS
450
UK Open Banking (Open Data)
Santander (UK)
676
US Government
Department of Commerce
594
Only one change from last month with Santander (UK) replacing Halifax at the bottom of UK Open Banking (Open Banking). NHS are bottom overall replacing Oasis Dex. It’s clear that there tends to less competition at the bottom. Good APIs – and there are a lot of them - tend to stay good whilst bad ones stay horrid.
Something of interest
The overall best performer in February was BNP Paribas with an excellent CASC score of 972. What made BNP Paribas so good? It has a 100% pass rate, tops its category for fewest outliers (0.15%) and the fast median latency (136 ms). If you win at everything, you are going to win overall! We get a 18 ms difference between the best cloud for BNP Paribas (Azure) and the worst (AWS). That’s 14% slower, although given the size of differences we often see between clouds, probably not something to worry too much about. We can see that a lot of the difference is down to handshake time with AWS 12 ms slower than Azure or more than 50%. If you want to get the best possible performance out of your APIs for all your users no matter which cloud they are using, it often comes down to the details of network configuration. That’s why it is important to actively monitor your APIs, so that you can discover the rate-determining steps and squeeze out all the improvements in speed. See you again in a week to see how things are shaping up as head towards the spring equinox.