April is here (and sadly COVID-19 is likely to still be here for several months to come), so we gather together again to discuss the State of the APIs. We recently launched API.expert as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. The basic service is free and will remain free, but other, related services will be built on top of it, including your very own API.expert dashboards if you want them! Each week, month and year we’ll look at the APIs we track in a variety of sectors for their quality (using our patent pending CASC score) and other factors like uptime and availability. If something is missing that you think should be here, don’t hesitate to drop us a line! Now, onto what the last seven days and last month can tells us about where the world of APIs is in early 2020.
API Performance Headlines
We look at over 200 APIs and pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular category. We do see that certain providers consistently vie for top spot in their category including Twitter, Slack and Github with others doing less well such as Cisco’s Spark services. Top Performers based on overall quality in each category were:
CASC score
Week ending March 30, 2020
Category
Organization
CASC score
Corporate Infrastructure
GitHub
960
Cryptocurrency Exchanges
HitBTC
935
PSD2 Banks
ABN AMRO Bank
956
Search
940
Social Networks
966
UK Government
GOV.UK
988
UK Open Banking (Open Data)
Danske Bank
967
US Government
Department of Justice
972
Three changes this week with HitBTC claiming the top spot in Cryptocurrency Exchanges from long-time favorite LATOKEN. Danske Bank takes the crown in UK Open Banking (Open Data) from perennial contender, Bank of Ireland. GOV.UK swaps places with Police.UK in UK Government. GOV.UK takes the overall title keeping the it in the public sector. A CASC score of over 900 is very good and one of 950 or more exceptional. Six of the eight categories are headed by organization with a CASC score of 950 or more this week, which is a good showing. Sustaining a CASC score of >925 over a period of several weeks is an excellent effort and congratulations to those organizations that have achieved it.
Top performers by latency
Week ending March 30, 2020
Category
Organization
Median latency
Corporate Infrastructure
Slack
193 ms
Cryptocurrency Exchanges
FTX
208 ms
PSD2 Banks
Open Bank Project
197 ms
Search
384 ms
Social Networks
122 ms
UK Government
Police.UK
76 ms
UK Open Banking (Open Data)
HSBC
92 ms
US Government
Department of Justice
70 ms
No changes again this week for median latency. This is an exceptionally stable category. This shows that APIs are more consistent in latency than they are in overall performance.
An important caveat: medians can be misleading!
An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score! As so often, FTX might have a low enough latency to top the Cryptocurrency Exchange category, but they also have a lot of outliers and a high variance on their latency, which is why they come out 6th out of 18 in terms of the CASC score this week, up from 8th last week. They do though have a CASC score of 880, which is a reasonable CASC score. It’s no good being fast if you are flaky, although being fast does help, but FTX do certainly seem to be going in the right direction at the moment.
Worst quality among all categories
Week ending March 30, 2020
Category
Organization
CASC Score
Corporate Infrastructure
Cisco Spark
753
Cryptocurrency Exchanges
Oasis Dex
184
PSD2 Banks
Visa
651
Search
Nobody below 800!
Social Networks
Nobody below 800!
UK Government
NHS
361
UK Open Banking (Open Data)
Halifax
653
US Government
Department of Commerce
542
Three changes this week with the Department of Commerce swapping places again with the Bureau of Labor Statistics in US Government, Halifax replacing HSBC at the bottom of UK Open Banking (Open Data) and no API in Social Networks scoring in the Amber Zone. Oasis Dex takes the overall last prize for the fourth week in a row, and has somehow managed to “top” last week’s appalling CASC score of 187 with one of 184. They have a pass rate of 68.01%, whilst 12 of their 17 peers manage 100% (next lowest is Coinlore at No. 17 with 97.62%). They also have 35.86% outliers (next worse Coinlore again with 7.59%). It is hard to believe that Oasis Dex’s performance hasn’t been having a serious impact on its users over the last few weeks…
Something of interest
As we discussed last week with respect to Cisco Spark, Corporate Infrastructure is an interesting category at the moment because of the massive increase in homeworking because of the COVID-19 pandemic (we are going to be adding Zoom to Corporate Infrastructure this week). If we look at the Corporate Infrastructure table, it looks as though the APIs are keeping up with the strain on them. 
Only 4 out of 14 APIs are down for the week with 4 up and 6 the same. As usual, only Cisco Spark in is the Amber Zone. 9 of the APIs have a CASC score of over 900 and 4 of 950 or more. 7 out of the 14 APIs have a median latency of greater than 400 ms. The general advice since the 1970s, is that a UI should respond with 400 ms to be considered instantaneous (this is the Doherty Threshold). This is just a rule of thumb and less important when there is no human in the loop, but it is a useful metric to aim for, although whether it is achievable always or at all will depend on the exact nature of what the API is doing. With more people using these APIs at the moment, this is aspect corporate infrastructure API providers might find their users do start noticing. See you again in a week to look at the State of the APIs for the week (and the month of March) as we bound confidently into April (the cruelest month).