This Week’s API Highlights
- Google in the Social Networks API category and Department of Justice in the US Government category share the overall title this week with a CASC score of 9.69.
- UK in the UK Government API category is the fastest API this week with a median latency of 70 ms.
- MailChimp in the Corporate Infrastructure API category is the overall worst quality API this week with a CASC score of 5.52.
- We take a good hard look at just what went wrong with MailChimp last week.
Something of interest
We all remember the MailChimp adverts from the original Serial back at the start of the Second Golden Age of Podcasting in 2014-5. But MailChimp is a pretty serious global player. Most of us probably receive several emails a day via MailChimp. So we expect a performant API. But MailChimp crashed from an excellent CASC score of 9.23 last week to the bottom of the overall rankings this week with a CASC score of just 5.52. What went wrong? 
The usual suspect, of course, is an outage. An extended one can ruin your CASC score in a few hours, but, no, MailChimp has 100% availability. What’s dragging it down is its consistency. There are quite a lot of outliers and the standard deviation is high. It’s not a particular quick API, but that would be OK as long as it was consistent. But this week it wasn’t. 
We can see that there is significant variation in latency between cloud locations. Interestingly, IBM Cloud is the slowest location in all four regions in which there are IBM Cloud locations (Google is the slowest in the other two). 
We see the problem when we look at the DNS Lookup Time. Only from AWS Europe is it 4 ms (and 6 locations 12 or 13 ms). From IBM Cloud Oceania, the DNS Lookup Time is 509 ms(!) and the same for Google South Asia. Optimizing the DNS Lookup Time so that it is 4 ms or 12 ms everywhere. And that’s why you need to be actively monitoring your APIs, so you know exactly which locations are underperforming and need to optimized immediately. Just looking from a single cloud or location won’t tell you that. 
Looking at the other latency component confirms that it is the DNS Lookup Time that is the rate-determining step and providing most of the variance that is driving up the standard deviation and thus driving down the CASC score. For Process Time, there’s not much variation within regions, which is generally a good sign, but there is some variation between regions and there’s the odd outlier, notably Google South America, so looking to optimize Process Time will also help to some degree with improving the CASC score.
API Analysis: Tops in Overall Quality
Week Ending 10 May 2021
Category
Organization
CASC score
Corporate Infrastructure
Box
9.49
COVID-19
covid19api.com