API.expert Weekly API Analysis: MailChimp

This Week’s API Highlights

• Google in the Social Networks API category and Department of Justice in the US Government category share the overall title this week with a CASC score of 9.69.
• UK in the UK Government API category is the fastest API this week with a median latency of 70 ms.
• MailChimp in the Corporate Infrastructure API category is the overall worst quality API this week with a CASC score of 5.52.
• We take a good hard look at just what went wrong with MailChimp last week.

Something of interest

We all remember the MailChimp adverts from the original Serial back at the start of the Second Golden Age of Podcasting in 2014-5.

But MailChimp is a pretty serious global player. Most of us probably receive several emails a day via MailChimp. So we expect a performant API.

But MailChimp crashed from an excellent CASC score of 9.23 last week to the bottom of the overall rankings this week with a CASC score of just 5.52.

What went wrong?

The usual suspect, of course, is an outage. An extended one can ruin your CASC score in a few hours, but, no, MailChimp has 100% availability.

What’s dragging it down is its consistency. There are quite a lot of outliers and the standard deviation is high.

It’s not a particular quick API, but that would be OK as long as it was consistent. But this week it wasn’t.

We can see that there is significant variation in latency between cloud locations. Interestingly, IBM Cloud is the slowest location in all four regions in which there are IBM Cloud locations (Google is the slowest in the other two).

We see the problem when we look at the DNS Lookup Time. Only from AWS Europe is it 4 ms (and 6 locations 12 or 13 ms). From IBM Cloud Oceania, the DNS Lookup Time is 509 ms(!) and the same for Google South Asia.

Optimizing the DNS Lookup Time so that it is 4 ms or 12 ms everywhere. And that’s why you need to be actively monitoring your APIs, so you know exactly which locations are underperforming and need to optimized immediately.

Just looking from a single cloud or location won’t tell you that.

Looking at the other latency component confirms that it is the DNS Lookup Time that is the rate-determining step and providing most of the variance that is driving up the standard deviation and thus driving down the CASC score.
For Process Time, there’s not much variation within regions, which is generally a good sign, but there is some variation between regions and there’s the odd outlier, notably Google South America, so looking to optimize Process Time will also help to some degree with improving the CASC score.

API Analysis: Tops in Overall Quality

Week Ending 10 May 2021

Category	Organization	CASC score
Corporate Infrastructure	Box	9.49
COVID-19	covid19api.com	9.54
Search	Google	9.28
Social Networks	Google	9.69
UK Government	Met Office	8.92
US Government	Department of Justice	9.69
Messaging APIs	RingCentral (Sandbox)	9.05

Google in the Social Networks API category and Department of Justice in the US Government API category share the overall title this week with a CASC score of 9.69. A CASC score of over 9.00 is very good and one of 9.50 or more exceptional. Four of the seven categories are headed by organizations with a CASC score of 9.50 or more this week, up from three last week. Sustaining a CASC score of >9.25 over a period of several weeks is a good showing and congratulations to those organizations that achieved it. This week, five out of seven categories are topped by an API with a CASC score of 9.28 or more. Nobody has a CASC score of 9.00 or over in the UK Government API category.

API Analysis: Top Performers by Latency

Week Ending 10 May 2021

Category	Organization	Median Latency
Corporate Infrastructure	Google OpenID Connect	102 ms
COVID-19	WHO	89 ms
Search	Duck Duck Go	136 ms
Social Networks	Google	56 ms
UK Government	Police.UK	70 ms
US Government	Department of Justice	104 ms
Messaging APIs	Slack	279 ms

Police.UK in the UK Government is the fastest API this week with a median latency of 70 ms. An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. These won’t affect the median, but they mean that users can experience many calls that were unacceptably slow. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score!

API Analysis: Worst Quality

Week Ending 10 May 2021

Category	Organization	CASC score
Corporate Infrastructure	MailChimp	5.52
COVID-19	nubentos.com	6.47
Search	Nobody under 8.00!
Social Networks	Foursquare	7.05
UK Government	NHS	6.63
US Government	Environmental Systems Research Institute	6.14
Messaging	Nobody under 8.00!

MailChimp in the Corporate Infrastructure API category is the overall worst quality API this week with a CASC score of 5.52. Two out of the seven categories don’t have an API outside the Green Zone and thus don’t appear in the table, the same as last week.

Reliable API monitoring is hard.

Make it simple with API.expert.

API.expert provides actionable API performance and quality API metrics for leading APIs including API uptime, API quality and regional cross-cloud API latency analysis.

Whether you’re an API owner or an API developer, API.expert will help you save time, money and customers by delivering access to clear API rankings for all the services you depend on.

But you don’t have to be an API expert to use API.expert - simply sign up with one click, and start monitoring leading APIs in your industry today.