API Ratings: Dec 7-14 | Pager Duty Outliers

API Ratings: Performance

For our API ratings, we look at over 100 APIs and pull all the metrics together to give you a general feeling for the service quality for an organization’s APIs in a particular API ratings category with our patented CASC score.

Something of interest

Pager Duty has a lot of outliers – 7.10% – more than twice the number of Slack (3.38%). If we look at Total Time API ratings, we can see there is a lot of variation between locations.

AWS North America is more than four times quicker than Azure South America and we see that the 95^th percent time is nearly three times faster. This indicates that we may get a lot of the outliers from South America.

We can see that it is the PageDuty List Incidents endpoint that is pulling down PagerDuty overall in terms of outliers. This endpoint has 18.4% outliers. Digging down we can that there is quite a big variation between clouds in terms of percentage of outliers with IBM Cloud being nearly three times more likely to produce an outlier than Azure.

That’s potentially going to have a big impact on user experience. This is why you need to actively monitor your APIs from a range of clouds and locations. Just because an API works fine from one cloud location does not mean that your users who are making calls from other ones are going to be getting the same performance.

50% of calls from South America are outliers this week, which is a serious deterioration from the previous week, but over 36% outliers is still not good. These results suggest that Pager Duty need to take a good hard look at the configuration around this endpoint in particular and especially whether there are any issues around it for IBM Cloud and South America.

We don’t have any IBM Cloud locations in South America, so that can’t be the worst over cloud location.

There are always going to variations in performance between cloud locations because the world is large and the speed of light finite, but the goal of API engineering should be minimizing them as much as possible. Actively monitoring your APIs is a vital step in ensuring that all your users get the performance they expect.

API Ratings: Overall Quality

Week Ending 14 December 2020

Category	Organization	CASC score
Corporate Infrastructure	Pivotal Tracker	9.55
COVID-19	nubentos.com	9.42
Search	Microsoft Bing	9.33
Social Networks	Google	9.62
UK Government	GOV.UK	9.50
US Government	Department of Justice	9.71
Messaging	Slack	9.52

Department of Justice in the US Government category takes the overall API ratings title again this week with a CASC score of 9.71, down from 9.81. A CASC score of over 9.00 is very good and one of 9.50 or more exceptional. Three of the six categories are headed by organization with a CASC score of 9.50 or more this week. Sustaining a CASC score of >9.25 over a period of several weeks is a good showing and congratulations to those organizations that achieved it. This week, all of the API ratings categories are topped by an API with a CASC score of more than 9.33.

API Ratings: Top Performers by Latency

Week Ending 14 December 2020

Category	Organization	Median Latency
Corporate Infrastructure	Microsoft Office	213 ms
COVID-19	WHO	84 ms
Search	Google	399 ms
Social Networks	Google	61 ms
UK Government	Police.UK	84 ms
US Government	Department of Justice	46 ms
Messaging	Slack	260 ms

Department of Justice in the US Government takes the overall title again with a median latency of 46 ms, the same as last week. An important caveat: medians can be misleading! An API might have a fast median latency but produce many slow outliers. So just being fast isn’t everything. You have to be reliable too if you want to have good APIs and get a high CASC score!

API Ratings: Worst Quality

Week Ending 14 December 2020

Category	Organization	CASC score
Corporate Infrastructure	Nobody under 8.00!
COVID-19	thevirustracker.com	6.58
Search	Nobody under 8.00!
Social Networks	Flickr	7.67
UK Government	NHS	6.18
US Government	NASA	6.05
Messaging	Cisco Webex Teams	7.35

NASA in the US Government category takes the overall Wooden Spoon this week with a CASC score of 6.14. We launched API.expert earlier in the year as a simple way to provide everybody with insights into the API economy and the leading APIs in different categories in particular. We are expanding our coverage of Open Finance APIs from around the world, so one of the major changes we made recently to these round ups has been to pull out the financial APIs. See you again in a week as we look at API ratings when it will be almost Christmas! Also don’t forget to look out for our monthly report for November and the weekly and monthly financial API reports.

Do more with API monitoring!

If you are new to APImetrics, sign up for a free trial so see how they could work for you with our free API set.

If you have any questions, please don't hesitate to contact us.