Impact of Location and Cloud Service Host on API Performance

Summary

Each day, APImetrics software agents in more than 60 locations around the world regularly exercise a wide variety of API endpoints. APImetrics and its customers use the results of the calls to measure the performance and quality of business-critical APIs. On average, over 600,000 API calls are made by our agents every day.

Because the APIs that are probed are real-world APIs, APImetrics has been able to build an unrivalled historical dataset over the past 18 months that represents the real state of the global API ecosystem. In this report, we present the conclusions of the largest ever analysis of general API performance by cloud service and location performed to date.

Because the APIs that are probed are real-world APIs, APImetrics has been able to build an unrivalled historical dataset over the past 18 months that represents the real state of the global API ecosystem. In this report, we present the conclusions of the largest ever analysis of general API performance by cloud service and location performed to date.

The report was created from an analysis of more than 285,000,000 API tests to more than 5,600 API endpoints, which generated 5.7 billion separate data points in the 18-month period from January 2016 to June 2017. Because of the very large sample size, we are confident that we can make a statistically valid comparison between clouds and regions.

API performance varies, often dramatically, between clouds and location, meaning it is essential, when planning where to host services, who your customers are, where they are and what they will be using to access your services.

In this report, we will examine these findings in more detail, and provide detailed results.

Key Findings

API Calls are Getting Faster

In general, API response times are getting faster. Network infrastructure is improving continuously in all regions.

It is clear from the decrease in total API call time across all cloud services that something resembling a Moore's Law is in effect for call latency. More fiber bandwidth, both trans-oceanic and trans- continental, will improve overall latency by reducing bottlenecks. Cloud services without their own fiber still benefit from the constantly improving network capacity of third-party providers. Advances in switching and routing technology also help reduce latencies.

It will be interesting to see if this trend continues through the rest of 2017, and whether it is possible to calculate a characteristic timescale for the process.

Chart: Total API call time less DNS, by region. Series: East Asia, Europe, India, North America, Oceania and South America. Source note: the PDF shows values only as visual lines, so none are transcribed.

India

The only region where performance decreased significantly during the survey was India. Latency doubled from the introduction of the Indian cloud services toward the end of the data sample in June 2017. Our hypothesis, based on the trend lines, is that the introduction of the India region has been popular, leading to rapid increases in use and what is described as the noisy neighbor problem. This has been a reported issue from multiple cloud hosting providers as services become more and more popular.

Chart: Total API call time less DNS, India. Series: India. Source note: the PDF shows values only as a visual line, so none are transcribed.

Google Has a Significant Latency Advantage

Google has invested heavily in Cloud capacity and it is showing in improvements in their results. Faster performance in 2016 could explained by Google only having a small number of data centers from which to call APIs, however, as 2017 has continued, more Google locations have come online and calls from those to common services have continued to be faster.

As of June 2017, we see a 50% performance difference, on average, between calls made from Google's Cloud services and other cloud hosts and apart from a degraded period in April 2017, the gap has remained consistent. There is statistically very little difference now between the other 3 measured hosting services.

Chart: Total API call time less DNS, by cloud service. Series: AWS, Azure, Softlayer and Google. Source note: the PDF shows values only as visual lines, so none are transcribed.

Connection Time and Total Time Do Not Correlate

Network access and cloud set-up show that TCP connect times have almost no correlation with total API call time.

The most extreme differentials were in South America, the slowest region for overall call times at more than 450ms per total call, but with a TCP connection time of just 6ms. In contrast, calls from Australia took a total time of 350ms on average with a connection time of 125ms.

This raises a significant flag. Using tools that only measure the ping response from a target endpoint don't analyze the full transaction.

Chart: South America, total call time v TCP connect. Series: AWS (connect time) and AWS (total call time). Source note: the chart has a few small data labels that are too small to read reliably in the PDF, so they are not transcribed. The figures in the text are transcribed.

Location Matters

The performance of different cloud services changes continuously. A service or a country that performs well one month might see degraded performance the next. In some cases, this might be because more and more services are migrating to the cloud, increasing the load on network infrastructure. When load exceeds capacity, serious performance degradations can occur.

An analogy is the paradox of the freeway. Widening a congested road adds capacity. Added capacity attracts new traffic. At first the increased capacity can cope with the increasingly higher volume of traffic, but as the volume exceeds that capacity, bottlenecks appear and traffic jams, resulting in congestion yet again.

A similar pattern can be seen with network infrastructure. Adding a trans-oceanic fiber link will provide a boost to API performance, but the increased bandwidth will soon reach capacity as more services are introduced. Consequently, organizations should look to maximize flexibility in locations of servers. The general rule is to be as close as possible to your users. If many of your API users are on other continents than where your service is hosted, it makes sense to multi-host in those regions.

Organizations should actively measure the performance of the APIs they expose and consume to understand how their end users are affected by location and cloud services. Depending on the nature of the service, it might be possible to change cloud service or location if significant degradation of performance is seen over a certain period. Alternatively, organizations might want to multi-host not just between regions, but between cloud services. But above all, it is essential to remember that performance can and does vary significantly. What might have the best choice of cloud service location 12 months ago might not be today.

Chart: Variation in Performance, UK v Ireland. Series: Ireland and United Kingdom. Source note: the PDF shows values only as visual lines, so none are transcribed.

The Great Firewall of China

Known generally as the ‘Great Firewall’ the infrastructure deployed by the Chinese government to block certain websites (such as Google or Facebook) does have a significant impact on API calls. Measured over the course of 2016 and into 2017, a typical API call took in excess of 1100ms, a full 400ms slower than a similar call made from elsewhere in the Asia-Pacific region.

If your service is provided inside China and you are unable to host locally, it is important to review the connections in the Asia Pacific region to determine if there is a better local hosting or co-location cloud which would provide better connectivity rather than dealing with the latency of a trans-pacific connection.

Chart: China API call impact. Series: China, Hong Kong, Japan, Korea and Singapore. Source note: the PDF shows values only as visual lines, so none are transcribed.

Technical Methodology

APImetrics makes calls to user-specified API endpoints and records the response and other metrics, including the duration of the various components of the call. These endpoints include a wide variety of social networks, developer tools, private APIs, and government/ corporate infrastructure APIs. Calls are made from the APImetrics software agents located in various cloud locations using synthetic transactions utilizing the correct API authorizations and authentications for the call.

The calls used in this study were made from different cloud locations between January 1, 2016 and June 30, 2017. We made calls from the AWS, Azure, Google and SoftLayer cloud services located in Asia, Europe, North America, Oceania and South America. We brought on line new cloud locations as soon as they became available. For instance, locations in India were added in early 2017.

Only passing calls are included in the results. Call times for failing calls have a different distribution (they are often either much shorter or longer than passing calls). We therefore have excluded them to prevent the results from being skewed by outliers associated with failing calls. Given the sample size, we have determined that statistically the sample accounts for variations in individual API call types.

DNS lookup time was also excluded from the calculated call times. This is because DNS lookup time is observed not to vary significantly between locations, and thus tends to obscure the fundamental underlying variation in performance between locations.

All latencies shown are the median of the results recorded for calls to a particular API endpoint from a cloud location in a given period. Time is noted in milliseconds (ms).

We have a reasonably balanced distribution of calls across different cloud services. There are fewer Google locations than for other cloud services, but more than 30 million calls were made from Google locations. SoftLayer (IBM Cloud) has the most locations, but some of these locations tend to be less utilized by APImetrics than other cloud locations.

The six regions can be divided into a group of large regions (East Asia, Europe and North America) and a group of small regions (Oceania, South America and India). Because the India locations only became available in early 2017, the overall number of calls from this region are much smaller than even small regions as of mid-2017.

Chart: Call made by Cloud. Series: Softlayer, AWS, Azure and Google. Source note: the pie chart shows no printed values, so the slices are not transcribed.

Chart: Distribution by Region. Series: Europe, North America, Oceania, South America, India and Asia. Source note: the pie chart shows no printed values, so the slices are not transcribed.

Detailed Results

Fastest Cloud by Service (June 2017)

Total API Call Time: Google

• Over the past 18 months, Google has consistently been the fastest cloud, both for connection time and total API call time:

Chart: Total API Call Time less DNS, by cloud service. Series: AWS, Azure, Google and Softlayer. Source note: the PDF shows values only as visual lines, so none are transcribed.

Fastest Region (June 2017)

Total API Call Time: North America

• It is noteworthy that the connection time for South America is currently much lower than other regions. This suggests that new infrastructure has recently come on line for that region. It might also be related to lower overall usage in that region, and links are not heavily loaded at present. Connection time is only one aspect of overall latency as it relates to the handshake with the API gateway. Overall latency is affected by server-side processing time and payload download time. Payload download time is affected by local bandwidth and might be slow even if connection time is short.
• North America and Europe are the fastest regions, reflecting distance between requesting and serving systems and the amount of bandwidth available. Europe is likely slower because many serving systems are located in the US.
• The steady increase in latency for the India region likely reflects high levels of popularity for the new cloud locations launched in early 2017. It will be interesting to see if this trend continues over the next few months. It is possible that additional bandwidth will be brought into the region (although this could take a significant length of time), or some users may relocate to other locations if latencies continue to increase as capacity is reached.

Chart: Total API Call Time less DNS, by region. Series: Asia, Europe, India, North America, Oceania and South America. Source note: the PDF shows values only as visual lines, so none are transcribed.

Regional Performance by Cloud (June 2017)

Fastest Cloud

East Asia

Total API Call Time: Google

• China Hosting is slow for both connection time (~350ms) and total time (~1100ms). This is a consequence of the additional latency from the Great Firewall of China. Although China Hosting shows the characteristic reduction in latency over the past 18 months, it is persistently ~400ms slower than other cloud services from East Asia.
• Google has had the fastest connect time over the past 18 months.
• For total time, Google was much slower from April-June 2017 than it had been in 2016 and Q1 2017. It is not obvious why total time suddenly shifted to the regional mean in April 2017. AWS, Azure and SoftLayer have followed a long term downward trend. Total time for Google also decreased in May and June 2017 to become lower than its competitors again, albeit only slightly.

Chart: Total API call time less DNS, East Asia region. Series: AWS, Azure, China VPS Hosting, Google and Softlayer. Source note: the PDF shows values only as visual lines, so none are transcribed.

Europe

Total API Call Time: Google

• Azure connection time has gotten persistently worse over the last 18 months.
• Total time for the four cloud services have been fairly similar over the past six months.

Chart: Total API Call Time less DNS, Europe region. Series: AWS, Azure, Google and Softlayer. Source note: the PDF shows values only as visual lines, so none are transcribed.

North America

Total API Call Time: Google (narrowly)

• We observed a significant deterioration in AWS and Azure over the last six months. This might suggest that both services are using the same network for a significant portion of their traffic.
• This is potentially also reflected in the increases in total time for AWS and Azure in Q1 2017 and April 2017.
• As of June 2017, AWS and Google are almost 100ms faster than Azure and SoftLayer.

Chart: Total API Call Time less DNS, North America region. Series: AWS, Azure, Google and Softlayer. Source note: the PDF shows values only as visual lines, so none are transcribed.

Oceania

Total API Call Time: Tie

• As of June 2017, AWS, Azure and SoftLayer have locations in Australia. The increase in connection time from Q1 2017 might represent an increase in usage of cloud services.
• The performance of the three cloud services in 2017 has so far been essentially identical. This suggests that all three services are using the same network infrastructure.

Chart: Total API Call Time less DNS, Oceania region. Series: AWS, Azure and Softlayer. Source note: the PDF shows values only as visual lines, so none are transcribed.

South America

Total API Call Time: AWS (narrowly)

• Connection time has been quick from Q1 2017 for AWS and Azure, the only two cloud services operating in South America (both from Brazil). This suggests they are both connecting to the same new network edge that has recently been provided to Brazil.
• Again, the similarity in total time indicates that both cloud services are using the same network infrastructure. At ~400ms, total time is sufficiently longer than any other region including Oceania. This indicates that even through the connection time is fast, the amount of bandwidth to South America is still limited.

Chart: Total API Call Time less DNS, South America region. Series: AWS and Azure. Source note: the PDF shows values only as visual lines, so none are transcribed.

Regional Performance by Country

Fastest Country

East Asia

Total API Call Time: Japan/Singapore (tie)

• China has by far the slowest connection time because of the Great Firewall.
• Japan, Singapore and Taiwan are much faster than Hong Kong and Korea, as these countries have quick edge connections.
• Total time for Taiwan has increased compared to 2016, perhaps because of bandwidth saturation.
• As of June 2017, total time appears to have converged for all territories expect China. Although China is also getting faster, the Great Firewall adds ~400ms time lag to the total time, and thus any services provided by China-based servers to users outside China.

Chart: Connection Time, countries in East Asia region. Series: Hong Kong, Korea, Japan, Singapore and Taiwan. Source note: the PDF shows values only as visual lines, so none are transcribed.

Chart: Total API Call Time less DNS, countries in East Asia region. Series: Hong Kong, Korea, Japan, Singapore and Taiwan. Source note: the PDF shows values only as visual lines, so none are transcribed.

Europe

Total API Call Time: Germany (narrowly)

• UK connection time has significantly degraded since April 2017.
• UK is also the slowest for total time. This might indicate bandwidth saturation or issues with network infrastructure. As of June 2017, all countries have total times within the 150-300ms range.

Chart: Total API Call Time less DNS, countries in Europe region. Series: U.K., France, Italy, Ireland, Belgium, Netherlands and Germany. Source note: the PDF shows values only as visual lines, so none are transcribed.

North America

Total API Call Time: Canada (narrowly)

• Connection time is now longer for North American countries than in 2016. This might suggest heavy loads on edge connects.
• Total time for all countries in North America is still higher in June 2017 than the means for 2016 and Q1 2017, although down from an April 2017 peak. Is North American network infrastructure struggling to keep up with demand?

Chart: Total API Call Time less DNS, countries in North America region. Series: Canada, Mexico and U.S. Source note: the PDF shows values only as visual lines, so none are transcribed.

Best and worst countries by Region

It is difficult to state which countries are best by region, because there are significant changes month on month. North America is the fastest region, with the three countries close in terms of total time. Countries within East Asia and Europe appear to have converged to within a ~200ms band.

Degradations in performance in Taiwan and the UK might be related to the saturation of network resources or other issues with network infrastructure.

About APImetrics

APImetrics are leaders in API performance solutions, providing accurate, independent performance data for APIs and cloud services around the world. Their platform uses machine learning technologies to measure and report on overall latency and performance quality using the CASC (Cloud API Service Consistency) scoring system which provides a score, out of 1000, relating to API quality.

Author

Dr. Paul Cray is a computational physicist working with APImetrics on the application of Machine Learning technologies to API quality.

[email protected]

Glossary

Terms this paper defines or explains in its own text.

Great Firewall

The infrastructure deployed by the Chinese government to block certain websites, such as Google or Facebook. The paper reports that it adds a significant performance and speed overhead to API calls.

CASC (Cloud API Service Consistency)

The scoring system APImetrics uses to report overall latency and performance quality. It provides a score, out of 1000, relating to API quality.

Median latency

In this study, the median of the results recorded for calls to a particular API endpoint from a cloud location in a given period. Time is noted in milliseconds (ms).

About APIContext

APIContext eliminates blind spots for enterprises across the digital delivery chain with proactive synthetic monitoring, performance analytics, and automated conformance validation. Our platform delivers actionable insights so connected systems perform and conform—ensuring every interaction is trusted, secure, and compliant.

What are your APIs saying to AI? Contact us to find out.