Approximately two years ago, we published an article comparing the performance of SpreadsheetWeb (5.3) with Google Sheets and Microsoft Office 365 Excel APIs. Since then we have significantly improved the performance of SpreadsheetWeb's calculation engine and its API. We decided to repeat the same tests and compare the result.
We will skip the details of the testing methodology and experimental setup since they are identical to those from two years ago. We will jump right into the results. Following chart compares average response times from SpreadsheetWeb 6.11, Google Sheets, and Office 365 APIs for 1, 5, and 25 concurrent user scenarios.
Performance of SpreadsheetWeb API is significantly faster than both Google Sheets and Office 365 APIs as expected. Note that, because response times of SpreadsheetWeb are under 1 sec for all concurrency scenarios, results don't appear visually in this linear chart. Performance figures of Google Sheets are also about 40% faster from 2 years ago. Similarly, Office 365 API is also about 40-60% faster than previous test results.
We also compared the accuracy of results (chart below). Results from SpreadsheetWeb were 100% correct on all concurrent user scenarios similar to the previous tests except for 25 concurrent users. Results from SpreadsheetWeb 5.3 were 99.6% accurate in this case due to server overload. SpreadsheetWeb 6.11 is now 100% accurate.
As for Google Sheets and Office 365, accuracy of results diminish significantly after 1 user scenario. This is mainly due to the reasons explained in the previous article that neither APIs were designed to handle these type of usage. Both of those APIs require two consecutive calls to set the inputs and get the outputs of the calculations. When there are too many calls to the API, it cannot complete the calculation for a given input set before its output is requested. Hence, the only way to improve the accuracy is either to lock the workbook or introduce an delay of a few seconds between each write and read calls. However, this would further slow down the response times for both APIs.
The reason SpreadsheetWeb is so accurate compared to both Google Sheets and Office 365 is that it takes only one API call to complete a single calculation (write and read). It is designed to handle necessary locking mechanism in the internal engine. As a result, it can handle large concurrent user scenarios with 100% accuracy.
The chart below compares the performance of SpreadsheetWeb 6.11 with 5.3. SpreadsheetWeb 6.11 calculation engine and API is significantly faster than 5.3. Average response times for all concurrent user scenarios were under 1 sec. This improvement can be attributed to significant optimizations in SpreadsheetWeb's calculation engine as well as a complete overhaul of its API.
