In the past, we have compared the performance of SpreadsheetWeb’s public API to Office 365 and Google Sheet’s public APIs. We have run various concurrency tests to emulate real life, multi-user web applications scenarios. While those tests sought to emulate the response times for a web application using SpreadsheetWeb vs. Office 365 APIs as a back-end calculation engine, they also include factors that are subject to some conditions and variation, like network overhead, server configuration, and capacity.
We wanted to test and compare the core performance of SpreadsheetWeb against Excel, so we needed to aim for the lower-level interfaces: the calculation engines. We ran our tests directly using Excel’s Interop against SpreadsheetWeb’s internal engine API.
Methodology
We developed a .NET application that performs the following set of actions:
- Load a workbook into memory.
- Configure the set of input cells and output cells.
- Set inputs.
- Calculate the results.
- Retrieve the outputs.
We ran the application with Office 365 Interop and SpreadsheetWeb’s new calculation engine. In both cases, the time to load the workbook into memory were excluded since our goal was to compare the actual calculation speeds. In practice, whether you use Excel or SpreadsheetWeb, loading times occur only once when the application or file is first opened. Calculation times are relevant for all subsequent operations.
We selected 46 different Excel files containing a variety of complex formulas and dependencies. In each file, we narrowed down the input cells that triggered the majority of the calculation dependency tree to execute. Then, we set random but relevant input values to those cells and measured the time to calculate the outputs.
In each file, we repeated the calculations five times, and took the average. Results are measured and presented in seconds. We ran these tests on a computer with Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz (Turbo 2.39 GHz) and 8GB of RAM running 64-bit Windows 10 operating system.
Results
We found that in 35 out of 46 files (~76%), SpreadsheetWeb was faster at performing the calculations.
| Calculation Times (sec) | |||
| File # | Size (MB) | SpreadsheetWeb | Excel |
| 1 | 11.1 | 1.517 | 1.089 |
| 2 | 1.9 | 0.158 | 0.043 |
| 3 | 0.9 | 0.367 | 3.637 |
| 4 | 12.0 | 0.015 | 0.049 |
| 5 | 0.8 | 0.017 | 0.031 |
| 6 | 1.1 | 0.019 | 0.034 |
| 7 | 0.7 | 0.013 | 0.071 |
| 8 | 0.5 | 0.026 | 0.052 |
| 9 | 3.8 | 1.454 | 0.799 |
| 10 | 3.8 | 0.042 | 0.855 |
| 11 | 0.7 | 0.020 | 0.048 |
| 12 | 2.3 | 0.363 | 0.441 |
| 13 | 1.0 | 0.045 | 0.024 |
| 14 | 3.3 | 0.148 | 0.850 |
| 15 | 0.4 | 0.013 | 1.878 |
| 16 | 0.4 | 0.093 | 0.045 |
| 17 | 0.3 | 0.017 | 0.028 |
| 18 | 0.7 | 0.007 | 0.041 |
| 19 | 2.1 | 0.016 | 0.078 |
| 20 | 0.2 | 0.006 | 0.020 |
| 21 | 6.1 | 0.129 | 0.318 |
| 22 | 14.1 | 0.196 | 1.289 |
| 23 | 0.5 | 0.004 | 0.061 |
| 24 | 0.3 | 0.017 | 0.025 |
| 25 | 5.5 | 0.054 | 0.071 |
| 26 | 2.9 | 0.647 | 0.294 |
| 27 | 0.1 | 0.012 | 0.052 |
| 28 | 6.8 | 0.007 | 0.114 |
| 29 | 0.3 | 0.002 | 0.051 |
| 30 | 2.2 | 0.048 | 0.021 |
| 31 | 3.2 | 0.033 | 0.056 |
| 32 | 1.2 | 0.032 | 0.023 |
| 33 | 4.7 | 0.272 | 0.572 |
| 34 | 0.9 | 0.071 | 0.112 |
| 35 | 14.1 | 0.169 | 0.961 |
| 36 | 3.0 | 0.027 | 0.085 |
| 37 | 5.6 | 0.053 | 0.184 |
| 38 | 0.5 | 0.003 | 0.025 |
| 39 | 3.3 | 0.054 | 0.262 |
| 40 | 3.0 | 0.059 | 0.187 |
| 41 | 1.5 | 0.080 | 0.063 |
| 42 | 22.7 | 0.163 | 0.102 |
| 43 | 0.1 | 0.014 | 0.064 |
| 44 | 7.4 | 0.057 | 0.033 |
| 45 | 2.3 | 0.062 | 0.153 |
| 46 | 5.3 | 0.060 | 0.654 |
The chart below compares calculations times for each file. Each dot in the chart represents one of the 46 Excel files. The x-axis represents the time, in seconds, that SpreadsheetWeb performed the calculations, and the number on the y-axis represents how fast Excel performed the same calculations. The blue line signifies where the calculation speeds would be the same.
Note that the chart axes are logarithmic since there is quite a bit of variation between the calculation speeds across all the files.
We also wanted to quantify the calculation speed variance across the files (i.e. when x is faster, how much faster). We took the 35 files wherein SpreadsheetWeb was the faster engine and divided Excel’s calculations times by SpreadsheetWeb’s calculation times. We did the same calculation inversely for the 11 files where Excel was faster the faster engine.
When SpreadsheetWeb is faster, it is about 9.6 times faster than Excel; when Excel is faster, it is about 1.9 times faster than SpreadsheetWeb. You can see this trend in the chart, as the dots below the blue line are closer to the line whereas the dots above the line are more spread out.
Conclusion
One of the main focuses of SpreadsheetWeb is to execute complex Excel formulas in a web application as fast as possible, which is why we are always working to optimize our calculation engine. We spent the past three years rewriting and reoptimizing our calculation engine, which will be a part of our next major SpreadsheetWeb release.
These results show that our new calculation engine is relatively faster than Excel. The new engine will be part of SpreadsheetWeb’s 7.0 release.
