In the past, we have compared the performance of SpreadsheetWeb’s public API to Office 365 and Google Sheet’s public APIs. We have run various concurrency tests to emulate real life, multi-user web applications scenarios. While those tests sought to emulate the response times for a web application using SpreadsheetWeb vs. Office 365 APIs as a back-end calculation engine, they also include factors that are subject to some conditions and variation, like network overhead, server configuration, and capacity.

We wanted to test and compare the core performance of SpreadsheetWeb against Excel, so we needed to aim for the lower-level interfaces: the calculation engines. We ran our tests directly using Excel’s Interop against SpreadsheetWeb’s internal engine API.

 

Methodology

We developed a .NET application that performs the following set of actions:

  1. Load a workbook into memory.
  2. Configure the set of input cells and output cells.
  3. Set inputs.
  4. Calculate the results.
  5. Retrieve the outputs.

We ran the application with Office 365 Interop and SpreadsheetWeb’s new calculation engine. In both cases, the time to load the workbook into memory were excluded since our goal was to compare the actual calculation speeds. In practice, whether you use Excel or SpreadsheetWeb, loading times occur only once when the application or file is first opened. Calculation times are relevant for all subsequent operations.

We selected 46 different Excel files containing a variety of complex formulas and dependencies. In each file, we narrowed down the input cells that triggered the majority of the calculation dependency tree to execute. Then, we set random but relevant input values to those cells and measured the time to calculate the outputs.

In each file, we repeated the calculations five times, and took the average. Results are measured and presented in seconds. We ran these tests on a computer with Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz (Turbo 2.39 GHz) and 8GB of RAM running 64-bit Windows 10 operating system.

 

Results

We found that in 35 out of 46 files (~76%), SpreadsheetWeb was faster at performing the calculations.

 

Calculation Times (sec)
File #Size (MB)SpreadsheetWebExcel
111.11.5171.089
21.90.1580.043
30.90.3673.637
412.00.0150.049
50.80.0170.031
61.10.0190.034
70.70.0130.071
80.50.0260.052
93.81.4540.799
103.80.0420.855
110.70.0200.048
122.30.3630.441
131.00.0450.024
143.30.1480.850
150.40.0131.878
160.40.0930.045
170.30.0170.028
180.70.0070.041
192.10.0160.078
200.20.0060.020
216.10.1290.318
2214.10.1961.289
230.50.0040.061
240.30.0170.025
255.50.0540.071
262.90.6470.294
270.10.0120.052
286.80.0070.114
290.30.0020.051
302.20.0480.021
313.20.0330.056
321.20.0320.023
334.70.2720.572
340.90.0710.112
3514.10.1690.961
363.00.0270.085
375.60.0530.184
380.50.0030.025
393.30.0540.262
403.00.0590.187
411.50.0800.063
4222.70.1630.102
430.10.0140.064
447.40.0570.033
452.30.0620.153
465.30.0600.654

 

The chart below compares calculations times for each file. Each dot in the chart represents one of the 46 Excel files. The x-axis represents the time, in seconds, that SpreadsheetWeb performed the calculations, and the number on the y-axis represents how fast Excel performed the same calculations. The blue line signifies where the calculation speeds would be the same.

Performance Comparison: SpreadsheetWeb vs Excel

Note that the chart axes are logarithmic since there is quite a bit of variation between the calculation speeds across all the files.

We also wanted to quantify the calculation speed variance across the files (i.e. when x is faster, how much faster). We took the 35 files wherein SpreadsheetWeb was the faster engine and divided Excel’s calculations times by SpreadsheetWeb’s calculation times. We did the same calculation inversely for the 11 files where Excel was faster the faster engine.

When SpreadsheetWeb is faster, it is about 9.6 times faster than Excel; when Excel is faster, it is about 1.9 times faster than SpreadsheetWeb. You can see this trend in the chart, as the dots below the blue line are closer to the line whereas the dots above the line are more spread out.

 

Conclusion

One of the main focuses of SpreadsheetWeb is to execute complex Excel formulas in a web application as fast as possible, which is why we are always working to optimize our calculation engine. We spent the past three years rewriting and reoptimizing our calculation engine, which will be a part of our next major SpreadsheetWeb release.

These results show that our new calculation engine is relatively faster than Excel. The new engine will be part of SpreadsheetWeb’s 7.0 release.