Relative Performance of Rich Media Content across Browsers and Operating Systems
Two of the things that Flash is often criticized for is that it:
- Uses too much CPU
- Performs significantly worse on the Mac than on Windows
This got me thinking about whether some quick tests would bear this out, and if so, whether it was isolated to just Flash content.
Below are some raw numbers showing CPU usage of various rich content across different browsers and operating systems. This includes video deployed via HTML 5, JavaScript / Canvas animations, Flash Video, and Flash animations. This is by no means a scientific study, but I do think the results show that:
- Flash does not necessarily perform worse on Mac as opposed to PC
- All rich media content, including that created with JavaScript / HTML 5 content show (sometimes widely) varying levels of performance across browsers and operating systems
All numbers below show CPU usage as a total of all CPU resources available on the system. This means that if you are running on a machine with more than one core processor, then CPU usage can be over 100%. This is how Mac shows CPU usage, and I feel it gives a clearer picture of how much CPU any individual item is using. Windows CPU usage have been normalized to show CPU usage in terms of this overall CPU usage.
All tests were run on the following hardware:
| Model Identifier: | MacPro3,1 |
| Processor Name: | Quad-Core Intel Xeon |
| Processor Speed: | 3 GHz |
| Number of Processors: | 2 |
| Total Number of Cores: | 8 |
| L2 Cache: | 12 MB |
| Memory: | 8 GB |
| Bus Speed: | 1.6 GHz |
I have both a Windows 7 and OS X partition on this computer, and Windows 7 tests were run while booted into Windows 7.
For browsers, I used the latest release browser versions. For Flash Player, I used the latest labs release (Flash Player 10.1 Beta 3 : 10,1,51,95).
Mac OS X Versions
| Mac OS X | 10.6.2 |
| Google Chrome | 5.0.307.9 beta |
| Safari | 4.0.4 (6531.21.10) |
| Firefox | 3.6 |
| Flash Player | MAC 10,1,51,95 |
Windows 7 Versions
| Windows 7 | 7600 |
| Google Chrome | 4.0.249.89 (38071) |
| Safari | 4.0.4 (531.21.10) |
| Firefox | 3.6 |
| Internet Explorer | 8.0.7600.16385 |
| Flash Player | WIN 10,1,51,95 |
IMPORTANT, THE ONLY RELEVANT COMPARISONS ARE BETWEEN PERFORMANCE OF THE SAME CONTENT ACROSS DIFFERENT BROWSERS AND OPERATING SYSTEMS. COMPARING FLASH CONTENT TO JAVASCRIPT CONTENT IN THIS TEST IS NOT VALID GIVEN THE DIFFERENCES IN THE CONTENT AND / OR FUNCTIONALITY.
Below are the pages / content run, and the amount of CPU they took to run. Lowest CPU usage is highlighted in green, and highest CPU usage is highlighted in red. I have also included a platform delta line, which shows the range in performance between the high and lowest CPU usage per operating system. Note, that as time passes, the content at some of the URLs may change, and not reflect the content tested.
Note: I am looking into why Internet Explorer is reporting 0% CPU usage on some of the JavaScript and Flash examples.
MacHeist Dynamic Canvas / JavaScript animation example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 95% | 80% |
| Safari | 25% | 80% |
| Firefox | 100% | 40% |
| Internet Explorer | NA | 0% |
9elements JavaScript / Canvas Dynamic Animation Example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 100% | 96% |
| Safari | 98% | 104% |
| Firefox | 100% - 170% | 72% |
| Internet Explorer | NA | Did Not Work |
Sublime HTML 5 HD Video Example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 85% | 24% |
| Safari | 21% | 104% |
| Firefox | 85% | 80% |
| Internet Explorer | NA | Did Not Work |
HYPE Dynamic Flash Animation Example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 25% | 16% |
| Safari | 17% | 48% |
| Firefox | 23% | 18% |
| Internet Explorer | NA | 0% |
| Mac | Windows | |
|---|---|---|
| Google Chrome | 88% | 56% |
| Safari | 58% | 64% |
| Firefox | 70% | 64% |
| Internet Explorer | NA | 104% |
Overall Comparative Performance (shows how often each platform performed the best per content type)
| Mac | Windows | |
|---|---|---|
| HTML / JavaScript | 3 | 6 |
| Flash | 2 | 4 |
| All | 5 | 10 |
I am not going to draw any definitive conclusions from this, but I do have some observations based on the results above.
- From these tests, Flash content does not perform consistently worse on Mac than on Windows.
- There is a wide range of CPU usage for HTML 5 video, depending on the browser / operating system it is being played back on, with Mac generally being slower.
- Canvas / JavaScript animations (at least those tested) seem to have high CPU usage, and generally run slower on Mac than on Windows (although not in all cases).
- Some of the HTML / JavaScript content would not run across all browsers.
- There seems to be some bug in the tested Flash Player when playing back video in Firefox. CPU usage would climb very high, and then drop. (I have reported this to the team).
- Availability of hardware acceleration for playing back video looks like it makes a huge difference in CPU usage (duh!).
- Regardless of technology (Flash and JavaScript / HTML), performance of rich media can vary widely depending on which browser / operating system it runs on.
AGAIN, AS I STATED BEFORE THE ONLY RELEVANT COMPARISONS ARE BETWEEN PERFORMANCE OF THE SAME CONTENT ACROSS DIFFERENT BROWSERS AND OPERATING SYSTEMS. COMPARING FLASH CONTENT TO JAVASCRIPT CONTENT IN THESE TESTS IS NOT VALID GIVEN THE DIFFERENCES IN THE CONTENT AND / OR FUNCTIONALITY.
Please keep comments on topic. Off topic comments will be moderated / deleted.
Update (March 10, 2010) : You can find similar (but separate) tests and discussions on this topic here and here.
Update (March 11, 2010) : Updated to make it clearer which Flash Player version was being used.
