BlackBoard & Oracle benchmarks on SSD storage
by bitpushr
I spent Friday benchmarking two platforms against each other: our production BlackBoard instance (9.1 SP6) versus a development BlackBoard instance (also 9.1 SP6) that I spun up for this test. The unique thing about this test was that I put the development instance entirely on SSD storage.
Production instance:
- BlackBoard 9.1 SP6 on VMware ESXi VM (Linux)
- Oracle 10g R2 on HP BL465c G6 Blade (Linux)
- BlackBoard on NetApp filer; 1 volume on a 45-disk RAID-DP aggregate
- Oracle on local HP SAS RAID-1 disk set
Development instance:
- BlackBoard 9.1 SP6 on VMware ESXi VM (Linux)
- Oracle 10g R2 on VMware ESXi VM (Linux)
- BlackBoard on FusionIO ioDrive SSD
- Oracle also on (the same) FusionIO ioDrive SSD
With the exception of workloads (itself a significant difference, to be sure), I tried to keep everything else the same — same patches, same OS revision, same LDAP & SSL configurations, etc.
To test performance, I had a suggestion from Steve (@Seven_Seconds) at BlackBoard: hit the /webapps/login/ page with ab, the Apache benchmark tool.
The results were, well, strange. With each test of em running 5000 requests to /webapps/login/, here is a graph:
You can see that the mean response times SSD (blue line) equals or out-performs the NetApp (red line) at every concurrency value tested. What is particularly interesting, though, is the incredibly slow rate at which the standard deviation of the SSD (purple bars) grows; compare that with the rate at which the standard deviation of the response times of the NetApp (green bars) grow.
Right now, I don’t have a clear answer as to why the deviations from the mean grow so differently. The NetApp filer has a significantly different workload to the SSD (the SSD has no workload other than this test), but the exponential growth of the NetApp’s standard deviation is something I’ll need to investigate further.
Last but not least, there were no SSD-specific tuning options put in place, nor where there any NetApp-specific tuning options. This was as vanilla as both installs could get in order to keep everything relatively comparable.
This test is interesting. I wouldn’t think that storage backends would make that much difference for a non-dynamic page like /webapps/login/. Then again, Bb writes both a LOGOUT and a PAGE_ACCESS event to ACTIVITY_ACCUMULATOR_QUEUE for each request (and possibly other DB read/writes). It looks like in your production instance the Oracle DB is only using a RAID-1 disk set (instead of the 45-disk RAID-DP aggregate). It’s probably slower because the page you’re benchmarking is writing to the DB multiple times and the DB’s HDD isn’t as fast on the production instance (dev SSD outperforms production RAID-1?).
BTW, I think “SSD (blue line)” should be “SSD (red line)” and “NetApp (red line)” should be “NetApp (blue line)”.