BlackBoard & Oracle benchmarks on SSD storage

I spent Friday benchmarking two platforms against each other: our production BlackBoard instance (9.1 SP6) versus a development BlackBoard instance (also 9.1 SP6) that I spun up for this test. The unique thing about this test was that I put the development instance entirely on SSD storage.

Production instance:

  • BlackBoard 9.1 SP6 on VMware ESXi VM (Linux)
  • Oracle 10g R2 on HP BL465c G6 Blade (Linux)
  • BlackBoard on NetApp filer; 1 volume on a 45-disk RAID-DP aggregate
  • Oracle on local HP SAS RAID-1 disk set

Development instance:

  • BlackBoard 9.1 SP6 on VMware ESXi VM (Linux)
  • Oracle 10g R2 on VMware ESXi VM (Linux)
  • BlackBoard on FusionIO ioDrive SSD
  • Oracle also on (the same) FusionIO ioDrive SSD

With the exception of workloads (itself a significant difference, to be sure), I tried to keep everything else the same — same patches, same OS revision, same LDAP & SSL configurations, etc.

To test performance, I had a suggestion from Steve (@Seven_Seconds) at BlackBoard: hit the /webapps/login/ page with ab, the Apache benchmark tool.

The results were, well, strange. With each test of em running 5000 requests to /webapps/login/, here is a graph:

NetApp v SSD

You can see that the mean response times SSD (blue line) equals or out-performs the NetApp (red line) at every concurrency value tested. What is particularly interesting, though, is the incredibly slow rate at which the standard deviation of the SSD (purple bars) grows; compare that with the rate at which the standard deviation of the response times of the NetApp (green bars) grow.

Right now, I don’t have a clear answer as to why the deviations from the mean grow so differently. The NetApp filer has a significantly different workload to the SSD (the SSD has no workload other than this test), but the exponential growth of the NetApp’s standard deviation is something I’ll need to investigate further.

Last but not least, there were no SSD-specific tuning options put in place, nor where there any NetApp-specific tuning options. This was as vanilla as both installs could get in order to keep everything relatively comparable.


One thought on “BlackBoard & Oracle benchmarks on SSD storage

  1. This test is interesting. I wouldn’t think that storage backends would make that much difference for a non-dynamic page like /webapps/login/. Then again, Bb writes both a LOGOUT and a PAGE_ACCESS event to ACTIVITY_ACCUMULATOR_QUEUE for each request (and possibly other DB read/writes). It looks like in your production instance the Oracle DB is only using a RAID-1 disk set (instead of the 45-disk RAID-DP aggregate). It’s probably slower because the page you’re benchmarking is writing to the DB multiple times and the DB’s HDD isn’t as fast on the production instance (dev SSD outperforms production RAID-1?).

    BTW, I think “SSD (blue line)” should be “SSD (red line)” and “NetApp (red line)” should be “NetApp (blue line)”.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s