Performance in strange places

Sometimes, you find performance improvements in places where you weren’t looking for an improvement. I will give an example: at a meeting last week, our webmaster was passing on complaints from users that their Content Management System was performing poorly. He had done a bunch of investigative work and concluded that the problem was likely an Active Directory problem. While that’s not exactly what he meant, it’s what he said, and our Active Directory administrator was a little annoyed.

What the Webmaster actually meant was that, when publishing several thousand files that live on a NetApp filer that is Active Directory-aware, there may be issues if you’re performing thousands of LDAP lookups serially. Which is a valid point: that can get computationally expensive. So, we got to talking about it, and another engineer mentioned nscd, the name services caching daemon — something I didn’t know existed outside NIS (i.e., legacy platforms).

Long story short, it made for an amusing (and satisfying) test.

/home#  time ls -al | wc -l

real    0m19.547s
user    0m3.729s
sys     0m1.700s

Almost 20 seconds to perform a lookup on an NFS-mounted directory that has almost 4,000 unique Active Directory users. That’s a lot of UIDs and GIDs! Now, we install nscd, and run the same test once to populate the cache.

Then we run the test again, with a populated cache. Care to see the results?

/home# time ls -al | wc -l     

real    0m2.241s
user    0m0.360s
sys     0m0.570s

From 19 seconds down to less than 3 seconds. Quite the improvement! Obviously you may need to adjust your cache parameters, particularly if this is a fairly inactive system most of the time but does have the occasional need for performance — in such a case, you may set your cache to expire unusually slowly. The lesson from this, though, is that sometimes you find gains when you weren’t looking. Or even in places you didn’t know gains could be found.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s