We’ve implemented a short-term/long-term caching strategy in 4.0 (Build 603) which has significantly boosted performance.

I’ve included a bit of technical information for anyone interested. If this kind of thing puts you to sleep, the above sentence pretty much says it all: with build 603 you should be finding the system even faster than it was already. During the maintenance window last weekend we updated our hosted Cerb4 helpdesks to this version.

Now for the technical details:

Long-term vs. Short-Term Caching

Long-term caching stores frequently-read and infrequently-changed information from the database. The cache is invalidated and re-cached when the source is updated in the database. This long-term cache can be shared between all the helpdesk workers and community tool visitors, and it drastically reduces the load on the database. This caching strategy is especially useful for lists of workers, groups, plugins, mail routing rules or global settings. Any time we have an ID of one of these objects we can simply refer to the cache (to display names, labels and other information) rather than doing expensive joins from the database.

Short-term caching exists for the life of a single page request by a single worker. Often a single request will need the same information (like a list of workers) multiple times when rendering a page. After the initial request is made from the long-term cache, it’s stored in short-term cache so any subsequent requests are lightning fast. Even though the long-term cache is almost always faster than talking to the database, there’s still some overhead involved in talking to it rather than memory. This depends on the caching options from framework.config.php — by default the long-term cache is on the filesystem, though more advanced options like Memcached can be easily enabled.

The Impact on Cerb4

The long-term cache strategy has existed in Cerb4 since the beginning; but we’ve recently realized it could become pretty inefficient on high-load systems as the filesystem became a bottleneck for frequently-requested caches.

This was happening because the long-term cache was retrieved from the disk every time it was requested. For a process like the parser, which is loosely-coupled and modular, some caches are requested from events each time an e-mail message is parsed. If you download 100 messages you have 300+ cache requests. It obviously doesn’t make a lot of sense to be accessing files on the disk 300 times during the same page load. You can imagine the performance-drain effect this has if you have a helpdesk with a lot of concurrent workers all causing frequent hits on the disk cache as well.

The first impulse many programmers have to address this is to “cache the cache” in a local variable. This isn’t as simple as it sounds when you’re dealing with loosely-coupled, object-oriented code. You can’t simply pass references around to methods; and it doesn’t make a lot of sense to implement static patterns like Singleton for every object, or to bloat the API with another specialized global registry just to store cache output.

In prior builds, we were directly exposing the Zend_Cache class (of the Zend Framework) from Devblocks to handle caching. This left short-term caching up to the programmers.

With this build we’ve abstracted the cache manager (DevblocksPlatform::getCacheService()) to automatically handle short-term and long-term caching without requiring any special code. Existing cache code should continue to work exactly as it was written, just much faster under higher loads.

Enjoy!