A characteristic of cache servers is the pre-set limit on the amount of memory available for storing objects. Unlike the content server, this memory space does not "grow" (but the size of the cache can of course be increased at any time).

If the cache server fills up to the level where it does not have enough space to store the next object, it starts deleting "old" objects from the memory. To facilitate this, the cache carries out a statistical analysis of accesses to cached objects. It then deletes the documents that have been unaccessed the longest, until there is enough space in the memory to store the current document. This method is also called the least recently used (LRU) mechanism.

SAP’s own analyses have shown that the cache provides a consistent, significant advantage in terms of speed, taking into account both the frequency and size of requests. In the diagram below, this is shown in the comparison of the response times of the cache server and the content server. From the diagram, it is clear that both curves run parallel, regardless of the request size, and that the response times of the content server exceed those of the cache server by a factor of 10. This result is, of course, independent of the bandwidth available to the content server. For example, SAP has a high-bandwidth connection between Walldorf and Palo Alto. In a customer environment, much higher improvements on speed can be expected.

The relationship between cache hits and cache misses is an essential criteria in cache efficiency. When the hit rate is near 100%, cache performance is at optimum level. If the misses are in the majority, or if misses make up 20-30% of the total number of requests, you should consider taking performance optimization measures.

What causes a sharp increase in cache misses?

• An almost full cache
• A large number of widely dispersed requests; that is, documents that are not located in the cache are being constantly requested

Both factors cause the cache to behave in the following ways.

Stored documents are deleted to make way for new objects. The lack of space in the cache and the non-homogenous nature of the requests means that deletion continually takes place.
In extreme cases, the cache content is in a constant state of flux (also known as "thrashing"). This greatly degrades cache performance.

The only solution to this problem is to extend the cache space, so that the maximum occupancy of the cache can be maintained at 75%. Backbone caches are particularly susceptible to thrashing, as these serve as concentrators for other caches.