BackgroundMemory Management (Garbage Collection)

 

The generational nature of the SAP JVM's memory system provides the flexibility to use specific garbage collection algorithms suited to the needs of a diverse set of applications. The SAP JVM supports several different garbage collection algorithms designed to serve different pause time and throughput requirements.

Background

A major attraction of the Java programming language for programmers is that it is the first mainstream programming language to provide built-in automatic memory management, or garbage collection (GC). In traditional languages, dynamic memory is allocated using an explicit allocate/free model. In practice, this turns out to be not only a major source of memory leaks, program bugs, and crashes in programs written in traditional languages, but also a performance bottleneck and a major impediment to modular, reusable code. (Determining free points across module boundaries is almost impossible without explicit and hard-to-understand cooperation between modules.) In the Java programming language, garbage collection is also an important part of the "safe" execution semantics required to support the security model.

A garbage collector automatically handles freeing of unused object memory behind the scenes by reclaiming an object only when it can prove that the object is no longer accessible to the running program. Automation of this process completely eliminates not only the memory leaks caused by freeing too little memory, but also the program crashes and hard-to-find reference bugs caused by freeing too much memory.

Traditionally, garbage collection has been considered an inefficient process that impedes performance, relative to an explicit-free model. In fact, with modern garbage collection technology, performance has improved so much that overall performance is actually substantially better than that provided by explicit freeing of objects.

The SAP JVM Garbage Collectors

The SAP JVM comes with a set of different garbage collectors suited for different runtime scenarios. The major features of this garbage collector family are presented below. Overall, these capabilities are well-suited both for applications where the highest possible performance is needed and for long-running applications where memory leaks and memory inaccessibility due to fragmentation are highly undesirable.

Accuracy

The SAP JVM garbage collectors are fully accurate collectors. In contrast, many other garbage collectors are conservative or partially accurate. While conservative garbage collection can be attractive because it is very easy to add to a system without garbage collection support, it has certain drawbacks. In general, conservative garbage collectors are prone to memory leaks, disallow object migration, and can cause heap fragmentation.

Because the SAP JVM collectors are fully accurate, they can make several strong design guarantees that a conservative collector cannot make:

  • All inaccessible object memory can be reclaimed reliably.

  • All objects can be relocated, allowing object memory compaction, which eliminates object memory fragmentation and increases memory locality.

An accurate garbage collection mechanism avoids accidental memory leaks, enables object migration, and provides for full heap compaction. The GC mechanism in the SAP JVM scales well to very large heaps.

Generational Copying Collection

The SAP JVM employs a state-of-the-art generational copying collector, which provides two major benefits:

  • Increased allocation speed and overall garbage collection efficiency for most programs, compared to non-generational collectors

  • Corresponding decrease in the frequency and duration of user-perceivable garbage collection pauses

A generational collector takes advantage of the fact that in most programs, the vast majority of objects (often greater than 95 percent) are very short lived (for example, they are used as temporary data structures). By segregating newly created objects into an object nursery, a generational collector can accomplish several things. First, because new objects are allocated contiguously in stack-like fashion in the object nursery, allocation becomes extremely fast, since it merely involves updating a single pointer and performing a single check for nursery overflow. Secondly, by the time the nursery overflows, most of the objects in the nursery are already dead, allowing the garbage collector to simply move the few surviving objects elsewhere, and avoid doing any reclamation work for dead objects in the nursery.

Parallel Young Generation Collector

The single-threaded copying collector described above, while suitable for many deployments, could become a bottleneck to scaling in an application that is otherwise parallelized to take advantage of multiple processors. To take full advantage of all available CPUs on a multi-processor machine, the SAP JVM offers an optional multithreaded collector for the young generation, in which the tracing and copying of live objects is accomplished by multiple threads working in parallel. The implementation has been carefully tuned to balance the collection work between all available processors, allowing the collector to scale up to large numbers of processors. This reduces the pause times for collecting young space and maximizes garbage collection throughput.

When moving objects, the parallel collector tries to keep related objects together, resulting in improved memory locality and cache utilization, and leading to improved mutator performance. This is accomplished by copying objects in depth first order.

The parallel collector also uses available memory more efficiently. It does not need to keep a portion of the old object space in reserve to guarantee space for copying all live objects. Instead, it uses a novel technique to speculatively attempt to copy objects. If old object space is scarce this technique allows the collector to switch smoothly to compacting the heap without the need for holding any space in reserve. This results in better utilization of the available heap space.

Finally, the parallel collector is able to dynamically adjust its tunable parameters in response to the application's heap allocation behavior, leading to improved garbage collection performance over a wide range of applications and environments.

Mark-Compact Old Object Collector

Although the generational copying collector collects most dead objects efficiently, longer-living objects still accumulate in the old object memory area. Occasionally, because of low memory conditions or programmatic requests, an old object garbage collection must be performed. The SAP JVM by default uses a standard mark-compact collection algorithm, which traverses the entire graph of live objects from its roots, then sweeps through memory, compacting away the gaps left by dead objects. By compacting gaps in the heap, rather than collecting them into a free-list, memory fragmentation is eliminated, and old object allocation is streamlined by eliminating free-list searching.

Concurrent Mark and Sweep Collector

To provide support for applications that need low response times (like dialog-oriented applications, where the users will not accept sporadic longer response times due to a running “stop-the-world” garbage collection), the SAP JVM provides a largely concurrent garbage collection, which runs in parallel to the processed application and minimizes the use of stop-the-world pauses. These kinds of pauses are only necessary for identifying the initial set of root objects for object graph traversal and for marking of objects modified during the (concurrent) garbage collection run.

Since the concurrent mark and sweep collector does not compact the old generation, it utilizes free-lists for managing the free space. Potentially more heap memory is necessary, since garbage is created while the collector is running. The concurrent mark and sweep collector employs heuristics to minimize the possible fragmentation due to the missing compaction and to start the garbage collection early enough to reduce the probability of running into a full old generation.

Parallel Old Generation Collector

In addition, the SAP JVM provides a parallel mark-compact collector for the old generation designed to improve scalability for applications with large heaps. Where the concurrent mark-sweep collector focuses on decreasing pause times, the parallel old collector focuses on increasing throughput by using many threads simultaneously to collect the old generation during stop-the-world pauses.