Java Performance Tuning, Profiling, and Memory Management

Join the DZone community and get the full member experience.

Java application performance is an abstract word until you face its real implications. It may vary depending on your interpretation of the word 'performance'. This article is meant to give the developer a perspective of the various aspects of the JVM internals, the controls, and switches that can be altered to optimal effects that suit your application. There is no single size that can fit all. You need to customize to suit your application.

You may be facing one of the issues listed below:

1. The dreaded java.lang.OutOfMemory Error
2. Your application is literally crawling.

Before we take the plunge into solving the issues, we first need to understand some of the theory behind the issues.

Theory

What does the JVM do? The Java Virtual Machine has two primary jobs:

1. Executes Code
2. Manages Memory
   This includes allocating memory from the OS, managing Java allocation including heap compaction,
   and removal of garbaged objects

Besides the above, the JVM also does stuff like managing monitors.

Very Basic Java Theory

An object is created in the heap and is garbage-collected after there are no more references to it. Objects cannot be reclaimed or freed by explicit language directives. Objects become garbage when they’re no longer reachable from the root set (e.g static objects)

Objects inside the blue square are reachable from the thread root set, while objects outside the square (in red) are not.

The sequence of the garbage collection process is as follows:

1. Root set tracing and figure out objects that are not referenced at all.

2. Put the garbage objects from above in finalizer Q

3. Run finalize() of each of these instances

Infant mortality in Java

Most of the objects (80%) in a typical Java application die young. But this may not be true for your application. Hence there is a need to figure out this rough infant mortality number so that you can tune the JVM accordingly.

JVM flavors

• classic: disables the Hotspot JIT compiler.
• client (default): activates the Hotspot JIT for "client" applications.
• server: activates the "server" Hotspot JIT: it requires a fair amount of time to warm up, but delivers best performance for server.

Don't forget that, if you use them, -server or -client must be the first argument to Java.

The Hotspot JVM uses adaptive optimization

• JVM begins by interpreting all code, but it monitors the HotSpot
• Fires off a background thread that compiles hotspot bytecode to native code
• Hotspot JVM is only compiling and optimizing the "hot spot". Hotspot JVM has more time than a traditional JIT to perform optimizations
• The Hotspot JVM keeps the old bytecodes around in case a method moves out of the hot spot.

Java Garbage Collector

The following describes what the Java Garbage Collector does.

Sun Classic (1.1 JVM) . for historical reasons

• Mark, Sweep & Compact
  Mark: identify garbage
  Sweep: Find garbage on heap, de-allocate it
  Compact: collect all empty memory together
• Eligibility for garbage collection is determined by walking across memory, determining reachability and then compacting the heap
• Compaction is just copying the live objects so that they’re adjacent in memory
• there’s one large, contiguous block of free memory
• The main problem with classic mark, sweep and compact is that all other threads have to be suspended while the garbage collector runs
• Pause time is proportional to the number of objects on the heap

• Sun improved memory management in the Java 2 VMs by switching to a generational garbage collection scheme.
• The JavaHeap is separated into two regions(we will exclude the Permanent Generation for the time being):
  New Objects
  Old Objects
• The New Object Regions is subdivided into three smaller regions:
  1. Eden , where objects are allocated
  2. Survivor semi-spaces: From and To
• The Eden area is set up like a stack - an object allocation is implemented as a pointer increment. When the Eden area is full, the GC does a reachability test and then copies all the live objects from Eden to the To region.
• The labels on the regions are swapped
• To becomes From - now the From area has objects.

JVM Heap:

Java Heap is divided into 3 generations: Young(Eden), Old(Tenured), and Permanent.

The diagram below shows how objects get created in New generation and then move to survivor Spaces at every GC run, and if they survive for long to be considered old, they get moved to the Tenured generation. The number of times an object need to survive GC cycles to be considered old enough can be configured.

By default, Java has 2 separate threads for GC, one each for young(minor GC) and old generation(major GC). The minor GC (smaller pause, but more frequent) occurs to clean up garbage in the young generation, while the major GC (larger pause, but less frequent) cleans up the garbage in the old generation. If the major GC too fails to free required memory, the JVM increases the current memory to help create new object. This whole cycle can go on till the current memory reaches the MaxMemory for the JVM (default is 64MB for client JVM), after which JVM throws OutOfMemory Error.

Permanent Generation

Class information is stored in the perm generation. Also constant strings are stored there. Strings created dynamically in your application with String.intern() will also be stored in the perm generation. Reflective objects (classes, methods, etc.) are stored in perm. It holds all of the reflective data for the JVM

JVM process memory

The windows task manager just shows the memory usage of the java.exe task/process. It is not unusual for the total memory consumption of the VM to exceed the value of -Xmx

What you see in the TaskManager is the total PAS, while what the profiler shows is the Java Heap and the PERM(optionally)

1. x86 / Redhat Linux 32 bit 2 GB
2. x86 / Redhat Linux 64 bit 3 GB
3. x86 / Win98/2000/NT/Me/XP 2 GB
4. x86 / Solaris x86 (32 bit) 4 GB
5. Sparc / Solaris 32 bit 4 GB

Why GC needs tuning

Limits of Vertical scaling

If F is the fraction of a calculation that is sequential (i.e. cannot benefit from parallelization), and (1 − F) is the fraction that can be parallelized, then the maximum speedup that can be achieved by using N processors is:

1
------------ Amdahl's law
F + (1-F)/N

In the limit, as N -> infinity, the maximum speedup tends to 1/F. If F is only 10%, the problem can be sped up by only a maximum of a factor of 10, no matter how large the value of N used.

So we assume that there is a scope of leveraging benefits of multiple CPUs or multithreading.All right, enough of theory. can it solve my problem??

Problem Statements

Your application may be crawling because it's spending too much time cleaning up the garbage , rather than running the app.

Java.lang.OutOfMemoryError can occur due to 3 possible reasons:

1. JavaHeap space low to create new objects . Increase by -Xmx (java.lang.OutOfMemoryError: Java heap space).
java.lang.OutOfMemoryError: Java heap space
MaxHeap=30528 KB TotalHeap=30528 KB FreHeap=170 KB UsedHeap=30357 KB

2. Permanent Generation low. Increase by XX:MaxPermSize=256m (java.lang.OutOfMemoryError: PermGen space)
java.lang.OutOfMemoryError: PermGen space
MaxHeap=65088 KB TotalHeap=17616 KB FreeHeap=9692 KB UsedHeap=7923 KB

def new generationtotal 1280K, used 0K [0x02a70000, 0x02bd0000, 0x02f50000)

eden space 1152K,0% used [0x02a70000, 0x02a70000, 0x02b90000)

from space 128K,0% used [0x02bb0000, 0x02bb0000, 0x02bd0000)

tospace 128K,0% used [0x02b90000, 0x02b90000, 0x02bb0000)

tenured generationtotal 16336K, used 7784K [0x02f50000, 0x03f44000, 0x06a70000)

the space 16336K,47% used [0x02f50000, 0x036ea3f8, 0x036ea400, 0x03f44000)

compacting perm gen total 12288K, used 12287K [0x06a70000, 0x07670000, 0x07670000)

the space 12288K, 99% used [0x06a70000, 0x0766ffd8, 0x07670000, 0x07670000)

3. java.lang.OutOfMemoryError: . Out of swap space .

JNI Heap runs low on memory, even though the JavaHeap and the PermGen have memory. This typically happens if you are meking lots of heavy JNI calls, but the JavaHeap objects occupy little space. In that scenario the GC might not feel the urge to cleanup JavaHeap, while the JNI Heap keeps on increasing till it goes out of memory.

If you use java NIO packages, watch out for this issue. DirectBuffer allocation uses the native heap.

The NativeHeap can be increasded by -XX:MaxDirectMemorySize=256M (default is 128)

Diagnosis:

There are some starting points to diagnose the problem.You may start with the '-verbose:gc' flag on the java command and see the memory footprint as the application progresses, till you find a spike. You may analyze the logs or use a light profiler like JConsole (part of JDK) to check the memory graph. If you need the details of the objects that are occupying the memory at a certain point, then you may use JProfiler or AppPerfect which can provide the details of each object instance and all the in/out bound references to/from it. This is a memory intensive procedure and not meant for production systems. Depending upon your application, these heavy profilers can slow down the app upto 10 times.

Below are some of the ways you can zero-in on the issue.

A) GC outputs

This flag starts printing additional lines to the console, like given below

[GC 65620K -> 50747K(138432K), 0.0279446 secs]
[Full GC 46577K -> 18794K(126848K), 0.2040139 secs]
Combined size of live objects before(young+tenured) GC -> Combined size of live objects(young+tenured) after GC (Total heap size, not counting the space in the permanent generation
-XX:+PrintHeapAtGC : More details
•-XX:+PrintGCTimeStamps will additionally print a time stamp at the start of each collection.
111.042: [GC 111.042: [DefNew: 8128K->8128K(8128K), 0.0000505 secs]
111.042: [Tenured: 18154K->2311K(24576K), 0.1290354 secs]
26282K->2311K(32704K), 0.1293306 secs]
The collection starts about 111 seconds into the execution of the application. The tenured generation usage was reduced to about 10%
18154K->2311K(24576K)

B) hprof output file

java –Xrunhprof:heap=sites,cpu=samples,depth=10,thread=y,doe=y
The heap=sites tells the profiler to write information about memory utilization on the heap, indicating where it was allocated.
cpu=samples tells the profiler to do statistical sampling to determine CPU use.
depth=10 indicates the depth of the trace for threads.
thread=y tells the profiler to identify the threads in the stack traces.
doe=y tells the profiler to produce dump of profiling data on exit.

C) -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=C:\OOM.txt

Dump the heap on OOM, and then analyze the OOM.txt (Binary file) with jhat tool (bundled with JDK)

The command below will launch http server @port 7777 . Open a browser with the URL 'http://localhost:7777' to see the results.

jhat -port 7777 c:\OOM.txt

D) Profiling the app

You can profile the application to figure out Memory Leaks.

Java memory leaks (or what we like to call unintentionally retained objects), are often caused by saving an object reference in a class level collection and forgetting to remove it at the proper time. The collection might be storing 100 objects, out of which 95 might never be used. So in this case those 95 objects are creating the memory leak, since the GC cannot free them as they are referenced by the collection.

There are also other kinds of problems with managing resources that impact performance, such as not closing JDBC Statements/ResultSets in a finally block (many JDBC drivers store a Statement reference in the Connection object).

A java "memory leak" is more like holding a strong reference to an object though it would never be needed anymore. The fact that you hold a strong reference to an object prevents the GC from deallocating it.. Java "memory leaks" are objects that fall into category (2). Objects that are reachable but not "live" can be considered memory leaks.

JVMPI for Profiling applications give a high level of detailing
Profilers: Hprof, JConsole, JProfiler, AppPerfect, YourKit, Eclipse Profiler, NetBeans Profiler ,JMP, Extensible Java Profiler (EJP), TomcatProbe, Profiler4j

JConsole is good for summary level info, tracking the memory footprint, checking Thread deadlocks etc. It does not provide details of the Heap object. For Heap details you may use AppPerfect (licensed) or JProfiler.

E) For NativeHeap issues.

JRockit JDK (from BEA) provides better tools than the SUN JDK to peep inside the JNI Heap(atleast on Windows).

JRockt Runtime Analyzer . this is part of the jrockit install.
jrcmd PSID print_memusage
JRMC.exe . launch from /bin and start recording.

Try to get some Solution:

Based on the findings from the diagnosis, you may have to take these actions:

1. Code change - For memory leak issues, it has to be a code change.
2. JVM parameters tuning - You need to find the behavior of your app in terms of the ratio of young to old objects, and then tune the JVM accordingly. We ll talk abt when to tune a parameter as we discuss the relevant params below.

Memory Size: overall size, individual region sizes

-ms, -Xms
sets the initial heap size (young and tenured generation ONLY, NOT Permanent)

If the app starts with a large memory footprint, then you should set the initial heap to a large value so that the JVM does not consume cycles to keep expanding the heap.

-mx, -Xmx
sets the maximum heap size(young and tenured gen ONLY,NOT Perm) (default: 64mb)

This is the most frequently tuned parameter to suit the max memory requirements of the app. A low value overworks the GC so that it frees space for new objects to be created, and may lead to OOM. A very high value can starve other apps and induce swapping. Hence, Profile the memory requirements to select the right value.

-Xminf [0-1], -XX:MinHeapFreeRatio [0-100]

sets the percentage of minimum free heap space - controls heap expansion rate

-Xmaxf [0-1], -XX:MaxHeapFreeRatio [0-100]

sets the percentage of maximum free heap space - controls when the VM will return unused heap memory to the OS

-XX:NewRatio

sets the ratio of the old and new generations in the heap. A NewRatio of 5 sets the ratio of new to old at 1:5, making the new generation occupy 1/6th of the overall heap
defaults: client 8, server 2

-XX:SurvivorRatio

Garbage Collector Tuning:

Types of GarbageCollectors (not complete list)

1. Throughput collector: (default for Server JVM)
   •parallel version of the young generation collector.
   •-XX:+UseParallelGC
   •The tenured gc is the same as the serial collector (default GC for client JVM).
   •multiple threads to execute a minor collection
   •application has a large number of threads allocating objects / large Eden
   •-XX:+UseParallelOldGC (major also in parallel)
2. Concurrent low pause collector :
   •collects the tenured generation and does most of the collection concurrently with the execution of the application. Attempts to reduce the pause times needed to collect the tenured generation
   •-Xincgc™ or -XX:+UseConcMarkSweepGC
   •The application is paused for short periods during the collection. A parallel version of the young generation copying collector is used with the concurrent collector.
   •Multiprocessor; apps that have a relatively large set of long-lived data (a large tenured generation;
   •Apps where response time is more important than overall throughput e.g. JAVA_OPTS= -Xms128M -Xmx1024M -XX:NewRatio=1 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:E:\loggc.txt

Performance Solution

1. Application Software profiling
2. Server and JVM tuning
3. Right Hardware and OS
4. Code improvement as per the Behaviour of your application & profiling results….…. easier said than done
5. Use JVM the right way : optimal JVM params
6. Client / server application
7. •-XX:+UseParallelGC if u have multiprocessors

Some Tips

• Unless you have problems with pauses, try granting as much memory as possible to the virtual machine
• Setting -Xms and -Xmx to the same value ….but be sure about the application behaviour
• Be sure to increase the memory as you increase the number of processors, since allocation can be parallelized
• Don’t forget to tune the Perm generation
• Minimize the use of synchronization
• Use multithreading only if it benefits. Be aware of the thread overheads. e.g a simple task like counter incrementing from 1 to billion . use single thread. Multiple threads will ruin to mutiple of 10. I tested it out on dual CPU WinXP with 8 threads.
• Avoid premature object creation. Creation should be as close to the actual place of use as possible. Very basic concept that we tend to overlook.
• JSPs are generally slower than servlets.
• Too many custom CLs, reflection : increase Perm generation. Don't be PermGen-agnostic.
• Soft References for memory leakages. They enable smart caches and yet do not load memory. GC will flush out SoftReferences automatically if the JVM runs low on memory.
• StringBuffer instead of String concat
• Minimize JNI calls in your code
• XML APIs – be careful …SAX or DOM- make correct choice. Use precompiled xpaths for better performance of the queries.

Conclusion:

There can be various bottlenecks for the entire application, and application JVM may be one of the culprits.There can be various reasons like JVM not tuned optimally to suit your application, Memory leakages, JNI issues etc. They need to be diagnosed, analyzed and then fixed.

Java (programming language) Memory (storage engine) Java virtual machine garbage collection Object (computer science) application Java performance

Opinions expressed by DZone contributors are their own.