When locating JVM performance problems, you may encounter memory leaks and cause JVM OutOfMemory. If the parameter reloadable="true" is set when using the Tomcat container, you may also encounter memory overflow when frequently deploying applications. The principle of hot deployment of Tomcat is to detect that the files in the WEB-INF/classes or WEB-INF/lib directory are changed and the application will be stopped first and then started. Since Tomcat assigns a WebAppClassLoader to each application by default, the principle of hot replacement is to create a new ClassLoader to load the class. Since the uniqueness of a class in the JVM is determined by its class file and its class loader, reloading the class can achieve the purpose of hot replacement. When the number of hot deployments is more frequent, it will cause more classes loaded by the JVM. If the previous class is not unloaded in time for some reason (such as memory leaks), it may lead to permanent generation or MetaSpace OutOfMemory. This article briefly introduces the scenario where ThreadLocal and ClassLoader lead to memory leaks and ultimately OutOfMemory through a demo.
Uninstalling the class
After the class is used, if the following situation is satisfied, it will be uninstalled:
1. All instances of this class in the heap have been recycled, that is, instance objects of this class do not exist in the heap.
2. The classLoader loading this class has been recycled.
3. The Class object corresponding to this class cannot be referenced anywhere, and the Class object cannot be accessed through reflection.
If the class meets the uninstallation conditions, the JVM uninstalls the class when it is in GC, that is, clears the class information in the method area.
Scene introduction
In the previous article, I introduced the principle of ThreadLocal. Each thread has a ThreadLocalMap. If the life cycle of the thread is relatively long, the Entry in ThreadLocalMap may not be recycled. The ThreadLocal object has always been strongly referenced by the thread. Since the instance object will hold the reference of the Class object, the Class object will hold the reference of the ClassLoader that loads it, which will cause the Class to be unloaded. When there are enough classes loaded, a permanent generation or MetaSpace memory overflow may occur. If the class has large objects, such as a larger byte array, it will cause the memory overflow of the Java heap area.
Source code introduction
Here is an internal class Inner. The Inner class has a static ThreadLocal object, which is mainly used to make threads hold strong references to the Inner class, so that the Inner class cannot be recycled. A custom class loader is defined to load the Inner class, as shown below:
public class MemoryLeak { public static void main(String[] args) { //Because the thread is running all the time, the Inner object in ThreadLocalMap has been strongly referenced by Thread object new Thread(new Runnable() { @Override public void run() { while (true) { //Every time a new ClassLoader instance is created to load the Inner class CustomClassLoader classLoader = new CustomClassLoader ("load1", MemoryLeak.class.getClassLoader(), "com.ezlippi.MemoryLeak$Inner", "com.ezlippi.MemoryLeak$Inner$1"); try { Class<?> innerClass = classLoader.loadClass("com.ezlippi.MemoryLeak$Inner"); innerClass.newInstance(); //Help GC for reference processing innerClass = null; classLoader = null; Thread.sleep(10); } catch (ClassNotFoundException | IllegalAccessException | InstantiationException | InterruptedException e) { e.printStackTrace(); } } } }).start(); } //In order to reach the heap area faster public static class Inner { private byte[] MB = new byte[1024 * 1024]; static ThreadLocal<Inner> threadLocal = new ThreadLocal<Inner>() { @Override protected Inner initialValue() { return new Inner(); } }; //To call ThreadLocal.get() to initialize an Inner object static { threadLocal.get(); } public Inner() { } } //Source code omit private static class CustomClassLoader extends ClassLoader {}Heap area memory overflow
In order to trigger the heap memory overflow, I set up a 1MB byte array in the Inner class, and at the same time, I have to call threadLocal.get() in the static block. Only the call will trigger initialValue() to initialize an Inner object, otherwise I will just create an empty ThreadLocal object, and there is no data in the ThreadLocalMap.
The JVM parameters are as follows:
-Xms100m -Xmx100m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintClassHistogram -XX:+HeapDumpOnOutOfMemoryError
After the last 814 executions, the JVM heap area memory overflows, as shown below:
java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid11824.hprof ...Heap dump file created [100661202 bytes in 1.501 secs]Heap par new generation total 30720K, used 30389K [0x000000000f9c000000, 0x00000000fbd500000) eden space 27328K, 99% used [0x00000000f9c000000, 0x00000000fb6ad450, 0x00000000fb6b0000) from space 3392K, 90% used [0x0000000000fb6b0000, 0x00000000fb9b0030, 0x00000000fba00000) to space 3392K, 0% used [0x0000000000fba00000, 0x00000000fbd500000) concurrent mark-sweep generation total 68288K, used 67600K [0x000000000fbd500000, 0x0000001000000000, 0x000000010000000) Metaspace used 3770K, capacity 5134K, committed 5248K, reserved 1056768K class space used 474K, capacity 578K, committed 640K, reserved 1048576KException in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space at com.ezlippi.MemoryLeak$Inner.<clinit>(MemoryLeak.java:34) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) Source) at java.lang.Class.newInstance(Unknown Source) at com.ezlippi.MemoryLeak$1.run(MemoryLeak.java:20) at java.lang.Thread.run(Unknown Source)
You can see that the JVM has no memory to create a new Inner object, because the heap area stores many 1MB byte arrays. Here I printed out the histogram of the class (the following figure is a scene with a heap size of 1024M), omitting some insignificant classes. It can be seen that the byte array occupies 855M of space, and 814 instances of com.ezlippi.MemoryLeak$CustomClassLoader are created, which basically match the size of the byte array:
num #instances #bytes class name------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ com.ezlippi.MemoryLeak$CustomClassLoader 12: 820 53088 [Ljava.util.Hashtable$Entry; 15: 817 39216 java.util.Hashtable 16: 915 36600 java.lang.ref.SoftReference 17: 543 34752 java.net.URL 18: 697 33456 java.nio.HeapCharBuffer 19: 817 32680 java.security.ProtectionDomain 20: 785 31400 java.util.TreeMap$Entry 21: 928 29696 java.util.Hashtable$Entry 22: 1802 28832 java.util.HashSet 23: 817 26144 java.security.CodeSource 24: 814 26048 java.lang.ThreadLocal$ThreadLocalMap$Entry
Metaspace overflow
In order to make Metaspace overflow, you must reduce the space of MetaSpace a little, and load enough classes before the heap overflow. Therefore, I adjusted the JVM parameters and adjusted the size of the byte array to 1KB, as shown below:
private byte[] KB = new byte[1024];-Xms100m -Xmx100m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintClassHistogram -XX:MetaspaceSize=2m -XX:MaxMetaspaceSize=2m
From the GC log, we can see that when the Meraspace reaches the GC threshold (that is, the size of the MaxMetaspaceSize configuration) FullGC will be triggered:
java.lang.OutOfMemoryError: Metaspace <<no stack trace available>>{Heap before GC invocations=20 (full 20): par new generation total 30720K, used 0K [0x000000000f9c000000, 0x00000000fbd500000) eden space 27328K, 0% used [0x000000000f9c000000, 0x00000000f9c00000, 0x0000000f9c00000, 0x0000000f9c00000, 0x0000000fb6b0000) from space 3392K, 0% used [0x0000000000fb6b0000, 0x00000000fb6b0000, 0x00000000fba00000) to space 3392K, 0% used [0x0000000000fba00000, 0x000000000fbd500000) to space 3392K, 0% used [0x0000000000fba00000, 0x000000010000000000) Metaspace used 1806K, capacity 1988K, committed 2048K, reserved 1056768K class space used 202K, capacity 384K, committed 384K, reserved 1048576K[Full GC (Metadata GC Threshold) [CMSProcess finished with exit code 1From the above example, we can see that if the class loader and ThreadLocal are used improperly, it will indeed lead to memory leakage. The complete source code is in github