Heap and memory optimization
Today I tested the automatic data sorting function of a project, sorting out tens of thousands of records and pictures in the database. When the operation was close to the end, java.lang.outOfMemoryError, an error in java heap space was revealed. In the past, I rarely encountered such memory errors in writing programs, because Java has a garbage collector mechanism, so I have not paid much attention to it. Today I searched for some information online and sorted it out on this basis.
1. Stack and stack
Heap-built with new, garbage collector is responsible for recycling
1. When the program starts running, the JVM gets some memory from the OS, part of which is heap memory. Heap memory is usually arranged upwards at the bottom of the storage address.
2. The heap is a "runtime" data area, and the object instantiated in the class allocates space from the heap;
3. Allocating space on the heap is established through instructions such as "new". The heap is the dynamically allocated memory size, and the lifetime does not need to be told to the compiler in advance;
4. Unlike C++, Java automatically manages the heap and stack, and the garbage collector can automatically recycle heap memory that is no longer used;
5. The disadvantage is that since memory is dynamically allocated at runtime, memory access speed is slower.
Stack - Store basic types and reference types, fast
1. The first-in and then out data structure is usually used to save parameters and local variables in the method;
2. In Java, all variables of basic types (short, int, long, byte, float, double, boolean, char) and reference types are stored in the stack;
3. The living space of data in the stack is generally in the current scopes (the area enclosed by {...};
4. The access speed of the stack is faster than the heap, second only to the registers directly located in the CPU;
5. The data in the stack can be shared, and multiple references can point to the same address;
6. The disadvantage is that the data size and lifetime of the stack must be determined and lack flexibility.
2. Memory settings
1. Check the virtual machine memory status
long maxControl = Runtime.getRuntime().maxMemory();//Get the maximum amount of memory that the virtual machine can control long currentUse = Runtime.getRuntime().totalMemory();//Get the amount of memory currently used by the virtual machine
By default, maxControl=66650112B=63.5625M of java virtual machine;
If you do nothing, the currentUse measured on my machine=5177344B=4.9375M;
2. Command to set memory size
-Xms<size> set initial Java heap size: Set the JVM initialization heap memory size; this value can be set the same as -Xmx to avoid the JVM redistribution memory every time the garbage collection is completed.
-Xmx<size> set maximum Java heap size: Set the maximum heap memory size of the JVM;
-Xmn<size>: Set the size of the young generation, the entire heap size = the size of the young generation + the size of the old generation + the size of the last generation.
-Xss<size> set java thread stack size: set the JVM thread stack memory size;
3. Specific operations (1) JVM memory settings:
Open MyEclipse(Eclipse) window-preferences-Java-Installed JREs-Edit-Default VM Arguments
Enter: -Xmx128m -Xms64m -Xmn32m -Xss16m
(2) IDE memory settings:
Modify the configuration under -vmargs in myeclipse.ini (or eclipse.ini in the Eclipse root directory):
(3) Tomcat memory settings
Open the bin folder in the root directory of Tomcat and edit catalina.bat
Modify to: set JAVA_OPTS= -Xms256m -Xmx512m
3. OutOfMemoryError Error Analysis in Java Heap
When the JVM is started, the heap memory set by the -Xms parameter is used. As the program continues and creates more objects, the JVM begins to expand the heap memory to hold more objects. The JVM also uses a garbage collector to recycle memory. When the maximum heap memory set by -Xmx is almost reached, if no more memory can be allocated to the new object, the JVM will throw java.lang.outofmemoryerror and the program will crash. Before throwing OutOfMemoryError, the JVM will try to free up enough space with the garbage collector, but will throw this error when it finds that there is still not enough space. To solve this problem, you need to be clear about the information about the program objects, such as which objects have created, which objects occupy how much space, etc. You can use a profiler or a heap analyzer to handle OutOfMemoryError errors. "java.lang.OutOfMemoryError: Java heap space" means that the heap does not have enough space and cannot continue to expand. "java.lang.OutOfMemoryError: PermGen space" means that the permanent generation is full, and your program can no longer load the class or allocate a string.
4. Heap and garbage collection
We know that objects are created in heap memory, garbage collection is a process that clears dead objects out of heap space and returns these memory to the heap. In order to use the garbage collector, the heap is mainly divided into three areas, namely New Generation, Old Generation or Tenured Generation, and Perm space. New Generation is a space used to store newly created objects and is used when the object is newly created. If used for a long time, they will be moved to Old Generation (or Tenured Generation) by the garbage collector. Perm space is where the JVM stores Meta data, such as classes, methods, string pools, and class-level details.
5. Summary:
1. Java heap memory is part of the memory allocated to the JVM by the operating system.
2. When we create objects, they are stored in Java heap memory.
3. To facilitate garbage collection, the Java heap space is divided into three areas, called New Generation, Old Generation or Tenured Generation, and Perm Space.
4. You can adjust the size of the Java heap space by using the JVM command line options -Xms, -Xmx, and -Xmn.
5. You can use JConsole or Runtime.maxMemory(), Runtime.totalMemory(), and Runtime.freeMemory() to view the size of heap memory in Java.
6. You can use the command "jmap" to obtain the heap dump, and use "jhat" to analyze the heap dump.
7. Java heap space is different from stack space. Stack space is used to store call stacks and local variables.
8. The Java garbage collector is used to reclaim the memory occupied by dead objects (objects that are no longer used) and release it into the Java heap space.
9. When encountering java.lang.outOfMemoryError, you don’t have to worry. Sometimes you just need to increase the heap space. But if it happens frequently, you have to see if there is a memory leak in the Java program.
10. Use Profiler and Heap dump analysis tools to view the Java heap space, and you can see how much memory is allocated to each object.
Detailed explanation of stack storage
Java stack storage has the following characteristics:
1. The data size and life cycle in the stack must be determined.
For example, the storage of the basic type: int a = 1; This variable contains a literal value, a is a reference to the int type, pointing to the literal value of 3. Due to the size and lifetime of these literal data, these literal values are fixedly defined in a program block, and after the program block exits, the literal values disappear), exist in the stack for the sake of pursuing speed.
2. The data existing in the stack can be shared.
(1) Basic type data storage:
like:
int a = 3; int b = 3;
The compiler first processes int a = 3; first it will create a reference to the variable a in the stack, and then find out if there is an address with a literal value of 3. If it is not found, it will open an address with the literal value of 3, and then point a to the address of 3. Then process int b = 3; after creating the reference variable of b, since there is already a literal value of 3 in the stack, b is directly pointed to the address of 3. In this way, a and b both point to 3 at the same time.
Note: This literal reference is different from that of class objects. Assuming that the references of two class objects point to an object at the same time, if one object reference variable changes the internal state of the object, the other object reference variable immediately reflects this change. Instead, modifying its value through a literal reference will not cause another value to be changed accordingly. As in the above example, after we define the values of a and b, let a=4; then, b will not be equal to 4, or equal to 3. Inside the compiler, when a=4 is encountered, it will re-search whether there is a literal value of 4 in the stack. If not, re-open the address to store the value of 4; if it already exists, directly point a to this address. Therefore, the change in value a will not affect the value b.
(2) Packaging data storage:
Classes that wrap the corresponding basic data types, such as Integer, Double, String, etc. All of these class data exists in the heap. Java uses the new() statement to display the compiler and only creates dynamically as needed at runtime, so it is more flexible, but the disadvantage is that it takes up more time.
For example: Take String as an example.
String is a special packaging data. That is, it can be created in the form of String str = new String("abc"); or it can be created in the form of String str = "abc";. The former is the standardized class creation process, that is, in Java, everything is an object, and an object is an instance of the class, all created in the form of new(). Some classes in Java, such as the DateFormat class, can return a newly created class through the getInstance() method of the class, which seems to violate this principle. Actually, it is not the case. This class uses the singleton pattern to return an instance of the class, but this instance is created inside the class through new(), which hides this detail from the outside.
Then why is the instance not created through new() in String str = "abc";? Is it violated the above principle? Actually, there isn't.
About the internal work of String str = "abc". Java internally converts this statement into the following steps:
a. First define an object reference variable named str to the String class: String str;
b. Find whether there is an address with the value "abc" in the stack. If not, open an address with the literal value "abc", then create a new object O of the String class, and point the string value of O to this address, and note the reference object O next to this address in the stack. If there is an address with the value "abc", look for object O and return the address of O.
c. Point str to the address of object O.
It is worth noting that usually the string values in the String class are stored directly. But in situations like String str = "abc";, its string value holds a reference to the data present in the stack (i.e.: String str = "abc"; both stack storage and heap storage).
To better illustrate this problem, we can verify it through the following codes.
String str1 = "abc"; String str2 = "abc"; System.out.println(str1==str2); //true
(The truth value is returned only if both references point to the same object. Are str1 and str2 pointing to the same object)
The result shows that the JVM created two references str1 and str2, but only one object was created, and both references pointed to this object.
String str1 = "abc"; String str2 = "abc"; str1 = "bcd"; System.out.println(str1 + "," + str2); //bcd, abc System.out.println(str1==str2); //false
This means that the change in assignment results in the change in the reference of the class object, str1 points to another new object, while str2 still points to the original object. In the above example, when we change the value of str1 to "bcd", the JVM found that there is no address to store the value in the stack, so it opened this address and created a new object whose string value points to this address.
In fact, the String class is designed to be an immutable class. If you want to change its value, you can, but the JVM quietly creates a new object based on the new value at runtime (it cannot be changed based on the original memory), and then returns the address of this object to the reference of the original class. Although this creation process is completely automatic, it takes up more time after all. In an environment that is more sensitive to time requirements, it will have certain adverse effects.
String str1 = "abc"; String str2 = "abc"; str1 = "bcd"; String str3 = str1; System.out.println(str3); //bcd String str4 = "bcd"; System.out.println(str1 == str4); //true
The reference to str3 object directly points to the object pointed to by str1 (note that str3 does not create a new object). After str1 has changed its value, create a String reference str4 and point to a new object created by str1 modifying the value. It can be found that this time str4 did not create a new object, thus realizing the sharing of data in the stack again.
String str1 = new String("abc"); String str2 = "abc"; System.out.println(str1==str2); //falseTwo references were created. Two objects were created. The two references point to two different objects.
String str1 = "abc"; String str2 = new String("abc"); System.out.println(str1==str2); //falseTwo references were created. Two objects were created. The two references point to two different objects.
The above two codes indicate that as long as the object is created with new(), it will be created in the heap, and its strings are stored separately. Even if they are the same as the data in the stack, they will not be shared with the data in the stack.
Summarize:
(1) When we define a class using a format such as String str = "abc";, we always take it for granted that we created an object str of the String class. Worry about the trap! The object may not have been created! The only thing that is certain is that a reference to the String class is created. As for whether this reference points to a new object, it must be considered based on the context, unless you use the new() method to explicitly create a new object. Therefore, to be more precise, we create a reference variable str to an object of the String class, which points to a String class with a value of "abc". Being aware of this is helpful in troubleshooting difficult bugs in programs.
(2) Using String str = "abc"; can improve the running speed of the program to a certain extent, because the JVM will automatically decide whether it is necessary to create a new object based on the actual situation of the data in the stack. For the code of String str = new String("abc");, new objects are created in the heap regardless of whether their string values are equal or not, whether it is necessary to create new objects, thereby increasing the burden on the program.
(3) Due to the immutable nature of the String class (because the value of the wrapper class cannot be modified), when the String variable needs to be transformed frequently, the StringBuffer class should be considered to improve program efficiency.