Detailed explanation of nodeJs memory leak problem

I accidentally discovered that when react is rendered in server, when NODE_ENV != production, it will cause memory leaks. Specific issues: https://github.com/facebook/react/issues/7406. With the widespread use of node and react isomorphism and other technologies, issues such as node-side memory leakage should attract our attention. Why is the node prone to memory leaks and how to troubleshoot after it occurs? The following is a brief introduction and example.

First of all, node is based on the v8 engine, and its memory management method is consistent with v8. The following is a brief introduction to the relevant memory effects of v8.

V8 memory limit

Node is built on V8 and can allocate and manage js objects through V8. V8 has limitations on memory usage (about 1.4G for the old generation of memory 64-bit system, about 0.7G for the 32-bit system, about 32MB for the new generation of memory 64-bit system, and about 16MB for the 32-bit system). Under such restrictions, large memory objects will be unable to operate. If this boundary is accidentally touched, the process will exit.

Cause: V8 blocks the JavaScript application logic when performing garbage collection, and then re-executes the JavaScript application logic until the garbage collection is over. This behavior is called "stop-the-world". If the heap memory of V8 is 1.5GB, it takes more than 50ms for V8 to do a small garbage collection, and it takes more than 1 second for a non-incremental garbage collection.

Set the new generation memory and the old generation memory to crack the default memory limit by setting node --max-old-space-size=xxx (unit MB) and node --max-new-space-size=xxx (unit KB).

V8 heap composition

The V8 heap is not actually composed of two parts: the old generation and the new generation. The heap can be divided into several different regions:

1. New generation memory area: Most objects are allocated here, this area is very small but garbage collection is particularly frequent
2. Old Generation Pointer Area: belongs to the Old Generation, which contains most objects that may have pointers to other objects. Most objects promoted from the New Generation will be moved here.
3. Old Generation Data Area: belongs to the old Generation, where only the original data objects are saved, these objects have no pointers to other objects
4. Large object area: Objects with a size exceeding the size of other areas are stored here. Each object has its own memory. It will not move large objects in garbage collection.
5. Code area: Code object, that is, the object containing instructions after JIT, will be allocated here. The only memory area with execution permissions
6. Cell area, attribute Cell area, Map area: store cell, attribute Cell and map. Each area stores elements of the same size, with a simple structure.

GC recycling type

Incremental GC

Indicates whether the garbage collector collects (adds) garbage when scanning memory space and clears garbage at the end of the scan cycle.

Non-incremental GC

When using a non-incremental garbage collector, the garbage is empty as soon as it is collected.

The garbage collector will only conduct garbage collection for the new generation memory area, the old generation pointer area and the old generation data area. The object first enters the new generation memory that takes up less space. Most objects will fail quickly, and non-incremental GC directly recycles these small amounts of memory. If some objects cannot be recycled for a period of time, they will be entered into the old generation memory area. This area executes infrequent incremental GC and takes a long time.

Then when will the memory leak occur?

Memory leak paths

1. Memory leak
2. cache
3. Untimely queue consumption
4. Scope not released

The memory composition of Node is mainly the part allocated through V8 and the part allocated by Node itself. The main limitation of V8's garbage collection is V8's heap memory. The main reasons for memory leaks: 1. Cache; 2. Queue consumption is not timely; 3. Scope not released

Memory Leak Analysis

Check V8 memory usage (unit byte)

 process.memoryUsage(); { ress: 47038464, heapTotal: 34264656, heapUsed: 2052866 }

ress: The resident memory part of the process

heapTotal, heapUsed: V8 heap memory information

Check system memory usage (unit byte)

os.totalmem()

os.freemem()

Returns the total system memory and idle memory

View garbage collection log

node --trace_gc -e "var a = []; for( var i = 0; i < 1000000; i++ ) { a.push(new Array(100)); }" >> gc.log //Output garbage collection log

node --prof //Output node execution performance log. Use windows-tick.processor to view.

Analytical monitoring tools

v8-profiler captures snapshots of v8 heap memory and analyzes CPU

node-heapdump grabs snapshots of v8 heap memory

node-mtrace analysis stack usage

node-memwatch listens to garbage collection situation

node-memwatch

 memwatch.on('stats',function(info){ console.log(info)})memwatch.on('leak',function(info){ console.log(info)})

stats event: Each time a full heap garbage collection is performed, the stats event will be triggered. This event will pass memory statistics.

 {"num_full_gc": 17, //How many full-stack garbage collection "num_inc_gc": 8, //How many incremental garbage collection "heap_compactions": 8, //How many times the old generation are sorted out "estimated_base": 2592568, //The estimated cardinality "current_base": 2592568, //The current cardinality "min": 2499912, //Minimum "max": 2592568, //Maximum "usage_trend": 0 //User trend}

Observe num_full_gc and num_inc_gc reflect garbage collection.

leak event: If the memory is still not released after 5 consecutive garbage collections, it means that memory leaks occur. This time a leak event will be triggered.

 { start: Fri, 29 Jun 2012 14:12:13 GMT,end: Fri, 29 Jun 2012 14:12:33 GMT,growth: 67984,reason: 'heap growth over 5 consecutive GCs (20s) - 11.67 mb/hr'}

Heap Diffing Heap Memory Comparison Troubleshooting Memory Overflow Code.

Below, we use an example to demonstrate how to troubleshoot memory leaks:

First we create an example that causes memory leaks:

 //app.jsvar app = require('express')();var http = require('http').Server(app);var heapdump = require('heapdump');var leakobjs = [];function LeakClass(){ this.x = 1;}app.get('/', function(req, res){ console.log('get /'); for(var i = 0; i < 1000; i++){ leakobjs.push(new LeakClass()); } res.send('<h1>Hello world</h1>');});setInterval(function(){ heapdump.writeSnapshot('./' + Date.now() + '.heapsnapshot');}, 3000);http.listen(3000, function(){ console.log('listening on port 3000');});

Here we simulate memory leaks by setting up an array that is constantly increasing and does not retrieve.

Use the heap-dump module to record memory snapshots regularly, and import snapshots through the chrome developer tool profiles for comparison and analysis.

We can see that after the browser accesses localhost:3000 and refreshes it many times, the size of the snapshot has been growing, and even if it is not requested, it does not decrease, indicating that a leak has occurred.

Then we import snapshots through the chrome developer tool profiles. By setting comparison, compare the initial snapshot, send requests, and then send requests to send memory snapshots in these three stages. You can find that LeakClass has been increasing in the new on the right. Always positive in delta, it means that it has not been recycled.

summary

For memory leaks, you can use memwatch to implant, or report the process.memoryUsage memory usage to monitor regularly, and set an alarm threshold for monitoring.

When memory leaks are found, if allowed, you can run node-heapdump locally and use timed memory snapshots to be generated. And use the snapshot to analyze the cause of the leakage through chrome profiles. If local debugging is not possible, use v8-profiler to output memory snapshots on the test server to compare and analyze json (code intrusion is required).

Under what circumstances should be considered, memwatch/heapdump is enabled. Consider the frequency of heapdump to avoid running out of CPU. Other ways to detect memory growth can also be considered, such as directly monitoring process.memoryUsage().

Beware of misjudgment, short-term memory usage peaks behave like memory leaks. If your app suddenly consumes a lot of CPU and memory, the processing time may span several garbage collection cycles, and then memwatch may misjudge it as a memory leak. However, in this case, once your app uses these resources, the memory consumption will drop back to normal levels. So it is important to note that memory leaks are continuously reported, and one or two sudden alarms can be ignored.