Node.js is naturally asynchronous and event-driven, and is ideal for handling I/O-related tasks. If you are dealing with I/O-related operations in your application, you can take advantage of streams in Node.js. Therefore, let’s take a look at streams in detail and understand how they simplify I/O operations.
What is flow
Streams are unix pipelines that allow you to easily read data from a data source and then flow to another destination.
Simply put, streaming is not something special, it is just an EventEmitter that implements some methods. According to the method it implements, the stream can become a readable stream (Readable), a writable stream (Writable), or a bidirectional stream (Duplex, readable and writable).
A readable stream allows you to read data from a data source, while a writeable stream allows you to write data to your destination.
If you have used Node.js, you are likely to have encountered streaming.
For example, in a Node.js HTTP server, request is a readable stream and response is a writable stream.
You may have also used the fs module, which can help you handle readable and writable streams.
Now let you learn some basics and understand different types of flows. This article will discuss readable and writable streams. Bidirectional streams are beyond the scope of this article and we will not discuss them.
Readable Streams
We can use a readable stream to read data from a data source, which can be anything, such as a file in the system, a buffer in memory, or even other streams. Because streams are EventEmitters, they send data with various events. We will use these events to make the flow work.
Read data from stream
The best way to read data from a stream is to listen for data events and add a callback function. When data flows over, the readable stream will send a data event and the callback function will be triggered. Take a look at the following code snippet:
var fs = require('fs');var readableStream = fs.createReadStream('file.txt');var data = '';var readableStream.on('data', function(chunk){ data += chunk;});readableStream.on('end', function(){ console.log(data);});fs.createReadStream will give you a readable stream.
At the beginning, this stream was not flow dynamic. When you add data's event listener and add a callback function, it will become flowing. After this, it will read a small piece of data and pass it into your callback function.
The implementer of the stream determines the trigger frequency of the data event. For example, HTTP request will trigger the data event when several KBs of data are read. When you read data from a file, you may decide to trigger the data event when a line is finished.
When there is no data to read (when it is read at the end of the file), the stream will send an end event. In the example above, we listened to this event and printed out the data when we finished reading the file.
There is another way to read streams. You just need to keep calling the read() method in the stream instance before reading to the end of the file.
var fs = require('fs');var readableStream = fs.createReadStream('file.txt');var data = '';var chunk;readableStream.on('readable', function(){ while ((chunk = readableStream.read()) != null) { data += chunk; }});readableStream.on('end', function(){ console.log(data);});The read() method reads data from the internal buffer, and returns null when there is no data to read.
Therefore, in the while loop we check whether read() returns null, and when it returns null, the loop will be terminated.
It should be noted that when we can read data from the stream, the readable event will be fired.
Set encoding
By default, what you read from the stream is a Buffer object. If you are reading a string, this is not for you. Therefore, you can set the encoding of the stream by calling Readable.setEncoding() like in the following example:
var fs = require('fs');var readableStream = fs.createReadStream('file.txt');var data = '';readableStream.setEncoding('utf8');readableStream.on('data', function(chunk){ data += chunk;});readableStream.on('end', function(){ console.log(data);});In the above example, if we set the encoding of the stream to utf8, the data will be parsed into utf8, and the chunk in the callback function will be a string.
Piping
Pipelines are a great mechanism where you can read data from a data source without managing the state of the stream yourself and write it to your destination. Let's take a look at the following example:
var fs = require('fs');var readableStream = fs.createReadStream('file1.txt');var writableStream = fs.createWriteStream('file2.txt');readableStream.pipe(writableStream);The above example uses the pipe() method to write the content of file1 to file2. Because pipe() will help you manage data flow, you don't need to worry about the speed of data flow. This makes pipe() very simple and easy to use.
It should be noted that pipe() returns the destination stream, so you can easily link multiple streams!
Link (Chaining)
Suppose there is an archive file and you want to unzip it. There are many ways to accomplish this task. But the simplest way is to use pipelines and links:
var fs = require('fs');var zlib = require('zlib');fs.createReadStream('input.txt.gz') .pipe(zlib.createGunzip()) .pipe(fs.createWriteStream('output.txt'));First, we create a readable stream via input.txt.gz and then let it stream the zlib.createGunzip() stream, which decompresses the content. Finally, we add a writable stream to write the decompressed content to another file.
Other methods
We've discussed some important concepts in readable streams, and here are some ways you need to know:
1.Readable.pause() This method will pause the flow of the flow. In other words, it will not trigger the data event again.
2.Readable.resume() method is the opposite of the above and will allow the pause flow to resume flow.
3.Readable.unpipe() method will remove the destination. If there are parameters passed in, it will let the readable stream stop Liu Xiang's specific destination, otherwise it will remove all destinations.
Writable Streams
A writable stream allows you to write data to your destination. Just like readable streams, these are EventEmitters, and they also trigger different events. Let's take a look at the events and methods that will be triggered in the writable stream.
Write to stream
To write data as a writable stream, you need to call the write() method in the writable stream instance, and see the following example:
var fs = require('fs');var readableStream = fs.createReadStream('file1.txt');var writableStream = fs.createWriteStream('file2.txt');readableStream.setEncoding('utf8');readableStream.on('data', function(chunk){ writableStream.write('chunk');});The above code is very simple, it just reads the data from the input stream and writes it to the destination with write().
This method returns a Boolean value to indicate whether the write is successful. If the returned true, it means that the write is successful and you can continue to write more data. If it is false, it means something error occurred and you can't continue writing now. A writable stream triggers a drain event to tell you that you can continue writing data.
After writing the data
When you don't need to write data, you can call the end() method to tell the stream that you have finished writing. Assuming res is an HTTP response object, you will usually send a response to the browser:
res.write('Some Data!!');
res.end();
When end() is called, all data is written and the stream will then trigger a finish event. Note that after calling end(), you can no longer write data into the writeable stream. For example, the following code will report an error:
res.write('Some Data!!');
res.end();
res.write('Trying to write again'); //Error !
Here are some important events related to writable streams:
1.error fires when an error occurs in writing or linking
2.pipe When the readable stream is linked to the writable stream, this event will trigger
3.unpipe will fire when unpipe is called by readable stream
The above is all the content of this article. I hope it will be helpful to everyone's learning and I hope everyone will support Wulin.com more.