Background knowledge
Synchronous, asynchronous, blocking, non-blocking
First of all, these concepts are very easy to confuse, but they are involved in NIO, so let's summarize it.
Synchronization: When the API call returns, the caller knows how the operation is resulted (how many bytes are actually read/write).
Asynchronous: Compared with synchronization, the caller does not know the result of the operation when the API call returns, and the callback will notify the result later.
Blocking: When there is no data to be read or all data cannot be written, suspend the current thread waiting.
Non-blocking: When reading, you can read as much as the data you can read and then return. When writing, you can write as much as the data you can write and then return.
For I/O operations, according to the documents of Oracle's official website, the synchronization and asynchronous division standard is "whether the caller needs to wait for the I/O operation to complete". This "waiting for the I/O operation to complete" does not mean that data must be read or all data must be written, but rather whether the caller needs to wait when the I/O operation is actually carried out, such as the time when the data is transmitted between the TCP/IP protocol stack buffer and the JVM buffer.
Therefore, our commonly used read() and write() methods are synchronous I/O. Synchronous I/O is divided into two modes: blocking and non-blocking. If it is a non-blocking mode, it will be returned directly when it detects that there is no data to be read, and the I/O operation is not really performed.
In summary, in Java, there are actually only three mechanisms: synchronous blocking I/O, synchronous non-blocking I/O and asynchronous I/O. What we are talking about below is the first two. JDK1.7 has begun to introduce asynchronous I/O, which is called NIO.2.
Traditional IO
We know that the emergence of a new technology is always accompanied by improvements and improvements, and so is the emergence of JavaNIO.
Traditional I/O is blocking I/O, and the main problem is the waste of system resources. For example, in order to read the data of a TCP connection, we call the read() method of InputStream, which will cause the current thread to be suspended and will not be awakened until the data arrives. The thread occupies memory resources (storage thread stack) during the period of data arrival, but does nothing. This is what the saying goes, occupying the pit and not poop. In order to read the data of other connections, we have to start another thread. This may be fine when there are not many concurrent connections, but when the number of connections reaches a certain scale, memory resources will be consumed by a large number of threads. On the other hand, thread switching requires changing the status of the processor, such as the values of program counters and registers, so switching between a large number of threads is also a waste of resources.
With the development of technology, modern operating systems provide new I/O mechanisms that can avoid this waste of resources. Based on this, JavaNIO was born, and the representative feature of NIO is non-blocking I/O. Immediately afterwards, we found that simply using non-blocking I/O cannot solve the problem, because in non-blocking mode, the read() method will return immediately when the data is not read. We don’t know when the data will arrive, so we can only keep calling the read() method to try again. This is obviously a waste of CPU resources. From the following, we can know that the Selector component was born to solve this problem.
JavaNIO core components
1.Channel
concept
All I/O operations in JavaNIO are based on Channel objects, just as stream operations are based on Stream objects, so it is necessary to first understand what Channel is. The following content is excerpted from the documentation of JDK1.8
Achannel represents the annex connection to annexity suchasahardwareDevice, afile, annetworksocket, oroprogram component that can be performed on one ormory distinctive I/Ooperations, forexamplereading orwriting.
From the above content, we can see that a Channel represents a connection to a certain entity, which can be a file, a network socket, etc. In other words, the channel is a bridge provided by JavaNIO for our programs to interact with the underlying I/O services of the operating system.
Channels are a very basic and abstract description, interact with different I/O services, perform different I/O operations, and implement different implementations. Therefore, the specific ones include FileChannel, SocketChannel, etc.
The channel is similar to Stream when used. It can read data into a Buffer or write data in the Buffer to the channel.
Of course, there are also differences, which are mainly reflected in the following two points:
A channel can be read and written, while a Stream is one-way (so divided into InputStream and OutputStream)
The channel has non-blocking I/O mode
accomplish
The most commonly used channel implementations in JavaNIO are as follows, and it can be seen that they correspond to traditional I/O operation classes one by one.
FileChannel: Read and write files
DatagramChannel: UDP protocol network communication
SocketChannel: TCP protocol network communication
ServerSocketChannel: Listen to TCP connections
2.Buffer
The buffer used in NIO is not a simple byte array, but an encapsulated Buffer class. Through the API it provides, we can flexibly manipulate data. Let's take a closer look.
Corresponding to Java basic types, NIO provides a variety of Buffer types, such as ByteBuffer, CharBuffer, IntBuffer, etc. The difference is that the unit length of the buffer is different when reading and writing (reading and writing in units of the corresponding type variables).
There are 3 very important variables in the Buffer. They are the key to understanding the working mechanism of the Buffer, namely
capacity (total capacity)
position (the current position of the pointer)
limit (read/write boundary position)
The working method of Buffer is very similar to character arrays in C. In analogy, capacity is the total length of the array, position is the subscript variable for us to read/write characters, and limit is the position of the ending character. The situation of the 3 variables at the beginning of the Buffer is as follows
During the process of reading/writing the Buffer, the position will move backwards, and limit is the boundary of the position movement. It is not difficult to imagine that when writing to a Buffer, limit should be set to the size of capacity, and when reading to a Buffer, limit should be set to the actual end position of the data. (Note: Writing Buffer data to the channel is a Buffer read operation, and reading data from the channel to the Buffer is a Buffer write operation)
Before reading/write operations on a Buffer, we can call some auxiliary methods provided by the Buffer class to correctly set the values of position and limit, mainly as follows
flip(): Set limit to the value of position, and then set position to 0. Calls before reading the Buffer.
rewind(): Just set position 0. It is usually called before rereading the Buffer data, for example, it will be used when reading the data of the same Buffer and writing it to multiple channels.
clear(): Return to the initial state, that is, limit is equal to capacity, position set to 0. Call the Buffer before writing.
compact(): Move unread data (data between position and limit) to the beginning of the buffer and set position to the next position at the end of this data. In fact, it is equivalent to writing such a piece of data to the buffer again.
Then, look at an example, use FileChannel to read and write text files, and use this example to verify the channel's readable and writable characteristics and the basic usage of Buffer (note that FileChannel cannot be set to non-blocking mode).
FileChannel channel = new RandomAccessFile("test.txt", "rw").getChannel();channel.position(channel.size());// Move the file pointer to the end (append write) ByteBuffer byteBuffer = ByteBuffer.allocate(20);// Write data to BufferbyteBuffer.put("Hello, world!/n".getBytes(StandardCharsets.UTF_8));// Buffer -> ChannelbyteBuffer.flip();while (byteBuffer.hasRemaining()) {channel.write(byteBuffer);}channel.position(0);// Move the file pointer to the beginning (read from the beginning) CharBuffer charBuffer = CharBuffer.allocate(10); CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder();// Read out all data byteBuffer.clear(); while (channel.read(byteBuffer) != -1 || byteBuffer.position() > 0) {byteBuffer.flip();// Decode charBuffer.clear();decoder.decode(byteBuffer, charBuffer, false);System.out.print(charBuffer.flip().toString());byteBuffer.compact();// There may be data remaining}channel.close();In this example, two Buffers are used, where byteBuffer is the data buffer for channel reading and writing, and charBuffer is used to store decoded characters. The usage of clear() and flip() is as mentioned above. It should be noted that the last compact() method is, even if the size of the charBuffer is completely sufficient to accommodate the decoded data byteBuffer, this compact() is also essential. This is because the UTF-8 encoding of commonly used Chinese characters accounts for 3 bytes, so there is a high probability that it will occur in the middle truncation. Please see the figure below:
When the Decoder reads 0xe4 at the end of the buffer, it cannot be mapped to a Unicode. The third parameter of the decode() method, false, is used to make the Decoder treat the unmappable bytes and the subsequent data as additional data. Therefore, the decode() method will stop here, and the position will fall back to the 0xe4 position. In this way, the first byte encoded by the word "medium" is left in the buffer, and it must be compacted to the front and spliced together with the correct and subsequent sequence data. Regarding character encoding, you can refer to " Explanation of ANSI, Unicode, BMP, UTF and other encoding concepts "
BTW, the CharsetDecoder in the example is also a new feature of JavaNIO, so you should have discovered a little bit. NIO operations are buffer-oriented (traditional I/O is stream-oriented).
At this point, we have learned about the basic usage of Channel and Buffer. Next, we will talk about important components of letting a thread manage multiple channels.
3.Selector
What is Selector
Selector is a special component used to collect the state (or events) of each channel. We first register the channel to the selector and set the event we care about, and then we can quietly wait for the event to occur by calling the select() method.
The channel has the following 4 events for us to listen to:
Accept: There is an acceptable connection
Connect: Connect successfully
Read: There is data to read
Write: You can write data
Why use Selector
As mentioned above, if you use blocking I/O, you need to multi-thread (a waste of memory), and if you use non-blocking I/O, you need to constantly try again (a consumption of CPU). The emergence of Selector solves this embarrassing problem. In non-blocking mode, through Selector, our threads only work for ready channels, and there is no need to try blindly. For example, when no data is reached in all channels, no Read event occurs, and our thread will be suspended at the select() method, thus giving up CPU resources.
How to use
As shown below, create a Selector and register a Channel.
Note: To register the Channel to Selector, you must first set the Channel to non-blocking mode, otherwise an exception will be thrown.
Selector selector = Selector.open();channel.configureBlocking(false);SelectionKey key = channel.register(selector, SelectionKey.OP_READ);
The second parameter of the register() method is called "interest set", which is the event set you are concerned about. If you care about multiple events, separate them with a "bital or operator", e.g.
SelectionKey.OP_READ | SelectionKey.OP_WRITE
This writing method is not unfamiliar with it. It is played in programming languages that support bit operations. Using an integer variable can identify multiple states. How is it done? It is actually very simple. For example, first predefined some constants, and their values (binary) are as follows
It can be found that the bits with their value of 1 are all staggered, so the values obtained after performing bitwise or calculations on them have no ambiguity, and they can be deduced inversely which variables are calculated from. How to judge, yes, it is the "bits and" operation. For example, there is now a state set variable value of 0011. We only need to determine whether the value of "0011&OP_READ" is 1 or 0 to determine whether the set contains the OP_READ state.
Then, note that the register() method returns a SelectionKey object, which contains the information for this registration, and we can also modify the registration information through it. From the complete example below, we can see that after select(), we also get the channels with state ready by obtaining a collection of SelectionKeys.
A complete example
The concepts and theoretical things have been explained (actually, after writing them here, I found that I didn’t write much, which is so embarrassing (⊙ˍ⊙)). Let’s take a look at a complete example.
This example uses JavaNIO to implement a single-threaded server. The function is very simple. It listens to the client connection. When the connection is established, it reads the client's message and responds to a message to the client.
It should be noted that I use the character '/0' (a byte with a value of 0) to identify the end of the message.
Single threaded server
public class NioServer {public static void main(String[] args) throws IOException {// Create a selectorSelector selector = Selector.open();// Initialize the TCP connection listening channel ServerSocketChannel listenChannel = ServerSocketChannel.open(); listenChannel.bind(new InetSocketAddress(9999)); listenChannel.configureBlocking(false);// Register to selector (listen to its ACCEPT event) listenChannel.register(selector, SelectionKey.OP_ACCEPT);// Create a buffer ByteBuffer buffer = ByteBuffer.allocate(100); while (true) {selector.select();//Block until an event is listened to occurs Iterator<SelectionKey> keyIter = selector.selectedKeys().iterator();// Access the channel event selected through an iterator in turn while (keyIter.hasNext()) {SelectionKey key = keyIter.next();if (key.isAcceptable()) {// There is a connection to accept SocketChannel channel = ((ServerSocketChannel) key.channel()).accept();channel.configureBlocking(false);channel.register(selector, SelectionKey.OP_READ);System.out.println("Connection established with [" + channel.getRemoteAddress() + "]!");} else if (key.isReadable()) {// There is data to read buffer.clear();// Read to the end of the stream, indicating that the TCP connection has been disconnected, // Therefore, it is necessary to close the channel or cancel listening to the READ event// Otherwise, it will loop infinitely if (((SocketChannel) key.channel()).read(buffer) == -1) {key.channel().close();continue;}// traverse data by byte buffer.flip(); while (buffer.hasRemaining()) {byte b = buffer.get();if (b == 0) {// /0System.out.println() at the end of the client message;// Response client buffer.clear();buffer.put("Hello, Client!/0".getBytes());buffer.flip();while (buffer.hasRemaining()) {((SocketChannel) key.channel()).write(buffer);}} else {System.out.print((char) b);}}}// For events that have been processed, you must manually remove keyIter.remove();}}}}Client
This client is purely used for testing. In order to make it less difficult, it uses traditional writing methods, and the code is very short.
If you need to be more rigorous in testing, you should run a large number of clients concurrently to count the response time of the server, and do not send data immediately after the connection is established, so as to give full play to the advantages of non-blocking I/O on the server.
public class Client {public static void main(String[] args) throws Exception {Socket socket = new Socket("localhost", 9999);InputStream is = socket.getInputStream();OutputStream os = socket.getOutputStream();// Send data to the server first os.write("Hello, Server!/0".getBytes());// Read the data sent by the server int b; while ((b = is.read()) != 0) {System.out.print((char) b);}System.out.println();socket.close();}}Summarize
The above is all about a quick understanding of NIO core components in Java. I hope it will be helpful to everyone. Interested friends can continue to refer to other relevant contents of this website. If there are any shortcomings, please leave a message to point it out. Thank you friends for your support for this site!