ssxrver

ssxrver

     _______.     _______.___   ___ .______     ____    ____  _______ .______      
    /       |    /       |   /  / |   _           /   / |   ____||   _       
   |   (----`   |   (----`   V  /  |  |_)  |       /   /  |  |__   |  |_)  |    
                        >   <   |      /            /   |   __|  |      /     
.----)   |   .----)   |    /  .    |  |  ----.      /    |  |____ |  |  ----.
|_______/    |_______/    /__/ __ | _| `._____|   __/     |_______|| _| `._____|

ssxrver is a high-performance, high-concurrency network library running on the Linux platform. It is written in C++17 and supports TCP and UDP protocols.

Advantages

1. The optimized one-loop-per-thread + fine-grained lock thread-pool model is used.
2. High performance and high concurrency, pressure measurement data is higher than Nginx/1.14.2 and Apache/2.4.28
3. The main thread only performs accept operations to distribute events through eventfd to avoid competition from locks, and IO threads read and write data.
4. According to the analysis in different scenarios, different multiplexing methods of epoll are used to improve performance.
5. Use RAII mechanism to control the object life cycle, and all memory allocation operations use smart pointers to avoid memory leakage.
6. Use timerfd provided by the Linux kernel to handle timed events and IO time uniformly, and implement timer management through the C++11 standard libraries std::chrono and std::priority_queue to implement nanosecond-level timing tasks.
7. Use non-blocking sockets to prevent threads from being blocked by a single connection.
8. Encapsulate the http module, and a high-performance HTTP Server can be configured with simple operations, using Ragel (finite state machine) to parse HTTP requests, adjusting efficiency, supporting GET and POST requests for HTTP/1.0 and HTTP/1.1, and supporting long connections.
9. Encapsulate high-performance buffer classes for data transmission and reception.
10. File sending uses sendfile zero copy technology to improve file sending performance.
11. Encapsulate the database operation module, which can simply parse and generate SQL statements corresponding to the MySQL database, and can implement database connection pooling with fine-grained lock thread-pool.
12. Implement multi-buffer asynchronous log library, supporting the setting of log levels, log scroll size and other functions.
13. Use new features of C++11 14 17 syntax to improve performance, such as std::make_shared , std::make_unique , std:string_view , explicit , [[nodiscard]] , emplace_back , etc.
14. Use unified style code style and naming specifications, and add more than 10 compilation parameters to standardize code implementation, improving code quality and compiler optimization possibilities.
15. Multiple designs are used to reuse objects to reduce the frequent application of certain objects for release.
16. Using object-based programming ideas, the project code structure is clear and clear, and functions that are frequently called to each other should be put together as much as possible, increasing the CPU cache hit rate, loose coupling between modules, making it easy to add new functional modules.
17. Use singleton mode, policy mode, adapter mode and other design mode to reduce code redundancy and make the implementation of code more elegant.
18. Encapsulate the configuration file module and use the json format to quickly configure it.
19. The CPU affinity can be configured through configuration files, thereby reducing the number of direct thread context switching times and improving performance.
20. Supports UDP protocol.

Development Environment

• Operating system hairstyle version: deepin v20.1 Community version (1030)
• Kernel version: 5.4.70-amd64-desktop (64-bit)
• Compiler version: gcc 8.3
• Language: c++ 17
• cmake version: 3.11.2
• boost library version: 1.72
• Database version: MySQL 5.7.21-1

How to run

1. Please try to match the same development environment as me. If you do not need a database module, please modify CMakeLists.txt accordingly.

   • cmake installation

      # debian/ubuntu
     sudo apt-get install cmake

   • boost library installation

     wget http://sourceforge.net/projects/boost/files/boost/1.72.0/boost_1_72_0.tar.bz2
     tar -xvf boost_1_72_0.tar.bz2
     cd ./boost_1_72_0
     ./bootstrap.sh --prefix=/usr/local
     sudo ./b2 install --with=all

2. Run ./build.sh in the ssxrver directory, you can modify build.sh to choose to generate the Debug version or the Release version (the default Release version)

   ./build.sh

3. The compiled successfully will generate the build/ directory, and the executable file is in the corresponding version directory. For example, when you select the Release version, the executable file is in /build/Release/ssxrver.

4. Imitate the format of conf/ssxrver.json.example to create your configuration file (note that the configuration file cannot be commented, not commented, not commented). I will explain the options of each configuration file below. I actually set the default values ​​for many parameters. If not configured, it will not affect it.

   {
     " port " : 4507, # 端口号,不填的话默认4507
     " address " : " 127.0.0.1 " , # 绑定的地址
     " worker_processes " : 4, # IO 线程数量,不填默认为 4 个
     " worker_connections " : -1, # 一个 IO 线程最多支持多少连接, -1 表示最多能创建多少就创建多少,不做限制
     " task_processes " : 0, # 任务线程,不填的话默认为 0 
     " cpu_affinity " : " off " , # cpu 亲和度 ,默认关闭
     " http " : { # http 模块
       " max_body_size " : 67108864, # 单个 http 包最大支持大小
       " root_path " : " /home/randylambert/sunshouxun/ssxrver/html/ " # 文件访问根路径
     },
     " log " : { # log 模块
       " level " : " INFO " , # 输出等级,可填三种等级, DEBUG,INFO,WARN 不填默认为 INFO 等级
       " ansync_started " : " off " , # 是否打开异步日志线程,不填默认关闭
       " flush_second " : 3, # 异步线程每隔多久持久化一次
       " roll_size " : 67108864, # 日志文件滚动大小
       " path " : " /home/randylambert/sunshouxun/ssxrver/logs/ " , # 日志文件存放路径
       " base_name " : " ssxrver " # 日志文件基础名
     },
     " mysql " : { # 数据库模块
       " mysql_started " : " off " , # 是否打开数据库模块,默认关闭
       " address " : " 127.0.0.1 " , # 以下是对应数据库连接信息
       " user " : " root " ,
       " password " : " 123456 " ,
       " database_name " : " ttms " ,
       " port " : 0,
       " unix_socket " : null,
       " client_flag " : 0
     },
     " blocks_ip " : [ " 122.0.0.2 " , " 198.1.2.33 " ] # 可屏蔽部分恶意 IP
   }

5. Run the executable file.

./ssxrver -f /配置文件的路径
# 例如
./build/Release/ssxrver -f ./conf/ssxrver.json

Pressure test

Test environment	Value
Operating system hairstyle version	deepin v20.1 Community Edition (1030)
Kernel version	5.4.70-amd64-desktop (64-bit)
Compiler version	gcc 8.3
boost library version	1.72
processor	Intel(R) Core(TM) i7-8750H CPU @2.20GHz
L1 Cache Size	32K
L2 Cache Size	256K
L3 Cache Size	9216K
Hard disk speed	1.8 TiB mechanical hard drive 5400 rpm
Hard disk read and write speed	370 MB in 3.03 seconds = 122.27 MB/sec
Memory	7.6GB
Swap partition	4.7GB
Logical core count	12 cores

Test scenario

1. To control variables, restart the computer before testing to ensure that the test environment does not have other applications with high CPU load and high IO load.

2. The test tool is webbench1.5. Remove the first warm-up data. The test command is as follows (100 clients have been accessed continuously for 15 seconds).

   ./webbench -c 100 -t 15 http://127.0.0.1:8081/

3. The test objects are Apache/2.4.38, nginx/1.14.2, ssxrver.

   • Apache/2.4.38 adopts default configuration
   • nginx/1.14.2 Close log printing, open 4 worker processes, open sendfile, and the rest default configurations.
   • The ssxrver LOG level is set to INFO, open the asynchronous log thread, and open 4 IO threads.

Note: Whether using webbench or ab, the data measured by this pressure measurement tool can only be used as a simple reference. Pressure measurement is a test that requires all-round and multi-angle, rather than simply running a command. Even during pressure measurement, the data is not transmitted through the network at all, but just goes around in the kernel.

Test results

Network library	Speed(pages/min)	Requests success rate
ssxrver returns the response generated in memory	7107414	100%
ssxrver returns static files	5114376	100%
Apache/2.4.28	2884072	100%
nginx/1.14.2	4728748	100%

• ssxrver returns the response generated in memory
• ssxrver returns static files
• Apache/2.4.28
• nginx/1.14.2

The test results of ssxrver are pretty good, but strangely, I thought the data would be higher, because when I was developing in the early days, I didn't do many optimizations at that time. When I returned the response generated directly in memory, it was measured at most close to 8000000 pages/min (the test results of 8000000 pages/min were not taken in screenshots, leaving a 7550778). At that time, nginx/1.14.2 had a maximum of 5000000 pages/min. However, no matter whether it was ssxrver or nginx/1.14.2, I couldn't find such a high value. I don't know what was the reason, which led to such a big gap in the final result (Is it because my computer is aging? ￣□￣｜｜)

About trade-offs

1. When I was writing thread pools, I hesitated for a long time whether to use lock-free thread pools or fine-grained lock thread pools. In the end, I chose the fine-grained lock thread pool. Although the lock-free thread pool will enter the kernel state less in high concurrency scenarios and generally has higher performance, it will consume CPU resources in vain when the number of tasks is small. In order to ensure the universality of ssxrver in any scenario, I chose the fine-grained lock thread pool.
2. When implementing the timer function, the best performance of my task should be the timer implemented using a fine-grained time wheel. The time granularity is easy to configure, and the time complexity of adding and obtaining timing tasks is close to O(1). However, in the scenario of network library, I found it difficult to control the time wheel to scroll forward according to a fixed time unit, and sleep for a while with sleep? This will directly block normal IO events. Using signals? In multi-threaded programming, signal processing is very difficult, and there is no advantage in performance, which is not worth the loss. Use epoll_wait() to set the timeout time? Whenever a readable event is triggered, a new timeout time must be modified. If the processing time is too long, the unit time will be exceeded, the accuracy will be reduced. Directly opening a separate timer thread is only responsible for the timing task? This can perfectly solve the above problem. The timer thread is only responsible for adding and triggering the corresponding timing task, and after triggering, the task is passed to the IO after triggering. The execution of threads or calculation threads will not cause the accuracy to decrease, but I feel that it is not necessary to do so, so I simply hand over the timing tasks to kernel management, using the combination of priority queue + timefd, which ensures good time complexity (O(log(n))), timefd ensures extremely high accuracy, and can also handle timing tasks and IO time together. Although I did not implement the timer using the time wheel, the idea of ​​time wheel is actually in use. For example, in some scenarios, TCP KeepAlive cannot meet our requirements for disconnection of idle long connections. If we want to implement the user-mode KeepAlive, we need to create a timing task for each connection, so as to kick off the idle long connection that has not been communicated for a long time, or create a timing task, traversing the entire connection pool each time (of course, you can use a sorted connection List , so there is no need to traverse the entire connection pool) to determine whether the corresponding connection is to be kicked off. However, neither of these methods is elegant enough. At this time, we can borrow the idea of ​​the time wheel and place the connection in the roulette slot. By setting the timing task, we control the time wheel to scroll forward. Each scrolling step will be processed. In this way, we will not traverse all connections every time, and there will be too many timing tasks.

About the Future

1. At present, I personally will modify the Buffer module and Log module of ssxrver if I have time.

   • First of all, the easiest way to modify the Buffer module is to change it to a cyclic buffer, thereby effectively reducing the number of times the Buffer moves data forward, or directly abandoning this Buffer implementation and re-implementing a high-performance Buffer.

   • Secondly, the current Log module is written in the stream form of C++. Although it is definitely higher in performance than using C++ directly with iostream, overloading the Log in the << symbolic form will still cause inconvenient format control and performance problems caused by function call chains. Both of these problems can be solved by implementing the Log in the form of printf.

2. Due to time reasons, SSXRver does not implement the memory management module, so it is almost impossible to write a general high-performance memory management module (it is better to go directly to jemalloc or tcmalloc). However, by analyzing the network library scenario, it is still a little chance to write a memory management module with higher performance in this scenario. If I have time, I will take a look at the implementation in nginx and learn it.

3. When I was querying the information, I came to a conclusion that in C++17, you can use std::string_view to replace const string&, which will improve some efficiency. Therefore, I tried to replace all the places where const string& in my project with std::string_view. However, when I finally used perf -top to view the changed load, I unexpectedly discovered that some functions actually increased after I used std::string_view to replace it. I was very puzzled why this situation occurred. Due to time reasons, I will not investigate the specific cause of this problem for the time being. I have the opportunity to check the underlying implementation to check the specific reason.

• Before replacement
• After replacement

4. When implementing the http parsing module, I used a handwritten state machine that directly matches strings in the first version. Then I replaced it with the state machine implemented by Ragel. However, during recent tests, I found that the load of the http parsing function is very exaggerated, reaching 10%. Could it be that using Ragel has caused performance degradation? (If parsing the header will cause such a high system load, then it seems that HTTP/2.0 will still improve performance significantly) Unfortunately, when I handwritten the state machine before, I did not test the load of the corresponding parsing function. Now I can't get the data comparison between the two at once, and I have the opportunity to write a BenchMark test.

5. Ssxrver supports simple UDP transmission, but I personally think that a UDP framework without congestion control, traffic control, and packet loss retransmission functions can basically be said to be unable to be used normally. In the future, I have time to learn QUIC and KCP protocols. I will supplement UDP related knowledge. I believe that the more efficient and flexible UDP protocol will be more and more widely used in the future!

About the framework

In fact, I actually think the best network framework at present should be that port multiplexing address multiplexing plus multiple threads (multi-process) binds the same address and port, and the kernel automatically performs accept load balancing. At the same time, blocks system calls through coroutine framework + hook. After using this framework, it can ensure high performance without using the main thread to distribute connections, and there is no need to fall into asynchronous callback hell.

In addition, if you can use the asynchronous IO mechanism io_uring added after Linux kernel 5.1, I believe that the performance of the server will be higher. However, I don’t know much about io_uring at present, and I don’t have the ability to design an asynchronous IO network library based on io_uring.