Through this article, we will mainly explain the problems encountered in Servlet 3.0 asynchronous processing in JAVA development and the solutions. The following are the specific contents:
Servlet 3.0 has begun to provide AsyncContext to support asynchronous processing of requests. So what benefits can asynchronous processing of requests bring?
Generally speaking, the way web containers handle requests is to assign a thread to each request. We all know that the creation of threads is not without cost, and the thread pool of web containers has an upper limit.
A very predictable problem is that under high load conditions, the thread pool is occupied, so the subsequent request can only be waited. If you are not lucky, the client will report a waiting timeout error.
Before AsyncContext appeared, the only way to solve this problem was to expand the thread pool of the web container.
But there is still a problem with this, consider the following scenarios:
There is a web container with a thread pool size of 200. There is a web app that has two servlets, the time it takes Servlet-A to handle a single request is 10s, and the time it takes Servlet-B to handle a single request is 1s.
Now we encounter a high load, with more than 200 requests to Servlet-A. If Servlet-B is requested at this time, we will wait because all HTTP threads have been occupied by Servlet-A.
At this time, the engineer discovered the problem and expanded the thread pool size to 400, but the load continued to rise. Now there are 400 requests to Servlet-A, and Servlet-B still cannot respond.
Have you seen the problem? Because HTTP thread and Worker thread are coupled together, it will fill up the HTTP thread when a large number of requests are made to a time-consuming operation, resulting in the entire web container being unable to respond.
However, if we use AsyncContext, we can hand over the time-consuming operation to another thread, so that the HTTP thread will be released and we can handle other requests.
Note that only using AsyncContext can achieve the effect mentioned above. If you directly use new Thread() or similar methods, the HTTP thread will not be returned to the container.
Here is an official example:
@WebServlet(urlPatterns={"/asyncservlet"}, asyncSupported=true)public class AsyncServlet extends HttpServlet { /* ... Same variables and init method as in SyncServlet ... */ @Override public void doGet(HttpServletRequest request, HttpServletResponse response) { response.setContentType("text/html;charset=UTF-8"); final AsyncContext acontext = request.startAsync(); acontext.start(new Runnable() { public void run() { String param = acontext.getRequest().getParameter("param"); String result = resource.process(param); HttpServletResponse response = acontext.getResponse(); /* ... print to the response ... */ acontext.complete(); } }); }} trap
In this official example, each HTTP thread will open another Worker thread to process the request, and then return the HTTP thread to the web container. But look at the javadoc of the AsyncContext.start() method:
Causes the container to dispatch a thread, possibly from a managed thread pool, to run the specified Runnable.
In fact, there is no regulation here where the Worker thread comes from. Maybe it is another thread pool other than the HTTP thread pool? Or is it just an HTTP thread pool?
The Limited Usefulness of AsyncContext.start() article reads: Different web containers have different implementations for this, but Tomcat actually uses HTTP thread pool to handle AsyncContext.start().
This means that we originally wanted to release the HTTP thread, but in fact it did not, because the HTTP thread is still used as the Worker thread, but this thread is not the same as the HTTP thread receiving the request.
We can also see this conclusion through the Jmeter benchmark of AsyncServlet1 and SyncServlet, and the results of the two throughput are similar. Start method: Start Main, and then use Jmeter to start benchmark.jmx (HTTP thread pool=200 under Tomcat default configuration).
Using ExecutorService
I saw earlier that Tomcat does not maintain the Worker thread pool separately, so we have to find a way to do it ourselves, see AsyncServlet2, which uses an ExecutorService with Thread pool to handle AsyncContext.
Other ways
Therefore, there is no fixed way to use AsyncContext. You can use different methods to deal with it according to actual needs. For this reason, you need some knowledge of Java concurrent programming.
Misunderstanding of performance
The purpose of AsyncContext is not to improve performance, nor does it directly provide performance improvement. It provides a mechanism to decouple HTTP thread and Worker thread, thereby improving the responsiveness of web containers.
However, AsyncContext can improve performance at some point, but this depends on how your code is written.
For example: the number of HTTP thread pools in a web container is 200, and a Servlet uses a 300 Worker thread pool to handle AsyncContext.
Compared with the Sync method Worker thread pool=HTTP thread pool=200, in this case we have a Worker thread pool of 300, so it will definitely bring some performance improvements (after all, there are more people working).
On the contrary, if the number of Worker threads is <= the number of HTTP threads, there will be no performance improvement, because the bottleneck for processing requests is at Worker thread.
You can modify the thread pool size of AsyncServlet2 and compare it with SyncServlet benchmark results to verify this conclusion.
Don't think that the Worker thread pool must be larger than the HTTP thread pool. The reasons are as follows:
The two responsibilities are different. One is that the web container is used to receive external requests , and the other is to process business logic.
Creating threads comes at a cost. If the HTTP thread pool is already large, then creating a larger Worker thread pool will cause too much Context switch and memory overhead.
The purpose of AsyncContext is to release the HTTP thread to avoid long-term use of operations and thus causing the web container to be unable to respond.
So in most cases, the Worker thread pool will not be very large, and different Worker thread pools will be built according to different businesses.
For example: the web container thread pool size is 200, and the Worker thread pool size is 10 for a slow servlet. In this way, no matter how many requests are used to slow operations, it will not fill the HTTP thread and cause other requests to be unable to process.