This article describes the multi-threaded Java download. Share it for your reference, as follows:
Using multi-threaded files to download files can complete the download of files faster. The reason why multi-threaded files are fast is that they occupy a lot of server resources. For example: Suppose the server serves up to 100 users at the same time, and one thread in the server corresponds to one user. 100 threads are not executed concurrently in the computer, but are executed in turn by the CPU. If Application A uses 99 threads to download files, it is equivalent to occupying the resources of 99 users. Suppose that the average execution time allocated by the CPU to each thread within one second is 10ms. Application A obtains 990ms execution time in one second in the server, while other applications only have 10ms execution time in one second. Just like a faucet, when the water output per second is equal, it is definitely more water for 990 milliseconds than for 10 milliseconds.
The implementation process of multi-threaded download:
1. First get the length of the downloaded file, and then set the length of the local file.
HttpURLConnection.getContentLength();RandomAccessFile file = new RandomAccessFile("youdao.exe","rw");file.setLength(filesize);//Set the length of the local file2. Calculate the data length and download location of each thread based on the file length and number of threads. For example: the length of the file is 6M and the number of threads is 3, then the length of data downloaded by each thread is 2M, and the location where each thread starts downloading is shown in the figure below.
3. Use the Range header field of Http to specify where each thread starts downloading from, such as: specify the file download from the 2M position of the file, the code is as follows:
The code copy is as follows: HttpURLConnection.setRequestProperty("Range", "bytes=2097152-");
4. Save the file and use the RandomAccessFile class to specify where each thread starts to write data from in the local file.
RandomAccessFile threadfile = new RandomAccessFile("<span style="font-family: Arial, Helvetica, sans-serif;">youdao.exe</span><span style="font-family: Arial, Helvetica, sans-serif;"> ","rw");</span>threadfile.seek(2097152);//Where to write data from where in the fileThe following is the specific implementation class:
Before writing the implementation class, we must first place the file to be downloaded on the server and deploy:
I put it here in the directory D:/Tomcat/apache-tomcat-7.0.37/webapps/doudou and start the startup.bat under D:/Tomcat/apache-tomcat-7.0.37/bin
1.DownLoadTest.java
package www.csdn.net.down;import java.io.File;import java.io.FileInputStream;import java.io.FileOutputStream;import java.io.IOException;import java.io.InputStream;import java.io.RandomAccessFile;import java.net.HttpURLConnection;import java.net.MalformedURLException;import java.net.URL;public class DownloadLoadTest { public File file; public RandomAccessFile accessFile; // Number of threads public static int threadNum = 3; // Each thread is responsible for downloading int blockSize; // Create the access path public String path = "http://localhost:8080/doudou/youdao.exe"; public static int threadCount;// Quantity public void testDown() { try { // Create URL object URL url = new URL(path); // Create HttpURLConnection object HttpURLConnection httpURLConnection = (HttpURLConnection) url .openConnection(); // Set the method of sending requests httpURLConnection.setRequestMethod("GET"); // Set whether the request timeout httpURLConnection.setConnectTimeout(5000); // Set httpURLConnection .setRequestProperty("User-Agent", " Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0)"); // Whether the response is successful if (httpURLConnection.getResponseCode() == 200) { // Get the file size int size = httpURLConnection.getContentLength(); System.out.println("File size" + size); // Create file file = new File("youdao.exe"); accessFile = new RandomAccessFile(file, "rwd"); // Set the file size accessFile.setLength(size); // Download the size of each thread blockSize = size / threadNum; // Open three threads to operate this file for (int i = 1; i <= threadNum; i++) { // 1 2 3 // Calculate the starting position of each thread int startSize = (i - 1) * blockSize; // End position int endSize = (i) * blockSize; // When the thread is the last thread if (i == threadNum) { // Determine whether the file size is greater than the calculated end position if (size > endSize) { // The end position is equal to the file size endSize = size; } } // Create a random read for each thread RandomAccessFile threadAccessFile = new RandomAccessFile( file, "rwd"); new Thread(new DownloadThread(i, threadAccessFile, startSize, endSize, path)).start(); } } } catch (MalformedURLException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } public static void main(String[] args) { DownloadTest downLoadTest = new DownloadTest(); // Call the download method downLoadTest.testDown(); }}class DownloadLoadThread implements Runnable { // Encapsulation of download file public RandomAccessFile accessFile; // Each thread has an accessFile file object Thread1 Thread2 Thread3 // The starting position of the thread download file public int startSize; public int endSize; // The path path of the file download public String path; public int threadId; // The identification of thread public DownloadThread(int threadId, RandomAccessFile accessFile, int startSize, int endSize, String path) { this.threadId = threadId; this.accessFile = accessFile; this.startSize = startSize; this.endSize = endSize; this.path = path; } @Override public void run() { // Execute the run method try { // Create a file File threadFile = new File(threadId + ".txt"); if (threadFile.exists()) { // Read the content of the file// Create the file's input stream object FileInputStream fis = new FileInputStream(threadFile); // Use tool class to read byte data[] = StreamTools.isToData(fis); // Convert to string String threadLen = new String(data); if ((threadLen != null) && (!"".equals(threadLen))) { startSize = Integer.valueOf(threadLen); // Solve the error of 416bug if (startSize > endSize) { startSize = endSize - 1; } } } // Create URL object URL url = new URL(path); // Create HttpURLConnection object HttpURLConnection httpURLConnection = (HttpURLConnection) url .openConnection(); // Set the request header httpURLConnection.setRequestMethod("GET"); // Set whether the request timeout httpURLConnection.setConnectTimeout(5000); // Set httpURLConnection .setRequestProperty("User-Agent", " Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0)"); // Key settings httpURLConnection.setRequestProperty("Range", "bytes=" + startSize + "-" + endSize); // Output the current thread System.out.println("Current thread" + threadId + " Download start position:" + startSize + " Download end position:" + endSize); // Successful response// Set the start position of random read files accessFile.seek(startSize); // Get the corresponding stream object InputStream is = httpURLConnection.getInputStream(); // Create output stream object byte buffer[] = new byte[1024]; int len = 0; int threadTotal = 0;// Save record after each thread is downloaded/ while ((len = is.read(buffer)) != -1) { accessFile.write(buffer, 0, len); threadTotal += len;// Record the length you write//xml file// Record the length of the file download through the file FileOutputStream fos = new FileOutputStream(threadFile); fos.write((threadTotal + "").getBytes()); fos.flush(); fos.close(); } accessFile.close(); is.close(); System.out.println(threadId + "Thread execution is completed"); //Thread operation synchronized (DownLoadTest.class) { DownloadLoadTest.threadCount++; if (DownLoadTest.threadCount >= DownloadLoadTest.threadNum) { for(int i=1;i<=DownLoadTest.threadNum;i++){ File file = new File(i+".txt"); if(file.exists()){ file.delete(); } } } } } catch (MalformedURLException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } }}2. StreamTools.java encapsulation of streaming tools
package www.csdn.net.down;import java.io.ByteArrayOutputStream;import java.io.IOException;import java.io.InputStream;public class StreamTools { public static byte[] isToData(InputStream is) throws IOException{ // ByteArrayOutputStream bops = new ByteArrayOutputStream(); // The buffer area for reading data byte buffer[] = new byte[1024]; // Read length record int len = 0; // Loop reading while ((len = is.read(buffer)) != -1) { bops.write(buffer, 0, len); } // Convert the read content into a byte array byte data[] = bops.toByteArray(); bops.flush(); bops.close(); is.close(); return data; }}I hope this article will be helpful to everyone's Java programming.