My friend asked me to help me write a program to import data from a text document into an oracle database. It is not technically difficult. The format of the document is fixed. As long as the corresponding field analysis in the database is enough, the key lies in performance.
The data volume is very large and millions of records. Therefore, considering that you need to use multi-thread concurrent execution, and you encounter problems during the writing process, I want to count the total time spent on all child processes after the execution is completed. Record the current time before the first child process is created. Use System.currentTimeMillis() to record the current time after the last child process is completed. The time difference obtained by subtraction twice is the total time. The code is as follows
long tStart = System.currentTimeMillis(); System.out.println(Thread.currentThread().getName() + "Start");//Print the start mark for (int ii = 0; ii < threadNum; ii++) {//Open threads Runnable r = new Runnable(){ @Override public void run(){ System.out.println(Thread.currentThread().getName() + "Start"); //Do something... System.out.println(Thread.currentThread().getName() + "end."); } } Thread t = new Thread(r); t.start(); } System.out.println(Thread.currentThread().getName() + "end.");//Print the end tag long tEnd = System.currentTimeMillis(); System.out.println("Total time:"+ (tEnd - tStart) + "millions"); The result is that the statement that the main thread prints the total time is executed almost the moment the for loop ends. The reason is that all the child threads are executed concurrently, and the main thread is also running when they run, which raises a question: how to "make the main thread wait for all the child threads to execute." I tried adding t.join() after each child thread starts, and the result is that all threads execute sequentially, which loses the meaning of concurrency, which is obviously not what I want.
Google has been finding a solution for a long time. Has no one ever encountered such a need? Or is this problem too simple? Wu Nai had to think of a solution...
Finally, my solution is to customize an ImportThread class inherits from java.lang.Thread, overload the run() method, and use a List property to save all generated threads. In this way, as long as you judge whether the List is empty, you will know whether there are still child threads that have not been executed. The class code is as follows:
public class ImportThread extends Thread { private static List<Thread> runningThreads = new ArrayList<Thread>(); public ImportThread() { } @Override public void run() { registration(this);//Register System.out.println(Thread.currentThread().getName() + "Start...");//Print the start marker//Do something... unRegist(this);//Unregister System.out.println(Thread.currentThread().getName() + "end.");//Print the end tag} public void registration(Thread t){ synchronized(runningThreads){ runningThreads.add(t); } } public void unRegist(Thread t){ synchronized(runningThreads){ synchronized(runningThreads){ runningThreads.remove(t); } } public static boolean hasThreadRunning() { return (runningThreads.size() > 0);//By judging whether runningThreads is empty, you can know whether there are still threads that have not been executed} } Code in main thread:
long tStart = System.currentTimeMillis(); System.out.println(Thread.currentThread().getName() + "Start");//Print the start mark for (int ii = 0; ii < threadNum; ii++) {//Open threads Thread t = new ImportThread(); t.start(); } while(true){//Waiting for all child threads to complete execution if(!ImportThread.hasThreadRunning()){ break; } Thread.sleep(500); } System.out.println(Thread.currentThread().getName() + "end.");//Print end tag long tEnd = System.currentTimeMillis(); System.out.println("Total time:"+ (tEnd - tStart) + "millions");The print result is:
Main Start
Thread-1 starts...
Thread-5 starts...
Thread-0 starts...
Thread-2 starts...
Thread-3 starts...
Thread-4 starts...
Thread-5 ends.
Thread-4 ends.
Thread-2 ends.
Thread-0 ends.
Thread-3 ends.
Thread-1 ends.
The end of main.
Total time: 20860millions
You can see that the main thread only starts execution after all the child threads have been executed.
=========================================================================================================================================
The above method has a hidden danger: if thread 1 starts and ends, and other threads have not started yet, the size of runningThreads is also 0, the main thread will think that all threads have been executed. The solution is to replace List type runningThreads with a non-simple type counter, and the counter value should be set before the thread is created.
MyCountDown class
public class MyCountDown { private int count; public MyCountDown(int count){ this.count = count; } public synchronized void countDown(){ count--; } public synchronized boolean hasNext(){ return (count > 0); } public int getCount() { return count; } public void setCount(int count) { this.count = count; } } ImportThread class
public class ImportThread extends Thread { private MyCountDown c; public ImportThread(MyCountDown c) { this.c = c; } @Override public void run() { System.out.println(Thread.currentThread().getName() + "Start...");//Print the start marker//Do something c.countDown();//Timer minus 1 System.out.println(Thread.currentThread().getName() + "End. And" + c.getCount() + "Threads");//Print the end mark} } In the main thread
System.out.println(Thread.currentThread().getName() + "Start");//Print the start tag MyCountDown c = new MyCountDown(threadNum);//Initialize countDown for (int ii = 0; ii < threadNum; ii++) {//Open threads Thread t = new ImportThread(c); t.start(); } while(true){//Waiting for all child threads to execute if(!c.hasNext()) break; } System.out.println(Thread.currentThread().getName() + "end.");//Print the end markPrint result:
Main Start
Thread-2 starts...
Thread-1 starts...
Thread-0 starts...
Thread-3 starts...
Thread-5 starts...
Thread-4 starts...
Thread-5 ends. 5 more threads
Thread-1 ends. There are 4 more threads
Thread-4 ends. There are 3 more threads
Thread-2 ends. 2 more threads
Thread-3 ends. 1 thread is still there
Thread-0 ends. There are 0 threads left
The end of main.
Easier way: Use java.util.concurrent.CountDownLatch instead of MyCountDown, use await() method instead of while(true){...}
ImportThread class
public class ImportThread extends Thread { private CountDownLatch threadsSignal; public ImportThread(CountDownLatch threadsSignal) { this.threadsSignal = threadsSignal; } @Override public void run() { System.out.println(Thread.currentThread().getName() + "Start..."); //Do something threadsSignal.countDown();//The counter is reduced by 1 at the end of the thread System.out.println(Thread.currentThread().getName() + "end. There are also " + threadsSignal.getCount() + " threads"); } } In the main thread
CountDownLatch threadSignal = new CountDownLatch(threadNum);//Initialize countDown for (int ii = 0; ii < threadNum; ii++) {//Open threads final Iterator<String> itt = it.get(ii); Thread t = new ImportThread(itt,sql,threadSignal); t.start(); } threadSignal.await();//Waiting for all child threads to complete execution System.out.println(Thread.currentThread().getName() + "End.");//Print the end markPrint result:
Main Start
Thread-1 starts...
Thread-0 starts...
Thread-2 starts...
Thread-3 starts...
Thread-4 starts...
Thread-5 starts...
Thread-0 ends. 5 more threads
Thread-1 ends. There are 4 more threads
Thread-4 ends. There are 3 more threads
Thread-2 ends. 2 more threads
Thread-5 ends. 1 thread is still there
Thread-3 ends. There are 0 threads left
The end of main.
The above is all the content of this article. I hope it will be helpful to everyone's learning and I hope everyone will support Wulin.com more.