1. Introduction
Pooling technology is widely used in Java. In short, it uses an object pool to store an instance with limited number of instances. Developers obtain instances from the object pool and then switch back to the object pool after use, thereby reducing the overhead of the system's frequent creation of objects and destroying objects to a certain extent. Java thread pool and database connection pool are typical applications, but not all objects are suitable for pooling. Pooling objects with relatively small overhead will affect performance, because maintaining object pools also requires a certain resource overhead. For objects with high overhead and frequent creation and use of use, using pooling technology will greatly improve performance.
There are many mature database connection pools in the industry, such as C3P0, DBCP, Proxool and Alibaba's Druid. There are many and open source, and you can find source code on GitHub. Developers can choose based on their own needs and their characteristics and performance. This article is just to understand the learning pooling technology and implement a simple database connection pool. If there are any errors, I hope to criticize and correct them.
2. Design
Main classes and interfaces
.ConnectionParam - Database connection pool parameter class, responsible for configuring database connections and connection pool related parameters. Implementation using Builder.
driver url user password - required to connect to the database
minConnection - Minimum number of connections
maxConnection - Maximum number of connections
minIdle - Minimum number of idle connections
maxWait - Maximum waiting time
private final String driver; private final String url; private final String user; private final String password; private final int minConnection; private final int maxConnection; private final int minIdle; private final long maxWait;
.ConnectionPool - Database connection pool
The ConnectionPool constructor is declared as protection, prohibits external creation, and is managed uniformly by ConnectionPoolFactory.
ConnectionPool implements the DataSource interface and re-getConnection() method.
ConnectionPool holds two containers - one Queue stores the idle Connection and the other Vector (considering synchronization) stores the Connection in use.
When the developer uses a database connection, it is retrieved from the Queue, and if there is no, it returns empty; when the close connection is completed, it is put back to Vector.
ConnectionPool provides a simple dynamic expansion mechanism based on minIdle and maxConnection.
private static final int INITIAL_SIZE = 5; private static final String CLOSE_METHOD = "close"; private static Logger logger; private int size; private ConnectionParam connectionParam; private ArrayBlockingQueue<Connection> idleConnectionQueue; private Vector<Connection> busyConnectionVector;
.ConnectionPoolFactory - Connection Pool Management Class
ConnectionPoolFactory holds a static ConcurrentHashMap to store connection pool objects.
ConnectionPoolFactory allows creating multiple connection pools with different configurations of different databases.
The developer needs to register (bind) the connection pool with a specific name for the first time, and then obtain the Connection from the specified connection pool each time.
If the connection pool is no longer in use, the developer can log out (unbind) the connection pool.
private static Map<String, ConnectionPool> poolMap = new ConcurrentHashMap<>(); public static Connection getConnection(String poolName) throws SQLException { nameCheck(poolName); ConnectionPool connectionPool = poolMap.get(poolName); return connectionPool.getConnection(); } public static void registerConnectionPool(String name, ConnectionParam connectionParam) { registerCheck(name); poolMap.put(name, new ConnectionPool(connectionParam)); } // Let GC public static void unRegisterConnectionPool(String name) { nameCheck(name); final ConnectionPool connectionPool = poolMap.get(name); poolMap.remove(name); new Thread(new Runnable() { @Override public void run() { connectionPool.clear(); } }).start(); }Core code
The core code of the database connection pool is the getConnection() method. Usually, after the developer has processed the database operations, the close() method will be called. The Connection should be closed and resources should be released. In the database connection pool, when the user calls the close() method, the Connection should not be closed directly, but should be put back into the pool and reused. Here, the Java dynamic proxy mechanism is used. The getConnection returns not a "real" Connection, but a custom proxy class (anonymous class is used here). When the user calls the close() method, it intercepts and puts it back into the pool. For dynamic proxy, you can refer to another blog "Java Dynamic Proxy Simple Application"
@Override public Connection getConnection() throws SQLException { try { final Connection connection = idleConnectionQueue.poll(connectionParam.getMaxWait(), TimeUnit.MILLISECONDS); if (connection == null) { logger.info(emptyMsg()); ensureCapacity(); return null; } busyConnectionVector.add(connection); return (Connection) Proxy.newProxyInstance(this.getClass().getClassLoader(), new Class[]{Connection.class}, new InvocationHandler() { @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { if (!method.getName().equals(CLOSE_METHOD)) { return method.invoke(connection, args); } else { idleConnectionQueue.offer(connection); busyConnectionVector.remove(connection); return null; } } }); } catch (InterruptedException e) { e.printStackTrace(); } return null; }2. Use
First, the user builds the database connection pool parameters (ConnectionParam), including driver, url, user, password required items. You can customize options such as minConnection, maxConnection, etc. If not set, use the system default value. This is the benefit of using Builder to build a large number of attributes, including required attributes and optional attributes. Then register the connection pool with the ConnectionPoolFactory with a specific name, and finally get the Connection by calling the ConnectionPoolFactory static factory method.
String driver = "com.mysql.jdbc.Driver"; String url = "jdbc:mysql://localhost:3306/test"; String user = "root"; String password = "root"; ConnectionParam connectionParam = new ConnectionParam.ConnectionParamBuilder(driver, url, user, password).build(); ConnectionPoolFactory.registerConnectionPool("test", connectionParam); Connection connection = ConnectionPoolFactory.getConnection("test");3. Code
.ParamConfiguration
package database.config;import java.io.Serializable;/** * DataBase Connection Parameters * Created by Michael Wong on 2016/1/18. */public class ParamConfiguration implements Serializable { public static final int MIN_CONNECTION = 5; public static final int MAX_CONNECTION = 50; public static final int MIN_IDLE = 5; public static final long MAX_WAIT = 30000; private ParamConfiguration() {}}.Builder
package database;/** * Builder * Created by Michael Wong on 2016/1/18. */public interface Builder<T> { T build();}.ConnectionParam
package database;import database.config.ParamConfiguration;/** * DataBase Connection Parameters * Created by Michael Wong on 2016/1/18. */public class ConnectionParam { private final String driver; private final String url; private final String user; private final String password; private final int minConnection; private final int maxConnection; private final int minIdle; private final long maxWait; private ConnectionParam(ConnectionParamBuilder builder) { this.driver = builder.driver; this.url = builder.url; this.user = builder.user; this.password = builder.password; this.minConnection = builder.minConnection; this.maxConnection = builder.maxConnection; this.minIdle = builder.minIdle; this.maxWait = builder.maxWait; } public String getDriver() { return this.driver; } public String getUrl() { return this.url; } public String getUser() { return this.user; } public String getPassword() { return this.password; } public int getMinConnection() { return this.minConnection; } public int getMaxConnection() { return this.maxConnection; } public int getMinIdle() { return this.minIdle; } public long getMaxWait() { return this.maxWait; } public static class ConnectionParamBuilder implements Builder<ConnectionParam> { // Required parameters private final String driver; private final String url; private final String user; private final String password; // Optional parameters - initialized to default value private int minConnection = ParamConfiguration.MIN_CONNECTION; private int maxConnection = ParamConfiguration.MAX_CONNECTION; private int minIdle = ParamConfiguration.MIN_IDLE; // Getting Connection wait time private long maxWait = ParamConfiguration.MAX_WAIT; public ConnectionParamBuilder(String driver, String url, String user, String password) { this.driver = driver; this.url = url; this.user = user; this.password = password; } public ConnectionParamBuilder minConnection(int minConnection) { this.minConnection = minConnection; return this; } public ConnectionParamBuilder maxConnection(int maxConnection) { this.maxConnection = maxConnection; return this; } public ConnectionParamBuilder minIdle(int minIdle) { this.minIdle = minIdle; return this; } public ConnectionParamBuilder maxWait(int maxWait) { this.maxWait = maxWait; return this; } @Override public ConnectionParam build() { return new ConnectionParam(this); } }}.ConnectionPool
package database.factory;import database.ConnectionParam;import javax.sql.DataSource;import java.io.PrintWriter;import java.lang.reflect.InvocationHandler;import java.lang.reflect.Method;import java.lang.reflect.Proxy;import java.sql.Connection;import java.sql.DriverManager;import java.sql.SQLException;import java.sql.SQLFeatureNotSupportedException;import java.util.Vector;import java.util.concurrent.ArrayBlockingQueue;import java.util.concurrent.TimeUnit;import java.util.logging.Logger;/** * Connection Pool * Created by Michael Wong on 2016/1/18. */public class ConnectionPool implements DataSource { private static final int INITIAL_SIZE = 5; private static final String CLOSE_METHOD = "close"; private static Logger logger; private int size; private ConnectionParam connectionParam; private ArrayBlockingQueue<Connection> idleConnectionQueue; private Vector<Connection> busyConnectionVector; protected ConnectionPool(ConnectionParam connectionParam) { this.connectionParam = connectionParam; int maxConnection = connectionParam.getMaxConnection(); idleConnectionQueue = new ArrayBlockingQueue<>(maxConnection); busyConnectionVector = new Vector<>(); logger = Logger.getLogger(this.getClass().getName()); initConnection(); } private void initConnection() { int minConnection = connectionParam.getMinConnection(); int initialSize = INITIAL_SIZE < minConnection ? minConnection : INITIAL_SIZE; try { Class.forName(connectionParam.getDriver()); for (int i = 0; i < initialSize + connectionParam.getMinConnection(); i++) { idleConnectionQueue.put(newConnection()); size++; } } catch (Exception e) { throw new ExceptionInInitializerError(e); } } @Override public Connection getConnection() throws SQLException { try { final Connection connection = idleConnectionQueue.poll(connectionParam.getMaxWait(), TimeUnit.MILLISECONDS); if (connection == null) { logger.info(emptyMsg()); ensureCapacity(); return null; } busyConnectionVector.add(connection); return (Connection) Proxy.newProxyInstance(this.getClass().getClassLoader(), new Class[]{Connection.class}, new InvocationHandler() { @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { if (!method.getName().equals(CLOSE_METHOD)) { return method.invoke(connection, args); } else { idleConnectionQueue.offer(connection); busyConnectionVector.remove(connection); return null; } } }); } catch (InterruptedException e) { e.printStackTrace(); } return null; } private Connection newConnection() throws SQLException { String url = connectionParam.getUrl(); String user = connectionParam.getUser(); String password = connectionParam.getPassword(); return DriverManager.getConnection(url, user, password); } protected int size() { return size; } protected int idleConnectionQuantity() { return idleConnectionQueue.size(); } protected int busyConnectionQuantity() { return busyConnectionVector.size(); } private void ensureCapacity() throws SQLException { int minIdle = connectionParam.getMinIdle(); int maxConnection = connectionParam.getMaxConnection(); int newCapacity = size + minIdle; newCapacity = newCapacity > maxConnection ? maxConnection : newCapacity; int growCount = 0; if (size < newCapacity) { try { for (int i = 0; i < newCapacity - size; i++) { idleConnectionQueue.put(newConnection()); growCount++; } } catch (InterruptedException e) { e.printStackTrace(); } } size = size + growCount; } protected void clear() { try { while (size-- > 0) { Connection connection = idleConnectionQueue.take(); connection.close(); } } catch (InterruptedException | SQLException e) { e.printStackTrace(); } } private String emptyMsg() { return "Database is busy, please wait..."; } @Override public Connection getConnection(String username, String password) throws SQLException { return null; } @Override public PrintWriter getLogWriter() throws SQLException { return null; } @Override public void setLogWriter(PrintWriter out) throws SQLException { } @Override public void setLoginTimeout(int seconds) throws SQLException { } @Override public int getLoginTimeout() throws SQLException { return 0; } @Override public Logger getParentLogger() throws SQLFeatureNotSupportedException { return null; } @Override public <T> T unwrap(Class<T> iface) throws SQLException { return null; } @Override public boolean isWrapperFor(Class<?> iface) throws SQLException { return false; }}.ConnectionPoolFactory
package database.factory;import database.ConnectionParam;import java.sql.Connection;import java.sql.SQLException;import java.util.Map;import java.util.concurrent.ConcurrentHashMap;/** * Connection Pool Factory * Created by Michael Wong on 2016/1/18. */public class ConnectionPoolFactory { private ConnectionPoolFactory() {} private static Map<String, ConnectionPool> poolMap = new ConcurrentHashMap<>(); public static Connection getConnection(String poolName) throws SQLException { nameCheck(poolName); ConnectionPool connectionPool = poolMap.get(poolName); return connectionPool.getConnection(); } public static void registerConnectionPool(String name, ConnectionParam connectionParam) { registerCheck(name); poolMap.put(name, new ConnectionPool(connectionParam)); } // Let GC public static void unRegisterConnectionPool(String name) { nameCheck(name); final ConnectionPool connectionPool = poolMap.get(name); poolMap.remove(name); new Thread(new Runnable() { @Override public void run() { connectionPool.clear(); } }).start(); } public static int size(String poolName) { nameCheck(poolName); return poolMap.get(poolName).size(); } public static int getIdleConnectionQuantity(String poolName) { nameCheck(poolName); return poolMap.get(poolName).idleConnectionQuantity(); } public static int getBusyConnectionQuantity(String poolName) { nameCheck(poolName); return poolMap.get(poolName).busyConnectionQuantity(); } private static void registerCheck(String name) { if (name == null) { throw new IllegalArgumentException(nullName()); } } private static void nameCheck(String name) { if (name == null) { throw new IllegalArgumentException(nullName()); } if (!poolMap.containsKey(name)) { throw new IllegalArgumentException(notExists(name)); } } private static String nullName() { return "Pool name must not be null"; } private static String notExists(String name) { return "Connection pool named " + name + " does not exists"; }} 4. Test
JUnit Unit Testing
package database.factory;import database.ConnectionParam;import org.junit.Test;import java.sql.Connection;import java.sql.SQLException;import java.util.ArrayList;import java.util.List;import static org.junit.Assert.*;/** * ConnectionPoolFactory Test * Created by Michael Wong on 2016/1/20. */public class ConnectionPoolFactoryTest { @Test public void testGetConnection() throws SQLException { String driver = "com.mysql.jdbc.Driver"; String url = "jdbc:mysql://localhost:3306/test"; String user = "root"; String password = "root"; ConnectionParam connectionParam = new ConnectionParam.ConnectionParamBuilder(driver, url, user, password).build(); ConnectionPoolFactory.registerConnectionPool("test", connectionParam); List<Connection> connectionList = new ArrayList<>(); for(int i = 0; i < 12; i++) { connectionList.add(ConnectionPoolFactory.getConnection("test")); } print(); close(connectionList); print(); connectionList.clear(); for(int i = 0; i < 12; i++) { connectionList.add(ConnectionPoolFactory.getConnection("test")); } print(); close(connectionList); ConnectionPoolFactory.unRegisterConnectionPool("test"); } @Test(expected = IllegalArgumentException.class) public void testException() { try { ConnectionPoolFactory.getConnection("test"); } catch (SQLException e) { e.printStackTrace(); } } private void close(List<Connection> connectionList) throws SQLException { for(Connection conn : connectionList) { if (conn != null) { conn.close(); } } } private void print() { System.out.println("idle: " + ConnectionPoolFactory.getIdleConnectionQuantity("test")); System.out.println("busy: " + ConnectionPoolFactory.getBusyConnectionQuantity("test")); System.out.println("size: " + ConnectionPoolFactory.size("test")); }}The above is all about this article, I hope it will be helpful to everyone's learning.