The same project sometimes involves multiple databases, that is, multiple data sources. Multiple data sources can be divided into two situations:
1) Two or more databases have no correlation and are independent of each other. In fact, this can be developed as two projects. For example, in game development, one database is a platform database, and the other databases corresponding to the games under the platform are also available;
2) Two or more databases are master-slave relationships, such as mysql builds a master-master, followed by multiple slaves; or master-slave copy built with MHA;
Currently, there are roughly two ways to build Spring multi-data sources, and you can choose according to the situation of multi-data sources.
1. Use spring configuration files to directly configure multiple data sources
For example, in case of no correlation between two databases, you can directly configure multiple data sources in the spring configuration file and then perform transaction configurations, as shown below:
<context:component-scan base-package="net.aazj.service,net.aazj.aop" /><context:component-scan base-package="net.aazj.aop" /><!-- Introduce property files-><context:property-placeholder location="classpath:config/db.properties" /> <!-- Configure data source-><bean name="dataSource" init-method="init" destroy-method="close"> <property name="url" value="${jdbc_url}" /> <property name="username" value="${jdbc_username}" /> <property name="password" value="${jdbc_password}" /> <!-- Initialize the connection size --> <property name="initialSize" value="0" /> <!-- Maximum number of connections used by the connection pool--> <property name="maxActive" value="20" /> <!-- Maximum number of connections available--> <property name="maxIdle" value="20" /> <!-- Minimum number of connections available--> <property name="minIdle" value="0" /> <!-- Get the maximum waiting time for connections--> <property name="maxWait" value="60000" /></bean> <bean id="sqlSessionFactory"> <property name="dataSource" ref="dataSource" /> <property name="configLocation" value="classpath:config/mybatis-config.xml" /> <property name="mapperLocations" value="classpath*:config/mappers/**/*.xml" /></bean> <!-- Transaction manager for a single JDBC DataSource --><bean id="transactionManager"> <property name="dataSource" ref="dataSource" /></bean> <!-- Define transactions using annotation--><tx:annotation-driven transaction-manager="transactionManager" /> <bean> <property name="basePackage" value="net.aazj.mapper" /> <property name="sqlSessionFactoryBeanName" value="sqlSessionFactory"/></bean> <!-- Enables the use of the @AspectJ style of Spring AOP --><aop:aspectj-autoproxy/>Configuration of the second data source
<bean name="dataSource_2" init-method="init" destroy-method="close"> <property name="url" value="${jdbc_url_2}" /> <property name="username" value="${jdbc_username_2}" /> <property name="password" value="${jdbc_password_2}" /> <!-- Initialize the connection size--> <property name="initialSize" value="0" /> <!-- Maximum number of connections used in the connection pool--> <property name="maxActive" value="20" /> <!-- Maximum idle connection pool--> <property name="maxIdle" value="20" /> <!-- Minimum idle connection pool--> <property name="minIdle" value="0" /> <!-- Get maximum connection waiting time--> <property name="maxWait" value="60000" /></bean> <bean id="sqlSessionFactory_slave"> <property name="dataSource" ref="dataSource_2" /> <property name="configLocation" value="classpath:config/mybatis-config-2.xml" /> <property name="mapperLocations" value="classpath*:config/mappers2/**/*.xml" /></bean> <!-- Transaction manager for a single JDBC DataSource --><bean id="transactionManager_2"> <property name="dataSource" ref="dataSource_2" /></bean> <!-- Definition of transactions using annotation--><tx:annotation-driven transaction-manager="transactionManager_2" /> <bean> <property name="basePackage" value="net.aazj.mapper2" /> <property name="sqlSessionFactoryBeanName" value="sqlSessionFactory_2"/></bean> As shown above, we configure two dataSources, two sqlSessionFactory, two transactionManagers, and the key is the configuration of MapperScannerConfigurer - using the sqlSessionFactoryBeanName property to inject different sqlSessionFactory names. In this way, we inject the corresponding sqlSessionFactory into the mapper interfaces corresponding to different databases.
It should be noted that this configuration of multiple databases does not support distributed transactions, that is, multiple databases cannot be operated in the same transaction. The advantage of this configuration is that it is very simple, but it is not flexible. It is not very suitable for the configuration of multi-data source of master-slave type. The configuration of multi-data source of master-slave type needs to be particularly flexible and requires detailed configuration according to the type of business. For example, for some time-consuming select statements, we hope to put them on the slave, and for update, delete and other operations, we can only execute them on the master. In addition, for some select statements with high real-time requirements, we may also need to put them on the master - for example, in a scenario, I go to the mall to buy a weapon, and the purchase operation is definitely the master. After the purchase is completed, we need to re-query the weapons and gold coins I own. Then this query may also need to prevent them from being executed on the master, and cannot be executed on the slave, because there may be delays on the slave. We do not want players to find that after the purchase is successful, they cannot find weapons in the backpack.
Therefore, for the configuration of master-slave type multi-data source, flexible configuration is required according to the business, which selects can be placed on the slave, and which selects cannot be placed on the slave. Therefore, the above configuration of the data source is not very suitable.
2. Configuration of multi-data source based on AbstractRoutingDataSource and AOP
The basic principle is that we define a DataSource class ThreadLocalRountingDataSource ourselves to inherit AbstractRoutingDataSource, and then inject master and slave data sources into ThreadLocalRountingDataSource in the configuration file, and then flexibly configure through AOP, where to choose master data source and where to choose slave data source. Let's see the code implementation below:
1) First define an enum to represent different data sources:
package net.aazj.enums; /** * Category of data source: master/slave */public enum DataSources { MASTER, SLAVE} 2) Use the headLocal to save the key of which data source to choose for each thread:
package net.aazj.util; import net.aazj.enums.DataSources; public class DataSourceTypeManager { private static final ThreadLocal<DataSources> dataSourceTypes = new ThreadLocal<DataSources>(){ @Override protected DataSources initialValue(){ return DataSources.MASTER; } }; public static DataSources get(){ return dataSourceTypes.get(); } public static void set(DataSources dataSourceType){ dataSourceTypes.set(dataSourceType); } public static void reset(){ dataSourceTypes.set(DataSources.MASTER0); }} 3) Define ThreadLocalRountingDataSource and inherit AbstractRoutingDataSource:
package net.aazj.util; import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource; public class ThreadLocalRountingDataSource extends AbstractRoutingDataSource { @Override protected Object determineCurrentLookupKey() { return DataSourceTypeManager.get(); }}4) Inject master and slave data sources into ThreadLocalRountingDataSource in the configuration file:
<context:component-scan base-package="net.aazj.service,net.aazj.aop" /><context:component-scan base-package="net.aazj.aop" /><!-- Introduce property files-><context:property-placeholder location="classpath:config/db.properties" /> <!-- Configure data source Master --><bean name="dataSourceMaster" init-method="init" destroy-method="close"> <property name="url" value="${jdbc_url}" /> <property name="username" value="${jdbc_username}" /> <property name="password" value="${jdbc_password}" /> <!-- Initialize the connection size --> <property name="initialSize" value="0" /> <!-- Maximum number of connections used by the connection pool--> <property name="maxActive" value="20" /> <!-- Maximum number of connections available--> <property name="maxIdle" value="20" /> <!-- Minimum number of connections available--> <property name="minIdle" value="0" /> <!-- Get the maximum waiting time for connections--> <property name="maxWait" value="60000" /></bean> <!-- Configure data source Slave --><bean name="dataSourceSlave" init-method="init" destroy-method="close"> <property name="url" value="${jdbc_url_slave}" /> <property name="username" value="${jdbc_username_slave}" /> <property name="password" value="${jdbc_password_slave}" /> <!-- Initialize connection size --> <property name="initialSize" value="0" /> <!-- Maximum number of connections used by the connection pool--> <property name="maxActive" value="20" /> <!-- Maximum idle connection pool--> <property name="maxIdle" value="20" /> <!-- Minimum idle connection pool--> <property name="minIdle" value="0" /> <!-- Get maximum connection waiting time--> <property name="maxWait" value="60000" /></bean> <bean id="dataSource"> <property name="defaultTargetDataSource" ref="dataSourceMaster" /> <property name="targetDataSources"> <map key-type="net.aazj.enums.DataSources"> <entry key="MASTER" value-ref="dataSourceMaster"/> <entry key="SLAVE" value-ref="dataSourceSlave"/> <!-- You can add multiple dataSources here --> </map> </property></bean> <bean id="sqlSessionFactory"> <property name="dataSource" ref="dataSource" /> <property name="configLocation" value="classpath:config/mybatis-config.xml" /> <property name="mapperLocations" value="classpath*:config/mappers/**/*.xml" /></bean> <!-- Transaction manager for a single JDBC DataSource --><bean id="transactionManager"> <property name="dataSource" ref="dataSource" /></bean> <!-- Definition of transactions using annotation--><tx:annotation-driven transaction-manager="transactionManager" /> <bean> <property name="basePackage" value="net.aazj.mapper" /> <!-- <property name="sqlSessionFactoryBeanName" value="sqlSessionFactory"/> --></bean> In the spring configuration file above, we define dataSourceMaster and dataSourceSlave for the master database and slave database respectively, and then inject it into <bean id="dataSource"> so that our dataSource can select dataSourceMaster and dataSourceSlave according to the different keys.
5) Use Spring AOP to specify the key of dataSource, so dataSource will select dataSourceMaster and dataSourceSlave according to the key:
package net.aazj.aop; import net.aazj.enums.DataSources;import net.aazj.util.DataSourceTypeManager; import org.aspectj.lang.JoinPoint;import org.aspectj.lang.annotation.Aspect;import org.aspectj.lang.annotation.Before;import org.aspectj.lang.annotation.Pointcut;import org.springframework.stereotype.Component; @Aspect // for aop@Component // for auto scanpublic class DataSourceInterceptor { @Pointcut("execution(public * net.aazj.service..*.getUser(..))") public void dataSourceSlave(){}; @Before("dataSourceSlave()") public void before(JoinPoint jp) { DataSourceTypeManager.set(DataSources.SLAVE); } // ... } Here we define an Aspect class. We use @Before to call DataSourceTypeManager.set(DataSources.SLAVE) before the method in compliance with @Pointcut("execution(public * net.aazj.service..*.getUser(..))") is called, and the key type is set to DataSources.SLAVE, so dataSource will select dataSourceSlave according to key=DataSources.SLAVE. Therefore, the SQL statements for this method will be executed on the slave database.
We can continuously expand the DataSourceInterceptor Aspect, and make various definitions in it to specify the dataSource corresponding to the appropriate data source for a certain service method.
In this way, we can use the powerful functions of Spring AOP to configure it very flexibly.
6) Analysis of the principle of AbstractRoutingDataSource
ThreadLocalRountingDataSource inherits AbstractRoutingDataSource, implementing its abstract method protected abstract Object determineCurrentLookupKey(); thus implementing the routing function for different data sources. Let’s start with the source code to analyze the principles:
public abstract class AbstractRoutingDataSource extends AbstractDataSource implements InitializingBeanAbstractRoutingDataSource implements InitializingBean. Then when spring initializes the bean, it will call the interface of the InitializingBean void afterPropertiesSet() throws Exception; Let's see how AbstractRoutingDataSource implements this interface: @Override public void afterPropertiesSet() { if (this.targetDataSources == null) { throw new IllegalArgumentException("Property 'targetDataSources' is required"); } this.resolvedDataSources = new HashMap<Object, DataSource>(this.targetDataSources.size()); for (Map.Entry<Object, Object> entry : this.targetDataSources.entrySet()) { Object lookupKey = resolveSpecifiedLookupKey(entry.getKey()); DataSource dataSource = resolveSpecifiedDataSource(entry.getValue()); this.resolvedDataSources.put(lookupKey, dataSource); } if (this.defaultTargetDataSource != null) { this.resolvedDefaultDataSource = resolveSpecifiedDataSource(this.defaultTargetDataSource); } } targetDataSources is the dataSourceMaster and dataSourceSlave that we inject into the xml configuration file. The afterPropertiesSet method is injected.
dataSourceMaster and dataSourceSlave to construct a HashMap - resolvedDataSources. It is convenient to obtain the corresponding dataSource from the map according to the key later.
Let's take a look at how Connection getConnection() throws SQLException; in the AbstractDataSource interface is implemented:
@Override public Connection getConnection() throws SQLException { return determineTargetDataSource().getConnection(); }The key is to determineTargetDataSource(), which can be seen based on the method name, and we should decide which dataSource to use here:
protected DataSource determineTargetDataSource() { Assert.notNull(this.resolvedDataSources, "DataSource router not initialized"); Object lookupKey = determineCurrentLookupKey(); DataSource dataSource = this.resolvedDataSources.get(lookupKey); if (dataSource == null && (this.lenientFallback || lookupKey == null)) { dataSource = this.resolvedDefaultDataSource; } if (dataSource == null) { throw new IllegalStateException("Cannot determine target DataSource for lookup key [" + lookupKey + "]"); } return dataSource;} Object lookupKey = determineCurrentLookupKey(); This method is implemented by us, where we get the key value saved in ThreadLocal. After obtaining the key, obtain the dataSource corresponding to the key in the map initialized resolvedDataSources from afterPropertiesSet(). The key value saved in ThreadLocal is set before calling the relevant methods in the service through AOP. OK, here it is done!
3. Summary
From this article, we can experience the power and flexibility of AOP.
The above is the information sorting of mybatis multi-data source processing. I hope it can help friends in need