summary:
Architecture, distributed, log queue, and the title itself is a log collection function, but a Redis is added in the middle to make a message queue. Why do I need a message queue? When factors such as "production" and "consumption" appear in the system are inconsistent, a message queue is needed to act as an abstract layer to bridge the differences between the two parties.
Architecture, distributed, log queue, and the title itself is a log collection function, but a Redis is added in the middle to make a message queue.
Why do I need a message queue?
When factors such as "production" and "consumption" appear in the system are inconsistent, a message queue is needed to act as an abstract layer to bridge the differences between the two parties.
For example, the common email and SMS sending in our system writes these functions that do not require timely response to the queue, process requests asynchronously, and reduce response time.
How to achieve it?
There are many mature JMS message queue middleware products on the market, but based on the current project architecture and deployment situation, we use Redis as message queue.
Why use Redis?
The list data structure in Redis has the characteristics of "double-ended queues", and redis has the ability to persist data, so redis is very safe and reliable to implement distributed queues.
It is similar to the "Queue" in JMS, except that the functionality and reliability (transactionality) are not as strict as JMS. Redis itself has high performance and "convenient" distributed design (replicas, sharding), which can provide a good foundation for implementing "distributed queues".
Provider side
The project uses a third-party redis plugin spring-data-redis. If you are not sure how to use it, please Google or Baidu.
redis.properties:
#redis Configuration Center redis.host=192.168.1.180redis.port=6379redis.password=123456redis.maxIdle=100 redis.maxActive=300 redis.maxWait=1000 redis.testOnBorrow=true redis.timeout=100000
Redis configuration:
<!-- redis configuration --> <bean id="jedisPoolConfig" /> <bean id="jedisConnectionFactory" > <property name="hostName" value="${redis.host}" /> <property name="port" value="${redis.port}" /> <property name="password" value="${redis.password}" /> <property name="timeout" value="${redis.timeout}" /> <property name="poolConfig" ref="jedisPoolConfig" /> <property name="usePool" value="true" /> </bean> <bean id="redisTemplate"> <property name="connectionFactory" ref="jedisConnectionFactory" /> </bean> Sectional log configuration (pseudocode):
/** * System log, facet processing class* Creator Xiaoqi 2012 * Creation time January 15, 2018*/@Component@Scope@Aspectpublic class SysLogAspect { @Autowired private RedisTemplate<String, String> redisTemplate; //Annotations are based on swagger API, and you can also define @Pointcut("@annotation(io.swagger.annotations.ApiOperation)") public void logPointCut() { } @Around("logPointCut()") public Object around(ProceedingJoinPoint point) throws Throwable { Object result = point.proceed(); //Write log messages to itsstyle_log channel redisTemplate.convertAndSend("itstyle_log","log data, process it by yourself"); return result; }}Consumer side
Redis configuration:
<!-- redis configuration --> <bean id="jedisPoolConfig" /> <bean id="jedisConnectionFactory" > <property name="hostName" value="${redis.host}" /> <property name="port" value="${redis.port}" /> <property name="password" value="${redis.password}" /> <property name="timeout" value="${redis.timeout}" /> <property name="poolConfig" ref="jedisPoolConfig" /> <property name="usePool" value="true" /> </bean> <bean id="redisTemplate" p:connection-factory-ref="jedisConnectionFactory"> <property name="keySerializer"> <bean /> </property> <property name="hashKeySerializer"> <bean /> </property> </bean> <!-- Listening to implementation class--> <bean id="listener"/> <bean id="stringRedisSerializer" /> <redis:listener-container connection-factory="jedisConnectionFactory"> <!-- topic represents the channel to monitor, which is a regular match, which is actually the channel you want to subscribe to --> <redis:listener ref="listener" serializer="stringRedisSerializer" method="handleLog" topic="itstyle_log"/> </redis:listener-container>Listening interface:
public interface MessageDelegateListener { public void handleLog(Serializable message);}Listening implementation:
public class MessageDelegateListenerImpl implements MessageDelegateListener { @Override public void handleLog(Serializable message) { if(message == null){ System.out.println("null"); }else { //Process log data} }}Q&A
【Question 1】Why use Redis?
Actually, there are already explanations above, although there are many very stable products on the market, such as Kafka, RabbitMQ and RocketMQ that you may think of. However, since the project itself uses Redis as distributed cache, Redis is selected based on the principle of saving troubles and feasibility.
[Question 2] How to store log data?
In principle, it is not recommended to store it in relational databases, such as MySql. After all, the number of logs generated is huge, so it is recommended to store it in non-relational databases such as Elasticsearch.
[Question 3] How is the collection of sectional logs implemented?
The section log needs to introduce spring-aspects-related Jar packages, and the configuration makes Spring adopt a CGLIB proxy.
Open source project source code (reference): https://gitee.com/52itstyle/spring-boot-mail
Summarize
The above is the Redis distributed log queue for the JavaWeb project architecture introduced to you by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply to you in time. Thank you very much for your support to Wulin.com website!