Detailed explanation of the method of integrating ElasticSearch into multi-version compatibility in Spring Boot

Preface

In the previous article, SpringBoot, Mybatis, Druid and PageHelper were integrated and operations of multiple data sources were implemented. This article mainly introduces and uses ElastiSearch, the most popular search engine, and uses it in combination with SpringBoot.

Introduction to ElasticSearch

ElasticSearch is a search server based on Lucene. It actually encapsulates Lucene and provides the operation interface of REST API. ElasticSearch is a highly scalable open source full-text search and analysis engine that can be used to quickly store, search and analyze big data.

The main features of ElasticSearch: distributed, high availability, asynchronous writing, multi-API, document-oriented.

ElasticSearch core concepts: near real-time, cluster, node (save data), index, shard (slicing index shards), replicas (slicing can set multiple replicas). It can quickly store, search and analyze massive amounts of data.

ElasticSearch use cases: Wikipedia, Stack Overflow, Github, etc.

SpringBoot integrates Elasticsearch

Before using SpringBoot to integrate Elasticsearch, we should understand the relationship between them.

The version of SpringBoot we are using here is 1.5.9, and the version of Elasticsearch is 2.3.5.

Using SpringBoot to integrate Elasticsearch, it is generally encapsulated using SpringData, and then the dao layer interface inherits the ElasticsearchRepository class. This class implements many methods, such as the commonly used CRUD method.

Use of SpringData

First, make relevant preparations before use.

Maven's configuration is as follows:

 <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <version>1.5.9.RELEASE</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> <version>1.5.9.RELEASE</version> </dependency>

Configuration of application.properties

 spring.data.elasticsearch.repositories.enabled = truespring.data.elasticsearch.cluster-nodes =127.0.0.1/:9300

Note: 9300 is the port of the Java client. 9200 is an interface that supports Restful HTTP.

More configurations:

spring.data.elasticsearch.cluster-name Elasticsearch cluster name. (Default: elasticsearch)
spring.data.elasticsearch.cluster-nodes List of cluster node addresses, separated by commas. If not specified, start a client node.
spring.data.elasticsearch.propertie is used to configure additional properties of the client.
spring.data.elasticsearch.repositories.enabled Enable the Elasticsearch repository. (Default: true.)

Code writing

Entity Class

 @Document(indexName = "userindex", type = "user")public class User implements Serializable{ /** * */ private static final long serialVersionUID = 1L; /** Number*/ private Long id; /** Name*/ private String name; /** Age*/ private Integer age; /** Description*/ private String description; /** Creation time*/ private String createtm; // getter and setter omitted }

When using SpringData, it needs to set indexName and type in the entity class. If compared with traditional databases, it is equivalent to libraries and tables.

It should be noted that both indexName and type must be in lowercase!!!

Dao Layer

 public interface UserDao extends ElasticsearchRepository<User, Long>{}

The dao layer is relatively simple here, just inherit the ElasticsearchRepository class. The main methods are save, delete and search. The save method is quite like insert and update. If there is no, it will be added and if there is one, it will be covered. The delete method is mainly to delete data and index libraries. As for search, it is a query, including some commonly used queries, such as pagination, weights, etc.

Service layer

 @Servicepublic class UserServiceImpl implements UserService { @Autowired private UserDao userDao; @Override public boolean insert(User user) { boolean falg=false; try{ userDao.save(user); falg=true; }catch(Exception e){ e.printStackTrace(); } return falg; } @Override public List<User> search(String searchContent) { QueryStringQueryBuilder builder = new QueryStringQueryBuilder(searchContent); System.out.println("Query statement:"+builder); Iterable<User> searchResult = userDao.search(builder); Iterator<User> iterator = searchResult.iterator(); List<User> list=new ArrayList<User>(); while (iterator.hasNext()) { list.add(iterator.next()); } return list; } @Override public List<User> searchUser(Integer pageNumber, Integer pageSize,String searchContent) { // Pagination parameters Pageable pageable = new PageRequest(pageNumber, pageSize); QueryStringQueryBuilder builder = new QueryStringQueryBuilder(searchContent); SearchQuery searchQuery = new NativeSearchQueryBuilder().withPageable(pageable).withQuery(builder).build(); System.out.println("Query statement:" + searchQuery.getQuery().toString()); Page<User> searchPageResults = userDao.search(searchQuery); return searchPageResults.getContent(); } @Override public List<User> searchUserByWeight(String searchContent) { // Query according to weights FunctionScoreQuiller functionScoreQueryBuilder = QueryBuilders.functionScoreQuery() .add(QueryBuilders.boolQuery(). should(QueryBuilders.matchQuery("name", searchContent)), ScoreFunctionBuilders.weightFactorFunction(10)) .add(QueryBuilders.boolQuery(). should(QueryBuilders.matchQuery("description", searchContent)), ScoreFunctionBuilders.weightFactorFunction(100)).setMinScore(2); System.out.println("Query statement:" + functionScoreQueryBuilder.toString()); Iterable<User> searchResult = userDao.search(functionScoreQueryBuilder); Iterator<User> iterator = searchResult.iterator(); List<User> list=new ArrayList<User>(); while (iterator.hasNext()) { list.add(iterator.next()); } return list; }}

Here I have simply written several methods, the main method is query. Queries include full-text search, pagination query, and weight query. What needs to be explained is the weight query. The higher the weight score, the higher the query result. If no score is set for other data, their default score is 1. If you don’t want to query these statements, just use setMinScore to set it to greater than 1.

Code Testing

Call the interface to add data

New data:

 POST http://localhost:8086/api/user{"id":1,"name":"Zhang San","age":20,"description":"Zhang San is a Java development engineer","createtm":"2018-4-25 11:07:42"}{"id":2,"name":"Li Si","age":24,"description":"Li Si is a test engineer","createtm":"1980-2-15 19:01:32"}{"id":3,"name":"Wang Wu","age":25,"description":"Wang Wu is an operation and maintenance engineer","createtm":"2016-8-21 06:11:32"}

Conduct full text query

ask

 http://localhost:8086/api/user?searchContent=Engineer

return

 [{"id":2,"name":"Li Si","age":14,"description":"Li Si is a test engineer","createtm": "1980-2-15 19:01:32"},{"id":1,"name":"Zhang San","age":20,"description":"Zhang San is a Java development engineer", "createtm": "2018-4-25 11:07:42"},{"id":3,"name":"Wang Wu","age":25,"description":"Wang Wu is an operation and maintenance engineer","createtm": "2016-8-21 06:11:32"}]

Perform a pagination query

ask

 http://localhost:8086/api/user?pageNumber=0&pageSize=2&searchContent=Engineer

return

 [{"id":2,"name":"Li Si","age":14,"description":"Li Si is a test engineer"},{"id":1,"name":"Zhang San","age":20,"description":"Zhang San is a Java development engineer"}]

Perform weight query

ask

 http://localhost:8086/api/user2?searchContent=Li Si

return

 [{"id":2,"name":"Li Si","age":24,"description":"Li Si is a test engineer","createtm":"1980-2-15 19:01:32"}]

The weight query print statement:

 Query statement:{{ "function_score" : { "functions" : [ { "filter" : { "bool" : { "bool" : { "should" : { "match" : { "name" : { "query" : "Li Si", "type" : "boolean" } } } } } } } }, "weight" : 10.0 }, { "filter" : { "bool" : { "should" : { "match" : { "description" : { "query" : "Li Si", "type" : "boolean" } } } } } }, "weight" : 100.0 } ], "min_score" : 2.0 }}

Note: In the test, since the minimum weight of setMinScore is set to be divided into 2, irrelevant data will not be displayed. If you want to display it, just remove it in the code.

After adding new data, you can enter: http://localhost:9200/_plugin/head/ in the browser

Then click Basic Query to view the added data. If you want to use statement query, you can paste the query statement printed by the console in the program to the query interface for query!

Note: I installed ElasticSearch here on Windows and installed the ES plugin head. The specific installation steps are at the end of the article.

In addition to SpringData, there are actually other methods to operate ElasticSearch.

For example, use native ElasticSearch API and use the TransportClient class to implement it.

Or use it to encapsulate by Spring, just inject the bean in the Service layer.

Example:

 @Autowired ElasticsearchTemplate elasticsearchTemplate;

However, the above methods have their limitations, that is, as the version of ElasticSearch is changed, the relevant Java API is also constantly adjusting, that is, after the server version of ElasticSearch is changed, the client's code may need to be re-written.

Therefore, it introduces a very useful third-party tool JestClient. It encapsulates ElasticSearch and fills the gap in the ElasticSearch HttpRest interface client. It is suitable for ElasticSearch 2.x or above versions, and there is no need to change the code due to the ElasticSearch server version change!

JestClient

First add the following dependencies in Maven:

 <dependency> <groupId>io.searchbox</groupId> <artifactId>jest</artifactId> <version>5.3.3</version> </dependency>

Then write relevant test code.

The comments in the code should be very complete, so I will not talk too much about the code here.

 import java.util.ArrayList;import java.util.List;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.search.builder.SearchSourceBuilder;import com.pancm.pojo.User;import io.searchbox.client.JestClient;import io.searchbox.client.JestClientFactory;import io.searchbox.client.JestResult;import io.searchbox.client.config.HttpClientConfig;import io.searchbox.core.Bulk;import io.searchbox.core.BulkResult;import io.searchbox.core.Delete;import io.searchbox.core.DocumentResult;import io.searchbox.core.Index;import io.searchbox.core.Search;import io.searchbox.indices.CreateIndex;import io.searchbox.indices.DeleteIndex;import io.searchbox.indices.mapping.GetMapping;import io.searchbox.indices.mapping.PutMapping;public class JestTest { private static JestClient jestClient; private static String indexName = "userindex"; // private static String indexName = "userindex2"; private static String typeName = "user"; private static String elasticIps="http://192.169.2.98:9200";// private static String elasticIps="http://127.0.0.1:9200"; public static void main(String[] args) throws Exception { jestClient = getJestClient(); insertBatch(); serach1(); serach2(); serach3(); jestClient.close(); } private static JestClient getJestClient() { JestClientFactory factory = new JestClientFactory(); factory.setHttpClientConfig(new HttpClientConfig.Builder(elasticIps).connTimeout(60000).readTimeout(60000).multiThreaded(true).build()); return factory.getObject(); } public static void insertBatch() { List<Object> objs = new ArrayList<Object>(); objs.add(new User(1L, "Zhang San", 20, "Zhang San is a Java development engineer","2018-4-25 11:07:42")); objs.add(new User(2L, "Li Si", 24, "Li Si is a test engineer","1980-2-15 19:01:32")); objs.add(new User(3L, "Wang Wu", 25, "Wang Wu is an operation and maintenance engineer","2016-8-21 06:11:32")); boolean result = false; try { result = insertBatch(jestClient,indexName, typeName,objs); } catch (Exception e) { e.printStackTrace(); } System.out.println("Batch new:"+result); } /** * Full text search*/ public static void serach1() { String query ="Engineer"; try { SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.queryStringQuery(query)); //Page setting searchSourceBuilder.from(0).size(2); System.out.println("Full-text search query statement:"+searchSourceBuilder.toString()); System.out.println("Full-text search returns result:"+search(jestClient,indexName, typeName, searchSourceBuilder.toString())); } catch (Exception e) { e.printStackTrace(); } } /** * Exact search*/ public static void serach2() { try { SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.termQuery("age", 24)); System.out.println("Exact search query statement:"+searchSourceBuilder.toString()); System.out.println("Exact search returns result:"+search(jestClient,indexName, typeName, searchSourceBuilder.toString())); } catch (Exception e) { e.printStackTrace(); } } /** * Interval search*/ public static void serach3() { String createtm="createtm"; String from="2016-8-21 06:11:32"; String to="2018-8-21 06:11:32"; try { SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.query(QueryBuilders.rangeQuery(createtm).gte(from).lte(to)); System.out.println("Interval search statement:"+searchSourceBuilder.toString()); System.out.println("Interval search returns result:"+search(jestClient,indexName, typeName, searchSourceBuilder.toString())); } catch (Exception e) { e.printStackTrace(); } } /** * Create index* @param indexName * @return * @throws Exception */ public boolean createIndex(JestClient jestClient,String indexName) throws Exception { JestResult jr = jestClient.execute(new CreateIndex.Builder(indexName).build()); return jr.isSucceeded(); } /** * New data* @param indexName * @param typeName * @param source * @return * @throws Exception */ public boolean insert(JestClient jestClient,String indexName, String typeName, String source) throws Exception { PutMapping putMapping = new PutMapping.Builder(indexName, typeName, source).build(); JestResult jr = jestClient.execute(putMapping); return jr.isSucceeded(); } /** * Query data* @param indexName * @param typeName * @return * @throws Exception */ public static String getIndexMapping(JestClient jestClient,String indexName, String typeName) throws Exception { GetMapping getMapping = new GetMapping.Builder().addIndex(indexName).addType(typeName).build(); JestResult jr =jestClient.execute(getMapping); return jr.getJsonString(); } /** * Add data in batches* @param indexName * @param typeName * @param objs * @return * @throws Exception */ public static boolean insertBatch(JestClient jestClient,String indexName, String typeName, List<Object> objs) throws Exception { Bulk.Builder bulk = new Bulk.Builder().defaultIndex(indexName).defaultType(typeName); for (Object obj: objs) { Index index = new Index.Builder(obj).build(); bulk.addAction(index); } BulkResult br = jestClient.execute(bulk.build()); return br.isSucceeded(); } /** * Full text search* @param indexName * @param typeName * @param query * @return * @throws Exception */ public static String search(JestClient jestClient,String indexName, String typeName, String query) throws Exception { Search search = new Search.Builder(query) .addIndex(indexName) .addType(typeName) .build(); JestResult jr = jestClient.execute(search); // System.out.println("--"+jr.getJsonString());// System.out.println("--"+jr.getSourceAsObject(User.class)); return jr.getSourceAsString(); } /** * Delete index* @param indexName * @return * @throws Exception */ public boolean delete(JestClient jestClient,String indexName) throws Exception { JestResult jr = jestClient.execute(new DeleteIndex.Builder(indexName).build()); return jr.isSucceeded(); } /** * Delete data* @param indexName * @param typeName * @param id * @return * @throws Exception */ public boolean delete(JestClient jestClient,String indexName, String typeName, String id) throws Exception { DocumentResult dr = jestClient.execute(new Delete.Builder(id).index(indexName).type(typeName).build()); return dr.isSucceeded(); }

Note: Before testing, let’s first explain that the ElasticSearch version installed on the local Windows system is 2.3.5, and the ElasticSearch version installed on the Linux server is 6.2.

Test results

Full text search

 Full-text search query statement: { "from" : 0, "size" : 2, "query" : { "query_string" : { "query" : "engineer" } }} Full-text search returns result: {"id":1,"name":"Zhang San","age":20,"description":"Zhang San is a Java development engineer","createtm":"2018-4-25 11:07:42"},{"id":2,"name":"Li Si","age":24,"description":"Li Si is a test engineer","createtm":"1980-2-15 19:01:32"}

Match search

 Accurate search query statement: { "query" : { "term" : { "age" : 24 } }} Accurate search returns result: {"id":2,"name":"Li Si","age":24,"description":"Li Si is a test engineer","createtm":"1980-2-15 19:01:32"}

Time interval search

 Interval search statement:{ "query" : { "range" : { "createtm" : { "from" : "2016-8-21 06:11:32", "to" : "2018-8-21 06:11:32", "include_lower" : true, "include_upper" : true } } }} interval search returns result: {"id":1,"name":"Zhang San","age":20,"description":"Zhang San is a Java development engineer","createtm":"2018-4-25 11:07:42"}

After adding new data, we can go to linux Kibana to conduct related queries, and the query results are as follows:

Note: Kibana is an open source software in ELK. Kibana provides log analysis-friendly web interfaces for Logstash and ElasticSearch to help summarize, analyze and search important data logs.

The results returned by the test in the above code are in line with our expectations. Among them, only a small part of JestClient is used. For more use, you can check the official documentation of JestClient.

Install ElasticSearch on Windows

1. Document preparation

Download address: https://www.elastic.co/downloads

Select the ElasticSearch related version, then select the suffix name ZIP file to download, and then unzip it after downloading.

2. Start Elasticsearch

Go to the bin directory and run elasticsearch.bat

Then enter: localhost:9200 on the browse

Successfully displaying the interface means success!

3. Install ES plug-in

Web management interface head installation

Enter the bin directory, open cmd, and enter the dos interface

Enter: plugin install mobz/elasticsearch-head

Make a download

After successful download, enter: http://localhost:9200/_plugin/head/

If the interface is displayed, the installation will be successful!

4. Register service

Enter the bin directory, open cmd, and enter the dos interface

Enter:

 service.bat installservice.bat start

After success, enter

 services.msc

Jump to the Service service interface to view the running status of es directly!

other

ElasticSearch official website API address:

https://www.elastic.co/guide/en/elasticsearch/client/java-api/2.3/index.html

JestClientGithub address:

https://github.com/searchbox-io/Jest (local download)

I put the project on github.

https://github.com/xuwujing/springBoot (local download)

Summarize

The above is the entire content of this article. I hope that the content of this article has certain reference value for everyone's study or work. If you have any questions, you can leave a message to communicate. Thank you for your support to Wulin.com.