The previous article introduced that ElasticSearch uses Repository and ElasticSearchTemplate to build complex query conditions, and briefly introduces the function of ElasticSearch to use geographic location.
In this article, let’s take a look at the function of using ElasticSearch to complete the large data query nearby people and search for data within the N-meter range.
Prepare the environment
The native test uses the latest ElasticSearch version 5.5.1, SpringBoot1.5.4, and spring-data-ElasticSearch2.1.4.
Create a new Springboot project, check ElasticSearch and web.
The pom file is as follows
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.tianyalei</groupId> <artifactId>elasticsearch</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <name>elasticsearch</name> <description>Demo project for Spring Boot</description> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>1.5.4.RELEASE</version> <relativePath/> <!-- lookup parent from repository --> </parent> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>com.sun.jna</groupId> <artifactId>jna</artifactId> <version>3.0.9</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>
Create a new model class Person
package com.tianyalei.elasticsearch.model; import org.springframework.data.annotation.Id; import org.springframework.data.elasticsearch.annotations.Document; import org.springframework.data.elasticsearch.annotations.GeoPointField; import java.io.Serializable; /** * model class*/ @Document(indexName="elastic_search_project",type="person",indexStoreType="fs",shards=5,replicas=1,refreshInterval="-1") public class Person implements Serializable { @Id private int id; private String name; private String phone; /** * Geographic location latitude and longitude* latitude, lon longitude "40.715,-74.011" * If you use an array, the opposite is true [-73.983, 40.719] */ @GeoPointField private String address; public int getId() { return id; } public void setId(int id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public String getPhone() { return phone; } public void setPhone(String phone) { this.phone = phone; } public String getAddress() { return address; } public void setAddress(String address) { this.address = address; } } I use the address field to represent the latitude and longitude position. Note that using String[] and String are different when denoting latitude and longitude, see the comments.
import com.tianyalei.elasticsearch.model.Person; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; public interface PersonRepository extends ElasticsearchRepository<Person, Integer> { } Take a look at the Service class and complete the function of inserting test data. I put the query function in the Controller. For the sake of convenience, it should be placed in the Service normally.
package com.tianyalei.elasticsearch.service; import com.tianyalei.elasticsearch.model.Person; import com.tianyalei.elasticsearch.repository.PersonRepository; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.elasticsearch.core.ElasticsearchTemplate; import org.springframework.data.elasticsearch.core.query.IndexQuery; import org.springframework.stereotype.Service; import java.util.ArrayList; import java.util.List; @Service public class PersonService { @Autowired PersonRepository personRepository; @Autowired ElasticsearchTemplate elasticsearchTemplate; private static final String PERSON_INDEX_NAME = "elastic_search_project"; private static final String PERSON_INDEX_TYPE = "person"; public Person add(Person person) { return personRepository.save(person); } public void bulkIndex(List<Person> personList) { int counter = 0; try { if (!elasticsearchTemplate.indexExists(PERSON_INDEX_NAME)) { elasticsearchTemplate.createIndex(PERSON_INDEX_TYPE); } List<IndexQuery> queries = new ArrayList<>(); for (Person person : personList) { IndexQuery indexQuery = new IndexQuery(); indexQuery.setId(person.getId() + ""); indexQuery.setObject(person); indexQuery.setIndexName(PERSON_INDEX_NAME); indexQuery.setType(PERSON_INDEX_TYPE); //The above steps can also be used to build using IndexQueryBuilder//IndexQuery index = new IndexQueryBuilder().withId(person.getId() + "").withObject(person).build(); queries.add(indexQuery); if (counter % 500 == 0) { elasticsearchTemplate.bulkIndex(queries); queries.clear(); System.out.println("bulkIndex counter : " + counter); } counter++; } if (queries.size() > 0) { elasticsearchTemplate.bulkIndex(queries); } System.out.println("bulkIndex completed."); } catch (Exception e) { System.out.println("IndexerService.bulkIndex e;" + e.getMessage()); throw e; } } }Pay attention to the bulkIndex method, which is used to batch insert data. bulk is also the method of batch insert data officially recommended by ES. Here, the bulk is inserted every 500 integer multiples.
package com.tianyalei.elasticsearch.controller; import com.tianyalei.elasticsearch.model.Person; import com.tianyalei.elasticsearch.service.PersonService; import org.elasticsearch.common.unit.DistanceUnit; import org.elasticsearch.index.query.GeoDistanceQueryBuilder; import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.search.sort.GeoDistanceSortBuilder; import org.elasticsearch.search.sort.SortBuilders; import org.elasticsearch.search.sort.SortOrder; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Pageable; import org.springframework.data.elasticsearch.core.ElasticsearchTemplate; import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder; import org.springframework.data.elasticsearch.core.query.SearchQuery; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; import java.text.DecimalFormat; import java.util.ArrayList; import java.util.List; import java.util.Random; @RestController public class PersonController { @Autowired PersonService personService; @Autowired ElasticsearchTemplate elasticsearchTemplate; @GetMapping("/add") public Object add() { double lat = 39.929986; double lon = 116.395645; List<Person> personList = new ArrayList<>(900000); for (int i = 100000; i < 1000000; i++) { double max = 0.00001; double min = 0.000001; Random random = new Random(); double s = random.nextDouble() % (max - min + 1) + max; DecimalFormat df = new DecimalFormat("######0.000000"); // System.out.println(s); String lons = df.format(s + lon); String lats = df.format(s + lat); Double dlon = Double.valueOf(lons); Double dlat = Double.valueOf(lats); Person person = new Person(); person.setId(i); person.setName("Name" + i); person.setPhone("Tel" + i); person.setAddress(dlat + "," + dlon); personList.add(person); } personService.bulkIndex(personList); // SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(QueryBuilders.queryStringQuery("spring boot OR book")).build(); // List<Article> articles = elas, ticsearchTemplate.queryForList(se, archQuery, Article.class); // for (Article article : articles) { // System.out.println(article.toString()); // } return "Add data"; } /** * geo_distance: Find the location within a certain range of a certain center point geo_bounding_box: Find the location within a rectangle area geo_distance_range: Find the location between min and max geo_polygon: Find the location within a polygon. sort can be used to sort */ @GetMapping("/query") public Object query() { double lat = 39.929986; double lon = 116.395645; Long nowTime = System.currentTimeMillis(); //Query within a certain latitude and longitude within 100 meters. GeoDistanceQueryBuilder builder = QueryBuilders.geoDistanceQuery("address").point(lat, lon) .distance(100, DistanceUnit.METERS); GeoDistanceSortBuilder sortBuilder = SortBuilders.geoDistanceSort("address") .point(lat, lon) .unit(DistanceUnit.METERS) .order(SortOrder.ASC); Pageable pageable = new PageRequest(0, 50); NativeSearchQueryBuilder builder1 = new NativeSearchQueryBuilder().withFilter(builder).withSort(sortBuilder).withPageable(pageable); SearchQuery searchQuery = builder1.build(); //QueryForList is paginated by default, and the queryForPage is used. The default is 10 List<Person> personList = elasticsearchTemplate.queryForList(searchQuery, Person.class); System.out.println("Timeconsuming:" + (System.currentTimeMillis() - nowTime)); return personList; } } Looking at the Controller class, in the add method, we insert 900,000 test data and randomly generate different latitude and longitude addresses.
In the query method, we construct a query condition that is within 100 meters of the query, sorted according to distance, and paging 50 items per page. If Pageable is not specified, the queryForList of ESTemplate is 10 by default, which can be seen through the source code.
Start the project, execute add first, and wait for millions of data to be inserted, for about a few dozen seconds.
Then execute the query and see the results.
The first query took more than 300ms, and the time dropped significantly after the query again, to about 30ms, because ES has been automatically cached to memory.
It can be seen that ES completes the query of geographical location very quickly. Suitable for queries such as nearby people and scope query.
Postscript: In later use, in the Elasticsearch version 2.3, the geotype cannot be indexed according to the above writing method. The es entry is String instead of the marked geofiled. Here is a record of the solution, modify the String type to GeoPoint, and it is packaged under org.springframework.data.elasticsearch.core.geo.GeoPoint. Then you need to explicitly call the mapping method when creating the index to correctly map to geofield.
as follows
if (!elasticsearchTemplate.indexExists("abc")) { elasticsearchTemplate.createIndex("abc"); elasticsearchTemplate.putMapping(Person.class); }The above is all the content of this article. I hope it will be helpful to everyone's learning and I hope everyone will support Wulin.com more.