movie seeker

movie-seeker

Lucene search engine based on Douban movie data.

1. Demo address

https://search.andrewpqc.xyz/service/

2. Technical points summary

• Python web crawler
• mysql (data store)
• lucene (full text search engine)
• java WEB/JSP
• Tomcat server
• docker (application environment encapsulation)
• nginx (reverse proxy)
• let's encrypt(https certificate)

3. Source code introduction

The directory tree is as follows

1. The contents in the lib directory in the root directory are the jar packages that the code depends on

2. The content in the out directory in the root directory is the compiled and output content, which can be packaged and published.

3. The src directory is the source code directory, which is also the main directory for implementing functions. The following is an introduction to the packages in this directory:

• LuceneIndex.java under indexer package: Use Lucene to create index demo

• LuceneRetrieval.java under retrieval package: a demo that uses Lucene for searching

• main package is the core file used by the application. SearchUtil.java organizes Lucene index creation and various types of searches into class member functions for direct call in JSP files. It mainly includes index establishment, single-field query, multi-field joint query, fuzzy query, scope-based query, etc. SearchTest.java is a test file for the above functions.

• The contents under other tester and other packages are codes written during the development process. They have no direct relationship with the implementation of app functions and must be ignored.

4. The contents in the web directory under the root directory are JSP pages

• index.jsp: Home page, mainly some forms, collecting user's query keywords and query logic
• main_single.jsp: Processing page for single-field query request
• mian_mult.jsp: Processing page for multi-field query requests
• nobodyknows.jsp: Index creation page, accessed only once when the service is opened, not open to the outside world

4. About the data source

This search engine is based on the data of more than 8,000 Douban movies that were crawled before. The following is an introduction table of the database structure and a single data template:

Fields	describe	Sample data
name	Title	The Shawshank Redemption
Screenwriter	screenwriter	Frank Derabont, Stephen King
actor	Starring	Tim Robbins, Morgan Freeman, Bob Gunton, William Seidler, Clancy Brown, Jill Beros, Mark Roston, James Whitmore, Jeffrey Deman, Larry Brandonberg, Neil Genetoly, Brian Libby, David Proval, Joseph Laugnog, Zude Secklera
type	type	Plot, crime
country	nation	USA
displaytime	Release time	1994-09-10, Toronto Film Festival
score	score	9.6
othername	Alias	The moon is dark and flying high (Hong Kong), stimulating 1995 (Taiwan), hell promises, iron window years, shaking the redemption of Shank
shortcut	Introduction	In the late 1940s, Andy (Tim Robbins), a young banker with a small achievement, was sent to jail for allegedly killing his wife and her lover. In this prison called Shawshank, hope seems illusory, and the punishment of life imprisonment undoubtedly destined Andy's next dark and desperate life. Not long after, Andy tried to approach the prestigious prisoner Reed (Morgan Freeman), and asked the other party to help him get a little hammer. Taking this as an opportunity, the two gradually became familiar with each other, and Andy seemed to have found his own way of survival in a prison where there were mixed, sins and confusions of black and white. He used his professional knowledge to help prison management evade taxes and launder money, and at the same time, he was gradually treated with courtesy among prisoners through his interactions with Reed. On the surface, he had changed from hatred to calmness towards the high wall like Red, but his desire for freedom still prompted him to move towards his hope and purpose. And the truth about his crimes seems to push all this forward one step further... The film is based on the famous writer Stephen Edwin King...
url	Details page link	https://movie.douban.com/subject/1292052/

5. Mirror usage

Install docker and run the following two commands:

docker pull  pqcsdockerhub/searchengine-image
docker run -d -p 8081:8080 pqcsdockerhub/searchengine-image

Then open the browser and visit https://localhost:8081/service/ to access the encapsulated service.

Note: When the service is running for the first time, you need to visit http://localhost:8081/service/nobodyknows.jsp . Access this url, let the program connect to the database in the background and conduct index creation work, and then the search service can run normally.