news search engine

News Search Engine

How to use

1. Install python 3.4+ environment (recommended Anaconda or Miniconda)
2. Install the lxml html parser, the command is pip install lxml
3. Install the jieba word participle component, the command is pip install jieba
4. Install Flask Web Framework, the command is pip install Flask
5. Enter the web folder and run the main.py file
6. Open the browser and visit http://127.0.0.1:5000/ Enter keywords to start testing

If you want to grab the latest news data and build an index, run ./code/setup.py with one click, and then test it according to the above method.

2020.4.5: Added a crawler program to capture China News Network. First run ./code/spider.chinanews.com.py to crawl the news in the last 5 days (about 2500); then comment ./code/setup.py line 38 and run it to automatically build the index.

Project Introduction

1. Building a search engine with me (I) Introduction
2. Build a search engine with me (II) Web crawler
3. Build a search engine with me (III) Build an index
4. Work with me to build a search engine (IV) search model
5. Build a search engine with me (Five) Recommended reading
6. Build a search engine with me (VI) system display
7. Building a search engine with me (7) Summary of the outlook
8. Build a search engine with me (eight) Update crawler & modify scoring & online deployment

grateful

• jieba
• scikit-learn
• flask