Open source web knowledge graph project
- Crawl the Chinese page of Baidu Encyclopedia
- Parsing triples and web content
- Building a Chinese Knowledge Graph
- Building an encyclopedia bot (under construction)
update 20200720
Deployment reference on Windows How to deploy on Windows, thanks to LMY-nlp0701!
update 20191121
- Migrate code to crawler framework scrapy
- Optimized extraction of some codes
- Data persistence migration to mongodb
- Fix chatbot failure issue
- Open neo4j backend interface to view the knowledge graph forming effect
Tips
- If it is a project problem, please ask an issue.
- If it involves inconvenient disclosure, please send an email.
- ChatBot Please visit the link
- The molded encyclopedia knowledge graph access link, username: neo4j, password: 123. The effects are as follows:
environment
- Python 3.6
- re:url regular match
- scrapy: web crawler and web parsing
- neo4j: Knowledge graph database, please refer to the link for installation
- pip install neo4j-driver: neo4j python driver
- pip install pymongodb: python support for mongodb
- mongodb database: installation reference link
Code execution:
cd WEB_KG/baike
scrapy crawl baike
Execution interface (press ctrl+c to stop): 
Knowledge graph renderings
Web page content stored by mongodb
Triples stored by mongodb
neo4j background interface
