AnimeBack

AnimeBack

Search the pictures. This is a search engine server for animation scenes. You can search in reverse through screenshots of a certain moment of the anime, and in reverse, which episode it came from and the exact time it appeared. Front-end of the website

Compared with trace.moe, this project may be able to provide more robust and more accurate search services because it uses deep learning models to extract image features. Therefore, the performance overhead is greater and the inclusion is slower. Still in the testing phase

Example

Test pictures

Search results

deploy

Environmental requirements

python3 needs to be installed; ffmpeg needs to be installed

Install dependencies:

pip install bilibili_api imagehash tensorflow keras flask pymilvus opencv-python sklearn bilili
# Linux
pip install plyvel
# Windows
pip install plyvel-win32

run

• Initialize the configuration file

python run.py

• Install and start milvus
• Configure config.json
• Run the download program (bilibili)

python run.py download-bilibili

• Run video processing/entry program

 # 将会处理已下载的视频
python run.py process

• Run the website backend

python app.py

This method is for testing only. Please refer to Flask deployment method for production environment

The following example uses gunicorn , 4 worker process ( -w 4 ) to run the Flask application and bind to the 4000 port of localhost ( -b 127.0.0.1:4000 ):

gunicorn -w 4 -b 127.0.0.1:4000 app:flask_app

Technology implementation

• Through the bilibili (or other website) API, automatically download the drama and initially save the drama information.

• Use ffmpeg to compress the video and convert it to mp4, and put it in the static directory of the website

• Use ffmpeg to convert the video into a picture at a certain sampling rate and put it in a temporary directory

• Read the picture frame by frame, filter out adjacent similar pictures through the phash algorithm, and extract the feature vectors using the model and insert them into milvus . The corresponding information of each frame added is stored in the leveldb database, such as id , time , and epid

• When searching, the image feature vector is also extracted, searched with milvus , return id of similar frames, and then query other information through the database.

To-do

• Support Xception pre-trained model and PCA dimensionality reduction
• Training more suitable models
• Implement op/ed optimization
• Automatically download from Sakura Anime
• Open Search API

Thanks

• Some of the processes are searched using pictures. Some ideas come from tuxiaobei
• The part downloaded from B station uses bilili
• The implementation of automatic cropping of black edges of images comes from crop.py of trace.moe
• Milvus index and search vectors are used
• Various deep learning models were used to extract image features