comic translate
1.0.0
English | 한국어 | Français | 简体中文 | 日本語 | Português Brasileiro
Many Automatic Manga Translators exist. Very few properly support comics of other kinds in other languages. This project was created to utilize the ability of State of the Art (SOTA) Large Language Models (LLMs) like GPT-4 and translate comics from all over the world. Currently, it supports translating to and from English, Korean, Japanese, French, Simplified Chinese, Traditional Chinese, Russian, German, Dutch, Spanish and Italian. It can translate to (but not from) Turkish, Polish, Portuguese and Brazillian Portuguese.
The State of Machine Translation
Preview
Getting Started
How it works
Acknowledgements
For a couple dozen languages, the best Machine Translator is not Google Translate, Papago or even DeepL, but a SOTA LLM like GPT-4o, and by far. This is very apparent for distant language pairs (Korean<->English, Japanese<->English etc) where other translators still often devolve into gibberish. Excerpt from "The Walking Practice"(보행 연습) by Dolki Min(돌기민)
GPT-4 as Translator. Note: Some of these also have Official English Translations
The Wretched of the High Seas
Journey to the West
The Wormworld Saga
Frieren: Beyond Journey's End
Days of Sand
Player (OH Hyeon-Jun)
Carbon & Silicon
Install Python (>=3.10). Tick "Add python.exe to PATH" during the setup.
https://www.python.org/downloads/Clone the repo (or download the folder), navigate to the folder
git clone https://github.com/ogkalu2/comic-translate
cd comic-translateand install the requirements
pip install -r requirements.txtIf you run into any issues, you can try running it in a virtual environment. Open the terminal/cmd in whatever directory you want the virtual environment installed (or cd 'path/to/virtual environment/folder'). Create your virtual environment with:
python -m venv comic-translate-venvNow activate the virtual environment. On Windows:
comic-translate-venvScriptsactivateOn Mac and Linux:
source comic-translate-venv/bin/activateNow you can run the Installation Commands again. When you are finished using the app, you can deactivate the virtul environment with:
deactivateTo re-activate, use the same commands with the terminal in the folder your virtual environment folder is located in.
If you have an NVIDIA GPU, then it is recommended to run
pip uninstall torch torchvision
pip install torch==2.1.0+cu121 -f https://download.pytorch.org/whl/torch_stable.html
pip install torchvision==0.16.0+cu121 -f https://download.pytorch.org/whl/torch_stable.htmlNote: The 121 in +cu121 represents the CUDA version - 12.1. Replace 121 with your CUDA version. E.g 118 if you are running CUDA 11.8
In the comic-translate directory, run
python comic.pyThis will launch the GUI
raise RarCannotExec("Cannot find working tool")In that case, Instructions for Windows, Linux, Mac
To following selections will require access to closed resources and subsequently, API Keys:
speech-bubble-detector, text-segmenter. Two yolov8m models trained on 8k and 3k images of comics (Manga, Webtoons, Western) respectively.
By Default:
Optional:
These can be used for any of the supported languages. An API Key is required.
A Manga/Anime finetuned lama checkpoint to remove text detected by the segmenter. Implementation courtsey of lama-cleaner
Currently, this supports using GPT-4o, GPT-4o mini, DeepL, Claude-3-Opus, Claude-3.5-Sonnet, Claude-3-Haiku, Gemini-1.5-Flash, Gemini-1.5-Pro, Yandex, Google Translate and Microsoft Translator.
All LLMs are fed the entire page text to aid translations. There is also the Option to provide the Image itself for further context.
PIL for rendering wrapped text in bounding boxes obtained from bubbles and text.
