Share My Creation ImageTrans - An Image and Comic Translation Tool

Hi there,

I've spent several months on this project. Now I would like to share with you what I've achieved with B4J in a few months.

ImageTrans is an image and comics translation tool. Below is an example of translation from English to Chinese of American golden age comics from Boy Loves Girls magazine.

It has the following major functions:

1. Localize and recognize text in images with state-of-art OCR technology provided by Tesseract, Microsoft, OCRSpace, Baidu and Youdao.
2. A homebrew comics text localization method based on connected-components grouping on binarized images and text/nontext image classification retrained on MobileNet with Tensorflow.
3. Text mask generation and text erasure using OpenCV's findContours and inpaint methods.
4. Foreground color and background color detection explained here: https://www.b4x.com/android/forum/threads/detect-dominant-colors-of-images.114547/
5. Computer-aided translation functions like translation memory, terminology management and machine translation.
6. Text reinjection utilizing controls like Pane, ImageView and Label. Use pane.snapshot to generate target images.

 

[Erel]

Looks amazing! Well done.

Tell us more. Each one of these points is a challenging task.

 

[xulihang]

This is for my graduation thesis and there will be too many details to explain for each point. I think a
straightforward list of functions would be enough for now.

 

[AnandGupta]

Below is an example of translation from English to Chinese of American golden age comics from Boy Loves Girls magazine.

Very very impressive.
good luck on your graduation thesis.

Regards,

Anand

 

[xulihang]

You can get it from here. It relies on opencv. Download the opencv java library from here.
The interface and documentation are in Chinese. I haven't had time to do the translation.

 

[xulihang]

An online version which uses jShell to run the GUI app: link.
Choose the image, adjust parameters like language code and wait for the transated version to come up.
Server's source: https://github.com/xulihang/ImageTrans_Server

 

[xulihang]

Update:

Created a Chrome Extension to translate images on webpages. It uses a websocket server as a bridge between chrome extension and ImageTrans.

 

[Erel]

It looks impressive but I'm not sure that I understand. You are running a B4J server which makes the translation?

 

[xulihang]

The server just receives commands from the Chrome extension (src) and calls ImageTrans to do the OCR, machine translation and type setting. OCR and machine translation APIs are provided by companies like Google.