There's a nice story behind this project too. My dear friend and colleague Tobi once pointed out to me that he'd found a great manga on the internet. He complained a bit that he only barely understood the English fan translation because his English isn't that great. He had to take screenshots, use OCR, and then translate every single speech bubble individually just to understand what was going on.
And I took that as a challenge. There's a better way to do this. So, I looked into the whole thing and realized that the manhua, as it is, consists of a bunch of images glued together. So, I took the original, which is freely available, downloaded it, and ran OCR software over it. I then marked the text and the positions found based on the speech bubbles. Then I had the content painted over and dropped in the German translation.
But that's not enough. After all that fun, I eventually assigned voices to the entire manga. I then used several AI runs to let the system understand the story – which, by the way, happened locally on my machine again. Then the AI determines the speakers, the main characters. It doesn't work perfectly yet, but it's good enough. Then sound FX are added, then music is layered underneath, and in the end, I had the entire comic voiced over or rather turned into a video. I've included one episode here to show you how it works.
Well, and now we practically have a quasi-automatic pipeline for how a normal manga becomes a video that you can just put on in the background. Take a look, it's quite nice, and I'll include my Manga Factory program code for you as well. All right, enjoy the fun and, as usual: feel free to take the source code apart, adapt it for yourself, and use it. Have fun!
Oh, one more thing as a postscript: it's not enough for me to just create full videos. No, I also have to build an automated player via my T-Scale network so that I can play the whole thing beautifully one after another directly from the website. Just a video file would be boring. No, if you're going to do it, do it right.
Gallery



