Colorization https://en.wikipedia.org/wiki/Film_colorization#Digital_colorization refers to the process of adding color to black-and-white photographs. This work was historically done by hand. These days, colorization is usually done digitally, with the support of specialized tooling. But it is still quite labor-intensive.
A forthcoming paper http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/en/ from researchers at Waseda University of Japan have developed a method for automatic image colorization using deep learning neural network. The results are both impressive and easy to reproduce, as the authors have published their code https://github.com/satoshiiizuka/siggraph2016_colorization to GitHub with a permissive license.
Someone has already taken this code and packaged it as a simple webapp, available at http://colorizr.io/ (NSFW). The webapp lets you upload black-and-white pictures and colorizes them for you. (The site is currently not safe for work because it displays a gallery of recent uploads. We know how that goes.)
In this thread, let's discuss how this technology could be integrated with the projects. Should we have a bot that can perform colorization on demand, the way Rotatebot https://commons.wikimedia.org/wiki/User:Rotatebot can rotate images?
On 05/03/2016 03:21 PM, Ori Livneh wrote:
A forthcoming paper http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/en/ from researchers at Waseda University of Japan have developed a method for automatic image colorization using deep learning neural network. The results are both impressive and easy to reproduce, as the authors have published their code https://github.com/satoshiiizuka/siggraph2016_colorization to GitHub with a permissive license.
Unfortunately, this is not an open source license, and thus we should not use it. It uses Creative Commons Attribution-NonCommercial-ShareAlike 4.0.
Creative Commons consistently recommends against any use of CC licenses for software, and this one in particular is not libre or open source because it has a non-commercial restriction.
Matt Flaschen
On Tue, May 3, 2016 at 2:59 PM, Matthew Flaschen mflaschen@wikimedia.org wrote:
On 05/03/2016 03:21 PM, Ori Livneh wrote:
A forthcoming paper http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/en/ from researchers at Waseda University of Japan have developed a method for automatic image colorization using deep learning neural network. The results are both impressive and easy to reproduce, as the authors have published their code https://github.com/satoshiiizuka/siggraph2016_colorization to GitHub with a permissive license.
Unfortunately, this is not an open source license, and thus we should not use it. It uses Creative Commons Attribution-NonCommercial-ShareAlike 4.0.
Creative Commons consistently recommends against any use of CC licenses for software, and this one in particular is not libre or open source because it has a non-commercial restriction.
Hi Ori and Matt,
Matt, I agree that they probably picked an inappropriate license. However, we shouldn't assume that the people picking the license have a very sophisticated understanding of licenses. It might be worthwhile to ask the authors why they chose CC BY-NC-SA 4.0 instead of a free license (like MIT, Apache, GPL or AGPL). If we approach them respectfully, we might convince them to learn more about our ideals, and change the license on their software.
Ori, this is a fantastic find! I haven't created this wiki page yet, but I think it should exist: https://mediawiki.org/wiki/Colorization
It'd be really awesome if https://mediawiki.org/wiki/Talk:Colorization contained a pointer back to this discussion.
That is, of course, that people reading this list agree is interesting. Anyone here against colors? ;-)
Rob
On Tue, May 3, 2016 at 3:31 PM, Rob Lanphier robla@wikimedia.org wrote:
It might be worthwhile to ask the authors why they chose CC BY-NC-SA 4.0 instead of a free license (like MIT, Apache, GPL or AGPL). If we approach them respectfully, we might convince them to learn more about our ideals, and change the license on their software.
Good idea. I sent a note to the authors to explain why the non-commercial clause is incompatible with "open source" and to encourage them to release the code and model under a permissive license. I will share any updates with the list.
On 05/03/2016 06:31 PM, Rob Lanphier wrote:
Matt, I agree that they probably picked an inappropriate license. However, we shouldn't assume that the people picking the license have a very sophisticated understanding of licenses. It might be worthwhile to ask the authors why they chose CC BY-NC-SA 4.0 instead of a free license (like MIT, Apache, GPL or AGPL).
You're right. This is a good point and I'm glad Ori contacted them.
Ori, this is a fantastic find! I haven't created this wiki page yet, but I think it should exist: https://mediawiki.org/wiki/Colorization
It'd be really awesome if https://mediawiki.org/wiki/Talk:Colorization contained a pointer back to this discussion.
That is, of course, that people reading this list agree is interesting. Anyone here against colors? ;-)
Not at all. I sent my previous message a bit too hastily since I was about to go into a meeting, so I neglected to mention:
This is a really cool idea and a neat project (with very impressive results if the example images on GitHub are representative). I hope the authors are willing to release their work under an open source license.
Matt
On 05/03/2016 03:21 PM, Ori Livneh wrote:
In this thread, let's discuss how this technology could be integrated with the projects. Should we have a bot that can perform colorization on demand, the way Rotatebot https://commons.wikimedia.org/wiki/User:Rotatebot can rotate images?
A couple notes:
* This could go in https://www.mediawiki.org/wiki/Extension:ImageTweaks when that's ready.
* This should go to a separate destination file (which it appears ImageTweaks already supports), since unlike rotation (which can be lossless and doesn't fundamentally change the essence of the image), colorization is a complete transformation, and it's worth preserving both old and new in some cases (e.g. historical images from archives).
Matt Flaschen
On 04/05/16 05:21, Ori Livneh wrote:
Colorization https://en.wikipedia.org/wiki/Film_colorization#Digital_colorization refers to the process of adding color to black-and-white photographs. This work was historically done by hand. These days, colorization is usually done digitally, with the support of specialized tooling. But it is still quite labor-intensive.
A forthcoming paper http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/en/ from researchers at Waseda University of Japan have developed a method for automatic image colorization using deep learning neural network. The results are both impressive and easy to reproduce, as the authors have published their code https://github.com/satoshiiizuka/siggraph2016_colorization to GitHub with a permissive license.
Impressive, yes, but with lots of ridiculous errors. For example, the ground often ends up green even when it's a road:
http://colorizr.io/image.php?uuid=cdcc0b2f-dc9e-4592-938b-b1146f75ecb5 http://colorizr.io/image.php?uuid=b868719e-b59a-42ed-ae9b-27f2a52fe246
Clothing is apparently always brown:
http://colorizr.io/image.php?uuid=a94e5ff7-25a1-4e61-b1be-b54f8301708d http://colorizr.io/image.php?uuid=3f5dadcb-912c-40fb-82fa-b52dde6d280b http://colorizr.io/image.php?uuid=687cb8e6-0031-443c-83b4-37d403a7fd34
Red is randomly splashed around with no apparent pattern:
http://colorizr.io/image.php?uuid=9567aab8-a94d-488d-a4b1-40b746649757 http://colorizr.io/image.php?uuid=3cce1ab2-b866-4ca6-b713-d7f49e392ab2 http://colorizr.io/image.php?uuid=d6d65eed-94e0-4a86-a772-057975d1a18c http://colorizr.io/image.php?uuid=debdc3f9-369b-494f-929d-5cf5a5b38712
Sometimes feature identification fails spectacularly:
http://colorizr.io/image.php?uuid=44d1c028-074d-4162-be65-4200569b89d2
Is it good enough for Wikipedia? Even the best examples have subtle defects.
Should we have a bot that can perform colorization on demand, the way Rotatebot https://commons.wikimedia.org/wiki/User:Rotatebot can rotate images?
Well, Rotatebot uploads images without review.
-- Tim Starling
wikitech-l@lists.wikimedia.org