Google security have announced that they have a working collision attack against the SHA-1 hash:
https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html
It's highly recommended to move to sha-256 where doable.
Note that MediaWiki uses sha-1 in a number of places; in some such as revision hashes it's advisory for tools only, but in other places like deleted files (filearchive table) we use it for addressing, and should consider steps to mitigate attacks swapping in alternate files during deletion/undeletion.
-- brion
I searched in phabricator, if we've a task already, but couldn't find any. However, as the phabricator search and me aren't really good friends, it's possible, that the search wasn't as honest to me, as I would wish and I missed something, so I ask on this list :) Do we've a task already to track the work on this topic? A short github search[1] showed some usages of sha1 (at least the string), so I suspect, that there're some places where we use it, right?
[1] https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93&q=SHA1
Best, Florian
-----Ursprüngliche Nachricht----- Von: Wikitech-l [mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Brion Vibber Gesendet: Freitag, 24. Februar 2017 18:57 An: Wikimedia-tech list wikitech-l@lists.wikimedia.org Betreff: [Wikitech-l] SHA-1 hash officially broken
Google security have announced that they have a working collision attack against the SHA-1 hash:
https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html
It's highly recommended to move to sha-256 where doable.
Note that MediaWiki uses sha-1 in a number of places; in some such as revision hashes it's advisory for tools only, but in other places like deleted files (filearchive table) we use it for addressing, and should consider steps to mitigate attacks swapping in alternate files during deletion/undeletion.
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Feb 24, 2017 at 10:34 AM, Florian Schmidt < florian.schmidt.welzow@t-online.de> wrote:
I searched in phabricator, if we've a task already, but couldn't find any. However, as the phabricator search and me aren't really good friends, it's possible, that the search wasn't as honest to me, as I would wish and I missed something, so I ask on this list :) Do we've a task already to track the work on this topic? A short github search[1] showed some usages of sha1 (at least the string), so I suspect, that there're some places where we use it, right?
[1] https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93&q=SHA1
Usages are listed at https://www.mediawiki.org/wiki/Manual:Hashing -- I've added a "purpose" column on the DB fields list. There might be a couple we missed, so feel free to edit!
It looks like img_sha1 and fa_storage_key are the biggest practical dangers of collision, in that pairs of images could be created such that, say, one is a cute kitten and the other is a shock image, such that deletion and undeletion might surface the wrong file from what the undeleting admin expected.
However since the attack requires creating a common prefix and suffix for the matching pair of files, there's little or no danger of an attack that replaces legitimate files already uploaded by someone else.
I would probably recommend migrating rev_sha1 and img_sha1's usages in collision detection over to a rev_sha256 and img_sha256 columns, and reworking filearchve to use a sha-256 hash for content addressing of new files, but this doesn't need to be a "drop everything" hurry task.
-- brion
but this doesn't need to be a "drop everything" hurry task.
Totally agree! :)
Thanks for the information with the page on mediawiki.org. I'll take a look and probably edit it, if I see something :P
Best, Florian
-----Ursprüngliche Nachricht----- Von: Wikitech-l [mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Brion Vibber Gesendet: Freitag, 24. Februar 2017 19:48 An: Wikimedia developers wikitech-l@lists.wikimedia.org Betreff: Re: [Wikitech-l] SHA-1 hash officially broken
On Fri, Feb 24, 2017 at 10:34 AM, Florian Schmidt < florian.schmidt.welzow@t-online.de> wrote:
I searched in phabricator, if we've a task already, but couldn't find any. However, as the phabricator search and me aren't really good friends, it's possible, that the search wasn't as honest to me, as I would wish and I missed something, so I ask on this list :) Do we've a task already to track the work on this topic? A short github search[1] showed some usages of sha1 (at least the string), so I suspect, that there're some places where we use it, right?
[1] https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93&q=SHA1
Usages are listed at https://www.mediawiki.org/wiki/Manual:Hashing -- I've added a "purpose" column on the DB fields list. There might be a couple we missed, so feel free to edit!
It looks like img_sha1 and fa_storage_key are the biggest practical dangers of collision, in that pairs of images could be created such that, say, one is a cute kitten and the other is a shock image, such that deletion and undeletion might surface the wrong file from what the undeleting admin expected.
However since the attack requires creating a common prefix and suffix for the matching pair of files, there's little or no danger of an attack that replaces legitimate files already uploaded by someone else.
I would probably recommend migrating rev_sha1 and img_sha1's usages in collision detection over to a rev_sha256 and img_sha256 columns, and reworking filearchve to use a sha-256 hash for content addressing of new files, but this doesn't need to be a "drop everything" hurry task.
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Added a tracking/epic task on phab: https://phabricator.wikimedia.org/T158986
Attach any specific things to that as subtasks, y'all.
-- brion
On Fri, Feb 24, 2017 at 10:51 AM, Florian Schmidt < florian.schmidt.welzow@t-online.de> wrote:
but this doesn't need to be a "drop everything" hurry task.
Totally agree! :)
Thanks for the information with the page on mediawiki.org. I'll take a look and probably edit it, if I see something :P
Best, Florian
-----Ursprüngliche Nachricht----- Von: Wikitech-l [mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Brion Vibber Gesendet: Freitag, 24. Februar 2017 19:48 An: Wikimedia developers wikitech-l@lists.wikimedia.org Betreff: Re: [Wikitech-l] SHA-1 hash officially broken
On Fri, Feb 24, 2017 at 10:34 AM, Florian Schmidt < florian.schmidt.welzow@t-online.de> wrote:
I searched in phabricator, if we've a task already, but couldn't find
any.
However, as the phabricator search and me aren't really good friends, it's possible, that the search wasn't as honest to me, as I would wish and I missed something, so I ask on this list :) Do we've a task already to track the work on this topic? A short github search[1] showed some usages of sha1 (at least the string), so I suspect, that there're some places where we use it, right?
[1] https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93&q=SHA1
Usages are listed at https://www.mediawiki.org/wiki/Manual:Hashing -- I've added a "purpose" column on the DB fields list. There might be a couple we missed, so feel free to edit!
It looks like img_sha1 and fa_storage_key are the biggest practical dangers of collision, in that pairs of images could be created such that, say, one is a cute kitten and the other is a shock image, such that deletion and undeletion might surface the wrong file from what the undeleting admin expected.
However since the attack requires creating a common prefix and suffix for the matching pair of files, there's little or no danger of an attack that replaces legitimate files already uploaded by someone else.
I would probably recommend migrating rev_sha1 and img_sha1's usages in collision detection over to a rev_sha256 and img_sha256 columns, and reworking filearchve to use a sha-256 hash for content addressing of new files, but this doesn't need to be a "drop everything" hurry task.
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Feb 24, 2017 at 10:57 AM Brion Vibber bvibber@wikimedia.org wrote:
Added a tracking/epic task on phab: https://phabricator.wikimedia.org/T158986
Attach any specific things to that as subtasks, y'all.
Thanks for getting the discussion started. I was talking about this yesterday with some people and yeah, file and revision sha1s immediately came to mind.
Also possibly a future issue: Git--but we'll mostly want to wait for various upstreams to make progress here. There's some interesting thoughts over at this fork[0] (h/t to Tim)
But, like you said (and Dan pointed out on the task), this is far from a drop everything situation...we just need to start making some inroads to this sooner rather than later. Since it's broken, making it faster is an optimization problem so the writing is on the wall.
-Chad
[0] https://github.com/peff/git/commit/b55aa200a0b3af58dc9e0accadc719bb47573ac3
About the git part: There's an interesting statement from Linus [1] with which I can totally agree (and additionally this one[2]). So probably even this is not a "high priority, do nothing else" task :P
[1] http://marc.info/?l=git&m=148787047422954 [2] https://public-inbox.org/git/Pine.LNX.4.58.0504291221250.18901@ppc970.osdl.o...
-----Ursprüngliche Nachricht----- Von: Wikitech-l [mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Chad Gesendet: Freitag, 24. Februar 2017 20:21 An: Wikimedia developers wikitech-l@lists.wikimedia.org Betreff: Re: [Wikitech-l] SHA-1 hash officially broken
On Fri, Feb 24, 2017 at 10:57 AM Brion Vibber bvibber@wikimedia.org wrote:
Added a tracking/epic task on phab: https://phabricator.wikimedia.org/T158986
Attach any specific things to that as subtasks, y'all.
Thanks for getting the discussion started. I was talking about this yesterday with some people and yeah, file and revision sha1s immediately came to mind.
Also possibly a future issue: Git--but we'll mostly want to wait for various upstreams to make progress here. There's some interesting thoughts over at this fork[0] (h/t to Tim)
But, like you said (and Dan pointed out on the task), this is far from a drop everything situation...we just need to start making some inroads to this sooner rather than later. Since it's broken, making it faster is an optimization problem so the writing is on the wall.
-Chad
[0] https://github.com/peff/git/commit/b55aa200a0b3af58dc9e0accadc719bb47573ac3 _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
2017-02-24 21:29 GMT+02:00 Florian Schmidt florian.schmidt.welzow@t-online.de:
About the git part: There's an interesting statement from Linus [1] with which I can totally agree (and additionally this one[2]). So probably even this is not a "high priority, do nothing else" task :P
Just FYI, this was confirmed in [3]
[1] http://marc.info/?l=git&m=148787047422954 [2] https://public-inbox.org/git/Pine.LNX.4.58.0504291221250.18901@ppc970.osdl.o...
[3] https://plus.google.com/+LinusTorvalds/posts/7tp2gYWQugL
On Fri, Feb 24, 2017 at 11:20 AM, Chad innocentkiller@gmail.com wrote:
But, like you said (and Dan pointed out on the task), this is far from a drop everything situation...we just need to start making some inroads to this sooner rather than later. Since it's broken, making it faster is an optimization problem so the writing is on the wall.
Well, technically it was broken in 2012, optimization is what we are seeing now. Google claims files generated with this technique are detectable and they will publish the detection library soon, so in the short term we just need to install that as a filter for file uploads.
wikitech-l@lists.wikimedia.org