On Fri, Aug 7, 2020 at 6:39 PM Ryan Kaldari rkaldari@wikimedia.org wrote:
Whatever danger is embodied in Amir's code, it's only a matter of time before this danger is ubiquitous. And for the worst-case scenario—governments using the technology to hunt down dissidents—I imagine this is already happening. So while I agree there is a moral consideration to releasing this software, I think the moral implications aren't actually that huge. Eventually, we will just have to accept that creating separate accounts is not an effective way to protect your identity.
Deanonymizing wiki accounts is one way of misusing the tool, and one which would indeed happen anyway. Another scenario is an attacker examining the tool with the intent of misleading it (such as using an adversarial network to construct edits which the tool would consistently misidentify as belonging to a certain user, which could be used to cast suspicion on a legitimate user). That specifically depends on the model being publicly available.