Thanks Amir for having this conversation here.
On Nathan's point: outside the Wikimedia projects, we of the free
culture movement tend to argue for full transparency on the functioning
of "automated decision making", "algorithmic tools" , "forensic
software" and so on, typically ensured by open data and free software.
Wikimedia wikis are not a court system, a block is not jail and so on,
but for instance EFF just argued that in the USA judiciary certain
rights ought to be ensured to respect the Sixth Amendment.
https://www.eff.org/deeplinks/2020/07/our-eu-policy-principles-procedural-j…
https://www.eff.org/deeplinks/2020/08/eff-and-aclu-tell-federal-court-foren…
<https://meta.wikimedia.org/wiki/EU_policy/Consultation_on_the_White_Paper_on_Artificial_Intelligence_(2020)>
Federico