Hi sitereq-l,
I'm looking for context regarding our upload-by-url allowlist in the hopes of reducing workload for the site request process. Does anyone know
* Why do we even have an allowlist for upload-by-url? I presume this is to make it harder to upload a large amount of non-free files, but I'm curious if there are any other reasons that I'm not aware of.
* If there aren't other reasons for having the allowlist, are there any reasons other than "someone needs to work on it" that would not let us to move the allowlist to a system message that Commons administrators can edit?
Thanks!
-- Taavi
On Fri, 28 Jan 2022 at 06:42, Taavi Väänänen hi@taavi.wtf wrote:
Hi sitereq-l,
I'm looking for context regarding our upload-by-url allowlist in the hopes of reducing workload for the site request process. Does anyone know
- Why do we even have an allowlist for upload-by-url? I presume this is
to make it harder to upload a large amount of non-free files, but I'm curious if there are any other reasons that I'm not aware of.
- If there aren't other reasons for having the allowlist, are there any
reasons other than "someone needs to work on it" that would not let us to move the allowlist to a system message that Commons administrators can edit?
Yeah, I filed T140040 https://phabricator.wikimedia.org/T140040 a few years ago to scrap the allowlist and just trust +sysop users (and let the community de-sysop them if they misuse or abuse it). Any change of this kind would need to be discussed in advance with the Commons community of course.
Switching the allowlist to an on-wiki page seems fine from my POV, though it might be worth exploring just setting it to * first before doing the extra work of migrating it?
J. -- *James D. Forrester* (he/him http://pronoun.is/he or they/themself http://pronoun.is/they/.../themself) Wikimedia Foundation https://wikimediafoundation.org/
Like James, I'd be fine with having the allowlist on wiki.
I don't think it's a good idea to *remove* the allowlist though. If you remove it, the upload-by-url feature might become a vector for an amplification DoS attack. As of today, upload_by_url can be used by any and all Commons users. With no allowlist, it'd be much easier to instruct our servers to request an excessive amount of data from a target server of your choice. This will (likely? didn't check) be restricted by our own rate limits on uploading, but the upload rate limits are virtually nonexistent for autopatrollers and above (which is a role reasonably easy to get; much easier than +sysop, for example). I'm not sure if this kind of abuse is likely to happen.
However, I recall it given as an explanation when I wondered why the allowlist exists a few years ago. What do you think Taavi?
Martin Urbanec
pá 28. 1. 2022 v 16:12 odesílatel James Forrester jforrester@wikimedia.org napsal:
On Fri, 28 Jan 2022 at 06:42, Taavi Väänänen hi@taavi.wtf wrote:
Hi sitereq-l,
I'm looking for context regarding our upload-by-url allowlist in the hopes of reducing workload for the site request process. Does anyone know
- Why do we even have an allowlist for upload-by-url? I presume this is
to make it harder to upload a large amount of non-free files, but I'm curious if there are any other reasons that I'm not aware of.
- If there aren't other reasons for having the allowlist, are there any
reasons other than "someone needs to work on it" that would not let us to move the allowlist to a system message that Commons administrators can edit?
Yeah, I filed T140040 https://phabricator.wikimedia.org/T140040 a few years ago to scrap the allowlist and just trust +sysop users (and let the community de-sysop them if they misuse or abuse it). Any change of this kind would need to be discussed in advance with the Commons community of course.
Switching the allowlist to an on-wiki page seems fine from my POV, though it might be worth exploring just setting it to * first before doing the extra work of migrating it?
J.
*James D. Forrester* (he/him http://pronoun.is/he or they/themself http://pronoun.is/they/.../themself) Wikimedia Foundation https://wikimediafoundation.org/ _______________________________________________ Sitereq-l mailing list -- sitereq-l@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/sitereq-l.lists.wikimedia.org/
Your explanations make sense, thank you! I agree with Martin here that we should not completely remove the allowlist as long as the feature is available to "normal" users (as in non-sysops).
Based on that I'm inclined to go ahead and write a patch that lets sysops manage the allowlist via a system message and filed https://phabricator.wikimedia.org/T300407 to track it. I don't think there is an issue with trusting Commons sysops to manage the list.
-- Taavi
PS. Looking at this brought me some memories! Writing tests for UploadFromUrl::isAllowedHost was my very first contribution to the core, back in January 2020 during GCI 2019. :-)
On 1/28/22 17:28, Martin Urbanec wrote:
Like James, I'd be fine with having the allowlist on wiki.
I don't think it's a good idea to /remove/ the allowlist though. If you remove it, the upload-by-url feature might become a vector for an amplification DoS attack. As of today, upload_by_url can be used by any and all Commons users. With no allowlist, it'd be much easier to instruct our servers to request an excessive amount of data from a target server of your choice. This will (likely? didn't check) be restricted by our own rate limits on uploading, but the upload rate limits are virtually nonexistent for autopatrollers and above (which is a role reasonably easy to get; much easier than +sysop, for example). I'm not sure if this kind of abuse is likely to happen.
However, I recall it given as an explanation when I wondered why the allowlist exists a few years ago. What do you think Taavi?
Martin Urbanec
pá 28. 1. 2022 v 16:12 odesílatel James Forrester <jforrester@wikimedia.org mailto:jforrester@wikimedia.org> napsal:
On Fri, 28 Jan 2022 at 06:42, Taavi Väänänen <hi@taavi.wtf> wrote: Hi sitereq-l, I'm looking for context regarding our upload-by-url allowlist in the hopes of reducing workload for the site request process. Does anyone know * Why do we even have an allowlist for upload-by-url? I presume this is to make it harder to upload a large amount of non-free files, but I'm curious if there are any other reasons that I'm not aware of. * If there aren't other reasons for having the allowlist, are there any reasons other than "someone needs to work on it" that would not let us to move the allowlist to a system message that Commons administrators can edit? Yeah, I filed T140040 <https://phabricator.wikimedia.org/T140040> a few years ago to scrap the allowlist and just trust +sysop users (and let the community de-sysop them if they misuse or abuse it). Any change of this kind would need to be discussed in advance with the Commons community of course. Switching the allowlist to an on-wiki page seems fine from my POV, though it might be worth exploring just setting it to *first before doing the extra work of migrating it? J. -- *James D. Forrester*(he/him <http://pronoun.is/he> or they/themself <http://pronoun.is/they/.../themself>) Wikimedia Foundation <https://wikimediafoundation.org/> _______________________________________________ Sitereq-l mailing list -- sitereq-l@lists.wikimedia.org <mailto:sitereq-l@lists.wikimedia.org> List information: https://lists.wikimedia.org/postorius/lists/sitereq-l.lists.wikimedia.org/ <https://lists.wikimedia.org/postorius/lists/sitereq-l.lists.wikimedia.org/>