[re: mass-shifting of free-licence images from en:wp to commons]
On 16/12/06, geni geniice@gmail.com wrote:
On 12/16/06, David Gerard dgerard@gmail.com wrote:
I mean for "this should be moved to Commons". Is there any reason I'm not thinking of for not generally moving free content to Commons, as the page on en:wp will continue to work just the same?
Wating until we can move complete image histories to commons? The effect of the current situations is that there are many images for which only a en.wikipedia admin can find out their true history.
Ooh yes. Do we need to wait on single login for that to be a reasonable thing to do? I understand a dev also has to explicitly allow transwiki to move histories, though I'm not at all clear on the details.
[cc'd to wikitech-l for my cluification]
- d.
On 16/12/06, David Gerard dgerard@gmail.com wrote:
Ooh yes. Do we need to wait on single login for that to be a reasonable thing to do? I understand a dev also has to explicitly allow transwiki to move histories, though I'm not at all clear on the details.
As far as I know, Special:Import won't handle moving images in the correct manner. Mind you, I've not been keeping 100% up to date with what's developed lately, so I may well be wrong.
Rob Church
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Rob Church wrote:
On 16/12/06, David Gerard dgerard@gmail.com wrote:
Ooh yes. Do we need to wait on single login for that to be a reasonable thing to do? I understand a dev also has to explicitly allow transwiki to move histories, though I'm not at all clear on the details.
As far as I know, Special:Import won't handle moving images in the correct manner. Mind you, I've not been keeping 100% up to date with what's developed lately, so I may well be wrong.
Special:Export and Special:Import do not handle images at this time. There's some specs for it in the XML format but currently it's only used by the OAI feed system.
- -- brion vibber (brion @ pobox.com)
On 17/12/06, Brion Vibber brion@pobox.com wrote:
Special:Export and Special:Import do not handle images at this time. There's some specs for it in the XML format but currently it's only used by the OAI feed system.
What might be the best approach to making this work, then? Chucking the filename in the XML somewhere and having Special:Import do a local copy, if possible?
Rob Church
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Rob Church wrote:
On 17/12/06, Brion Vibber brion@pobox.com wrote:
Special:Export and Special:Import do not handle images at this time. There's some specs for it in the XML format but currently it's only used by the OAI feed system.
What might be the best approach to making this work, then? Chucking the filename in the XML somewhere and having Special:Import do a local copy, if possible?
That's pretty much how it would work if you implemented the spec. I stashed an example in docs/export-demo.xml:
<page> <title>Image:Some image.jpg</title> [snip] <upload> <timestamp>2001-01-15T20:34:12Z</timestamp> <contributor><username>Foobar</username><id>42</id></contributor> <comment>My awesomeest image!</comment> <filename>Some_image.jpg</filename> <src>http://upload.wikimedia.org/commons/2/22/Some_image.jpg</src> <size>12345</size> </upload> </page>
Probably not too hard to hack up, actually, if someone wants to give it a try.
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
On 12/17/06, Brion Vibber brion@pobox.com wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Rob Church wrote:
On 17/12/06, Brion Vibber brion@pobox.com wrote:
Special:Export and Special:Import do not handle images at this time. There's some specs for it in the XML format but currently it's only used by the OAI feed system.
What might be the best approach to making this work, then? Chucking the filename in the XML somewhere and having Special:Import do a local copy, if possible?
That's pretty much how it would work if you implemented the spec. I stashed an example in docs/export-demo.xml:
<page> <title>Image:Some image.jpg</title> [snip] <upload> <timestamp>2001-01-15T20:34:12Z</timestamp> <contributor><username>Foobar</username><id>42</id></contributor> <comment>My awesomeest image!</comment> <filename>Some_image.jpg</filename> <src>http://upload.wikimedia.org/commons/2/22/Some_image.jpg</src> <size>12345</size> </upload> </page>
Probably not too hard to hack up, actually, if someone wants to give it a try.
Or, we could use query.php (or that other api thing...) :
http://en.wikipedia.org/w/query.php?what=imageinfo&titles=Image:Cetacea_...
to import the image history, and use SpecialImport for the description page only.
Magnus
On 18/12/06, Magnus Manske magnusmanske@googlemail.com wrote:
Or, we could use query.php (or that other api thing...) :
http://en.wikipedia.org/w/query.php?what=imageinfo&titles=Image:Cetacea_...
to import the image history, and use SpecialImport for the description page only.
No, I think it makes more sense if Special:Import handles the full, proper import itself.
Rob Church
"Rob Church" robchur@gmail.com wrote in message news:e92136380612180426w77762c6ftf0ece0adc563b330@mail.gmail.com...
On 18/12/06, Magnus Manske
magnusmanske@googlemail.com wrote:
Or, we could use query.php (or that other api thing...) :
http://en.wikipedia.org/w/query.php?what=imageinfo&titles=Image:Cetacea_... e_map_Chinese_River_Dolphin.PNG
to import the image history, and use SpecialImport for the description page only.
No, I think it makes more sense if Special:Import handles the full, proper import itself.
Rob Church
I agree, Special:Import should copy the full image history as well as the full page history. The way it is at the moment just feels broken, and making it a two stage process won't fix that.
We may also want an 'include files when relevant' check-box that can be turned off for those occasions when you just want the text, although I can't really think of a use for that - at least not for the way we use image pages on WM projects.
- Mark Clements (HappyDog)
On 12/19/06, Mark Clements gmane@kennel17.co.uk wrote:
"Rob Church" robchur@gmail.com wrote in message news:e92136380612180426w77762c6ftf0ece0adc563b330@mail.gmail.com...
On 18/12/06, Magnus Manske
magnusmanske@googlemail.com wrote:
Or, we could use query.php (or that other api thing...) :
http://en.wikipedia.org/w/query.php?what=imageinfo&titles=Image:Cetacea_... e_map_Chinese_River_Dolphin.PNG
to import the image history, and use SpecialImport for the description page only.
No, I think it makes more sense if Special:Import handles the full, proper import itself.
Rob Church
I agree, Special:Import should copy the full image history as well as the full page history. The way it is at the moment just feels broken, and making it a two stage process won't fix that.
OK. So, how do we handle the user ids from the other project? Do we assume single login is in place? Or do we create dummy users like "en:someguy", user_id=0? (I didn't bother to look how it's handled in Special:Import right now, easier to ask here;-)
Also, if we plan to do mass-moving, of images, should we add this function to the api? What about handling the deletion of the images on the "source wiki"? If these functions were available using "name=xxx&password=yyy" on the api, it would save a lot of clicks with a decent front-end (which I then would volunteer to develop;-)
We may also want an 'include files when relevant' check-box that can be turned off for those occasions when you just want the text, although I can't really think of a use for that - at least not for the way we use image pages on WM projects.
A checkbox/option won't hurt. Except when people set up a wikipedia mirror and leech the images from wikimedia servers that way...
Magnus
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Magnus Manske wrote:
OK. So, how do we handle the user ids from the other project? Do we assume single login is in place? Or do we create dummy users like "en:someguy", user_id=0? (I didn't bother to look how it's handled in Special:Import right now, easier to ask here;-)
As far as I remember, dummy users (user_id=0) are created if the users are not exist.
On 19/12/06, Magnus Manske magnusmanske@googlemail.com wrote:
OK. So, how do we handle the user ids from the other project? Do we assume single login is in place? Or do we create dummy users like "en:someguy", user_id=0? (I didn't bother to look how it's handled in Special:Import right now, easier to ask here;-)
Put the username in image.img_user_text and set image.img_user to 0.
Also, if we plan to do mass-moving, of images, should we add this function to the api? What about handling the deletion of the images on the "source wiki"? If these functions were available using "name=xxx&password=yyy" on the api, it would save a lot of clicks with a decent front-end (which I then would volunteer to develop;-)
The API is none of my business; I gather a separate group has taken the responsibility for designing and implementing it.
Rob Church
"Magnus Manske" magnusmanske@googlemail.com wrote in message news:fab0ecb70612190104y741e5bb6of789bb1ca3132d51@mail.gmail.com...
On 12/19/06, Mark Clements gmane@kennel17.co.uk
wrote:
I agree, Special:Import should copy the full image history as well as
the
full page history. The way it is at the moment just feels broken, and making it a two stage process won't fix that.
OK. So, how do we handle the user ids from the other project? Do we assume single login is in place? Or do we create dummy users like "en:someguy", user_id=0? (I didn't bother to look how it's handled in Special:Import right now, easier to ask here;-)
Also, if we plan to do mass-moving, of images, should we add this function to the api? What about handling the deletion of the images on the "source wiki"? If these functions were available using "name=xxx&password=yyy" on the api, it would save a lot of clicks with a decent front-end (which I then would volunteer to develop;-)
The current system works fine, as far as I am concerned. The import is a one-way process, and doesn't touch the material on the source wiki. This is as it should be, as there will be situations where you are trying to copy, not move the material. Currently you do the import and, if the content is being moved, login to the remote wiki and either delete the page or add a soft redirect. There should definitely _not_ be a mechanism for remotely deleting wiki content from a remote wiki!
- Mark Clements (HappyDog)
In SpecialUserlogin.php, it looks like after $wgAuth->updateUser( $u ) is called $u->saveSettings() is not called. This is not the same behavior as before (in MediaWiki 1.6). Is this the correct behavior, or should I put this into the bugzilla?
If this is the correct behavior, are plugin authors supposed to manually call saveSettings in updateUser(), but not in initUser()?
V/r,
Ryan Lane
On 12/19/06, Mark Clements gmane@kennel17.co.uk wrote:
The current system works fine, as far as I am concerned. The import is a one-way process, and doesn't touch the material on the source wiki. This is as it should be, as there will be situations where you are trying to copy, not move the material. Currently you do the import and, if the content is being moved, login to the remote wiki and either delete the page or add a soft redirect. There should definitely _not_ be a mechanism for remotely deleting wiki content from a remote wiki!
Why not? If I'm an admin on that remote wiki, it might be handy when moving images to Commons. At the moment of writing this mail, there are 657,331 media files on en.wikipedia. If 1/3 of them (rough guess) are suitable for Commons, that means >200,000 manual deletions when moving. If an api would allow image deletions by admins, it would save a lot of unneccessary work, without compromising safety (the images are kept, and the api could also offer undelete).
Magnus
On 12/19/06, Magnus Manske magnusmanske@googlemail.com wrote:
Why not? If I'm an admin on that remote wiki, it might be handy when moving images to Commons. At the moment of writing this mail, there are 657,331 media files on en.wikipedia. If 1/3 of them (rough guess) are suitable for Commons, that means >200,000 manual deletions when moving. If an api would allow image deletions by admins, it would save a lot of unneccessary work, without compromising safety (the images are kept, and the api could also offer undelete).
These images need to be reviewed by a human.. many of them obtained a free tag through license roulette and are obvious violations to even the most cursory examination. A move to commons is a good time to perform such a check.
This isn't an argument against the API allowing deletion, but just a factor pointing out that the dream of fully automated moves should never be made real. :)
On 12/19/06, Gregory Maxwell gmaxwell@gmail.com wrote:
These images need to be reviewed by a human.. many of them obtained a free tag through license roulette and are obvious violations to even the most cursory examination. A move to commons is a good time to perform such a check.
This isn't an argument against the API allowing deletion, but just a factor pointing out that the dream of fully automated moves should never be made real. :)
I'm not aiming for a "move everything" automaton, rather for a tool that can * list images on a "source wiki" (e.g., all images of a user) * shows thumbnails and description of each image * has a checkbox for each image * and a button "move selected images"
Basically, my "PushForCommons" tool, but pimped ;-)
For example, I remember there's some user (on de, I think) who uploaded hundreds of images of tractors, which he took during some exhibition. They all have a similar, complete description. I'd like to be able to take these and move them all at once, since I know they're all OK.
Magnus
"Magnus Manske" magnusmanske@googlemail.com wrote in message news:fab0ecb70612191032s4db8b922h4e121b4413b6c3ba@mail.gmail.com...
On 12/19/06, Mark Clements gmane@kennel17.co.uk
wrote:
The current system works fine, as far as I am concerned. The import is
a
one-way process, and doesn't touch the material on the source wiki.
This is
as it should be, as there will be situations where you are trying to
copy,
not move the material. Currently you do the import and, if the content
is
being moved, login to the remote wiki and either delete the page or add
a
soft redirect. There should definitely _not_ be a mechanism for remotely deleting wiki content from a remote wiki!
Why not? If I'm an admin on that remote wiki, it might be handy when moving images to Commons. At the moment of writing this mail, there are 657,331 media files on en.wikipedia. If 1/3 of them (rough guess) are suitable for Commons, that means >200,000 manual deletions when moving. If an api would allow image deletions by admins, it would save a lot of unneccessary work, without compromising safety (the images are kept, and the api could also offer undelete).
Because you are changing the trust model. PersonA is an admin on WikiA and has been trusted enough to delete articles and to perform a transwiki import. WikiB has not made PersonA an admin - they (implicitly) do not trust PersonA to delete articles. However, PersonA is allowed to delete articles by using the transwiki import + delete.
Admins at WikiA have full delete rights at WikiB, even if they are non-admins and explicitly blocked at WikiB!
I understand that you qualified your statement with "If I'm an admin on that remote wiki...", but I don't see how that can be done without explicitly requiring a username/password each time a user wants to delete (plus, you are then giving your remote details to the local wiki sysadmin - an unscrupulous sysadmin or maybe even a badly behaved extension would be able to access these).
- Mark Clements (HappyDog)
Admins at WikiA have full delete rights at WikiB, even if they are non-admins and explicitly blocked at WikiB!
As I understand this feature in no way would give deletion rights of WikiA at WikiB, they would still delete the file only at WikiA, regardless of the interface they use.
I understand that you qualified your statement with "If I'm an admin on that
remote wiki...", but I don't see how that can be done without explicitly requiring a username/password each time a user wants to delete (plus, you are then giving your remote details to the local wiki sysadmin - an unscrupulous sysadmin or maybe even a badly behaved extension would be able to access these).
Shouldn't single login (minus username conflicts), allow such identification?
Bence
On 12/19/06, Bence Damokos bdamokos@gmail.com wrote:
Admins at WikiA have full delete rights at WikiB, even if they are non-admins and explicitly blocked at WikiB!
As I understand this feature in no way would give deletion rights of WikiA at WikiB, they would still delete the file only at WikiA, regardless of the interface they use.
Exactly. You'll need upload rights at WikiB to upload/import there, and admin rights at WikiA to delete there.
The only potential problem could be from an evil admin who deletes thousands of images through the api. But they'd all be recoverable (maybe through the api); also, we could limit the number/speed of api-based deletions.
Magnus
"Magnus Manske" magnusmanske@googlemail.com wrote in message news:fab0ecb70612201329t76e9d930l5b2034effa86cb57@mail.gmail.com...
On 12/19/06, Bence Damokos
bdamokos@gmail.com wrote:
Admins at WikiA have full delete rights at WikiB, even if they are non-admins and explicitly blocked at WikiB!
As I understand this feature in no way would give deletion rights of
WikiA
at WikiB, they would still delete the file only at WikiA, regardless of
the
interface they use.
Exactly. You'll need upload rights at WikiB to upload/import there, and admin rights at WikiA to delete there.
Yes, but the transwiki is an _import_, not an _export_ process (i.e. you perform the operation from the target wiki), so how can this process perform the remote delete without compromising security?
Perhaps I do not understand how transwiki works. Is it possible for me, on my private wiki, to set up WP as an import source for transwiki?
- Mark Clements (HappyDog)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Mark Clements wrote:
Yes, but the transwiki is an _import_, not an _export_ process (i.e. you perform the operation from the target wiki), so how can this process perform the remote delete without compromising security?
Perhaps I do not understand how transwiki works. Is it possible for me, on my private wiki, to set up WP as an import source for transwiki?
Yes.
- -- brion vibber (brion @ pobox.com)
"Brion Vibber" brion@pobox.com wrote in message news:458B072A.6010108@pobox.com...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Mark Clements wrote:
Yes, but the transwiki is an _import_, not an _export_ process (i.e. you perform the operation from the target wiki), so how can this process
perform
the remote delete without compromising security?
Perhaps I do not understand how transwiki works. Is it possible for me,
on
my private wiki, to set up WP as an import source for transwiki?
Yes.
Well, in that case it would be very dangerous if the import option allowed importers to delete from the source wiki, and single-user sign-on does not help in this case. Either the user isn't verified, in which case users of the remote wiki can delete WP content willy nilly (even if not registered), or the importer has to enter their Wikipedia username & password at the target wiki, and thus give these details to the administrator of that wiki (or a malicious extension writer).
- Mark Clements (HappyDog)
On 22/12/06, Mark Clements gmane@kennel17.co.uk wrote:
Well, in that case it would be very dangerous if the import option allowed importers to delete from the source wiki, and single-user sign-on does not
[snip]
The problem is that someone took the "let's support true image importing" ball and ran with it, off to the "ooh, let's enable moving" camp.
I am of the opinion that while making the import process easier for Commons admins is going to end up being more or less essential, it is just as essential that a human has the final say over whether or not the image is deleted from the source wiki.
There are cases where you wouldn't want to be performing a *move*, anyway; the import process as it stands implies copying, rather than moving, and that is I how I think it should remain.
Rob Church
On 12/22/06, Rob Church robchur@gmail.com wrote:
On 22/12/06, Mark Clements gmane@kennel17.co.uk wrote:
Well, in that case it would be very dangerous if the import option allowed importers to delete from the source wiki, and single-user sign-on does not
[snip]
The problem is that someone took the "let's support true image importing" ball and ran with it, off to the "ooh, let's enable moving" camp.
I am of the opinion that while making the import process easier for Commons admins is going to end up being more or less essential, it is just as essential that a human has the final say over whether or not the image is deleted from the source wiki.
I don't think anyone suggested someting different.
There are cases where you wouldn't want to be performing a *move*, anyway; the import process as it stands implies copying, rather than moving, and that is I how I think it should remain.
Well, there will be tons of cases where you *do* want to move images, and for these cases, the subsequent deletion after copying should be as pain- and clickless as possible.
Magnus
On 12/21/06, Mark Clements gmane@kennel17.co.uk wrote:
Well, in that case it would be very dangerous if the import option allowed importers to delete from the source wiki, and single-user sign-on does not help in this case. Either the user isn't verified, in which case users of the remote wiki can delete WP content willy nilly (even if not registered), or the importer has to enter their Wikipedia username & password at the target wiki, and thus give these details to the administrator of that wiki (or a malicious extension writer).
I don't understand the difficulty.
1) You log in on Wiki A. Your username and password are the same as on Wiki B, since SUL is implemented, so no security breach occurs.
2) You say, through Wiki A's interface, that you would like to delete an image from Wiki B. Wiki A passes your username and password to Wiki B for authentication; since they're the same on both wikis, Wiki B will accept them and check if you're a sysop on Wiki B.
3) If you are, the image is deleted, and Wiki B tells Wiki A to acknowledge the deletion. Otherwise it tells Wiki A to return an error.
There can't be any security breach if the two wikis share the same database for usernames and passwords (i.e., SUL). If they don't, this doesn't have to work, but if it did, you could directly (but invisibly) connect to Wiki A and give it your cookie if you're already logged in there.
SUL won't help with local wikis (non-wp), so 1) simply isn't true.
-- chris
wikitech-l-bounces@wikimedia.org schrieb am 22.12.2006 03:08:46:
On 12/21/06, Mark Clements gmane@kennel17.co.uk wrote:
Well, in that case it would be very dangerous if the import option
allowed
importers to delete from the source wiki, and single-user sign-on does
not
help in this case. Either the user isn't verified, in which case users
of
the remote wiki can delete WP content willy nilly (even if not
registered),
or the importer has to enter their Wikipedia username & password at
the
target wiki, and thus give these details to the administrator of that
wiki
(or a malicious extension writer).
I don't understand the difficulty.
- You log in on Wiki A. Your username and password are the same as
on Wiki B, since SUL is implemented, so no security breach occurs.
- You say, through Wiki A's interface, that you would like to delete
an image from Wiki B. Wiki A passes your username and password to Wiki B for authentication; since they're the same on both wikis, Wiki B will accept them and check if you're a sysop on Wiki B.
- If you are, the image is deleted, and Wiki B tells Wiki A to
acknowledge the deletion. Otherwise it tells Wiki A to return an error.
There can't be any security breach if the two wikis share the same database for usernames and passwords (i.e., SUL). If they don't, this doesn't have to work, but if it did, you could directly (but invisibly) connect to Wiki A and give it your cookie if you're already logged in there. _______________________________________________ Wikitech-l mailing list Wikitech-l@wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
On 12/22/06, christoph.huesler@css.ch christoph.huesler@css.ch wrote:
SUL won't help with local wikis (non-wp), so 1) simply isn't true.
Random wikis will not be able to delete anything from Wikipedia under this scheme. Only wikis linked to it through SUL would be able to. I should have thought that was obvious from my clearly-stated premises. I don't think allowing admins of Uncyclopedia (or any other random wiki) who are also Wikipedia admins to delete remotely from Wikipedia would be terribly useful. Allowing Commons/Wikipedia admins (or just Wikipedia admins who are autoconfirmed on Commons) to delete remotely from Wikipedia when moving to Commons would be, and that's what this feature is aimed at.
"Simetrical" Simetrical+wikitech@gmail.com wrote in message news:7c2a12e20612220747s9627c23g4d7a7199ea9a638a@mail.gmail.com...
On 12/22/06, christoph.huesler@css.ch
christoph.huesler@css.ch wrote:
SUL won't help with local wikis (non-wp), so 1) simply isn't true.
Random wikis will not be able to delete anything from Wikipedia under this scheme. Only wikis linked to it through SUL would be able to. I should have thought that was obvious from my clearly-stated premises.
Except your premises were wrong. "Your username and password are the same as on Wiki B, since SUL is implemented" - this is not the case anywhere, and will never be the case in the example that I gave (importing WP content into a private wiki).
However, had you specified these as *requirements* for the feature, then that is a different matter (see below).
I don't think allowing admins of Uncyclopedia (or any other random wiki) who are also Wikipedia admins to delete remotely from Wikipedia would be terribly useful. Allowing Commons/Wikipedia admins (or just Wikipedia admins who are autoconfirmed on Commons) to delete remotely from Wikipedia when moving to Commons would be, and that's what this feature is aimed at.
If the feature is restricted to only work on wikis that share a single login then the security risk is removed. It may also simplify the code, e.g. instead of having to pass the images via XML/http it could perhaps copy them directly on the disk. My point was that in situations where SUL isn't available (e.g. current WM-wikis, or non-WM wikis in general) then there is a big risk in opening up this kind of functionality.
- Mark Clements (HappyDog)
On 12/23/06, Mark Clements gmane@kennel17.co.uk wrote:
If the feature is restricted to only work on wikis that share a single login then the security risk is removed. It may also simplify the code, e.g. instead of having to pass the images via XML/http it could perhaps copy them directly on the disk. My point was that in situations where SUL isn't available (e.g. current WM-wikis, or non-WM wikis in general) then there is a big risk in opening up this kind of functionality.
The most obvious application (and probably the only significant application) of this feature (deleting the file from the source wiki) would be for transwikiing media to Commons from other Wikimedia projects. As such I think it would be safe to confine the feature to our specific conditions, which include a central location for media files, regardless of which wiki they are on (so requiring no moving of files), and in the near future SUL.
wikitech-l@lists.wikimedia.org