Hi,
I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked).
I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons.
I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them.
I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick.
I see that it is in /scripts/archive/
Is this still a viable script - or is it deprecated in some way?
Does anyone have a guide for using it beyond the comments at the top of the script?
I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually.
Any suggestions (including ‘don’t do this’) with explanations would be welcome please.
Many thanks
Ian
Ian Watt ianwatt@gmail.com
watty62
Hi Ian,
data_ingestion.py is still available but is not tested. There might be some breaking changes since 5.6 and 7.0 which can cause the script failing.
Either you have to pretend the archive folder when calling the script or add the path to user_script_paths in your user-config.py [1].
You cannot go back to an older Pywikibot version (pre 6.0) because Pywikibot 6.6.1 is required for the current MW release used at commons. [2]
Possibly I find some time to recover the script shortly.
Best xqt
[1] https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?hig...
[2] https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility
Am 27.01.2022 um 17:07 schrieb Ian Watt ianwatt@gmail.com:
Hi,
I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked).
I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons.
I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them.
I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick.
I see that it is in /scripts/archive/
Is this still a viable script - or is it deprecated in some way?
Does anyone have a guide for using it beyond the comments at the top of the script?
I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually.
Any suggestions (including ‘don’t do this’) with explanations would be welcome please.
Many thanks
Ian
Ian Watt ianwatt@gmail.com
watty62
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
Thank you, xqt
That’s really helpful.
Ian
Ian Watt ianwatt@gmail.com
On 28 Jan 2022, at 06:13, info@gno.de wrote:
Hi Ian,
data_ingestion.py is still available but is not tested. There might be some breaking changes since 5.6 and 7.0 which can cause the script failing.
Either you have to pretend the archive folder when calling the script or add the path to user_script_paths in your user-config.py [1].
You cannot go back to an older Pywikibot version (pre 6.0) because Pywikibot 6.6.1 is required for the current MW release used at commons. [2]
Possibly I find some time to recover the script shortly.
Best xqt
[1] https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?hig... https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?highlight=config#external-script-path-settings
[2] https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility
Am 27.01.2022 um 17:07 schrieb Ian Watt ianwatt@gmail.com:
Hi,
I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked).
I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons.
I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them.
I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick.
I see that it is in /scripts/archive/
Is this still a viable script - or is it deprecated in some way?
Does anyone have a guide for using it beyond the comments at the top of the script?
I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually.
Any suggestions (including ‘don’t do this’) with explanations would be welcome please.
Many thanks
Ian
Ian Watt ianwatt@gmail.com mailto:ianwatt@gmail.com
watty62
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
Hi Ian,
I’ve recovered this script. Are you able to review it? [1] There is a known issue due to a wrong migration from compat to core but the script should work anyway.
Best xqt
[1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/757889
Am 28.01.2022 um 11:15 schrieb Ian Watt ianwatt@gmail.com:
Thank you, xqt
That’s really helpful.
Ian
Ian Watt ianwatt@gmail.com
On 28 Jan 2022, at 06:13, info@gno.de wrote:
Hi Ian,
data_ingestion.py is still available but is not tested. There might be some breaking changes since 5.6 and 7.0 which can cause the script failing.
Either you have to pretend the archive folder when calling the script or add the path to user_script_paths in your user-config.py [1].
You cannot go back to an older Pywikibot version (pre 6.0) because Pywikibot 6.6.1 is required for the current MW release used at commons. [2]
Possibly I find some time to recover the script shortly.
Best xqt
[1] https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?hig...
[2] https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility
Am 27.01.2022 um 17:07 schrieb Ian Watt ianwatt@gmail.com:
Hi,
I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked).
I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons.
I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them.
I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick.
I see that it is in /scripts/archive/
Is this still a viable script - or is it deprecated in some way?
Does anyone have a guide for using it beyond the comments at the top of the script?
I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually.
Any suggestions (including ‘don’t do this’) with explanations would be welcome please.
Many thanks
Ian
Ian Watt ianwatt@gmail.com
watty62
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
Hi xqt
Thanks for doing that.
Do you mean review it from a code-review standpoint? Perhaps I over-stated my Python competence!?
Of do you mean try to test the script from a real-world attempt to upload files with it? I guess I’d need to download the file as it is amended (some sort of git pull? ) and set it in my version of PWB on my computer?
Sorry if this sounds ignorant!
Ian
Ian Watt ianwatt@gmail.com
On 28 Jan 2022, at 13:38, info@gno.de wrote:
Hi Ian,
I’ve recovered this script. Are you able to review it? [1] There is a known issue due to a wrong migration from compat to core but the script should work anyway.
Best xqt
[1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/757889
Am 28.01.2022 um 11:15 schrieb Ian Watt ianwatt@gmail.com:
Thank you, xqt
That’s really helpful.
Ian
Ian Watt ianwatt@gmail.com mailto:ianwatt@gmail.com
On 28 Jan 2022, at 06:13, info@gno.de mailto:info@gno.de wrote:
Hi Ian,
data_ingestion.py is still available but is not tested. There might be some breaking changes since 5.6 and 7.0 which can cause the script failing.
Either you have to pretend the archive folder when calling the script or add the path to user_script_paths in your user-config.py [1].
You cannot go back to an older Pywikibot version (pre 6.0) because Pywikibot 6.6.1 is required for the current MW release used at commons. [2]
Possibly I find some time to recover the script shortly.
Best xqt
[1] https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?hig... https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?highlight=config#external-script-path-settings
[2] https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility
Am 27.01.2022 um 17:07 schrieb Ian Watt <ianwatt@gmail.com mailto:ianwatt@gmail.com>:
Hi,
I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked).
I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons.
I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them.
I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick.
I see that it is in /scripts/archive/
Is this still a viable script - or is it deprecated in some way?
Does anyone have a guide for using it beyond the comments at the top of the script?
I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually.
Any suggestions (including ‘don’t do this’) with explanations would be welcome please.
Many thanks
Ian
Ian Watt ianwatt@gmail.com mailto:ianwatt@gmail.com
watty62
pywikibot mailing list -- pywikibot@lists.wikimedia.org mailto:pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org mailto:pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org mailto:pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org mailto:pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
Hi Ian,
a real test would be fine and enough for me. There are automatic tests for the code. You can copy the script from here [1]. To prevent real changes you may use the -simulate option when calling the script.
Best xqt
[1] https://gerrit.wikimedia.org/r/plugins/gitiles/pywikibot/core/+/8b024efbe6d2...
Am 29.01.2022 um 18:00 schrieb Ian Watt ianwatt@gmail.com:
Hi xqt
Thanks for doing that.
Do you mean review it from a code-review standpoint? Perhaps I over-stated my Python competence!?
Of do you mean try to test the script from a real-world attempt to upload files with it? I guess I’d need to download the file as it is amended (some sort of git pull? ) and set it in my version of PWB on my computer?
Sorry if this sounds ignorant!
Ian
Ian Watt ianwatt@gmail.com
On 28 Jan 2022, at 13:38, info@gno.de wrote:
Hi Ian,
I’ve recovered this script. Are you able to review it? [1] There is a known issue due to a wrong migration from compat to core but the script should work anyway.
Best xqt
[1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/757889
Am 28.01.2022 um 11:15 schrieb Ian Watt ianwatt@gmail.com:
Thank you, xqt
That’s really helpful.
Ian
Ian Watt ianwatt@gmail.com
On 28 Jan 2022, at 06:13, info@gno.de wrote:
Hi Ian,
data_ingestion.py is still available but is not tested. There might be some breaking changes since 5.6 and 7.0 which can cause the script failing.
Either you have to pretend the archive folder when calling the script or add the path to user_script_paths in your user-config.py [1].
You cannot go back to an older Pywikibot version (pre 6.0) because Pywikibot 6.6.1 is required for the current MW release used at commons. [2]
Possibly I find some time to recover the script shortly.
Best xqt
[1] https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?hig...
[2] https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility
Am 27.01.2022 um 17:07 schrieb Ian Watt ianwatt@gmail.com:
Hi,
I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked).
I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons.
I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them.
I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick.
I see that it is in /scripts/archive/
Is this still a viable script - or is it deprecated in some way?
Does anyone have a guide for using it beyond the comments at the top of the script?
I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually.
Any suggestions (including ‘don’t do this’) with explanations would be welcome please.
Many thanks
Ian
Ian Watt ianwatt@gmail.com
watty62
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
Hi xqt
Apologies. This fell off my radar.
With Pattypan becoming available again, it meant that the familiar route to uploading images was again available. So, trying to script something to upload 4000+ images in an unfamiliar environment (PWB) was no longer necessary.
I hope that I didn’t put you to to much trouble!
Ian
Ian Watt ianwatt@gmail.com
On 28 Jan 2022, at 13:38, info@gno.de wrote:
Hi Ian,
I’ve recovered this script. Are you able to review it? [1] There is a known issue due to a wrong migration from compat to core but the script should work anyway.
Best xqt
[1] https://gerrit.wikimedia.org/r/c/pywikibot/core/+/757889
Am 28.01.2022 um 11:15 schrieb Ian Watt ianwatt@gmail.com:
Thank you, xqt
That’s really helpful.
Ian
Ian Watt ianwatt@gmail.com mailto:ianwatt@gmail.com
On 28 Jan 2022, at 06:13, info@gno.de mailto:info@gno.de wrote:
Hi Ian,
data_ingestion.py is still available but is not tested. There might be some breaking changes since 5.6 and 7.0 which can cause the script failing.
Either you have to pretend the archive folder when calling the script or add the path to user_script_paths in your user-config.py [1].
You cannot go back to an older Pywikibot version (pre 6.0) because Pywikibot 6.6.1 is required for the current MW release used at commons. [2]
Possibly I find some time to recover the script shortly.
Best xqt
[1] https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?hig... https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.config.html?highlight=config#external-script-path-settings
[2] https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility
Am 27.01.2022 um 17:07 schrieb Ian Watt <ianwatt@gmail.com mailto:ianwatt@gmail.com>:
Hi,
I’m assisting our local Museums and Galleries on a project to open up around 4,000 images as CC-0) via Commons. We picked a bad time to do it (Pattypan being borked).
I’m a PyWikiBot noob - although I have some long-term familiarity with Python. I’m trying to work out if using PWB might be a route to get these images onto Commons.
I have both downloaded image files and URLs which I can point a script at - as well as good metadata for them.
I’ve been looking at pre-canned PWB scripts and see that data_ingestion *might* do the trick.
I see that it is in /scripts/archive/
Is this still a viable script - or is it deprecated in some way?
Does anyone have a guide for using it beyond the comments at the top of the script?
I had a look at /tests/data/csv_ingestion.csv and it looks kind of bare - as I’d expect more fields etc. I’d rather construct something more like the metadata fields that I’d use with Patypan if using that - rather than be faced with 4,000 files uploaded and have to add metadata to them in a separate process or *shudder* manually.
Any suggestions (including ‘don’t do this’) with explanations would be welcome please.
Many thanks
Ian
Ian Watt ianwatt@gmail.com mailto:ianwatt@gmail.com
watty62
pywikibot mailing list -- pywikibot@lists.wikimedia.org mailto:pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org mailto:pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org mailto:pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org mailto:pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org
there was no trouble with it and now you are prepared for the next failure 😄
Best xqt
Am 15.02.2022 um 10:54 schrieb Ian Watt ianwatt@gmail.com:
Hi xqt
Apologies. This fell off my radar.
With Pattypan becoming available again, it meant that the familiar route to uploading images was again available. So, trying to script something to upload 4000+ images in an unfamiliar environment (PWB) was no longer necessary.
I hope that I didn’t put you to to much trouble!
Hi xqt
Thank you so much.
I am delighted to say that I have my first bot running on PAWS doing simple text replacement.
And it works!
Thanks again.
Ian
On Tue, 15 Feb 2022, 19:04 , info@gno.de wrote:
there was no trouble with it and now you are prepared for the next failure 😄
Best xqt
Am 15.02.2022 um 10:54 schrieb Ian Watt ianwatt@gmail.com:
Hi xqt
Apologies. This fell off my radar.
With Pattypan becoming available again, it meant that the familiar route
to uploading images was again available. So, trying to script something to upload 4000+ images in an unfamiliar environment (PWB) was no longer necessary.
I hope that I didn’t put you to to much trouble!
pywikibot mailing list -- pywikibot@lists.wikimedia.org To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org