Hello to all tool maintainers!
It was good to see several of you in person this weekend! Faces ftw! :)
Here is a status update with what is now working in Tool Labs:
First of all (looking at DaB.s survey and your comments): You _don't_ have to care about virtual instances, puppet or any of this. This is underneath/behind Tool Labs but you can just use the login server without having to know about it.
What you get: * login server made to resemble the toolserver * database replicas from s1-s6, s7 is following in June. Access to all data you can access as a registered, unprivileged user. * possibility to create your own user databases * shared project storage, home directories (nfs) * possibility to add maintainers to your project via the web interface on http://tools.wmflabs.org/ * possibility to restore your files to the state they were in 1, 2, 3 hours ago, 1, 2, 3 days ago and on the last 2 Sundays. * infrastructure to run web services * job system: OpenGridengine * two different environments to develop tools in: "Tools" project which is a stable environment, "Bots" project which can be used for experiments with the environment, root access is possible in the latter. * bug tracking in WMF's Bugzilla (product: Wikimedia Labs, components: bots and tools for the two projects) * more stuff that is mentioned in the road map (link below).
What we are still on: * documentation (let's crowdsource this!) * make Merlissimo's MerlLinkBot hunt down deprecated weblinks and point them to your tools' new URL.
Looking for things you can help with? * Write down you experiences like some already did: Magnus Manske: https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolse... or Russell Blau: https://wikitech.wikimedia.org/wiki/User:Russell_Blau/Using_pywikibot_on_Lab... * Help to improve the user interface of tools.wmflabs.org! Contact Coren/Marc-Andre Pelletier.
Links: Tool Labs help: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help FAQ (ongoing collection of what you ask): http://www.mediawiki.org/wiki/Wikimedia_Labs/Migration_of_Toolserver_tools Road map: http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_en (English) and http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_de (German)
What's next? Please come and try out the new database replication! If your tool is up and running in Tool Labs, _please consider documenting_ what you missed in the docs. By doing so you will help others a lot! Don't hesitate to ask! The best places to ask questions: * IRC (#wikimedia-labs), where Coren and petan (as Tool Labs admins) and many other experienced people hang around * the list labs-l (https://lists.wikimedia.org/mailman/listinfo/labs-l)
Do your tools need more adaptations than you think you can do? Wouldn't mind a helping hand? Ask me and I'll try to find you a helper!
Best, Silke
A bot to update URLs is a nice thing (except that some stupid wikis will surely block it), but a true solution would be setting up a system where redirects from toolserver to new URLs can be set up without too much pain.
Nemo
.htaccess is your friend :-)
On Tue, May 28, 2013 at 7:59 AM, Federico Leva (Nemo) nemowiki@gmail.comwrote:
A bot to update URLs is a nice thing (except that some stupid wikis will surely block it), but a true solution would be setting up a system where redirects from toolserver to new URLs can be set up without too much pain.
Nemo
______________________________**_________________ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.**orgToolserver-l@lists.wikimedia.org ) https://lists.wikimedia.org/**mailman/listinfo/toolserver-lhttps://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/** view/Mailing_list_etiquettehttps://wiki.toolserver.org/view/Mailing_list_etiquette
Magnus Manske magnusmanske@googlemail.com wrote:
.htaccess is your friend :-)
[...]
To be a bit more explicit :-):
| Redirect /~$USERONTOOLSERVER/ http://tools.wmflabs.org/$TOOLONTOOLS/
in ~/public_html/.htaccess should redirect all requests from the former to the latter while preserving the rest of the URL, i. e. http://toolserver.org/~$USERONTOOLSERVER/abc.php?def=ghi gets redirected to http://tools.wmflabs.org/$TOOLONTOOLS/abc.php?def=ghi. You can limit this to subdirectories or single files as well.
Every few days, you can then log into ortelius and wolfsbane and grep /var/log/http/access for $USERONTOOLSERVER. You should see requests with status (sixth field) 302, and the eighth field is the referring page.
Tim
On May 28, 2013 1:53 PM, "Tim Landscheidt" tim@tim-landscheidt.de wrote:
Magnus Manske magnusmanske@googlemail.com wrote:
.htaccess is your friend :-)
[...]
To be a bit more explicit :-):
| Redirect /~$USERONTOOLSERVER/ http://tools.wmflabs.org/$TOOLONTOOLS/
probably better to use Alias. (at least for most people)
-Jeremy
Jeremy Baron jeremy@tuxmachine.com wrote:
.htaccess is your friend :-)
[...]
To be a bit more explicit :-):
| Redirect /~$USERONTOOLSERVER/ http://tools.wmflabs.org/$TOOLONTOOLS/
probably better to use Alias. (at least for most people)
Eh, how? The Alias directive seems to allow only mapping to the local filesystem (http://support.zeus.com/zws/media/docs/ZWSUserGuide.pdf, page 361; http://httpd.apache.org/docs/current/mod/mod_alias.html#alias as well).
Tim
On May 28, 2013 2:20 PM, "Tim Landscheidt" tim@tim-landscheidt.de wrote:
Jeremy Baron jeremy@tuxmachine.com wrote:
.htaccess is your friend :-)
[...]
To be a bit more explicit :-):
| Redirect /~$USERONTOOLSERVER/ http://tools.wmflabs.org/$TOOLONTOOLS/
probably better to use Alias. (at least for most people)
Eh, how? The Alias directive seems to allow only mapping to the local filesystem (http://support.zeus.com/zws/media/docs/ZWSUserGuide.pdf, page 361; http://httpd.apache.org/docs/current/mod/mod_alias.html#alias as well).
I guess I meant Redirect (part of mod_alias as is Alias). (Don't have a shell handy here on my phone and haven't looked at the docs for a few weeks)
Would .htaccess work also for expired accounts then, and could operators/roots do something about them? We still have tons of users entering, say, interiot and soxred's tools and getting slightly unhelpful results.
Nemo
Are they up somewhere else? Otherwise, redirects won't be useful.
I tried to have a look at interiot's tools, but both web and shell access are blocked. There might be volunteers to port abandoned tools to labs (as I did for [1]), but only if they are still sought-after, and if the source code is accessible and properly licensed.
[1] http://tools.wmflabs.org/joanjoc/
On Fri, May 31, 2013 at 9:19 AM, Federico Leva (Nemo) nemowiki@gmail.comwrote:
Would .htaccess work also for expired accounts then, and could operators/roots do something about them? We still have tons of users entering, say, interiot and soxred's tools and getting slightly unhelpful results.
Nemo
______________________________**_________________ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.**orgToolserver-l@lists.wikimedia.org ) https://lists.wikimedia.org/**mailman/listinfo/toolserver-lhttps://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/** view/Mailing_list_etiquettehttps://wiki.toolserver.org/view/Mailing_list_etiquette
"Federico Leva (Nemo)" nemowiki@gmail.com wrote:
Would .htaccess work also for expired accounts then, and could operators/roots do something about them? We still have tons of users entering, say, interiot and soxred's tools and getting slightly unhelpful results.
The "User account expired" messages are generated by https://svn.toolserver.org/svnroot/puppet/trunk/puppet/modules/webserver/fil... which replaces the bottom of /etc/htaccess. It's certainly possible to exempt some user accounts and add special redi- rects for them. The bigger problem for a move of abandoned tools to /anywhere/ is however, as Magnus said, if the source code is properly licensed. soxred93 for example has set a default license of MIT (but http://toolserver.org/~soxred93/ looks pretty empty), while interiot has no default license, but /home/interiot/public_html/cgi-bin/Tool1/wannabe_kate de- clares itself as "[[Public domain]]". Just redirecting to a working tool should pose no problems, though.
Tim
Database HOWTO: https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolse...
On Mon, May 27, 2013 at 4:27 PM, Silke Meyer silke.meyer@wikimedia.dewrote:
Hello to all tool maintainers!
It was good to see several of you in person this weekend! Faces ftw! :)
Here is a status update with what is now working in Tool Labs:
First of all (looking at DaB.s survey and your comments): You _don't_ have to care about virtual instances, puppet or any of this. This is underneath/behind Tool Labs but you can just use the login server without having to know about it.
What you get:
- login server made to resemble the toolserver
- database replicas from s1-s6, s7 is following in June. Access to all
data you can access as a registered, unprivileged user.
- possibility to create your own user databases
- shared project storage, home directories (nfs)
- possibility to add maintainers to your project via the web interface
- possibility to restore your files to the state they were in 1, 2, 3
hours ago, 1, 2, 3 days ago and on the last 2 Sundays.
- infrastructure to run web services
- job system: OpenGridengine
- two different environments to develop tools in: "Tools" project
which is a stable environment, "Bots" project which can be used for experiments with the environment, root access is possible in the latter.
- bug tracking in WMF's Bugzilla (product: Wikimedia Labs, components:
bots and tools for the two projects)
- more stuff that is mentioned in the road map (link below).
What we are still on:
- documentation (let's crowdsource this!)
- make Merlissimo's MerlLinkBot hunt down deprecated weblinks and
point them to your tools' new URL.
Looking for things you can help with?
- Write down you experiences like some already did: Magnus Manske:
https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolse... or Russell Blau:
https://wikitech.wikimedia.org/wiki/User:Russell_Blau/Using_pywikibot_on_Lab...
- Help to improve the user interface of tools.wmflabs.org! Contact
Coren/Marc-Andre Pelletier.
Links: Tool Labs help: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help FAQ (ongoing collection of what you ask): http://www.mediawiki.org/wiki/Wikimedia_Labs/Migration_of_Toolserver_tools Road map: http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_en (English) and http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_de (German)
What's next? Please come and try out the new database replication! If your tool is up and running in Tool Labs, _please consider documenting_ what you missed in the docs. By doing so you will help others a lot! Don't hesitate to ask! The best places to ask questions:
- IRC (#wikimedia-labs), where Coren and petan (as Tool Labs admins)
and many other experienced people hang around
- the list labs-l (https://lists.wikimedia.org/mailman/listinfo/labs-l)
Do your tools need more adaptations than you think you can do? Wouldn't mind a helping hand? Ask me and I'll try to find you a helper!
Best, Silke
-- Silke Meyer Internes IT-Management und Projektmanagement Toolserver
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin Tel. (030) 219 158 260
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Cool, thanks for documenting, Magnus! Cheers, Silke
2013/5/28 Magnus Manske magnusmanske@googlemail.com:
Database HOWTO: https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolse...
On Mon, May 27, 2013 at 4:27 PM, Silke Meyer silke.meyer@wikimedia.de wrote:
Hello to all tool maintainers!
It was good to see several of you in person this weekend! Faces ftw! :)
Here is a status update with what is now working in Tool Labs:
First of all (looking at DaB.s survey and your comments): You _don't_ have to care about virtual instances, puppet or any of this. This is underneath/behind Tool Labs but you can just use the login server without having to know about it.
What you get:
- login server made to resemble the toolserver
- database replicas from s1-s6, s7 is following in June. Access to all
data you can access as a registered, unprivileged user.
- possibility to create your own user databases
- shared project storage, home directories (nfs)
- possibility to add maintainers to your project via the web interface
- possibility to restore your files to the state they were in 1, 2, 3
hours ago, 1, 2, 3 days ago and on the last 2 Sundays.
- infrastructure to run web services
- job system: OpenGridengine
- two different environments to develop tools in: "Tools" project
which is a stable environment, "Bots" project which can be used for experiments with the environment, root access is possible in the latter.
- bug tracking in WMF's Bugzilla (product: Wikimedia Labs, components:
bots and tools for the two projects)
- more stuff that is mentioned in the road map (link below).
What we are still on:
- documentation (let's crowdsource this!)
- make Merlissimo's MerlLinkBot hunt down deprecated weblinks and
point them to your tools' new URL.
Looking for things you can help with?
- Write down you experiences like some already did: Magnus Manske:
https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolse... or Russell Blau:
https://wikitech.wikimedia.org/wiki/User:Russell_Blau/Using_pywikibot_on_Lab...
- Help to improve the user interface of tools.wmflabs.org! Contact
Coren/Marc-Andre Pelletier.
Links: Tool Labs help: https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help FAQ (ongoing collection of what you ask): http://www.mediawiki.org/wiki/Wikimedia_Labs/Migration_of_Toolserver_tools Road map: http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_en (English) and http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_de (German)
What's next? Please come and try out the new database replication! If your tool is up and running in Tool Labs, _please consider documenting_ what you missed in the docs. By doing so you will help others a lot! Don't hesitate to ask! The best places to ask questions:
- IRC (#wikimedia-labs), where Coren and petan (as Tool Labs admins)
and many other experienced people hang around
- the list labs-l (https://lists.wikimedia.org/mailman/listinfo/labs-l)
Do your tools need more adaptations than you think you can do? Wouldn't mind a helping hand? Ask me and I'll try to find you a helper!
Best, Silke
-- Silke Meyer Internes IT-Management und Projektmanagement Toolserver
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin Tel. (030) 219 158 260
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Is there any how-to on moving SVN repo. along with it's history to TL? Is it possible/planned at all?
Also could you clarify on "...WMF doesn’t provide SVN hosting, but volunteers can create an SVN project in Labs.". I can create a repo in my home dir or what? How would I access it from my machine then? Can I just enable and configure WEBDAV on some public_html dir?
Yes I know Git is superior and all that, but I don't want to get into the discussion of history on a full tree versus history on a file. I just want to know if I will be able to use SVN on your servers and just switch my local directory without any needless conversions.
Regards, Nux.
Hello, At Saturday 01 June 2013 02:19:13 DaB. wrote:
Is there any how-to on moving SVN repo. along with it's history to TL? Is it possible/planned at all?
the Toolserver stores the different repos in different directories so it is no problem from our side to give you the files when you leave. But I have no idea if TL can import it and who could do that.
Sincerely, DaB.
Maciej Jaros egil@wp.pl wrote:
Is there any how-to on moving SVN repo. along with it's history to TL? Is it possible/planned at all?
Also could you clarify on "...WMF doesn’t provide SVN hosting, but volunteers can create an SVN project in Labs.". I can create a repo in my home dir or what? How would I access it from my machine then? Can I just enable and configure WEBDAV on some public_html dir?
As DaB. explained, the roots can provide you with a dump of your SVN repository. You can then import ("load") this into a repository in your home directory and access it via ssh (this seems to be possible from Windows as well, cf. http://sebastian.thiele.me/blog/tortoise-svn-putty-ssh-windows/2438 (German)). If you create a dummy tool and put the reposi- tory in the tool directory, all users in the tool group should be able to write to it as well.
You could also import the dump for example to SourceForge (cf. http://sourceforge.net/apps/trac/sourceforge/wiki/Subversion%20import%20inst...) and probably other hosters.
I don't know of any plans regarding WebDAV.
Yes I know Git is superior and all that, but I don't want to get into the discussion of history on a full tree versus history on a file. I just want to know if I will be able to use SVN on your servers and just switch my local directory without any needless conversions.
You can get the history of a file with "git log FILENAME"; if you want to follow the history through renames, use "git log --follow FILENAME".
Tim
On 2 June 2013 21:12, Tim Landscheidt tim@tim-landscheidt.de wrote:
Maciej Jaros egil@wp.pl wrote:
Is there any how-to on moving SVN repo. along with it's history to TL? Is it possible/planned at all?
As DaB. explained, the roots can provide you with a dump of your SVN repository. You can then import ("load") this into a repository in your home directory and access it via ssh
Note: you can also use svnsync to do this from your own computer - see https://github.com/pywikibot/svn2git/blob/master/scripts-pwb/sync for an example.
Merlijn
Slightly OT: Sorry if I missed that in the discussion, but is there a for-dummies howto for creating a labs git for the ported tool? I know they use gerrit, but that's the extend of my knowledge.
On Sun, Jun 2, 2013 at 9:22 PM, Merlijn van Deen valhallasw@arctus.nlwrote:
On 2 June 2013 21:12, Tim Landscheidt tim@tim-landscheidt.de wrote:
Maciej Jaros egil@wp.pl wrote:
Is there any how-to on moving SVN repo. along with it's history to TL? Is it possible/planned at all?
As DaB. explained, the roots can provide you with a dump of your SVN repository. You can then import ("load") this into a repository in your home directory and access it via ssh
Note: you can also use svnsync to do this from your own computer - see https://github.com/pywikibot/svn2git/blob/master/scripts-pwb/sync for an example.
Merlijn
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
---------- Původní zpráva ---------- Od: Magnus Manske magnusmanske@googlemail.com Datum: 2. 6. 2013 Předmět: Re: [Toolserver-l] Status update on Tool Labs
"
Slightly OT: Sorry if I missed that in the discussion, but is there a for-dummies howto for creating a labs git for the ported tool? I know they use gerrit, but that's the extend of my knowledge.
"
I've actually been told during Amsterdam hackathon, that you can choose if you want to use plain git or gerrit for your tools, if that helps a bit...
Danny B.
toolserver-l@lists.wikimedia.org