[Labs-l] Question: Is it allowed/okay to tunnel db query results to toolserver?

[Tim Landscheidt]

Silke Meyer <silke.meyer at wikimedia.de> wrote:

> I am trying to write down questions and answers I got in Amsterdam.
> But I didn't have answers for all of them, so here:

> 1) "If I run a proprietary software and I'm willing to change this but
> need some time for it can I use Tool Labs for my db queries and then
> tunnel the results to the toolserver where the actual tool is still
> running?"

You can, provided that you don't share your account with
others (Toolserver rule #7).  I think Tools roots don't
count as "another user".  The (draft of) Wikimedia Labs
Terms of Use
(cf. http://www.mediawiki.org/wiki/Wikimedia_Labs/Terms_of_use)
doesn't state whether you can store your ssh key on
Toolserver for the reverse direction.

Personally, I don't see why someone should rush to Tools
when they are clearly not ready to migrate.  Toolserver will
still be around for over a year.  If the replication lag is
the reason behind this question, the addition of the new
Toolserver admin and the ongoing migrations of tools should
cause that to drop over time.

> 2) What's the recommended way to develop tools as a group? If others
> want to modify my stuff but not the "original files", should they fork
> it? Should they create a branch?

In Git-speak, a fork and a branch aren't that much differ-
ent.  dbreps for example uses GitHub as a repository
(cf. https://github.com/mzmcbride/database-reports).
Developers can either push new changes to the master branch,
or post changes to a "pull request", that then can be merged
(or amended, or rejected) after discussion.  In the end,
there is always the Highlander principle: There can be only
one instance of the code that is actually running.

The more important point to collaborative development (and
for single-maintainer tools as well) is that the deployment
process has to be /documented/.  If the repository has a
nice "master" branch, but the actually deployed branch is
called "deployment", that needs to be written down some-
where.  If files from subdirectory "webstuff" need to be
copied to "~/public_html", and files from subdirectory
"data/app" need to be copied to "~/data-20130527", that
needs to be written down somewhere as well.  In most cases,
it's best to put this documentation in the form of a shell
script/Makefile/setup.py so that there is less ambiguity.
For dbreps, there is documentation at
https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Tools/dbreps
that might be of inspiration.

For deployment, the same basic rule as for backups applies:
Unless you have successfully checked that you can in fact
deploy the tool by following the documentation, you have no
documentation.

> I am collecting all this here:
> http://www.mediawiki.org/wiki/Wikimedia_Labs/Migration_of_Toolserver_tools
> The Labs and Tool Labs people might want to integrate it into the
> "official" docs somewhere...

Ceterum censeo: Any mention of Bots, especially such as:
"'Bots' is a more flexible environment where you can experi-
ment with changes in the environment itself for your tools",
will cause more confusion to developers planning to migrate
or create new tools.  I respect that demand for user support
correlates to job security for some, but as a volunteer I'd
like to keep unnecessary questions to a minimum, so that
more of the developers' effort can be spent on building and
maintaining great tools.

Tim