[Labs-l] RFC: Webtools setup

[Platonides]

On 17/02/13 21:27, Tim Landscheidt wrote:
> (anonymous) wrote:
> 
>> [...]
> 
>>>> Nova Resource:Webtools also proposes the public entry-point address to
>>>> be http://web.wmflabs.org/TOOL. As I also said on the talk page, I would
>>>> prefer tool(s).wmflabs.org. ;) Unfortunately, that went unnoticed. :(
> 
>>>> [...]
> 
>>> Sounds good to me, *but* we need to check if we can have a
>>> catch-all DNS entry for *.wmflabs.org and still other ser-
>>> vers in this domain.
> 
>> Wait-a-minute, you meant adding a dns entry *for each tool*??
> 
> No, I mean if we have a wildcard DNS entry for
> *.wmflabs.org, we need to ensure that
> non-webtools-project-1.wmflabs.org and
> non-webtools-project-2.wmflabs.org are still reachable.

No, I thought you wanted tools.wmflabs.org/$toolname but now I'm
wondering if you wanted tools to be a variable,  ie.
{$toolname}.wmflabs.org (which I don't consider a good idea, better to
balance it from varnish)



>>>> I would prefer not to require an admin in the process of creating a
>>>> tool. It should be possible to do through eg. a privileged helper.
>>>> Although if it's a standarised change, maybe it could be merged
>>>> automatically by jenkins.
> 
>>> We do require admin interaction and manual "review" to add
>>> feeds to Planet Wikimedia, and still get turn-around times
>>> in the range of minutes, so I don't think that we should
>>> spend much thought in optimizing the creation times of new
>>> tools :-).
> 
>> Well, perhaps if it was just a few minutes..., but  zero-time is still
>> infinitely faster. You don't want to put off that developer that awoke
>> at 5 am with a neat idea for a tool just because nobody was there to
>> approve its creation. :)
> 
> You're still in Toolserver land where everything hinges on
> DaB.'s sleep cycle (and health, and time, and mood,
> and ... :-)).  At 5 AM anywhere on the world, there will be
> probably an admin, but certainly an experienced user on IRC
> who can either guide the developer how to submit the change-
> set or do it for him, and a few minutes later it can be live
> as WMF ops will probably be staffed 24/7.

That's in an ideal world. But there are weekends, and hours with more
people than others, and changesets where you forgot to add the right
person, and ops too busy with server managing to check the pending
changesets for wikimedia/labtools/ ...

I bet that it wouldn't be hard to send such changeset at a time where it
will take hours (if not days) to get approved.
Yes, there will be more than just DaB to do it, +2 to puppet is still
restricted to ops, so we are still talking about a few people where I
don't expect more than three paying attention to deployment in the
day-to-day (Ryan, Andrew and Coren). We have two in toolserver (DaB and
Marlen), and had three at some points in the past. It's not that
different, if we detected a bottleneck and can remove it, that's better
than just making it wider.



> IMVHO, though, if developers want to work in a collaborative
> environment, they have to collaborate.  If the world must
> stop just because they think so, how will they handle issues
> where they definitely need to wait for other humans?

No, they don't want that. Those volunteers want to solve for us a
problem that our software doesn't handle.
It is our task to make that process easy and nice.



>> [...]
> 
> We discussed this in the past at
> http://thread.gmane.org/gmane.org.wikimedia.toolserver.announce/485
> (though to no avail then :-)): Servers are added and/or get
> updated.  So you need to know the requirements of tools, and
> if packages are no longer shipped in future versions of
> Ubuntu, you need to find out which tool needed them so that
> you can decide if the tool can be changed or you have to
> package (and maintain) the software yourself.  If you have
> already packaged software locally, on an update where you
> can't rebuild the package at the first try, you want to see
> which tool needs it, so you can (again) decide which path of
> action is most appropriate.
>
> What *I* don't understand is the resistance to having a Pup-
> pet entry for an editcounter: It will do no harm, it isn't
> rocket science, but it adds information and eases adminis-
> tration.  When I shop at IKEA, I get asked for my postal
> code.  When I visit the Miniatur Wunderland, they ask me for
> my country or, speaking German, for my state.  They could
> speed up the checkout process if they didn't, and probably
> save thousands of euros on wages per year, but they prefer
> to have some actual data to having to act on the hunch:
> "Most of our customers are locals/Germans/Europeans/etc."
> And especially PHP is one of those languages where you need
> a lot of discipline in coding because there are practically
> no safeguards against shooting yourself in the foot.  I
> can't imagine a PHP developer who has written more than a
> line of code, but fails to fill out a Puppet template for
> PHP tools :-).

I see now what you mean. Yes, it would be good to keep track of that,
although I would prefer that it's stored in the same folder (repository)
as the tool, instead of a different repository (and preferably
enforced), to keep the dependencies with the tool.
If we enforce it to go through puppet, we risk that it makes the process
slow. If we make it work without the dependencies, some package may go
silently breaking something. In both cases, the listed dependencies may
get stale because the needed dependencies are fullfiled by other package.



> No, I meant what is the advantage of setting up a NIS server
> to "sharing passwd" using Puppet's users?
I'm seeing your point.


> "Marc A. Pelletier" <marc at uberbox.org> wrote:
> 
>> [...]
> 
>> Automating is a fixed cost; manual processing is an
>> unbounded cost against a finite resource.  :-)
> 
> That is (sort of) true, but if you focus on, say, a human's
> lifetime, the fixed cost can be much higher.  IIRC Matthias
> Wandel of woodgears.ca once commented on a suggestion for a
> quick release for one of his jigs that if it took him
> 15 minutes to build on, he would have to use it hundreds of
> time just to break even.
> 
> Also, "automating" can be very different from "automating".
> Take Planet Wikimedia as a more non-solid example: Creating
> a web form to add a URL to a planet is easy.  But if you re-
> ally want to cut out the human part of the process, you have
> test the URL that it is actually a link to an Atom/RSS feed.
> And that it is neither a link to the planet's feed itself.
> Also, you want to exclude spam sites.  So you add a review
> queue.  Now you need permissions for that queue.  And you
> need to protect this queue against DOS attacks.  Oh, and if
> anything on the business end of the planet changes, you have
> to adapt the ("fixed-cost") automation as well :-).
> 
> Tim

Or you could make it read a page from meta and leave that ‘manual’ task
to the meta admins :)