[Labs-l] RFC: Webtools setup

[Platonides]

On 17/02/13 05:12, Tim Landscheidt wrote:
> (anonymous) wrote:
>> Imho adding another layer of directory clutter (/web/) is a bad idea (I
>> already wrote that on Nova Resource Talk:Webtools). As well as is
>> /htdocs/cgi-bin/. It would be better to place all files into
>> /data/project/TOOL (use subdirectories if you want to). That would also
>> be the toolserver way (~USER/public_html respectively
>> ~PROJECT/public_html), as a side note.
> 
> I don't have any opinion on /data/project/TOOL vs.
> /data/project/web/TOOL :-).  But IIRC there are some differ-
> ences between a "normal" "public_html" and an explicit
> "cgi-bin", so a subdirectory "cgi-bin" is probably neces-
> sary.

I don't personally see the need of cgi-bin directories, but if a project
needs it, why not?


>> Nova Resource:Webtools also proposes the public entry-point address to
>> be http://web.wmflabs.org/TOOL. As I also said on the talk page, I would
>> prefer tool(s).wmflabs.org. ;) Unfortunately, that went unnoticed. :(
> 
>> [...]
> 
> Sounds good to me, *but* we need to check if we can have a
> catch-all DNS entry for *.wmflabs.org and still other ser-
> vers in this domain.

Wait-a-minute, you meant adding a dns entry *for each tool*??



>> I would prefer not to require an admin in the process of creating a
>> tool. It should be possible to do through eg. a privileged helper.
>> Although if it's a standarised change, maybe it could be merged
>> automatically by jenkins.
> 
> We do require admin interaction and manual "review" to add
> feeds to Planet Wikimedia, and still get turn-around times
> in the range of minutes, so I don't think that we should
> spend much thought in optimizing the creation times of new
> tools :-).

Well, perhaps if it was just a few minutes..., but  zero-time is still
infinitely faster. You don't want to put off that developer that awoke
at 5 am with a neat idea for a tool just because nobody was there to
approve its creation. :)



>> Probably, most projects wouldn't need to be in puppet.
> 
> Yes, of course, but Webtools is certainly one of the others
> as it will be used by dozens of developers and receive prob-
> ably a million hits per day.  It should be as robust as pos-
> sible from the get-go.  We should definitely learn from the
> Toolserver (or the earlier WMF cluster) where ex post docu-
> mentation of setup, including all hot fixes, to survive re-
> boots and updates is painful and pushes the admin workload
> *much* higher than necessary.

I was refering to tools projects. Of course that the setup for creating
a webtools apache or mysql should be puppetized. What I don't see the
need is for having a puppet entry for an editcounter. It just needs to
be in the proper folder of an instance for php tools (and remember that
project storage is automatically shared).


>>> I think LDAP accounts would open up lots of new security
>>> questions.  The users only need to exist on the instances of
>>> the Webtools project, and so local accounts should be
>>> enough.
> 
>> Those are not local accounts, since they should be shared through the
>> several instances of the webtools project.
> 
> I meant local as the opposite of LDAP.

Ok, I think we have too many similar terms. Maybe the first task for
Coren should be to make a glossary for tool labs discussions :)


>> I thought about setting up nis for that, but it would be better if those
>> uids were also handled by ldap.
> 
> What's the advantage of NIS?

It provides a “shared passwd”. See
http://en.wikipedia.org/wiki/Network_Information_Service
There's not a special advantage of using it instead of LDAP other than
"it manages global accounts" and "It's not LDAP", you already have
libraries for that and it'd be trivial to chain ldap and nis.



>> Not being on that project group, those 'users' couldn't join to other
>> projects. Maybe a problem would appear if we created an project with the
>> same name as used later by an ubuntu package. We could prefix with
>> "tool-" the tool names to avoid that, although that'd be uglier IMHO.
> 
> Collisions between LDAP and local (or instance-shared :-))
> users could be a problem, yes.  We need to find out if a lo-
> cal user created by Puppet shadows an LDAP one of the same
> name.
> 
> The idea of the prefix is neat and it may be easy to black-
> list just all usernames with this prefix from LDAP creation
> to avoid any collisions; I initially thought that we then
> would have to name the directories accordingly as well, but
> as we will probably heavily rewrite URLs in any case, this
> shouldn't be a problem that is visible to the public.
> 
> One more thing that came to my mind is sudo.  We need to
> allow developers to sudo as their tools.  "/etc/sudoers.d"
> should be ideal for that.
> 
> Tim

Yes. I don't know if the toolserver uses a sudo rule per project or if
it uses some kind of wildcard rule (is it even possible?). We just need a
 %toolname ALL=(toolname) NOPASSWD: ALL
rule per tool.
become(1) is only a wrapper over sudo.