[Labs-l] RFC: Webtools setup

[Tim Landscheidt]

(anonymous) wrote:

> [...]

>> Dependencies of tools on other software are specified ex-
>> plicitely so that tools can be moved to other servers or
>> servers can be split by other software needed (i. e., a
>> server that only handles PHP, Ruby on Rails, etc.).  Depen-
>> dencies can be different for development (command line) and
>> deployment (web).

> This could be more complex to organise, since cli and web may still need
> to be grouped in the same tool.

Hmmm.  What do you mean?

> [...]

>>   In it, the subdirectory "htdocs" contains the web
>>   stuff ("htdocs/cgi-bin" for CGI), the rest of the direc-
>>   tory (structure) can be used for private data (including
>>   bot credentials), log files & Co.

> I used public_html, keeping the same name as used in the toolserver, but
> the name isn't really important.

I'm fine with public_html.  There doesn't seem any estab-
lished name for this scenario anyway :-).

>> - Each tool has a Puppet module à la:

>>   - webtools::TOOL::someuniqueserver:

>>     - ensure that the directory structure under
>>       /data/project/web/TOOL is set up

>>   - webtools::TOOL::loginserver:

>>     - user TOOL exists
>>     - group TOOL with members TOOL and developers exists
>>     - development dependencies for TOOL exist

>>   - webtools::TOOL::webserver:

>>     - user TOOL exists
>>     - group TOOL with members TOOL and developers exists
>>     - deployment dependencies for TOOL exist
>>     - configuration in /etc/apache2/conf.d/TOOL for
>>       URL "/TOOL/" -> directory
>>       "/data/project/web/TOOL/htdocs/" (plus CGI directory)

> This looks good, although I wonder if it's practical to ask users to add
> puppet configuration for each tool. Maybe it could be templated to the
> point where you only provide the tool name as a parameter to get all
> those points.

I had imagined a workflow where a user asks an admin: "I
want to create PHP tool TOOL!", and the admin then uses a
template (or maybe even freehands).  This has the added
bonus that the admin doesn't have to check whether the user-
supplied puppet module is correct, but can rely on the cor-
rectness of his own template.

It should be pretty easy to write a script that takes a tool
name and submits a change of the puppet configuration to
Gerrit, but I would wait with that until we're handling,
say, dozens of new tools per day :-).

>> My (first :-)) questions are:

>> - Can glusterfs handle local users and groups on
>>   /data/project, or do we need to synchronize uids/gids?

> I'm not sure about what you mean. The tools uids should not collision
> with the LDAP users, and we should have a central store of them. We
> talked about this in irc some time ago, with no clear results. Although
> I think it would be safe to start tool uids with eg. 50000.

The problem I see is that user TOOL on instance A creates a
file on /data/project, and on instance B it must appear that
this file belongs to user TOOL on that machine.  So either
glusterfs must somehow handle this with some magic, or we
have to synchronize users and groups project-wide so that
user TOOL has uid 50000 on all instances.  The latter should
be possible by changing the puppet module to check that
"user TOOL exists with uid 50000", but if glusterfs had some
automatic mapping that would be even greater.

>> - It's probable that some file beneath the "htdocs" direc-
>>   tory (or "public_html" or whatever) will at one point be
>>   owned by a developer, but they shouldn't be executed as
>>   his account.  Can we configure Apache to execute all
>>   scripts beneath "/data/project/web/TOOL/htdocs/" as TOOL?

> Good idea, I will take it into account. I don't know if you noticed that
> I configured webtools-apache-1 to read the tools space, but I didn't set
> the setuid yet (all the tools we don't have yet run as apache). It needs
> to be changed from mod_php5 to fcgi.

I hadn't.  I browsed some Apache documentation though, and
if we set user TOOL's HOME to /data/project/web/TOOL, and
use UserDir public_html, and mod_suexec, it looks as if it's
almost what I meant.

Tim