We currently
have no plans for having the user databases on the same
servers as the replicated databases. Direct joins will not be
possible, so tools will need to be modified.
-50
It's such a useful feature, that it would be worth making a local mysql
slaves for having them.
I know, the all-powerful labs environment is unable to run a mysql
instance, but we could use MySQL cluster, trading memory (available) to
get joins (denied).
I'm not the one setting up the databases. If you want information
about why this won't be available, talk to Asher (binasher in
#wikimedia-operations on Freenode). Maybe he can be convinced
otherwise.
Of course, in the production cluster we don't do joins this way. We
handle the joins in the app logic, which is a more appropriate way of
doing this.
SGE is a strong queue system. We have people and tools
already trained
to use it. It would be my first option.
That said, if the presented alternative has the same user interface, it
shouldn't be a problem. For instance, I don't have an opinion about
which of the SGE forks would be preferable.
In general in Labs we don't have a large need for a queuing system
right now. If Toolserver folks need it very badly, it's possible to
add, someone just needs to put the effort into it. It likely wouldn't
be amazingly hard to puppetize this to run in a single project. Making
things multi-project is difficult and takes effort. Anyone can do the
single-project version in a project, multi-project will likely take
engineering effort.
* no
support for servlets
I'm not sure what you mean by servlet?
J2EE, I guess.
Well, if it's available in the ubuntu repos, or if it's open source,
then it's available in Labs.
I'd love
DaB to help us improve Labs.
Everything about Labs is fully open. Anyone can help build it, even
the production portions.
Would it be worth our efforts? I sometimes wonder why we should work on
that (yes, I'm pessimistic right now).
For instance the squid in front of *.beta.wmflabs.org. It was configured
by Petan and me. We had absolutely no support from the WMF. The squid
wasn't purging correctly. It worked on production, so there was a config
error somewhere.
We begged to see the squid config for months. But as it was in the
private repository, no, it can't be shown, just in case it has something
secret (very unlikely for squid config). Yes, we will clean them up and
publish, eventually. Months passed (not to mention how publishing the
config had been requested years ago). It could have been quickly
reviewed before handing out, and we weren't going to abuse it if there
really something weird was there. Replicating the WMF setup was done
without viewing that same setup. I finally fixed it. I was quite proud
of having solved it.
And you should be. Your changes kept that project moving along for
months until I broke it.
Where is that file right now? It vanished. The file
was lost in one of
the multiple corruptions of labs instances. It was replaced with a copy
of the cluster config (which was finally published in the meantime).
So it feels like wasted effort now. I'd have liked to save a local copy
at least.
To be fair, there's only been a single occurrence of instance
corruption, which was due to a bug in KVM.
Also, yes, the squid configuration was finally published because ones
of the devs spent the time to do so. I was working on stabilizing
things most of that time.
Does this mean your efforts were wasted? Of course not. Your efforts
helped keep the project running, which is important. Just because your
file was replaced with the production copy doesn't mean the work put
into it was for nothing.
It's not enough to leave tools there and say
"It is fully open. Anyone
can help build it"
We're also putting effort into making the migration happen, but we're
focusing our efforts in different places. We can't do everything,
which is why I'm trying to encourage others to help out. If we work on
separate pieces of the work it'll go much quicker.
- Ryan