The point is it's not a single server testing them against localhost: it's a virtual machine testing against a number other VMs, each running a different OS/browser.
That is how Wikimedia will test, but we should also write the scripts in a generic enough way that they can run against a localhost as well. We can't let every developer use the tesla cluster for testing; I doubt we have enough capacity for that.
That said, I think it'll be difficult for people to run selenium tests against their own installs, as many tests are going to expect the wiki to be configured in a specific way. We plan on handling this on the automated test server by dynamically reconfiguring wikis to run tests against. There is probably a way to do this that makes it easy for people to run locally as well though.
Are we tracking the testing architecture plans anywhere? I have a bunch of notes written up that would be good to share. It'll also help clear up some of the confusion around the subject.
Respectfully,
Ryan Lane