I vote for making the toolserver the head-node to a much larger beowulf cluster that has a well configured job scheduler. The data that needs to be crunched is already right there - it makes sense to put a research cluster there as well.
There will always be a limited supply of resources. Perhaps there should be a public approval system for the resources, where the community gets to pick which jobs should get added to the queue based on public analysis of the code and a description of the computation.
There will be no shortage of participants ;)
On Wed, Mar 11, 2009 at 2:37 AM, River Tarnell river@loreley.flyingparchment.org.uk wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Robert Rohde:
In particular, I think it is useful to separate "tools" from "analysis".
why?
"Tools" need high availability and low lag relative to the live site, but "analysis" doesn't care if it gets out of date and should use scheduling etc. to balance large loads.
what is preventing people from using the current toolserver for this analysis? what do we need to change about the platform that will enable people to run it on the current toolserver?
- river. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (HP-UX)
iEYEARECAAYFAkm3eD0ACgkQIXd7fCuc5vJeNQCbB3zmpKh2jLmyJDqr6riSXtE5 1GMAoLjUPl28JgGFiXMAMKEEF2659DI8 =R0i8 -----END PGP SIGNATURE-----
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l