> We don't have a machine dedicated to stats generation yet; core
> operations still take priority, and old machines aren't suitable for a
> script that apparently needs massive amounts of memory to process its
data.
Brion, I assume you can't wait to stop doing whatever you are doing and
verify the 'apparently' yourself ;)
Scripts are online since years, albeit not (yet) in SVN.
Perl is known for its tendency to spend memory in order to save time (both
in execution and in development).
Hashes are the main culprit, or blessing, which way you look at it.
Disabling edit counts for anons would make a substantial difference, with
limited effect on the output.
But that would be a stop gap solution and shortsighted. There are other very
interesting stats that would fill the space.
To name one example: I would love to generate statistics on how the content
of Wikipedia becomes less geeky, by analysing and visualising trends in
edits/articles/views per category (cluster) per month.
There is a saying that instead of spending time on optimizing perl it is
more efficient to take a job washing cars to save for adequate hardware.
Had I taken that advice I would have saved a lot of time to do more useful
work than over-optimizing a job that has outgrown its current platform.
We are not talking tens of thousands of dollars, just a run of the mill
reasonably fast machine with above average memory and a above average
harddisk, yet both in the commodity range.
Many 15 year olds in the first world spend more on their 3D gaming machine.
Erik Zachte
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
From: Brion Vibber <brion(a)wikimedia.org>
| Subject: Re: [Foundation-l] Wikimedia Projects Growth Animated
| To: Wikimedia Foundation Mailing List
| <foundation-l(a)lists.wikimedia.org>
| Message-ID: <47EA8364.7030104(a)wikimedia.org>
| Content-Type: text/plain; charset=ISO-8859-1
|
| Brian wrote:
|> PS: Why doesn't the Foundation readily give Erik a server that he can use
|> whenever he needs to? From my own memory at Wikimanias he has been
promised
|> by both Jimbo and Sue, and yet he has still never gotten one.
|
| We don't have a machine dedicated to stats generation yet; core
| operations still take priority, and old machines aren't suitable for a
| script that apparently needs massive amounts of memory to process its
data.
|
| I should note that it would probably be useful to ensure that stats
| generation scripts are available in the public source repository.
|
| -- brion vibber (brion @ wikimedia.org)
Probably talking crazy here, but why don't we develop something alike
SETI@HOME?.
I would run something that will process a segment of data like SETI@HOME
no problems in my PC and every PC at work ;).
Regards,
Damian.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFH6rLbBJnO54tRUmoRAsi3AKC+USJpAEVKLiVxu0hRtghxAZymtgCgpY/I
jkalV3+ihfVKSUyJ3osVVr4=
=HP62
-----END PGP SIGNATURE-----
Well, what if another Foundation was set up. The WMF would focus on the wiki project and the second foundation would build an endowment and fulfill the mission from a physical perspective.
----- Original Message ----
From: Anthony <wikimail(a)inbox.org>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 7:47:57 PM
Subject: Re: [Foundation-l] Thinking for the Future
On Wed, Mar 26, 2008 at 10:26 PM, Brian <Brian.Mingus(a)colorado.edu> wrote:
> I'm not sure that we are anti-Microsoft.
Personally, I'm not at all anti-Microsoft. In fact, about a week ago
I was a bona-fide Microsoft shareholder (sold to pay off some bills).
But I thought their name was invariably brought up whenever allowing
advertisements is discussed.
> The Foundation has been open to accepting funding from the Bill and Melinda Gates Foundation.
Accepting funding from the Bill and Melinda Gates Foundation is quite
different from providing funding for Microsoft, though. But on that
front, I remember the Bill and Melinda Gates Foundation specifically
being brought up as an example of an organization we shouldn't
support, during the uproar over Virgin Unite.
I dunno, I think it'd be controversial, and I'm not sure just telling
the advisors not to invest in certain companies would eliminate the
controversy. In fact, doing so would probably just increase the
controversy, in deciding which companies to include in such a list.
Of course, I suppose that's a problem that's more easily solved once
the money is in place. After all, if a benefactor wants to donate
$100 million for an endowment, s/he'd most likely have the most say in
what types of restrictions, if any, would be put on the investments.
Not that $100 million would be anywhere near enough to sustain the
WMF. It would probably be enough to sustain wikipedia.org for a long
time (barring a total financial collapse in the invested markets), but
at the point where the WMF is staring at $100 million I hope its sites
would be set on loftier goals than just distributing an encyclopedia
over the Internet.
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs
What I am thinking here is getting a bunch of people together who are interested in it and making a proposal to put in the filing cabinet should the Foundation decide that an endowment is a good idea.
----- Original Message ----
From: Geoffrey Plourde <geo.plrd(a)yahoo.com>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 9:51:38 PM
Subject: Re: [Foundation-l] Thinking for the Future
That's a damned good idea. Should I do it? Or create an Endowment Group?
----- Original Message ----
From: Brian <Brian.Mingus(a)colorado.edu>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 6:44:53 PM
Subject: Re: [Foundation-l] Thinking for the Future
On Wed, Mar 26, 2008 at 7:35 PM, Thomas Dalton <thomas.dalton(a)gmail.com>
wrote:
> Investment experts? Probably not.
>
Why don't we just create an investment policy page on meta and let the
community work out the details?
...
:)
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
That's a damned good idea. Should I do it? Or create an Endowment Group?
----- Original Message ----
From: Brian <Brian.Mingus(a)colorado.edu>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 6:44:53 PM
Subject: Re: [Foundation-l] Thinking for the Future
On Wed, Mar 26, 2008 at 7:35 PM, Thomas Dalton <thomas.dalton(a)gmail.com>
wrote:
> Investment experts? Probably not.
>
Why don't we just create an investment policy page on meta and let the
community work out the details?
...
:)
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
Who particularly cares if it is Microsoft stock. Wouldn't anyone here love to have power over Bill Gates?
----- Original Message ----
From: Thomas Dalton <thomas.dalton(a)gmail.com>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 7:11:37 PM
Subject: Re: [Foundation-l] Thinking for the Future
On 27/03/2008, Anthony <wikimail(a)inbox.org> wrote:
> On Wed, Mar 26, 2008 at 9:08 PM, Thomas Dalton <thomas.dalton(a)gmail.com> wrote:
> > On 27/03/2008, Geoffrey Plourde <geo.plrd(a)yahoo.com> wrote:
> > > What exactly are you thinking of investing in? Tax free munis? T-Bills? Money Market Account?
> >
> > Whatever the foundation's professional expert advisers suggest.
>
>
> What if they suggest something icky, like Microsoft stock?
If we're that anti-Microsoft, we can just tell the advisers that and
they can tailor their advice accordingly.
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
My estimate was figurative, not literal. It is meant to show that thinking for the future is what we should be doing.
----- Original Message ----
From: Thomas Dalton <thomas.dalton(a)gmail.com>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 2:01:41 PM
Subject: Re: [Foundation-l] [Announcement] Sloan awards Wikimedia USD 1m annually for three years
On 26/03/2008, Geoffrey Plourde <geo.plrd(a)yahoo.com> wrote:
> Yes, saving for the future is wise, even if it is just $100 a month.
I'm not sure $100/month is going to cut it... it might help for
building up a contingency fund (slowly), but not an endowment. For an
endowment you need enough so that you get enough interest that you can
get by without having to touch the capital - that requires quite a
large sum.
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
Yes, saving for the future is wise, even if it is just $100 a month.
----- Original Message ----
From: Thomas Dalton <thomas.dalton(a)gmail.com>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 6:54:26 AM
Subject: Re: [Foundation-l] [Announcement] Sloan awards Wikimedia USD 1m annually for three years
Congratulations, all round! I'm especially pleased to see that this
isn't just a one off - we have greater financial security for 3 years,
guaranteed! I'd also like to repeat what others have said: This
greater security means we might soon be able to start building up an
endowment, which is essential to the long term survival and
independence of the foundation.
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
Or maybe put donate on the bottom of the article next to the MediaWiki shiny.
----- Original Message ----
From: David Gerard <dgerard(a)gmail.com>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Sent: Tuesday, March 25, 2008 8:25:32 AM
Subject: Re: [Foundation-l] Advertisements?
On 25/03/2008, Marco Chiesa <chiesa.marco(a)gmail.com> wrote:
> David Gerard ha scritto:
> > They had a box at the door with "Suggested
> > donation £3" with the "£3" really big. I put in £5. It was clearly
> > voluntary, but also made it clear it would have been somewhat dickish
> > not to put a few quid in if you could spare it.
> > I dunno - articles with "Was this article useful to you? Suggested
> > donation: 25c" or something.
> I agree, but a 25c donation via PayPal goes all to PayPal...,
> microdonations are not so effective online
I know ... when they've looked at 20 articles, suggest $5? I suggest
this for anons - logged-in editors will presumably contribute content
or editing.
- d.
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs
Sounds like a good idea to me. If we can do it for pennies, I like it even better.
----- Original Message ----
From: Domas Mituzas <midom.lists(a)gmail.com>
To: Wikimedia Foundation Mailing List <foundation-l(a)lists.wikimedia.org>
Cc: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Sent: Wednesday, March 26, 2008 11:57:40 AM
Subject: Re: [Foundation-l] Wikimedia Projects Growth Animated
Hi!
this may sound as a heresy, but for some jobs, that are short in time-
span, but need lots of CPU capacity we could try using Amazon's EC2
or any other grid computing service (maybe some university wants to
donate cluster time?).
That would be much cheaper than allocating high-performance-high-
bucks hardware to projects like this.
Really, we have a capable cluster that has extra-CPU capacity for
distributed tasks, but anything what needs lots-of-memory in single
location simply doesn't scale.
Most of our tasks are scaled out, where lots of smaller machines can
do lots of big work, so this wikistats job is the only one which
cannot be distributed this way.
Eventually we may run Hadoop,Gearman or similar framework for
statistics job distribution, but really, first of all the actual
tasks have to be minimized to smaller segments, for map/reduce
operation, if needed.
I don't see many problems (except setting the whole grid up)
allocating job execution resources during off peak, on 10, 20 or 100
nodes, as long as it doesn't have exceptional resource needs on a
single node. It would be very nice practice for many other future
jobs too.
BR,
--
Domas Mituzas -- http://dammit.lt/ -- [[user:midom]]
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping