I finally got around to taking a quick look at memcached. It strikes me as quite elegant, and ought to be fairly easy to drop in in a number of places. The PHP client library is GPL, and the server is BSD:
http://www.danga.com/memcached/
At the moment we don't exactly have a lot of free memory floating around to work with, but hardware upgrades are on the horizon.
Memcached is essentially a really big associative array. :) Rather than shared memory, clients talk to the server(s) by TCP/IP... but this means you can tie together multiple servers (for *lots* of RAM-based storage) and multiple clients, should we in the future end up using multiple front-end web servers.
-- brion vibber (brion @ pobox.com)
Brion-
At the moment we don't exactly have a lot of free memory floating around
Larousse seems to still have about 400 megs free, pliny 885, that is, used by Linux for caching, but Linux always grabs as much as it gets for caching -- that doesn't mean it has any substantial effect. For example, I have 640 MB of RAM, "free" tells me that 620 are used -- but minus buffers/cache, only about 200 MB are used.
Using an application-optimized cache like memcached would be orders of magnitude more efficient than relying on Linux' kernel-based caching.
Regards,
Erik
I've been playing about a bit with memcached, and have just checked into the unstable branch some initial experimental code to use it to cache user data (for logged in users) and individual lookups in the LinkCache. Not very efficient, not very complete, just trying things out.
The PHP memcached client class is under GPL, and I've just included a copy in for convenience. If we don't tell it to connect, all requests become no-ops, so we can simply ask it for data, and on a memcached-less installation we'll always immediately fall back to the database.
Note that the client requires socket support, which generally means recompiling PHP with --enable-sockets. If disabled it shouldn't get to any of the socket calls, so it should run fine without memcached enabled on a stock socketless PHP.
See docs/memcached.doc for more details.
(I've also changed User::loadFromSession() to be a static class method that returns an instance of User, rather than something that's run on a newly created instance. This way we can just dump a serialized object to the cache and get it back intact.)
I haven't thoroughly tested things at this point, but in brief it seems to work. Note: performance _sucks_ with the stock memcached 1.1.8; use the prerelease snapshot of 1.1.9: http://lists.danga.com/pipermail/memcached/2003-August/000092.html
-- brion vibber (brion @ pobox.com)
I wrote:
I've been playing about a bit with memcached, and have just checked into the unstable branch some initial experimental code to use it to cache user data (for logged in users) and individual lookups in the LinkCache. Not very efficient, not very complete, just trying things out.
Also now checks user_newtalk for anons from the cache.
This code is now running on test.wikipedia.org. It's far from complete, and it's dumping gobs of debug info to the log, so don't expect it to be super fast. ;)
-- brion vibber (brion @ pobox.com)
wikitech-l@lists.wikimedia.org