On 10/02/2014 09:22 AM, Nikolas Everett wrote:
On Thu, Oct 2, 2014 at 12:45 AM, Ori Livneh <ori(a)wikimedia.org
On Wed, Oct 1, 2014 at 2:46 PM, Daniel Kinzler
We currently use memcached to share cached objects across wikis, most
importantly, entity objects (like data items). Ori suggested we
should look into
alternatives. This is what he wrote:
[21:15] <ori> I was wondering if you think the way you use memcached
(this sounds like a loaded question but I mean it sincerely). And if
not, I was
going to propose that you identify an optimal distributed object
store, and I
was also going to offer to help push for procurement and deployment
of such a
service on the WMF cluster.
[21:17] <ori> memcached is a bit of a black box. it is very
difficult to get
comprehensible metrics about how much space and bandwidth you're
especially when your data is mixed up with everything else that goes
[21:18] <ori> and the fact that you're serializing objects using php
rather than simple values makes it even harder, because it means
that you can
only really poke around from php with wikidata code available
The other major problem with memcache is that it doesn't support complex
data structures like lists, queues, sets, or maps, so when you want to
do things like, say, push or pop an item from a queue, you end up having
to retrieve the entire collection, unserialize it, manipulate it
locally, re-serialize it, and transmit it back in its entirety.
Use more Redis maybe?
Or use actual storage.