Sending this out because I know many of your are wondering about the status of the Redis JobQueue work.
This happened today:
18:24 logmsgbot: aaron synchronized wmf-config/jobqueue-eqiad.php 'Enabled use of redis for null test jobs'
What that means is that null test jobs will be sent to the new Redis-based JobQueue. Aaron will also begin sending some real jobs to the Redis-based queue later today and monitor them.
If all goes well with the testing, we should be able to switch over fully by this coming Monday.
Greg
<quote name="Greg Grossmeier" date="2013-04-10" time="12:49:25 -0700">
If all goes well with the testing, we should be able to switch over fully by this coming Monday.
All went well, we're now fully on Redis.
You can rejoice now.
Greg
Greg Grossmeier wrote:
<quote name="Greg Grossmeier" date="2013-04-10" time="12:49:25 -0700"> > If all goes well with the testing, we should be able to switch over > fully by this coming Monday.
All went well, we're now fully on Redis.
You can rejoice now.
Hi Greg.
I looked at your original e-mail and this e-mail, but I'm still not sure what Redis is or how it relates to the MediaWiki job queue. I looked for more information at https://wikitech.wikimedia.org/wiki/Redis, https://wikitech.wikimedia.org/wiki/Redis_JobQueue, and https://wikitech.wikimedia.org/wiki/JobQueue, but didn't find anything. I did find https://wikitech.wikimedia.org/wiki/Job_queue, but it makes no mention of Redis (I think that page is a bit out-of-date at this point).
Are there docs somewhere for those interested? (Apologies if docs have been posted to this mailing list and I've simply missed them.)
(Hmmmm. Just before hitting send, I thought to check mediawiki.org. There are some notes at https://www.mediawiki.org/wiki/Redis, but I'm not sure if these relate to Redis/MediaWiki generally or Wikimedia wikis specifically.)
MZMcBride
On Monday, April 15, 2013 at 7:02 PM, MZMcBride wrote:
(Hmmmm. Just before hitting send, I thought to check mediawiki.org (http://mediawiki.org). There are some notes at https://www.mediawiki.org/wiki/Redis, but I'm not sure if these relate to Redis/MediaWiki generally or Wikimedia wikis specifically.)
Oh, oops. While the links may be of interest, the lead is incorrect. Our analytics stuff doesn't use redis. We tried it for a while and decided it wasn't a good fit. It is used for a number of other things, as you mentioned.
-- Ori Livneh
Hiya MZ,
<quote name="MZMcBride" date="2013-04-15" time="22:02:03 -0400">
I looked at your original e-mail and this e-mail, but I'm still not sure what Redis is or how it relates to the MediaWiki job queue. I looked for more information at
[snip lots of places]
Are there docs somewhere for those interested? (Apologies if docs have been posted to this mailing list and I've simply missed them.)
Not really, no (at least, not that I know of). This is a part of the general site performance improvement work, tracked under this project in our monthly reports: https://www.mediawiki.org/wiki/Site_performance_and_architecture
You can see the update for March was: "JobQueueRedis merged, JobQueueAggregatorRedis merged and deployed. These improvements to the jobqueue should help site performance (see Tech Operations section of the monthly report)."
The one for April will be something like "Migrated all jobs to the Redis-based JobQueue for the win."
The reason I sent out these two messages is because while general performance improvements aren't always big events, this one was a blocker for some further work on a couple projects; notably WikiData and Echo/Notifications (from the Editor Engagement team). Those two make use of the JobQueue extensively and there were a couple times that the JobQueue feel over in the past couple weeks pre-Redis. Now that we're on Redis things should be much much much more stable (and they seem to be so far).
I don't know the plan for documenting the switch over on mediawiki.org/wikitech; Aaron will have a better idea on that.
Hope that helps,
Greg
On 16/04/13 12:02, MZMcBride wrote:
Greg Grossmeier wrote:
<quote name="Greg Grossmeier" date="2013-04-10" time="12:49:25 -0700"> > If all goes well with the testing, we should be able to switch over > fully by this coming Monday.
All went well, we're now fully on Redis.
You can rejoice now.
Hi Greg.
I looked at your original e-mail and this e-mail, but I'm still not sure what Redis is or how it relates to the MediaWiki job queue.
Redis http://redis.io/ is an in-memory key/value store, a bit like memcached except with more features.
We've been using it as a memcached replacement for session storage since the eqiad switchover in January, because it has a replication feature which can be used to synchronise data between the two data centres. It allowed us to switch from Tampa to Ashburn without logging everyone out.
It's designed more as a persistent store than a cache. Memcached still wins for simple unreliable object caching, so we're still using memcached for that.
We previously stored the MW job queue in MySQL. This gave us lots of useful features, like replication and indexing for duplicate removal, but it has often been hard to manage the performance implications of the high insert rate.
Among its many features, Redis embeds a Lua interpreter on the server side. The new Redis job queue class provides a rich feature set superior to the MySQL job queue, primarily by the use of several server-side Lua scripts which provide high-level job queue functions.
-- Tim Starling
On Tue, Apr 16, 2013 at 12:14 AM, Tim Starling tstarling@wikimedia.orgwrote:
We've been using it as a memcached replacement for session storage since the eqiad switchover in January, because it has a replication feature which can be used to synchronise data between the two data centres. It allowed us to switch from Tampa to Ashburn without logging everyone out.
It's designed more as a persistent store than a cache. Memcached still wins for simple unreliable object caching, so we're still using memcached for that.
We previously stored the MW job queue in MySQL. This gave us lots of useful features, like replication and indexing for duplicate removal, but it has often been hard to manage the performance implications of the high insert rate.
Among its many features, Redis embeds a Lua interpreter on the server side. The new Redis job queue class provides a rich feature set superior to the MySQL job queue, primarily by the use of several server-side Lua scripts which provide high-level job queue functions.
I've taken the liberty of adapting this explanation and my own additions for the Redis page on MediaWiki.org
Steven
Steven Walling wrote:
On Tue, Apr 16, 2013 at 12:14 AM, Tim Starling tstarling@wikimedia.orgwrote:
We've been using it as a memcached replacement for session storage since the eqiad switchover in January, because it has a replication feature which can be used to synchronise data between the two data centres. It allowed us to switch from Tampa to Ashburn without logging everyone out.
It's designed more as a persistent store than a cache. Memcached still wins for simple unreliable object caching, so we're still using memcached for that.
We previously stored the MW job queue in MySQL. This gave us lots of useful features, like replication and indexing for duplicate removal, but it has often been hard to manage the performance implications of the high insert rate.
Among its many features, Redis embeds a Lua interpreter on the server side. The new Redis job queue class provides a rich feature set superior to the MySQL job queue, primarily by the use of several server-side Lua scripts which provide high-level job queue functions.
I've taken the liberty of adapting this explanation and my own additions for the Redis page on MediaWiki.org
Thank you both. :-) I'll try to help out with that MediaWiki.org page as well (and perhaps add some pointers from Wikitech... two wikis blergh).
I had no idea about Redis being used for user sessions. That's neat.
MZMcBride
On Mon, Apr 15, 2013 at 3:17 PM, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="Greg Grossmeier" date="2013-04-10" time="12:49:25 -0700"> > If all goes well with the testing, we should be able to switch over > fully by this coming Monday.
All went well, we're now fully on Redis.
You can rejoice now.
This is what the enwiki master's rejoice looks like:
https://ganglia.wikimedia.org/latest/graph.php?r=month&z=xlarge&c=My...
Greg
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Engineering mailing list Engineering@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering
wikitech-l@lists.wikimedia.org