Hello all,
I keep seeing references in WMF documents to using Redis for session storage and as the storage for the job queue going forward. However, LocalSettings doesn't have any references to Redis. It looks like the session changes for Redis were in 1.20, and I thought that the change for the job queue to be backed by Redis was coming in 1.21. It also seems that Notifications (Echo) may require Redis, but that's not real clear.
I'm wondering what the general status of this is? As a 3rd party mediawiki admin, I'm eager to get redis going primarily for the job queue improvements, but with nothing noted in local settings documentation I'm wondering what the plan is for these redis based features? Will $wgMainCacheType be getting a CACHE_REDIS option?
Jamie Thingelstad jamie@thingelstad.com mobile: 612-810-3699 find me on AIM Twitter Facebook LinkedIn
To use redis as a cache you can have something like:
// requires phpredis extension for PHP $wgObjectCaches['pecl-redis'] = array( 'class' => 'RedisBagOStuff', 'servers' => array( '127.0.0.1:6379' ), ); $wgMainCacheType = 'pecl-redis';
This would also require that the redis server would have allkeys-lru for its eviction policy in redis.conf.
To use redis for a jobqueue, one can have something like:
// requires phpredis extension for PHP $wgJobTypeConf['default'] = array( 'class' => 'JobQueueRedis', 'redisServer' => '127.0.0.1:6379', 'redisConfig' => array(), 'claimTTL' => 3600 );
This works best if the redis server uses rdb snapshots and/or append-only-file logging in redis.conf so that jobs are lost with power outages or restarts.
-- View this message in context: http://wikimedia.7.x6.nabble.com/Information-on-MW-and-Redis-tp5005659p50056... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
Awesome! Thank you!
Do you know the version required for these? Thinking maybe I could writeup a page on mw.o documenting the process of putting redis in place for these. Jamie Thingelstad jamie@thingelstad.com mobile: 612-810-3699 find me on AIM Twitter Facebook LinkedIn
On May 24, 2013, at 2:44 PM, Aaron Schulz aschulz4587@gmail.com wrote:
To use redis as a cache you can have something like:
// requires phpredis extension for PHP $wgObjectCaches['pecl-redis'] = array( 'class' => 'RedisBagOStuff', 'servers' => array( '127.0.0.1:6379' ), ); $wgMainCacheType = 'pecl-redis';
This would also require that the redis server would have allkeys-lru for its eviction policy in redis.conf.
To use redis for a jobqueue, one can have something like:
// requires phpredis extension for PHP $wgJobTypeConf['default'] = array( 'class' => 'JobQueueRedis', 'redisServer' => '127.0.0.1:6379', 'redisConfig' => array(), 'claimTTL' => 3600 );
This works best if the redis server uses rdb snapshots and/or append-only-file logging in redis.conf so that jobs are lost with power outages or restarts.
-- View this message in context: http://wikimedia.7.x6.nabble.com/Information-on-MW-and-Redis-tp5005659p50056... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
2.2.2 of the extensions works for me. I downloaded it from source and compiled it.
The redis server itself will need to be 2.6 or higher for the job queue.
Looking around, I forgot to mention that JobQueueRedis was actually removed from 1.21 (though it's in master and will be in 1.22).
-- View this message in context: http://wikimedia.7.x6.nabble.com/Information-on-MW-and-Redis-tp5005659p50056... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
If you implement this, is there any use for memcached still? I assume PHP also needs to be told that sessions are in redis? Jamie Thingelstad jamie@thingelstad.com mobile: 612-810-3699 find me on AIM Twitter Facebook LinkedIn
On May 24, 2013, at 2:44 PM, Aaron Schulz aschulz4587@gmail.com wrote:
To use redis as a cache you can have something like:
// requires phpredis extension for PHP $wgObjectCaches['pecl-redis'] = array( 'class' => 'RedisBagOStuff', 'servers' => array( '127.0.0.1:6379' ), ); $wgMainCacheType = 'pecl-redis';
This would also require that the redis server would have allkeys-lru for its eviction policy in redis.conf.
To use redis for a jobqueue, one can have something like:
// requires phpredis extension for PHP $wgJobTypeConf['default'] = array( 'class' => 'JobQueueRedis', 'redisServer' => '127.0.0.1:6379', 'redisConfig' => array(), 'claimTTL' => 3600 );
This works best if the redis server uses rdb snapshots and/or append-only-file logging in redis.conf so that jobs are lost with power outages or restarts.
Note that if you already use memcached for the main cache, there isn't really any reason to switch to redis unless you need replication or persistence.
Anyway, to use it for sessions, if you had $wgSessionCacheType explicitly set to something, then you'd need to change that too (like to 'pecl-redis'). In any case, it doesn't hurt to be explicit. This all assumes that $wgSessionsInObjectCache = true as well.
-- View this message in context: http://wikimedia.7.x6.nabble.com/Information-on-MW-and-Redis-tp5005659p50056... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
I can see how memcached and redis would have similar performance for sessions. Is that also true for job queue? I was assuming that job queue could move from database to redis, but not to memcached. Is that a correct assumption?
The main reason I could see moving sessions into redis would be to not manage both redis and memcached, assuming that redis is the key to making the job queue faster.
-- Jamie Thingelstad jamie@thingelstad.com mobile: 612-810-3699
On May 24, 2013, at 3:07 PM, Aaron Schulz aschulz4587@gmail.com wrote:
Note that if you already use memcached for the main cache, there isn't really any reason to switch to redis unless you need replication or persistence.
Indeed, the queue cannot use memcached. Redis will trivialize the time spent on actually queue operations, which could help if that is a bottleneck for job runners. If the actual jobs themselves are slow, of course it won't help too much.
Have you already tried setting the job run rate to 0 and is a background script instead?
-- View this message in context: http://wikimedia.7.x6.nabble.com/Information-on-MW-and-Redis-tp5005659p50056... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
wikitech-l@lists.wikimedia.org