Hello,
We are running an old version of mediawiki (1.24.2), and have noticed recently that overnight we are seeing that the disk utilisation is reaching nearly 100% for almost an hour. During the night, at different times, we run a database backup (mysqldump) and a database check (mysqlcheck). It turns out that these are causing the disk usage problem.
In particular the database table 'objectcache' has nearly 500,000 entries. Whilst the table has an expiry date/time field for the records, all but about 10 of the records have the expiry date set to the year 2038!
Looking at the key names for some of the records they seem to be of the form: 'SM:resourceloader:filter:minify-css:7:066ff155efe7761cc6472668436b9554'.
My questions are:
1) Is it right that 'minify-css' records should have a very long expiry date (such that they are basically going to accumulate over time)?
2) The mediawiki docs say that it is possible to purge the table as required since the records will be recreated. Will this have any effect on someone who is using the wiki at the time the table is purged?
Current plans are to modify our backup script to exclude the table, but include its 'structure' into the backup file. (Kudos to AmitP's answer at https://stackoverflow.com/questions/13593148/mysql-dump-exclude-some-table-d... ). In addition, our database check script will perform a table purge beforehand if it is safe to do so. (Plans also include updating the mediawiki version!)
Thanks,
John.
-- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]http://www.plymouth.ac.uk/worldclass
This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.
Generally speaking objectcache is used as a last resort form of caching if you dont have apcu, memcached, redis, or something else configured.
Truncating the contents of the table will make the next couple of requests to mediawiki slower as some of the cache entries will need to be rebuilt. But it probably wont be horrible as long as you dont do it too often. Id reccomend experimenting during off peak hours so you can see what the performance impact is.
In the long run, I would reccomend installing apcu so that cache stuff is stored there instead of the db.
-- Brian
On Tuesday, February 6, 2018, John Horne john.horne@plymouth.ac.uk wrote:
Hello,
We are running an old version of mediawiki (1.24.2), and have noticed
recently
that overnight we are seeing that the disk utilisation is reaching nearly
100%
for almost an hour. During the night, at different times, we run a
database
backup (mysqldump) and a database check (mysqlcheck). It turns out that
these
are causing the disk usage problem.
In particular the database table 'objectcache' has nearly 500,000 entries. Whilst the table has an expiry date/time field for the records, all but
about
10 of the records have the expiry date set to the year 2038!
Looking at the key names for some of the records they seem to be of the
form:
'SM:resourceloader:filter:minify-css:7:066ff155efe7761cc6472668436b9554'.
My questions are:
- Is it right that 'minify-css' records should have a very long expiry
date
(such that they are basically going to accumulate over time)?
- The mediawiki docs say that it is possible to purge the table as
required
since the records will be recreated. Will this have any effect on someone
who
is using the wiki at the time the table is purged?
Current plans are to modify our backup script to exclude the table, but
include
its 'structure' into the backup file. (Kudos to AmitP's answer at
https://stackoverflow.com/questions/13593148/mysql-dump-exclude-some-table-d...
). In addition, our database check script will perform a table purge
beforehand
if it is safe to do so. (Plans also include updating the mediawiki version!)
Thanks,
John.
-- John Horne | Senior Operations Analyst | Technology and Information
Services
University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<
http://www.plymouth.ac.uk/worldclass%3E
This email and any files with it are confidential and intended solely for
the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
On Tue, 2018-02-06 at 15:33 -0500, Brian Wolff wrote:
Generally speaking objectcache is used as a last resort form of caching if you dont have apcu, memcached, redis, or something else configured.
...
In the long run, I would reccomend installing apcu so that cache stuff is stored there instead of the db.
Hello,
Thanks for the reply.
At the moment we have the main cache type set to CACHE_NONE, but php does have apc enabled (not sure why we didn't use it). We are running a relatively small wiki, but I think we may install memcached (and increase the server memory a bit). We run memcached on some other, non-wiki, servers with no problem.
I note that with MW 1.27+ there seems to be an issue using apc as the main cache and with session caching. For that reason using memcache may be easier at the moment, and not cause problems when we upgrade mediawiki.
John.
-- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]http://www.plymouth.ac.uk/worldclass
This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.
On Wednesday, February 7, 2018, John Horne john.horne@plymouth.ac.uk wrote:
On Tue, 2018-02-06 at 15:33 -0500, Brian Wolff wrote:
Generally speaking objectcache is used as a last resort form of caching
if
you dont have apcu, memcached, redis, or something else configured.
...
In the long run, I would reccomend installing apcu so that cache stuff is stored there instead of the db.
Hello,
Thanks for the reply.
At the moment we have the main cache type set to CACHE_NONE, but php does
have
apc enabled (not sure why we didn't use it). We are running a relatively
small
wiki, but I think we may install memcached (and increase the server
memory a
bit). We run memcached on some other, non-wiki, servers with no problem.
I note that with MW 1.27+ there seems to be an issue using apc as the main cache and with session caching. For that reason using memcache may be
easier at
the moment, and not cause problems when we upgrade mediawiki.
Fwiw, i dont think theres actually any bugs with session handling at apcu in 1.27 - just that if apcu isnt working properly (e.g. apcu.shm_size too small or any other situation where apcu doesnt actually store values long enough) then sessions wont work (the other caching stuff wont work either but sessions being broken is much more noticable than site being "slow"). But memcached is probably the best caching approach so your plan sounds like a good idea.
One side effect of killing objectcache i didnt think of before is it will log everyone out.
-- brian
John.
-- John Horne | Senior Operations Analyst | Technology and Information
Services
University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<
http://www.plymouth.ac.uk/worldclass%3E
This email and any files with it are confidential and intended solely for
the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
On Tue, Feb 6, 2018 at 12:34 PM Brian Wolff bawolff@gmail.com wrote:
Generally speaking objectcache is used as a last resort form of caching if you dont have apcu, memcached, redis, or something else configured.
I wonder if swapping it for a MEMORY table would be beneficial? Then it'd behave much more like the other object cache options (apcu.shm_size is max_heap_table_size, doesn't survive restarts, etc...)
Filed a task to follow-up: https://phabricator.wikimedia.org/T186752
-Chad
mediawiki-l@lists.wikimedia.org