Wikitech-l October 2011

wikitech-l@lists.wikimedia.org

117 participants
108 discussions

Help setting up mysql.cnf for local WIki mirror
by Fred Zimmerman 06 Oct '11

06 Oct '11

Hi, I am hoping that someone here can help me - I realize there is an xmlwikidumps mailing list but it is pretty low volume and expertise relative to this one. THere is a lot of conflicting advice on the mirroring wikipedia page. C I am setting up a local mirror of english Wikipedia pages-articles...xml and I have been stopped by repeated failures in mySQL configuration. I have downloaded the .gz data from the dumped and run it through mwdumper to create an SQL file without problems, but things keep breaking down on the way into mysql. I have had a lot of agony with inno_db log files, etc. and have learned how to make them bigger, but I'm still apaprently missing some pieces. my target machine is an AWS instance 32 bit Ubuntu 1.7GB RAM, 1 Core, 200 GB, which is only for this project. I can make it bigger if necessary. Can someone take a look at this my.cnf file and tell me what I need to change to get this to work? this is what my.cnf file looks like: [mysqladmin] user= [mysqld] basedir=/opt/bitnami/mysql datadir=/opt/bitnami/mysql/data port=3306 socket=/opt/bitnami/mysql/tmp/mysql.sock tmpdir=/opt/bitnami/mysql/tmp character-set-server=UTF8 collation-server=utf8_general_ci max_allowed_packet=128M wait_timeout = 120 long_query_time = 1 log_slow_queries log_queries_not_using_indexes query_cache_limit=2M query_cache_type=1 query_cache_size=128M innodb_additional_mem_pool_size=8M innodb_buffer_pool_size=256M innodb_log_file_size=128M #tmp_table_size=64M #max_connections = 2500 #max_user_connections = 2500 innodb_flush_method=O_DIRECT #key_buffer_size=64M [mysqld_safe] mysqld=mysqld.bin [client] default-character-set=UTF8 port=3306 socket=/opt/bitnami/mysql/tmp/mysql.sock [manager] port=3306 socket=/opt/bitnami/mysql/tmp/mysql.sock pid-file=/opt/bitnami/mysql/tmp/manager.pid default-mysqld-path=/opt/bitnami/mysql/bin/mysqld.bin

3 3

Some post-1.18 issues found on Meta
by Arthur Richards 06 Oct '11

06 Oct '11

Just wanted to raise two separate incidents that have been reported to me by two separate users on meta.wikimedia.org after the 1.18 upgrade: 1) A user was attempting to look at all of the translations for a particular banner in CentralNotice (a common task carried out with no issue prior to the upgrade) when he saw this error: PHP fatal error in /usr/local/apache/common-local/php-1.18/extensions/CentralAuth/CentralAuth.i18n.php line 19586: Allowed memory size of 125829120 bytes exhausted (tried to allocate 7864320 bytes) 2) Another user was attempting to move a page on Meta when she saw this: PHP fatal error in /usr/local/apache/common-local/php-1.18/includes/GlobalFunctions.php line 1197: Object of class AFPData could not be converted to string -- Arthur Richards Software Engineer Fundraising/Features/Offline/Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687

5 6

Attention developers (Gadgets, user/site scripts, Toolserver tools, MediaWiki extensions, ..)
by Krinkle 05 Oct '11

05 Oct '11

Hi all, If you write, maintain or are otherwise active in development, and directly or indirectly make use of MediaWiki, then you should subscribe to: mediawiki-api-announce(a)lists.wikimedia.org That is a mailing list that only brings important announcements for developers that use MediaWiki as an API. Note that this list is not limited to API as "api.php" but about MediaWiki as a application programming interface (API) in general. Subjects that have announcements on that list: * Important changes in database schema (columns or tables added, removed or changed in such a way that you should change your queries. Think for example of the addition of rev_deleted, queries should most likely query for rev_deleted=0 now). * Changes in the JavaScript API (methods being deprecated or removed in the mediawiki.js library etc. as well as upgrades of third-party libraries that ship with MediaWiki, such as jQuery). * Major changes to the HTML layout (such as the change for the sidebar id to #mw-panel) * Hooks in MediaWiki PHP. Mostly for extension developers. Changes or deprecation of hooks. * And last but not least, the api.php itself. All major changes. === Although time will learn how the list is used, to readers and writers of this list: "All subjects should clearly indicate what needs changing and when!" For example "Vector skin sidebar html ID changes to '#mw-panel' in 1.17". Also, whenever Wikimedia has scheduled a deployment of revision(s) or entire branches that expose any change that was previously announced, a new mail should be sent here to remind/summarize upcoming changes (since gadgets should/can't be changed until the new version is deployed but new versions can be prepared or tested in advance) Hope to see you soon on mediawiki-api-announce(a)lists.wikimedia.org :) Please reply-to to wikitech to keep discussions about this central. -- Krinkle

4 3

Fundraising bug triage - 5 October
by Arthur Richards 05 Oct '11

05 Oct '11

What: Fundraising bug triage When: Wendesday, October 5, 2011, 17:00UTC (10:00am PDT) Where: #wikimedia-dev on irc.freenode.net Wednesday, October 5, we will be conducting a bug triage for open engineering issues relating to the Wikimedia Fundraiser. We will be tackling some low-hanging fruit focussing mostly on some of the fundraiser metric reporting tools, as well as some non-Mediawiki related tools. This is a great opportunity for new devs to get involved with MediaWiki development as well as for veterans to help out with one of the most critical projects for the Wikimedia Foundation. We also have some CiviCRM related bugs - so if you're a CiviCRM hacker or interested in exploring another awesome open source project, this is for you! Take a look at some of the open bugs we'll be addressing during the triage: http://bit.ly/qSGKn4 Looking forward to seeing you there! -- Arthur Richards Software Engineer Fundraising/Features/Offline/Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687

1 1

Mystery of most-viewed pages on En.Wikipedia
by Gregory Kohs 05 Oct '11

05 Oct '11

Does the Wikimedia Foundation's technology team have any insight or comment on the finding that (other than the Wikipedia Main Page and the "404 error" page), in September the most popular page on the English Wikipedia was "Mathematical descriptions of opacity", with over 5.1 million views? There was no discernible "bump" in interest in opacity due to outside news events or a book or movie release on the subject. The phenomenon is outlined here: http://www.examiner.com/wiki-edits-in-national/wikipedia-s-top-10-most-view… Do you think this is some sort of malicious probing activity by a hacker, or is it perhaps the deliberate testing of a developer employed by the WMF? Thank you, Greg

6 7

Re: [Wikitech-l] FW: RFC: Refactor on File/FileRepo/MediaHandler? (Separation of concerns, portability)
by Russell Nelson 04 Oct '11

04 Oct '11

On Mon, Oct 3, 2011 at 10:15 PM, Brion Vibber <brion(a)wikimedia.org> wrote: > I would *very* strongly recommend doing the internal refactoring before we > get anywhere near reviewing and deploying that bad boy; otherwise we'll > spend all the code review time pointing out things to refactor to avoid > future maintenance problems. :) I agree. The only question is whether the current system is running out of steam badly enough to require a two stage process. If the answer is "no, we're good", then definitely FileStore first.

2 2

Page Output branch: PageView, Special Page based actions, Functional logic separation from skin, ...
by Daniel Friesen 04 Oct '11

04 Oct '11

I've been working on a pageoutput branch. Centred around an earlier RFC. https://www.mediawiki.org/wiki/User:Dantman/Page_output_branch https://www.mediawiki.org/wiki/Requests_for_comment/Drop_actions_in_favour_… The general goals of the project are: - Actions die off and instead we have SpecialPages that take on that functionality. &action= urls become one method of accessing certain special pages. - We have a new PageView system that replaces Article which just handles the output of a page and SpecialPage becomes an implementation of these. - Logic for things like tabs, special links in the toolbox, canonical urls, etc... become part of the PageView implementation and Skin only handles the way they are laid out instead of also doing the actual creation. Some of this stuff is related to skinning and works better for the skinning rewrite if done before so I started on this before the template system and whatnot. It takes a day of work to do much in this branch and I only have about 1 day in a week I can go and start coding things at that level, so the project hasn't gone all too far yet. (not to mention merging is horrible, I wasted several hours at one point I could have been coding with on merging some conflicts from trunk). -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

1 0

Skin rewrite future plans
by Daniel Friesen 04 Oct '11

04 Oct '11

I finally got around to organizing my Skinning system page and cleaning out the ideas sitting there that I actually dropped or replaced. https://www.mediawiki.org/wiki/User:Dantman/Skinning_system Some notable things: - A template system is planned. The idea is slowly stabilizing. - I plan to replace things like bodytext, newtalk, sitenotice, catlinks, dataAfterContent, etc... with a 'regions' system that lets skins define their own areas with certain parameters, extensions define things they want in certain types of areas, and MediaWiki figures out what to put where. - I have an idea on how to eliminate content_{actions,navigation}, personal_urls, toolbox, language_urls, etc... in favour of a more flexible system: https://www.mediawiki.org/wiki/User:Dantman/Skinning_system/Link_lists_rewr… - I've dropped the idea of controlling the SEARCH/LANGUAGES/TOOLBOX from the navigation editing interface and perhaps even the skins defining the navigation blocks themselves. Instead I'm thinking of users being able to define navigation blocks of different types. Drop those in for use in different skins. And the sidebar would become a widgetized sidebar setup where from a separate interface you can drop navigation lists as widgets into the sidebar, alongside the default search/etc..., raw blocks of wikitext, and extensions will be able to implement alternative widgets like donation buttons, advertisement blocks, etc... also extensions may be able to implement context-sensitive types of navigation that can be dropped in. eg: To make something like what Blender's Wiki tried to do possible. An example of a possible way the Vector skin might be built with this system: https://gist.github.com/1239039 -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

1 0

Announcing Phil Chang as our Product Manager Mobile
by Erik Moeller 04 Oct '11

04 Oct '11

Hello all, It’s with great pleasure that I’m announcing Phil Chang as our new Product Manager for Mobile, starting today. Phil will be working closely with Tomasz’ engineering team and will report to me. As product manager, it’s Phil’s job to define the scope and the priorities of the improvements and feature additions that we’ll be making to our mobile experience -- in partnership with the community and the WMF team. Phil’s been in the mobile space since 2002. His first product development work on mobile was with mobile start-ups in the UK, including a company that released StealthText, a service for auto-expiring text messages, and later Picsel Technologies. At Picsel, he led development of two new products, a Mobile Content Solution and an embedded mobile web browser. Prior to that, Phil had a long career in the Bay Area as a product innovator at Macromedia, AT&T, and his own start-up, e-Acumen. In his spare time, Phil enjoys art exhibitions, musical performances, ice skating and rollerblading. During his time in the UK, he became a British citizen and traveled throughout Europe. He has been a student of the German, Japanese and Korean languages, and is married to a native Japanese. He and his wife have a young daughter who holds three citizenships: Japan, US and UK. Working for a nonprofit again is coming full circle for Phil. During his high-school and university years, for six years, he published and edited a magazine of the arts operated as an independent non-profit. He’s passionate about making culture accessible to all human beings, and helping us bring free knowledge to billions of people. He'll also help us explore how our mobile user community can be directly engaged in the projects by contributing text and media. Please join me in welcoming Phil to the Wikimedia movement. All the best, Erik -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate

3 2

RFC: Refactor on File/FileRepo/MediaHandler? (Separation of concerns, portability)
by Brion Vibber 03 Oct '11

03 Oct '11

There are a few issues with how our file/image handling works that make handling remote files a bit harder than it needs to be. Some things don't work consistently over InstantCommons because a local MediaHandler class is used that doesn't understand the remote data, and the whole FileRepo architecture is pretty awkward for actually storing "local" files outside of the filesystem (eg the Extension:SwiftMedia backend has to jump through a lot of hoops). I've started collecting some notes for an RFC on the wiki: https://www.mediawiki.org/wiki/Requests_for_comment/Refactor_on_File-FileRe… There's not a lot of specific work targets in there yet, as we'll want some more data from the folks working on the alternate backends as well as the frontend issues which are mostly what I've written on. But I would like to see if we can avoid having to have a local MediaHandler that's compatible with a remotely-hosted file. As long as the necessary metadata can be exposed, and any rendering can be encapsulated as a thumbnail or an iframe, client sites *should* be able to render anything we can push out from Commons without any special support on their end. -- brion

3 3

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l October 2011