Wikitech-l November 2012

wikitech-l@lists.wikimedia.org

130 participants
133 discussions

Question about 2-phase dump
by vitalif＠yourcmc.ru 25 Nov '12

25 Nov '12

Hello! While working on my improvements to MediaWiki Import&Export, I've discovered a feature that is totally new for me: 2-phase backup dump. I.e. the first pass dumper creates XML file without page texts, and the second pass dumper adds page texts. I have several questions about it - what it is intended for? Is it a sort of optimisation for large databases and why such method of optimisation was chosen? Also, does anyone use it? (does Wikimedia use it?)

4 7

Re: [Wikitech-l] Future of MediaWiki's bad image list
by billinghurst 25 Nov '12

25 Nov '12

> Future of MediaWiki's bad image list (MZMcBride) > > Hi. > > In the context of <https://bugzilla.wikimedia.org/show_bug.cgi?id=14281> > ("Suggestion: Rename "Bad image list" to better title"), it occurred to me > that a bad image list is a specialized (and hackish) feature that probably > wouldn't be accepted into MediaWiki core today. > > I'm wondering if it makes sense to split this feature (a bad image list) out > into a MediaWiki extension. > > If splitting this feature out of MediaWiki core makes sense, I'm wondering > whether it should be put in its own extension, be part of another existing > extension (such as Phalanx), or be part of a more generalized behavior > control extension. > > An alternate approach suggested on the bug would be to use an AbuseFilter > filter instead of a bad image list (thereby deprecating "MediaWiki:Bad image > list"). This gets slightly more complicated if you want to also replicate > the MediaWiki page's exceptions functionality (you can block inline usage of > an image except on specified approved pages). This exceptions functionality > may make an _exact_ replica of the bad image list in an AbuseFilter filter > impossible. That said, the AbuseFilter extension was seemingly designed for > exactly this type of behavior control and includes handy features such as > disallowing the edits altogether and logging attempted edits. > > Thoughts? > > MZMcBride I would like to see some simplicity to be available to be retained, and not the never-ending complexity of trying to write complex abuse filters. Keeping simple files like "Mediawiki:Bad file list" enables those admins not hacking-literate to be able to add an image and have it locally blocked. That said, I can see value in having a more sophisticated approach to both bad and good lists that we can store in a protected environment like Mediawiki: namespace AND to have both portability and the ability to develop new features. Is it possible to write AbuseFilter so that it is able to make use of both bad and good lists stored within the MW namespace? I especially like that if we are able to utilise that in the global space. We have had global vandals come along and start spamming or vandalising by the addition of an image, or a singular image with coding. To be able to slow down that process with a global filter, would be excellent. I would see that a simple AbuseFilter that says inhibit (in whichever flavour) 'this' edit based on finding a filename found in 'Mediawiki:Bad image list' as a base feature would be the simplest response. Default the behaviour to on (for WMF) and it replicates the present case. If wikis wish to then further develop these filters, excellent. I also see that such filter behaviour could allow wikis to battle other forms of vandalism, and one that I know that I have put into bugzilla previously. We had a vandal who was putting sexually explicit pictures on contributors' user and user talk pages. If this wiki was then able to write a filter that basically said block the addition of images to such and such local namespace, from such a such category at Commons, and/or matching such and such keywords from a filelist, rather than have to try and write an abusefilter that had all the series of swear words and references to sexual appendages, BRILLIANT! Billinghurst

4 3

Future of MediaWiki's bad image list
by MZMcBride 24 Nov '12

24 Nov '12

Hi. In the context of <https://bugzilla.wikimedia.org/show_bug.cgi?id=14281> ("Suggestion: Rename "Bad image list" to better title"), it occurred to me that a bad image list is a specialized (and hackish) feature that probably wouldn't be accepted into MediaWiki core today. I'm wondering if it makes sense to split this feature (a bad image list) out into a MediaWiki extension. If splitting this feature out of MediaWiki core makes sense, I'm wondering whether it should be put in its own extension, be part of another existing extension (such as Phalanx), or be part of a more generalized behavior control extension. An alternate approach suggested on the bug would be to use an AbuseFilter filter instead of a bad image list (thereby deprecating "MediaWiki:Bad image list"). This gets slightly more complicated if you want to also replicate the MediaWiki page's exceptions functionality (you can block inline usage of an image except on specified approved pages). This exceptions functionality may make an _exact_ replica of the bad image list in an AbuseFilter filter impossible. That said, the AbuseFilter extension was seemingly designed for exactly this type of behavior control and includes handy features such as disallowing the edits altogether and logging attempted edits. Thoughts? MZMcBride

1 0

Gallery / thumbnails generation performance impact
by Sébastien Santoro 23 Nov '12

23 Nov '12

Good afternoon, The he.wikipedia community would like to use custom image sizes for thumbnails and galleries. This configuration change would require the creation of 250px and 200px pictures. Hashar have expressed concern on the relevant bug* this change, with two different sizes, would cost have a cost in processing power. I would like to know if someone has a way to measure the ImageMagick thumbnails generation cost for he.wikipedia, and more generally, if we have somewhere a page with the information "who to contact about a performance issue?" (and if not, it would be a good idea to create it). (*) https://bugzilla.wikimedia.org/show_bug.cgi?id=41712 Bug 41712 - hewiki asks to change default image sizes for thumb and gallery -- Best Regards, Sébastien Santoro aka Dereckson http://www.dereckson.be/

1 0

Commits that make schema changes
by Aaron Schulz 22 Nov '12

22 Nov '12

In order to make commits that change the schema as discoverable is possible, I'd propose the following guidelines: * Update RELEASE-NOTES to briefly mention the change (this is not always necessary for follow up changes within the same release, since people upgrading don't care about intra-release changes). * Include "[Schema]" near the beginning of the first line of the commit. * Mention the names of the tables that the schema was changed for as a bullet point in the commit summary. Also, changes should always allow for a temporary rollback strategy when people upgrade from one major MediaWiki version to the next. The schema for version X+1 should work with the version X code. Mostly, this just involves doing additions in version X+1 and removals in version X+2 or higher. More specifically: * New tables are fine * New columns are fine *as long as* they have default values (also one should check for code that does SELECT * and does for loops on all fields, which should be avoided to begin with) * New indexes are fine (note that adding a new column and a unique index on it can cause problems even with DEFAULT NULL for some non-mysql DBMS) * Table removal is fine if it already wasn't used in the previous MediaWiki version and any data worth migrating is already first migrated in update.php with a "logged update" * Column removal is fine if it already wasn't used in the previous MediaWiki version and any data worth migrating is already first migrated in update.php with a "logged update" * Index removal is fine if it already wasn't used in the previous MediaWiki version (one could also remove index A and add index B which also handles the queries that used A *provided* there are no FORCE INDEX A statements used by MediaWiki) * Column changes that just fix prior problems or just expand the range of values are fine (e.g int => bigint, NOT NULL => NULL, varchar => blob), as long as it works with the previous MediaWiki version * Index changes that are fine as long as it works with the previous MediaWiki version (e.g. making an index not used for sorting go from (field_sha1) to (field_sha1(8)), or doing changing and index from (field_a) to (field_a,field_b)). The reason I say "temporary rollback" is that during such a rollback it might be OK for certain problems to exists. For example, in the past the data stuffed in page_restrictions was moved to a new page_restrictions table. Of course if some one upgraded for a while, and then rolled back, newly protected pages would magically become unprotected (until the upgrade was re-attempted or an admin re-protected the pages). The degree to which these problems are acceptable depends on: a) The likelihood of a rollback being needed (larger with more complex changes) b) Whether a rollback would be likely to only last for a brief time for a hotfix or would be likely to last a long time (more so with more complex changes) c) How annoying the problem would be I'd say that those things should be measured on a case-by-case basis. In some cases, version X+1 should write to both the old and new style locations if the risk is too high. -- View this message in context: http://wikimedia.7.n6.nabble.com/Commits-that-make-schema-changes-tp4990111… Sent from the Wikipedia Developers mailing list archive at Nabble.com.

1 0

Fwd: [Tech/Product] Engineering/Product org structure
by Erik Moeller 21 Nov '12

21 Nov '12

FYI ---------- Forwarded message ---------- From: Erik Moeller <erik(a)wikimedia.org> Date: Mon, Nov 5, 2012 at 5:38 PM Subject: [Tech/Product] Engineering/Product org structure To: Staff All <wmfall(a)lists.wikimedia.org> Hi folks, consistent with Sue's narrowing focus mandate, I’ve been thinking & talking the last few weeks a fair bit with a bunch of different people about the future organizational structure of the engineering/product department. Long story short, if we want to scale the dept, and take seriously our identity as a tech org (as stated by Sue), it’s my view that we need to split the current department into an engineering dept and a product dept in about 6-8 months. To avoid fear and anxiety, and to make sure the plan makes sense, I want to start an open conversation now. If you think any of the below is a terrible idea, or have suggestions on how to improve the plan, I’d love to hear from you. I’ll make myself personally available to anyone who wants to talk more about it. (I'm traveling a bit starting tomorrow, but will be available via email during that time.) We can also discuss it at coming tech lunches and such. There’s also nothing private here, so I’m forwarding this note to wikitech-l@ and wikimedia-l@ as well. That said, there’s no urgency in this note, so feel free to set it aside for later. Here’s why I’m recommending to Sue that we create distinct engineering and product departments: - It’ll give product development and the user experience more visibility at the senior mgmt level, which means we’ll have more conversations at that level about the work that most of the organization actually does. Right now, a single dept of ~70 people is represented by 1 person across both engineering and product functions - me. That was fine when it was half the size. Right now it’s out of whack. - It’ll give us the ability to add Director-level leadership functions as appropriate without making my head explode. - I believe that separating the two functions is consistent with Sue’s recommendation to narrow our focus and develop our identity as an engineering organization. It will allow for more sustained effort in managing product priorities and greater advocacy for core platform issues (APIs, site performance, search, ops improvements, etc.) that are less visible than our feature priorities. A split dept structure wouldn’t affect the way we assemble teams -- we’d still pull from required functions (devs, product, UI/UX, etc.), and teams would continue to pursue their objectives fairly autonomously. It’s not all roses -- we might see more conflict between the two functions, more us vs. them thinking, and more communications breakdowns or forum shopping. But net I think the positives would outweigh the negatives, and there are ways to mitigate against the negatives. The way we’d get there: I’m prepared to resign from my engineering management responsibilities and to focus solely on my remaining role as VP of Product, as soon as a successor for VP of Engineering has been identified. We would start that hiring process probably in early 2013. I’m recommending to Sue that we seriously consider internal candidates for the VP of Engineering role, as we have a strong engineering management team in place today. So realistically we'd probably identify that person towards the end of the fiscal year. Obviously I can’t make any promises to you that in that brave new world, you’ll love whoever gets hired into the VP of Engineering role, so there’s some unavoidable uncertainty there. I’ll support Sue in the search, though, and I’m sure she’d appreciate feedback from you on the kind of person who you think would be ideal for the job. The VP of Product role would encompass a combination of functions. Howie and I would work with the department to figure out what makes sense as an internal structure. My opening view is that Analytics and User Experience are potential areas that may benefit from dedicated Director-level support roles. (Analytics is tricky because it includes a strong engineering piece, but also a research/analyst piece working closely with product.) The new structure would therefore be as follows: * VP of Engineering -> Directors of Engineering * VP of Product -> Director of Product Development, plus new Director-level functions (we've discussed UX/Design as a likely new leadership function, and Analytics as a _potential_ area to centralize here because it works so closely with product) Why Product? I’m happy to help the org in whatever way I can; I believe I’d be most useful to it in focusing there and helping build this relatively new organizational function. Based on my past experience, Howie & I make a great team. I know how engineering operates, which could help mitigate against some of the aforementioned issues. Plus, our product priorities generally already reflect lots of thought and consideration, and we have no intent of reopening questions like "Is Visual Editor the top product priority". I look forward to hearing your thoughts & discussing this further in coming weeks. All best, Erik -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate

9 15

A Newbie!
by Hasini Abeywickrama 21 Nov '12

21 Nov '12

Hi all! I am a new member to this mailing group & I am interested in taking part in the Outreach Programme for Women! I would like to make my contribution to Wikimedia. I read through the webpage but I have no clear idea on where do download the code & source files etc. Can someone out there please help? Thanks a lot in advance! :) Hasi

4 3

Deleting tags
by Chad 21 Nov '12

21 Nov '12

Hi all, Early today, Nikerabbit noticed he couldn't delete a tag on a project he's owner of. This was a problem, so I've fixed it for everyone. If you're the owner of a project, you can now delete tags from the command line as expected. `git push origin :refs/tags/mytag` There's no way in the Gerrit UI to delete a tag, but this is the standard Git way to do it. -Chad

2 3

Thanksgiving - many staffers away tomorrow & Friday
by Sumana Harihareswara 21 Nov '12

21 Nov '12

Because of US Thanksgiving https://en.wikipedia.org/wiki/Thanksgiving , many Wikimedia Foundation staffers in the US will be unavailable tomorrow and Friday November 22 & 23. If you have an emergency, I believe #wikimedia-tech will be responsive, just as it is on weekends. -- Sumana Harihareswara Engineering Community Manager Wikimedia Foundation

1 0

What other FLOSS projects does the WMF contribute to/collaborate with?
by Arthur Richards 21 Nov '12

21 Nov '12

I'm helping to organize an 'intro to FLOSS' orientation module for new hires at the WMF with Tomasz Finc, and we're curious to know what all the projects are that the we collaborate with/contribute to. Just from my own experience working at the WMF, I know of a handful like Drupal, CiviCRM, Phonegap, Varnish, jQuery, Squid, Open Stack... but I'm very curious to know the others. Thanks! -- Arthur Richards Software Engineer, Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687

6 6

← Newer
1
2
3
4
5
6
7
8
...
14
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l November 2012