Wikitech-l December 2006

wikitech-l@lists.wikimedia.org

91 participants
156 discussions

by Travis Derouin

Hi, We're evaluating our server configuration and considering making some changes and I was wondering if anyone had any input. Right now we're running on 4 servers, (mostly 32 bit Xeon processors): 1 for Squid, 2 for Apache and 1 for MySQL. We have 4GB of RAM for each server, and run memcached on one of the Apache servers, and run eaccelerator on both Apache servers. We're starting to see some symptoms of less reliable service from Apache/PHP similar to what we were experiencing before we added our 2nd Apache server - occasional high loads on the servers that are solved by restarting Apache. Given our traffic has increased by 50% since then, we're thinking of adding some more hardware. We're running Squid 2.6 (no epoll -yet), Apache 2.2, PHP 5.1.2 and Redhat ES 4 as well. We are thinking of moving to all 64 bit servers and adding a 3rd Apache server. For the Apache/PHP environment, is the performance of the 64 bit processor substantially better than the 32-bit? Would we get much more performance by adding a 2nd 64-bit processor, or would we encounter other system bottlenecks with PHP before we would experience the benefits of the 2nd CPU? I assume mostly for Apache/PHP that this setup is mostly processor bound, right? Is there any calculation that we can do to measure how many Apache servers we should have based on traffic? Squid is only serving on average just less than 100 requests per second, but sometimes goes over 150. Traffic is relatively variable from day to day, and can vary as much as 50% between consecutive days. Squid is effective in shielding traffic from the Apache servers - but they still occasionally get a very slow and the load on the servers can get quite high. Also, is there a rule of thumb for how many Apache servers each Squid can handle before moving to multiple Squids? As for RAM - any recommendations? It doesn't seem like we're using a lot of the 4GB on any of our servers - so I don't think we need more than 4GB. Are there techniques we can be using to use RAM more efficiently in addition to using memcached? Thanks!

17 years, 4 months

Startup looking for Superstar Wiki Developer (SF Bay Area)

by Michael Sha

We are a consumer web startup that's building a lot of really interesting features on top of an existing wiki platform (Mediawiki is one of the five we've narrowed it down to) and we're looking for a superstar developer with wiki experience to join the team. *We are a company:* Who wants to use the power of Wiki to solve a big problem for consumers. Part of our vision involves extending the functionality of one of the major wiki platforms with innovative features that pose exciting technical challenges. Be one of the first people to join a consumer web startup which could put a lot of large companies out of business. *You are someone who:* - Is *scary smart* and someone we can *trust* will find the *right answers* to hard problems you may have never solved before. - Is *technically brilliant*, *versatile*, and* creative*. - Your * teammates* love working with. You also have *friends* like you who would *love* to *work with you* as we continue to grow. - Will *passionately bust your ass* at the prospect of building something that will revolutionize a $100B+ industry. Specifically, we need someone who has a lot of valuable* experience*building and extending the feature set of wiki platforms. You also need experience designing and building highly-available and fault tolerant *consumer web *applications with *performance*, *scalability*, * stability*, and *security* in mind. This means you're very adept at: - Building and extending the functionality of a wiki platform - Database experience (optimizing SQL and writing software that uses SQL to access databases) - Distributed systems (hands-on experience preferred, academic experience is OK) - C/C++ - LAMP (PHP, Python, Perl) Past experience with the following will make you even better able to contribute to the team: - Innovative approaches to UI design (AJAX, Flex/Actionscript/Flash) - Designing and implementing production websites (HTML, CSS, XML, Javascript, etc.) - Database admin experience (setting up, maintaining and optimizing Oracle, MySQL or PostgreSQL databases) *You will:* - Start with ideas, design products and features, architect a technical solution, then build, test, and launch it. - Help build and grow the engineering team as we expand over time. If you also want to lead this team, that's in the cards, if you show us you can. *We will:* - Be a group of the smartest, most trustworthy, passionate, and interesting people you know. - Make sure you have what you need to be happy and productive. - Compensate you with a flexible mixture of cash and equity. - Make great teammates united around a vision of how to solve some pretty important problems for people. Interested candidates should send your resume to stealth.job(a)gmail.com<stealth_mode(a)gmail.com>with Wiki Guru in the subject line. Please also include in your email a summary of your past work with wiki platforms (which ones you've worked with and what your contributions have been to them).

17 years, 4 months

MediaWiki automated test run failure 2006-12-04

by brion＠pobox.com

An automated run of parserTests.php showed the following failures: Reading tests from "/home/brion/src/wiki/phase3/maintenance/parserTests.txt"... Running test TODO: Table security: embedded pipes (http://mail.wikipedia.org/pipermail/wikitech-l/2006-April/034637.html)... FAILED! Running test TODO: Link containing double-single-quotes '' (bug 4598)... FAILED! Running test TODO: Template with thumb image (with link in description)... FAILED! Running test TODO: message transform: <noinclude> in transcluded template (bug 4926)... FAILED! Running test TODO: message transform: <onlyinclude> in transcluded template (bug 4926)... FAILED! Running test BUG 1887, part 2: A <math> with a thumbnail- math enabled... FAILED! Running test TODO: HTML bullet list, unclosed tags (bug 5497)... FAILED! Running test TODO: HTML ordered list, unclosed tags (bug 5497)... FAILED! Running test TODO: HTML nested bullet list, open tags (bug 5497)... FAILED! Running test TODO: HTML nested ordered list, open tags (bug 5497)... FAILED! Running test TODO: Parsing optional HTML elements (Bug 6171)... FAILED! Running test TODO: Inline HTML vs wiki block nesting... FAILED! Running test TODO: Mixing markup for italics and bold... FAILED! Running test TODO: 5 quotes, code coverage +1 line... FAILED! Running test TODO: dt/dd/dl test... FAILED! Running test TODO: Images with the "|" character in the comment... FAILED! Running test TODO: Parents of subpages, two levels up, without trailing slash or name.... FAILED! Running test TODO: Parents of subpages, two levels up, with lots of extra trailing slashes.... FAILED! Running test TODO: Don't fall for the self-closing div... FAILED! Running test TODO: Always escape literal '>' in output, not just after '<'... FAILED! Reading tests from "/home/brion/src/wiki/phase3/extensions/Cite/citeParserTests.txt"... Passed 451 of 471 tests (95.75%)... FAILED!

17 years, 4 months

Re: [Wikitech-l] [MediaWiki-CVS] SVN: [18144] trunk/extensions/SelectCategory

by Rob Church

On 03/12/06, leon(a)svn.wikimedia.org <leon(a)svn.wikimedia.org> wrote: > Minor fixes, to avoid warnings. It's too lazy to just surpress them :-P Doesn't it make you feel all warm and cosy inside, knowing you've written better code? Rob Church

17 years, 4 months

MediaWiki automated test run failure 2006-12-03

by brion＠pobox.com

17 years, 4 months

Re: [Wikitech-l] [MediaWiki-CVS] SVN: [18117] trunk/phase3/includes/Linker.php

by Rob Church

On 02/12/06, werdna(a)svn.wikimedia.org <werdna(a)svn.wikimedia.org> wrote: > Update linker to make contributions links (in userToolLinks and userLink) and block links (blockLink) use subpage notation, rather than pass the target as a parameter Please be careful when converting things *to* those "subpage" forms, since we've had some problems in the past caused by long titles/usernames which cause the resulting title to be rejected, apparently crippling users' ability to delete/move/block them. Rob Church

17 years, 4 months

External storage consistency error

by Brion Vibber

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A problem was reported to us today which I have tracked down to a configuration consistency problem. First some background: We use "external storage" servers to store the bulk of page text contents for Wikimedia's wikis. These are web servers where we make use of otherwise unused disk space by sticking on a copy of MySQL and storing compressed text blobs. For reliability, they come in clusters, using MySQL's replication. So if one dies, we can shuffle them about with no loss of data. New pages get saved into the most recent one or two ES clusters, and every once in a while we close an old one off and add a new one on the end. The current write clusters are cluster8 and cluster9. The DB master for cluster8 was srv89, which had recently had some problems: it was apparently overloaded, and didn't respond to login attempts. On November 28, we had it rebooted, but it didn't come back online. Cluster8 was reshuffled, with the master moved to srv87. At about 22:30 UTC, srv89 came back online. Unfortunately this meant that its web server came online as well, and apparently made it back into the load balancing pool -- but with old copies of the MediaWiki configuration. srv89's MediaWiki still had the configuration files telling it to save text blobs into the external storage server on... srv89, itself. So, some portion of pages saved during an hour or so had their items saved into srv89 instead of srv87. When other web servers then came to read the pages back, they read from srv87 or its clone srv88, and received a _different page_ with the _same blob ID number_. The good news is that the data is probably recoverable, on this procedure: * For each wiki, list blobs which differ between srv87 and srv88 * For each mismatched blob, locate the two (or more) revisions using it * Disambiguate them somehow... (probably mostly automatable... i hope!) * Copy the srv88 blobs onto srv87, and reassign the revisions to the appropriate new blob ID numbers I'm not sure how many pages are affected, but it appears to be 25 edits on fr.wikipedia.org, so perhaps a few hundred overall. This could most likely have been avoided by forcing web servers to sync their MediaWiki configurations before starting Apache on boot. - -- brion vibber (brion @ pobox.com / brion @ wikimedia.org) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFcMQcwRnhpk1wk44RAgp0AKDgCNa8JWjmGmK0QhSuloQMYLPjdgCglhf6 uPsFhWJFxhXuvOFUdLzlgPE= =HhJ5 -----END PGP SIGNATURE-----

17 years, 4 months

MediaWiki automated test run failure 2006-12-02

by brion＠pobox.com

17 years, 4 months

Strange issue with article [[Lifting body]] on en

by George Herbert

I am getting a strange problem with [[Lifting body]] on en.wikipedia ( http://en.wikipedia.org/wiki/Lifting_body ); the article has history, talk page, etc., but any attempt to go right to it gives the "This article does not exist" page. DB burp? Cache problem? -- -george william herbert george.herbert(a)gmail.com

17 years, 4 months

Re: [Wikitech-l] [WikiEN-l] Time for preformatted empty articles?

by David Gerard

On 29/11/06, MacGyverMagic/Mgm <macgyvermagic(a)gmail.com> wrote: > We should probably still kill a lot of unsourced edits. Such a preload > template is already in use at AFC and it is routinely ignored, but better to > try and convince some people to follow it. It didn't solve the problem, but > it cut back on problem cases slightly. Yeah. The nice thing about a preload template is that it can guide new editors without being a straightjacket on experienced ones - it's just text in the edit box, after all. Something like (off the top of my head): '''{{PAGENAME}}''' is ... [one-sentence definition]. It is ... ==More detail== [Put more detail about important aspects of {{PAGENAME}} here. Use ==section-name== around new sections.] ==References== [List the sources you used in writing this article] ==External links== [List the most important couple of external webpages on the subject.] This shows people how to give us the useful content for a good stub article without having to know all about editing wikitext and lovely formatting. wikitech-l - is it possible yet to do a preload when someone hits 'edit' on a red link? If so, what do we need for this to be switched on for en:wp? It is likely time for such a thing. - d.

17 years, 4 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l December 2006