Dear All,
I need some help tracking down a DB deadlock on a single-server,
"private" Wiki installation running latest MediaWiki download
from http://www.mediawiki.org/wiki/Download
I have been referred to this mailing list from #mediawiki at
irc.freenode.org, thanks for the pointer!
Installed versions:
-------------------
- MediaWiki: 1.23.2
- VisualEditor from Git, branch REL1_23, ID 9883566
- Parsoid from Git, branch REL1_23, ID 8da3673
- parsoid service from Git, branch master, ID df35443
- mysql: 5.5.35, Debian wheezy package
- apache: 2.2.22, Debian wheezy package
- PHP: 5.4.4, Debian wheezy package
- OS: Linux 3.2.0-4-686-pae #1 SMP Debian 3.2.57-3 i686 GNU/Linux
MediaWiki, Parsoid (nodejs) service and mysql database are all on the
same server. Setup followed the instructions on
https://www.mediawiki.org/wiki/Parsoid/Setup
Problem description:
--------------------
Logging in, reading and editing using the standard (not visual) edit
works fine.
Clicking "Edit", which starts the visual editor, results in no
reaction at all for approx 5 seconds, then the page goes dim and a
progress indicator appears at the top right corner.
After another approx. 100 seconds, the page returns to normal state
as if opened or reading.
Directly accessing the Parsoid service, here running on port 8000,
promptly delivers the page.
Preliminary investigation:
--------------------------
Enabling debugging in LocalSettings.php and tweaking wfDebugTimer()
to get absolute millisecond timestamps and client address/port info
in the log file showed the following sequences of activities
(anonymized log files available if needed):
Summarized:
- client request page by calling
"GET /wiki/api.php?format=json&action=visualeditor&paction=parse..."
- MediaWiki starts processing and class the Parsoid service with
"GET /testwiki/TestPage?oldid=4053 HTTP/1.0"
- Parsoid service calls
"GET /wiki/api.php?format=json&action=query&prop=revisions&..."
- call from Parsoid server into MediaWiki API deadlocks with
client call into API.
The question is why is it deadlocking and how to prevent the deadlock.
Detailed activities analysis:
- (1) client accesses
"GET /wiki/api.php?format=json&action=visualeditor&paction=parse..."
- (2) MediaWiki starts some session bookkeeping, including opening a
DB transaction with "BEGIN" and performing several queries in
the transaction.
- (3) MediaWiki/1.23.2 calls the parsoid service with
"GET /testwiki/TestPage?oldid=4053 HTTP/1.0"
- (4) Parsoid/0.1 calls
"GET /wiki/api.php?format=json&action=query&prop=revisions&..."
- (5) MediaWiki starts processing (in a new apache/PHP slave) and opens
a new DB transaction with "BEGIN" performing the same initial
"UPDATE `wiki_user` SET user_touched ..." query as first query in
the transaction as in step (2).
- (6) approx. 40 seconds pass
- (7) Parsoid service drops connection from (4) to MediaWiki
(TCP FIN packet observed by tcpdump while capturing the on-wire
conversation between MediaWiki and the client / the Parsiod service)
- (8) Parsoid/0.1 calls again
"GET /wiki/api.php?format=json&action=query&prop=revisions&..."
from a new connection (client port number changed)
- (9) MediaWiki starts processing (in a new apache/PHP slave) and opens
a new DB transaction with "BEGIN" performing the same initial
"UPDATE `wiki_user` SET user_touched ..." query as first query in
the transaction as in step (2) and (5).
- (10) approx. 10 seconds pass
- (11) MediaWiki receives a
"SQL ERROR: Lock wait timeout exceeded; try restarting transaction"
relating to transaction from step (5)
- (12) MediaWiki attempts a ROLLBACK of (5) and tries to send a
"HTTP 500" (content captured by tcpdump) but the connection is
already dead.
- (13) approx. 30 seconds pass
- (14) Parsoid service drops connection from (8) to MediaWiki
(TCP FIN packet observed by tcpdump while capturing the on-wire
conversation between MediaWiki and the client / the Parsiod service)
- (15) Parsoid/0.1 calls again
"GET /wiki/api.php?format=json&action=query&prop=revisions&..."
from a new connection (client port number changed)
- (16) MediaWiki starts processing (in a new apache/PHP slave) and
opens a new DB transaction with "BEGIN" performing the same initial
"UPDATE `wiki_user` SET user_touched ..." query as first query in
the transaction as in step (2), (5) and (9).
- (17) approx. 10 seconds pass
- (18) MediaWiki receives a
"SQL ERROR: Lock wait timeout exceeded; try restarting transaction"
relating to transaction from step (9)
- (19) MediaWiki attempts a ROLLBACK of (9) and tries to send a
"HTTP 500" (content captured by tcpdump) but the connection is
already dead.
- (20) approx. 10 seconds pass
- (21) MediaWiki closes the connection to the Parsoid service from
step (3), without any data exchange after receiving the initial
"GET /testwiki/TestPage?oldid=4053 HTTP/1.0"
- (22) MediaWiki performs a DB "COMMIT" in the original client session
from step (2), after 104 seconds.
- (23) MediaWiki continues processing the session from step (16), at
least in part executing the same set of queries as in step (2).
- (24) MediaWiki performs a DB "COMMIT" in the Parsoid client session
from step (15), after 20 seconds.
- (25) MediaWiki sends a "HTTP 200" with a JSON body to the Parsoid
service as answer to request from step (15).
- (26) Parsoid service logs messages "starting parsing", a parse tree
and "completed parsing in 101234 ms". (Note: It is unclear when in
the range (21) to (26) these messages are logged, but I think this
position "(26)" is most logical.) At earlier times, Parsoid logs
the two messages "Failed API request, 8 retries remaining." and
"Failed API request, 7 retries remaining." and not precisely known
time steps, probably between (7)/(8) and (14)/(15), respectively.
Question:
---------
What could cause this behavior and how should I configure my system to
prevent the deadlocks? If this is a Bug in either MediWiki or the
VisualEditor or Parsoid, how to further investigate and fix it?
Best regards
Björn
--
| Bjoern Kahl +++ Siegburg +++ Germany |
| "googlelogin@-my-domain-" +++ www.bjoern-kahl.de |
| Languages: German, English, Ancient Latin (a bit :-)) |
(Duplicating this mail, as I wasn't subscribed to mobile-l a minute ago.)
On Sat, 16 Aug 2014, at 12:45, Nkansah Rexford wrote:
> [...]
> The Wikipedia app is currently under good development and I think its doing
> great.
We don't need apps. We need mobile websites which work as good as an app does.
Oh, the waste of effort.
I truly pledge you to work together to make a website which is so good that an 'app' is redundant.
svetlana
I agree with you too on that. apps are redundant, unless one has the
resources to support. 'Nativity' is great and can unleash some potentials,
which are hard to accomplish via browsers.
I personally find the mobile site for Wikipedia great enough and I use it
for editing instead of the native app.
rexford | google.com/+Nkansahrexford | sent from smartphone
On Aug 16, 2014 9:28 AM, "svetlana" <svetlana(a)fastmail.com.au> wrote:
On Sat, 16 Aug 2014, at 12:45, Nkansah Rexford wrote:
> [...]
> The Wikipedia app is currently under good development and I think its
doing
> great.
We don't need apps. We need mobile websites which work as good as an app
does.
Oh, the waste of effort.
I truly pledge you to work together to make a website which is so good that
an 'app' is redundant.
svetlana
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hello and welcome to the latest edition of the WMF Engineering Roadmap
and Deployment update.
The full log of planned deployments next week can be found at:
<https://wikitech.wikimedia.org/wiki/Deployments#Week_of_Aug_18th>
A quick list of notable items...
== Tuesday ==
* MediaWiki deploy
** group1 to 1.24wmf17: All non-Wikipedia sites (Wiktionary, Wikisource,
Wikinews, Wikibooks, Wikiquote, Wikiversity, and a few other sites)
** <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf17>
* Wikidata
** Badges support will be rolled out, see:
*** <https://lists.wikimedia.org/pipermail/wikidata-l/2014-August/004331.html>
** Wikinews will get sitelinks via Wikidata (aka "Wikidata phase 1")
== Wednesday ==
* Weekly fundraising banner test
== Thursday ==
* MediaWiki deploy
** group2 to 1.24wmf17 (all Wikipedias)
** group0 to 1.24wmf18 (test/test2/testwikidata/mediawiki)
Thanks and as always, questions and comments welcome,
Greg
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Suddenly I cannot run PHPUnit tests on vagrant
> cd /vagrant/mediawiki/tests/phpunit && /usr/bin/php5 phpunit.php --configuration /vagrant/mediawiki/extensions/Echo/tests/echo.suite.xml --group=Echo
No MWMultiVersion instance initialized! MWScript.php wrapper not used?
The documentation doesn't seem up to date...
https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Running_the_unit_tes…
Hello!
This past week at the hackathon we got a lot of things done related to
configuration, thank you to everyone who contributed to discussions,
worked on patchsets, or did code review!
This is just a list of things I can remember, we did much more. Some of
these aren't directly related to configuration, but will make it easier
for people to use the Config classes, especially in extensions.
* Converted most of includes/specials/ to use Config interface instead
of globals - a bunch of patches
* A bunch of other misc. classes were converted to use Config instead of
globals - another bunch of patches
* API modules can now be registered with factory functions to allow for
dependency injection - https://gerrit.wikimedia.org/r/#/c/149183/
* Skins can now be registered with factory functions to allow for
dependency injection - https://gerrit.wikimedia.org/r/#/c/153060/
* Patch for SpecialPages to use a factory function was started -
https://gerrit.wikimedia.org/r/#/c/152755/
* Added a JSONContentHandler to core -
https://gerrit.wikimedia.org/r/#/c/152933/
* Discussed creating a MultiConfig class to handle fallback logic -
https://gerrit.wikimedia.org/r/153541 (WIP)
* Created a "Configuration" component in Bugzilla
* Created a tracking bug for converting things to use Config classes
instead of globals -
https://bugzilla.wikimedia.org/show_bug.cgi?id=config-obj
If you have time, code review would be appreciated:
<https://gerrit.wikimedia.org/r/#/q/topic:conf+status:open,n,z>!
Thanks,
-- Legoktm
Hi,
I propose some constructive ideas to improve the deployment of new features:
* granular deployments: create "user profiles" where the users can choose
if they want an overall appearance:
* "never ever change my interface": some experienced authors do not like
when one change every month their workflow if they are happy with it,
* "experienced editor": some experienced editors want new features or see
what the newbies see,
* "newbie": the newbies/editors-to-be could expect an editing environment
possibly different than the reader environment,
* "reader": the readers have their own expectations for easy reading,
* etc.
The features could be deployed only for some groups, giving more flexibility
to deploy "reader features" for readers, etc. Obviously there are
preferences, but the newbies have no experience about it, and the
experienced editors have to be discover new preferences on a case-by-case
basis, making it difficult to everybody to track the preferences.
* implement global preferences (and the possibility to change locally or
globally, like in Mailman) [bug 14950][]
* when a new feature is introduced, propose to users (not in "never ever
change my interface") if they want the new feature, locally or globally,
possibly using the Notifications bar, or with some message in the prefs
page and highlighting it on the prefs page
* work on a better organisation of the preferences, e.g. add an exhaustive
preference panel similarly to Firefox’s about:config to permit the
developers to add more preferences and hence offering more customisation
possibilities for advanced users, by nullifying the argument "the
preferences page is too complicated for new users"
* as it was proposed, add a review process for the gadgets+JS pages to avoid
performance, security, usability issues, possibly with the help of the tech
staff, and possibly with the general MediaWiki code review
(gerrit/Phabricator) with some gateway between it and the MediaWiki
websites [bug 69445][] [bug 20153][]
In other words, improve the deployment targets and give easy choices to users
to opt-in/opt-out/etc the new features depending on their willingness to
change their environment.
And although I’m neither a loudly people neither the community, I vote to
remove the superprotect right and any other enforcement of this type in the
future. It’s like an edit war where one party has the power to silence the
other, and like all edit wars there are at least two wrong versions.
[bug 69445]: https://bugzilla.wikimedia.org/show_bug.cgi?id=69445
[bug 14950]: https://bugzilla.wikimedia.org/show_bug.cgi?id=14950
[bug 20153]: https://bugzilla.wikimedia.org/show_bug.cgi?id=20153
~ Seb35
Hi everybody,
So we had a discussion about this a while ago, but just recently PHP let
out the final 5.3 release. [0] Back in the previous thread concerning PHP
5.3, there seemed to be general agreement toward upping our PHP minimum
requirements for the 1.24 release of MediaWiki.
Here are the stats:
* Soon Ubuntu (trusty) and Debian (jessie) releases will be running PHP 5.5
and 5.6, respectively.
* MediaWiki 1.23 ends support in May 2017
* PHP 5.4 is estimated to be supported until 2015
* Ubuntu 12.04 LTS (PHP 5.3) ends support on April 26, 2017
* Debian Squeeze (PHP 5.3) ends support in February 2016
* Debian Wheezy (PHP 5.4) *might* be supported until May 2018, depending on
the feedback received from the Squeeze LTS trial
The results of this timeline are that when MediaWiki 1.23 ends support,
most supported distros will be on PHP 5.5 or higher (yes, not 5.4, but 5.5).
With that in mind, I'd like to move to raise our PHP minimum version.
Unless there are people that disagree, it is no longer a question of
whether to raise it or not, but rather what we want to raise it to. I
believe we have a couple of options:
1) Raise to PHP 5.5 for MW 1.24
2) Raise to PHP 5.4 for MW 1.24, and then when a release with support past
2018 is made, go to 5.5.
Option 2 takes advantage of the possible gap in coverage we will face
should Debian Wheezy receive LTS support.
Thoughts?
[0] http://php.net/archive/2014.php?050329#id2014-08-14-1
*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
FYI, Lila had chosen to engage in discussion on her meta talk page.
Numerous editors are commenting there. Discussion also continues on the
meta RFC and on the English Wikipedia arbitration workshop page.
Pine
On Aug 14, 2014 12:03 AM, "Russavia" <russavia.wikipedia(a)gmail.com> wrote:
Erik
On Thu, Aug 14, 2014 at 5:32 AM, Erik Moeller <erik(a)wikimedia.org> wrote:
>
> This is why on all major sites, you see a gradual ramp-up of a new
> feature, and continued improvement once it's widely used. Often
> there's an opt-in and then an opt-out to ease users into the change.
> But once a change is launched, it very rarely gets rolled back unless
> it's just clearly not doing what it's supposed to.
Are you are familiar with the Flickr experience in the last 12 months by
any chance? I think that is a very pertinent and prominent example of what
goes against what you say. The Flickr attitude was much the same as the
WMF's. That ended up in a revolt, much like the WMF is seeing against it.
In the end, they ended up doing what Erik?
Also, the other day I received a Flickr email from someone wishing to use
an image which I had not taken, but which I had uploaded to Commons. They
mentioned that they saw the photo on Commons.
When I told them that I am not the author, and that they would need to
contact Joe Bloggs, their response: "I'm sorry, this is SO confusing to me."
I put that down to MediaViewer and its adding irrelevant information, and
also the fact that file information is more difficult to find.
Russavia
_______________________________________________
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
<mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>