Thanks for this link Nemo!
I've discovered a caveat over the weekend which seems to make these
results more unreliable.
In test A, the success event is logged and then the page reloaded.
Concerned that the event might be lost in the page reload and skew the
results I decided to look at the 'save' event which is logged whenever
the user hits the save button. In comparison the 'success' event is
only logged when the save is successful.
Pre A-B test, successful save events [1]
0 14350
1 16518
(30868 saves in both groups, 2168 more in group A, 7%)
Post A/B test successful save events [2]
0 13462
1 13273
(26735 saves in both groups, 189 more in group B - 0.07%)
New users pre A-B test successful save events [3]
0 6216
1 6362
(12578 saves in both groups, 146 more in group A - 1%)
New users post A-B test successful save events [4]
0 4870
1 4772
(9642 saves in both groups, 98 more in group B - 1%)
These show little variance amongst the groups and lead me to believe
that the save data I posted earlier cannot be relied on.
I've written a patch to ensue the events get logged appropriately [A]
but I think from this my own conclusion is that ajax loading a page
via JavaScript after an edit only makes a significant difference (if
no difference at all) to the number of edits a user goes on to make.
We can probably safely kill it without concern.
[A]
https://gerrit.wikimedia.org/r/162164
[1] select event_isTestA, count(*) from MobileWebEditing_8599025 where
event_action = 'save' and timestamp > 20140827000000 and timestamp
< 20140904000000 and event_editor = 'SourceEditor' and wiki ='enwiki'
and event_mobileMode = 'stable' group by event_isTestA
[2] select event_isTestA, count(*) from MobileWebEditing_8599025 where
event_action = 'save' and timestamp > 20140912000000 and timestamp
< 20140919000000 and event_editor = 'SourceEditor' and wiki ='enwiki'
and event_mobileMode = 'stable' group by event_isTestA
[3] select event_isTestA, count(*) from MobileWebEditing_8599025 where
event_action = 'save' and timestamp > 20140827000000 and timestamp
< 20140904000000 and event_editor = 'SourceEditor' and wiki ='enwiki'
and event_userEditCount < 5 and event_mobileMode = 'stable' group by
event_isTestA
[4] select event_isTestA, count(*) from MobileWebEditing_8599025 where
event_action = 'save' and timestamp > 20140912000000 and timestamp
< 20140919000000 and event_editor = 'SourceEditor' and wiki ='enwiki'
and event_userEditCount < 5 and event_mobileMode = 'stable' group by
event_isTestA
On Fri, Sep 19, 2014 at 11:27 PM, Federico Leva (Nemo)
<nemowiki(a)gmail.com> wrote:
Thanks. For reference, the tracking bug for this area
is: Bug 14123 – Make
MediaWiki more AJAXy
https://bugzilla.wikimedia.org/showdependencytree.cgi?id=14123&hide_res…
Hoo *tripled* the number of patrollers in en.wiki by making patrol AJAXy
(bug 7851/15936), so it's definitely an area worth working on.
Nemo