Hi list,
I recently helped shepherd a new skin into Mediawiki;
https://www.mediawiki.org/wiki/Skin:Erudite
I would like to think it is perfect in every way, but one area it
hasn't had any love is making sure it works well on mobile devices.
To the best of my knowledge it shouldn't do anything particularly
heinous, but I don't have any such devices to test it on. If anybody
has a few minutes and could have a play, and see if anything could
be improved, please do and let me know on list.
A live example of it is at
https://www.dur.ac.uk/nick.white/erudite-demo/mediawiki-current/
which should have full editing etc rights enabled. It's a dummy
wiki, so add whatever you like.
Thanks in advance!
Nick White
Re-trying to send this as the message was rejected several times and I
have no idea why.
---------- Forwarded message ----------
From: Guillaume Paumier <gpaumier(a)wikimedia.org>
Date: Thu, Mar 28, 2013 at 5:53 PM
Subject: Please read: Proposal to consolidate Mobile projects documentation
To: mobile-l(a)lists.wikimedia.org
Hi,
A few weeks ago, Tomasz asked me to investigate the possibility of
consolidating the Mobile projects documentation, currently split
between meta and mediawiki.org.
I've done some inventorizing and planning:
https://www.mediawiki.org/wiki/Technical_communications/Mobile_documentatio…
What I'd like to ask you guys now is the following:
* Do you see any reason not to move forward with this plan?
* Could you take a look at the content on meta and delete what's
obsolete (and doesn't need archiving)?
* When would be the best time to do the actual move? I expect the move
to take about a day, and I'd like to choose a time that minimizes
disruption.
Let me know if you have any concerns or questions :)
--
Guillaume Paumier
The "add image to article" functionality in the mobile view is such an
awesome idea. I liked it so much I created a simple page suitable for
viewing on a mobile device that highlights local articles that need an
image:
http://inkdroid.org/ici/
It uses geonames' wikipedia api to figure out what articles are
relevant to your current location, and then the Wikipedia API
(currently just en) to determine if the article needs an image. In the
process I noticed that some articles that lack images but don't have
the handy "add image to article" button. For example:
https://en.m.wikipedia.org/wiki/Coastline_%28sculpture%29
Would it be difficult to add image upload support for articles like this?
//Ed
I re-ran the sessions job including IP in the output. Several things:
- I'm happy to report that we are correctly filtering out the WMF public
IPs, though there are about 100k hits per day from 10.x.x.x IPs (about
0.5%, LVS health checks) that we missed. We'll update the filter to include
those.
- So, who is it? I ran the IPs of the top sessions through whois and tried
to extract the org name. The results (omitting IP for privacy reasons) are
here:
https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ai_u2wTiMldddH…
A pretty interesting list.
--
David Schoonover
dsc(a)wikimedia.org
On Thu, Apr 25, 2013 at 10:38 AM, Haitham Shammaa <hshammaa(a)wikimedia.org>wrote:
> Maryana, that Wikipedia article is about a TV series which is being
> broadcasted since 2006, but I don't think it's very popular.
>
> On the other hand, nobody seems to mention the crab Big Daddy in the
> Japanese internet culture.
>
> *--*
> *Haitham Shammaa*
> *Contribution Research Manager*
> *Wikimedia Foundation*
>
> *Imagine a world in which every single human being can freely share in
> the sum of all knowledge. *
> *Click the "edit" button now, and help us make it a reality!*
>
>
> On Thu, Apr 25, 2013 at 10:19 AM, Maryana Pinchuk <mpinchuk(a)wikimedia.org>wrote:
>
>> On Wed, Apr 24, 2013 at 9:17 PM, Dario Taraborelli <
>> dtaraborelli(a)wikimedia.org> wrote:
>>
>>> Dave,
>>>
>>> thanks for sharing this, the referral data is particularly fascinating.
>>> I mentioned during the quarterly review that I'd love to get a better sense
>>> of (1) the proportion of requests in the mobile request logs lacking a
>>> referral, (2) the possible causes of this gap and (3) to what extent these
>>> missing entries introduce a bias in the referral ranking.
>>>
>>> The 3rd most popular query (according to your dumps) is ビッグダディ (japanese
>>> for "Big Daddy"), which presumably refers to this guy:
>>> http://metro.co.uk/2013/03/20/giant-japanese-spider-crab-big-daddy-arrives-…
>>> What's interesting is that there's no such entry on the japanese
>>> Wikipedia and I am baffled that people may have landed on the website via a
>>> search engine query for a non-existing article.
>>> Do you have an explanation for this or am I misinterpreting what you
>>> mean by search query?
>>>
>>
>> There *is* an article on this on ja.wiki :) It may have been renamed
>> since then, but it's still the 2nd Google hit for ビッグダディ:
>> http://ja.wikipedia.org/wiki/%E7%97%9B%E5%BF%AB!%E3%83%93%E3%83%83%E3%82%B0…
>>
>>
>>>
>>> Dario
>>>
>>> On Apr 24, 2013, at 8:40 PM, David Schoonover <dsc(a)wikimedia.org> wrote:
>>>
>>> Hiya all,
>>>
>>> As promised earlier today in the Analytics weekly showcase, I've got a
>>> few interesting bits of data to share from playing with the new Mobile Site
>>> Sessions dataset.
>>>
>>>
>>> # Visits to Mobile Site, 4/21/2013
>>>
>>> - Total Visits: 51,624,103
>>> - Unique Visitors: 37,736,120
>>> - Total Pageviews: 104,972,033
>>> - Avg Pageviews per Session: 2.0334
>>> - Max Pageviews in one Session: 141,882
>>>
>>> ## Standard Site
>>> - Visits: 51,603,221
>>> - Unique Visitors: 37,723,188
>>> - Pageviews: 104,910,382
>>> - Avg Pageviews per Session: 2.033
>>>
>>> ## Alpha Site
>>> - Visits: 986
>>> - Unique Visitors: 822
>>> - Pageviews: 7,087
>>> - Avg Pageviews per Session: 7.188
>>>
>>> ## Beta Site
>>> - Visits: 19,896
>>> - Unique Visitors: 16,235
>>> - Pageviews: 54,564
>>> - Avg Pageviews per Session: 2.742
>>>
>>>
>>> ## Notes
>>> - A session (or "visit") is defined as all activity with less than 30
>>> minutes between each hit. Intuitively speaking, a session ends when the
>>> user hasn't done anything in 30m.
>>> - As we do not set visitor_id cookies for all users, the "unique
>>> visitors" metric was calculated using hash(ip_address + users_agent) as
>>> visitor_id.
>>> - This job looked at all requests to the mobile site on 4/21/2013, which
>>> is 75.17 GB of request logs.
>>> - The job took ~17 minutes to process the day into 15.3 GB of sessions.
>>> - The summary above took maybe 10 minutes to set up/write in Hive, and
>>> the job took maybe 7 minutes.
>>>
>>>
>>> In addition to that summary, I ran a few jobs on the entry_referer field
>>> -- the URL that referred the user to us when the session started. Obvious
>>> caveats: this is only one day of data, and it's only the mobile site. Draw
>>> conclusions with care.
>>>
>>> First, I pulled out the top referring domains. It's mostly as you'd
>>> expect -- search engines -- though you'll also note that several Wikipedia
>>> mobile sites show up. My working hypothesis is that people don't tend to
>>> close tabs on smartphones; when they later come back, it is often to an
>>> open Wikipedia tab: clicking a link or perform a search means the referrer
>>> is still us.
>>>
>>> Since -- as expected -- so much of the data pertained to search engines,
>>> I also calculated the top search queries and top keywords that sent people
>>> to us. (For keywords, I've filtered out common "stop words": de, of, in,
>>> is, la, and, el, es, to, en, di, los, le, da, se, las, les, il, du, a, i,
>>> o, y, e.) In both, you see the predictable: lots of searches for porn, for
>>> "facebook", for "wiki", etc. But you also see a few things that surprised
>>> me:
>>>
>>> - Tons of Japanese. Japan is the most mobile-enabled country in the
>>> world so I guess we should have expected to see many searches in Japanese
>>> show up in the top queries. I've left them URL-encoded in the results --
>>> you'll see them as weird lines with % in them.
>>>
>>> - Apparently people search for movies and TV so they can spoil their fun
>>> by reading about them on Wikipedia. Both of "movies" and "film" show up in
>>> the top keywords; Iron Man 1, 2, AND 3 all show up in the top search
>>> queries. I didn't expect this was a major use-case, but -- wikigroaning
>>> aside -- it's an interesting fact.
>>>
>>> I'm sure we're only scratching the surface here. This is an exciting
>>> dataset, and I'm sure there's lots more to learn!
>>>
>>> The full results:
>>> - Top Referring Entry Domains:
>>> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
>>> - Top Referring Entry Search Queries:
>>> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
>>> - Top Referring Entry Search Keywords:
>>> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
>>>
>>> Questions are welcome!
>>>
>>>
>>> --
>>> David Schoonover
>>> dsc(a)wikimedia.org
>>> _______________________________________________
>>> Analytics mailing list
>>> Analytics(a)lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/analytics
>>>
>>>
>>>
>>
>>
>> --
>> Maryana Pinchuk
>> Associate Product Manager, Wikimedia Foundation
>> wikimediafoundation.org
>>
>> _______________________________________________
>> Analytics mailing list
>> Analytics(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/analytics
>>
>>
>
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>
On Wed, Apr 24, 2013 at 9:17 PM, Dario Taraborelli <
dtaraborelli(a)wikimedia.org> wrote:
> Dave,
>
> thanks for sharing this, the referral data is particularly fascinating. I
> mentioned during the quarterly review that I'd love to get a better sense
> of (1) the proportion of requests in the mobile request logs lacking a
> referral, (2) the possible causes of this gap and (3) to what extent these
> missing entries introduce a bias in the referral ranking.
>
> The 3rd most popular query (according to your dumps) is ビッグダディ (japanese
> for "Big Daddy"), which presumably refers to this guy:
> http://metro.co.uk/2013/03/20/giant-japanese-spider-crab-big-daddy-arrives-…
> What's interesting is that there's no such entry on the japanese Wikipedia
> and I am baffled that people may have landed on the website via a search
> engine query for a non-existing article.
> Do you have an explanation for this or am I misinterpreting what you mean
> by search query?
>
There *is* an article on this on ja.wiki :) It may have been renamed since
then, but it's still the 2nd Google hit for ビッグダディ:
http://ja.wikipedia.org/wiki/%E7%97%9B%E5%BF%AB!%E3%83%93%E3%83%83%E3%82%B0…
>
> Dario
>
> On Apr 24, 2013, at 8:40 PM, David Schoonover <dsc(a)wikimedia.org> wrote:
>
> Hiya all,
>
> As promised earlier today in the Analytics weekly showcase, I've got a few
> interesting bits of data to share from playing with the new Mobile Site
> Sessions dataset.
>
>
> # Visits to Mobile Site, 4/21/2013
>
> - Total Visits: 51,624,103
> - Unique Visitors: 37,736,120
> - Total Pageviews: 104,972,033
> - Avg Pageviews per Session: 2.0334
> - Max Pageviews in one Session: 141,882
>
> ## Standard Site
> - Visits: 51,603,221
> - Unique Visitors: 37,723,188
> - Pageviews: 104,910,382
> - Avg Pageviews per Session: 2.033
>
> ## Alpha Site
> - Visits: 986
> - Unique Visitors: 822
> - Pageviews: 7,087
> - Avg Pageviews per Session: 7.188
>
> ## Beta Site
> - Visits: 19,896
> - Unique Visitors: 16,235
> - Pageviews: 54,564
> - Avg Pageviews per Session: 2.742
>
>
> ## Notes
> - A session (or "visit") is defined as all activity with less than 30
> minutes between each hit. Intuitively speaking, a session ends when the
> user hasn't done anything in 30m.
> - As we do not set visitor_id cookies for all users, the "unique visitors"
> metric was calculated using hash(ip_address + users_agent) as visitor_id.
> - This job looked at all requests to the mobile site on 4/21/2013, which
> is 75.17 GB of request logs.
> - The job took ~17 minutes to process the day into 15.3 GB of sessions.
> - The summary above took maybe 10 minutes to set up/write in Hive, and the
> job took maybe 7 minutes.
>
>
> In addition to that summary, I ran a few jobs on the entry_referer field
> -- the URL that referred the user to us when the session started. Obvious
> caveats: this is only one day of data, and it's only the mobile site. Draw
> conclusions with care.
>
> First, I pulled out the top referring domains. It's mostly as you'd expect
> -- search engines -- though you'll also note that several Wikipedia mobile
> sites show up. My working hypothesis is that people don't tend to close
> tabs on smartphones; when they later come back, it is often to an open
> Wikipedia tab: clicking a link or perform a search means the referrer is
> still us.
>
> Since -- as expected -- so much of the data pertained to search engines, I
> also calculated the top search queries and top keywords that sent people to
> us. (For keywords, I've filtered out common "stop words": de, of, in, is,
> la, and, el, es, to, en, di, los, le, da, se, las, les, il, du, a, i, o, y,
> e.) In both, you see the predictable: lots of searches for porn, for
> "facebook", for "wiki", etc. But you also see a few things that surprised
> me:
>
> - Tons of Japanese. Japan is the most mobile-enabled country in the world
> so I guess we should have expected to see many searches in Japanese show up
> in the top queries. I've left them URL-encoded in the results -- you'll see
> them as weird lines with % in them.
>
> - Apparently people search for movies and TV so they can spoil their fun
> by reading about them on Wikipedia. Both of "movies" and "film" show up in
> the top keywords; Iron Man 1, 2, AND 3 all show up in the top search
> queries. I didn't expect this was a major use-case, but -- wikigroaning
> aside -- it's an interesting fact.
>
> I'm sure we're only scratching the surface here. This is an exciting
> dataset, and I'm sure there's lots more to learn!
>
> The full results:
> - Top Referring Entry Domains:
> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
> - Top Referring Entry Search Queries:
> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
> - Top Referring Entry Search Keywords:
> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
>
> Questions are welcome!
>
>
> --
> David Schoonover
> dsc(a)wikimedia.org
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>
>
--
Maryana Pinchuk
Associate Product Manager, Wikimedia Foundation
wikimediafoundation.org
Dario, https://code.google.com/p/browsersec/wiki/Part2 may offer some clues
on missing Referer data in the common cases. Just search for the term *
Referer* on that page. I suspect, but would require David's expertise to
know, that app or otherwise out-of-band mobile context-to-user agent
transitions (e.g., invocation of protocol handlers from a native search
screen) may also have something to do with this, at least after filtering
out screen scrapers and other bots.
I see that Asher asks about filtering out that stuff on a separate reply,
which would allow would make the long tail more uniformly distributed in
the hit plot.
On Wed, Apr 24, 2013 at 9:17 PM, Dario Taraborelli <
dtaraborelli(a)wikimedia.org> wrote:
> Dave,
>
> thanks for sharing this, the referral data is particularly fascinating. I
> mentioned during the quarterly review that I'd love to get a better sense
> of (1) the proportion of requests in the mobile request logs lacking a
> referral, (2) the possible causes of this gap and (3) to what extent these
> missing entries introduce a bias in the referral ranking.
>
> The 3rd most popular query (according to your dumps) is ビッグダディ (japanese
> for "Big Daddy"), which presumably refers to this guy:
> http://metro.co.uk/2013/03/20/giant-japanese-spider-crab-big-daddy-arrives-…
> What's interesting is that there's no such entry on the japanese Wikipedia
> and I am baffled that people may have landed on the website via a search
> engine query for a non-existing article.
> Do you have an explanation for this or am I misinterpreting what you mean
> by search query?
>
> Dario
>
> On Apr 24, 2013, at 8:40 PM, David Schoonover <dsc(a)wikimedia.org> wrote:
>
> Hiya all,
>
> As promised earlier today in the Analytics weekly showcase, I've got a few
> interesting bits of data to share from playing with the new Mobile Site
> Sessions dataset.
>
>
> # Visits to Mobile Site, 4/21/2013
>
> - Total Visits: 51,624,103
> - Unique Visitors: 37,736,120
> - Total Pageviews: 104,972,033
> - Avg Pageviews per Session: 2.0334
> - Max Pageviews in one Session: 141,882
>
> ## Standard Site
> - Visits: 51,603,221
> - Unique Visitors: 37,723,188
> - Pageviews: 104,910,382
> - Avg Pageviews per Session: 2.033
>
> ## Alpha Site
> - Visits: 986
> - Unique Visitors: 822
> - Pageviews: 7,087
> - Avg Pageviews per Session: 7.188
>
> ## Beta Site
> - Visits: 19,896
> - Unique Visitors: 16,235
> - Pageviews: 54,564
> - Avg Pageviews per Session: 2.742
>
>
> ## Notes
> - A session (or "visit") is defined as all activity with less than 30
> minutes between each hit. Intuitively speaking, a session ends when the
> user hasn't done anything in 30m.
> - As we do not set visitor_id cookies for all users, the "unique visitors"
> metric was calculated using hash(ip_address + users_agent) as visitor_id.
> - This job looked at all requests to the mobile site on 4/21/2013, which
> is 75.17 GB of request logs.
> - The job took ~17 minutes to process the day into 15.3 GB of sessions.
> - The summary above took maybe 10 minutes to set up/write in Hive, and the
> job took maybe 7 minutes.
>
>
> In addition to that summary, I ran a few jobs on the entry_referer field
> -- the URL that referred the user to us when the session started. Obvious
> caveats: this is only one day of data, and it's only the mobile site. Draw
> conclusions with care.
>
> First, I pulled out the top referring domains. It's mostly as you'd expect
> -- search engines -- though you'll also note that several Wikipedia mobile
> sites show up. My working hypothesis is that people don't tend to close
> tabs on smartphones; when they later come back, it is often to an open
> Wikipedia tab: clicking a link or perform a search means the referrer is
> still us.
>
> Since -- as expected -- so much of the data pertained to search engines, I
> also calculated the top search queries and top keywords that sent people to
> us. (For keywords, I've filtered out common "stop words": de, of, in, is,
> la, and, el, es, to, en, di, los, le, da, se, las, les, il, du, a, i, o, y,
> e.) In both, you see the predictable: lots of searches for porn, for
> "facebook", for "wiki", etc. But you also see a few things that surprised
> me:
>
> - Tons of Japanese. Japan is the most mobile-enabled country in the world
> so I guess we should have expected to see many searches in Japanese show up
> in the top queries. I've left them URL-encoded in the results -- you'll see
> them as weird lines with % in them.
>
> - Apparently people search for movies and TV so they can spoil their fun
> by reading about them on Wikipedia. Both of "movies" and "film" show up in
> the top keywords; Iron Man 1, 2, AND 3 all show up in the top search
> queries. I didn't expect this was a major use-case, but -- wikigroaning
> aside -- it's an interesting fact.
>
> I'm sure we're only scratching the surface here. This is an exciting
> dataset, and I'm sure there's lots more to learn!
>
> The full results:
> - Top Referring Entry Domains:
> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
> - Top Referring Entry Search Queries:
> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
> - Top Referring Entry Search Keywords:
> http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
>
> Questions are welcome!
>
>
> --
> David Schoonover
> dsc(a)wikimedia.org
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>
>
Hiya all,
As promised earlier today in the Analytics weekly showcase, I've got a few
interesting bits of data to share from playing with the new Mobile Site
Sessions dataset.
# Visits to Mobile Site, 4/21/2013
- Total Visits: 51,624,103
- Unique Visitors: 37,736,120
- Total Pageviews: 104,972,033
- Avg Pageviews per Session: 2.0334
- Max Pageviews in one Session: 141,882
## Standard Site
- Visits: 51,603,221
- Unique Visitors: 37,723,188
- Pageviews: 104,910,382
- Avg Pageviews per Session: 2.033
## Alpha Site
- Visits: 986
- Unique Visitors: 822
- Pageviews: 7,087
- Avg Pageviews per Session: 7.188
## Beta Site
- Visits: 19,896
- Unique Visitors: 16,235
- Pageviews: 54,564
- Avg Pageviews per Session: 2.742
## Notes
- A session (or "visit") is defined as all activity with less than 30
minutes between each hit. Intuitively speaking, a session ends when the
user hasn't done anything in 30m.
- As we do not set visitor_id cookies for all users, the "unique visitors"
metric was calculated using hash(ip_address + users_agent) as visitor_id.
- This job looked at all requests to the mobile site on 4/21/2013, which is
75.17 GB of request logs.
- The job took ~17 minutes to process the day into 15.3 GB of sessions.
- The summary above took maybe 10 minutes to set up/write in Hive, and the
job took maybe 7 minutes.
In addition to that summary, I ran a few jobs on the entry_referer field --
the URL that referred the user to us when the session started. Obvious
caveats: this is only one day of data, and it's only the mobile site. Draw
conclusions with care.
First, I pulled out the top referring domains. It's mostly as you'd expect
-- search engines -- though you'll also note that several Wikipedia mobile
sites show up. My working hypothesis is that people don't tend to close
tabs on smartphones; when they later come back, it is often to an open
Wikipedia tab: clicking a link or perform a search means the referrer is
still us.
Since -- as expected -- so much of the data pertained to search engines, I
also calculated the top search queries and top keywords that sent people to
us. (For keywords, I've filtered out common "stop words": de, of, in, is,
la, and, el, es, to, en, di, los, le, da, se, las, les, il, du, a, i, o, y,
e.) In both, you see the predictable: lots of searches for porn, for
"facebook", for "wiki", etc. But you also see a few things that surprised
me:
- Tons of Japanese. Japan is the most mobile-enabled country in the world
so I guess we should have expected to see many searches in Japanese show up
in the top queries. I've left them URL-encoded in the results -- you'll see
them as weird lines with % in them.
- Apparently people search for movies and TV so they can spoil their fun by
reading about them on Wikipedia. Both of "movies" and "film" show up in the
top keywords; Iron Man 1, 2, AND 3 all show up in the top search queries. I
didn't expect this was a major use-case, but -- wikigroaning aside -- it's
an interesting fact.
I'm sure we're only scratching the surface here. This is an exciting
dataset, and I'm sure there's lots more to learn!
The full results:
- Top Referring Entry Domains:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
- Top Referring Entry Search Queries:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
- Top Referring Entry Search Keywords:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/views/sessions/m…
Questions are welcome!
--
David Schoonover
dsc(a)wikimedia.org
Forwarding to mobile-l in case anyone is interested. Analytics for
mobile including statistics around alpha and beta usage.
---------- Forwarded message ----------
From: David Schoonover <dsc(a)wikimedia.org>
Date: Tue, Apr 23, 2013 at 2:45 AM
Subject: Mobile Site Session Data (with Bonus Analytics!)
To: mobile-tech <mobile-tech(a)wikimedia.org>, Analytics List
<analytics(a)lists.wikimedia.org>
Cc: Diederik van Liere <dvanliere(a)wikimedia.org>, Maryana Pinchuk
<mpinchuk(a)wikimedia.org>, Arthur Richards <arichards(a)wikimedia.org>
Howdy all,
Now that the stars have aligned and:
- The Mobile Frontend release with X-Analytics logging the site mode
cookie (mf-m) has been out for a bit, and shows up in our logs;
- We've deployed the patch to fix a critical Hadoop bug[1][2] that was
blocking the job
- I've personally "conquered" the wikiplague again (after being out
half last week)
- And finally, the script works and its results look valid and complete
...I'm happy to report the Mobile site sessions job[3] is ready to
ship. I'm pretty sure this is the first view of mobile site sessions
ever, so I was pretty excited. I've included some Bonus Stats from my
test run which I generated 'cause I was curious :)
Before we get too excited, though, I've held off on enabling the daily
job (as well as backfilling March) because it turns out that a day's
worth of data generates about 16GB worth of sessions. This isn't a
problem for the cluster, but we'd pretty rapidly compromise stat1001's
public data storage with daily syncs. So to go forward, access to the
data would probably have to be provided via private rsync. A third
option is to work with the data on the cluster itself via any of the
available tools; I've been using a SQL tool called Hive to validate
various job runs and I can't say I'm missing MySQL. (If people are
interested, I'd be happy to go over the options in more detail.)
So, we're looking for guidance on going forward.
- Is the granular session output still the desired result, given the
job's size? Current the job ends by coalescing the data into one giant
TSV; instead it could generate a summary, or a selection of stats
about the run.
- If so:
- Is it helpful to backfill March?
- Does the data need to be publicly accessible via HTTP, or can we
explore other options for providing access to the team?
I'm happy to answer any other questions as well.
Thanks!
Dave for Team Analytics
[1] The bug: https://issues.cloudera.org/browse/DISTRO-461
[2] The fix: https://mingle.corp.wikimedia.org/projects/analytics/cards/595
[3] Feature request:
https://mingle.corp.wikimedia.org/projects/analytics/cards/240
---
BONUS STATS!
Notes:
- As a reminder, a session (or "visit") is defined as all activity
with less than 30 minutes between each hit.
- The test job looked at all requests on 4/21, which is 75.17 GB of
request logs.
- It took ~17 minutes to process the day into 15.3 GB of sessions. (It
then took 51m44s to concatenate those 28 files into one monstrous TSV
for "ease" of delivery to y'all.)
- The summary below took maybe 10 minutes to set up/write in Hive, and
the job took maybe 7 minutes.
Visits to Mobile Site, 4/21/2013
- Total Visits: 51,624,103
- Unique Visitors: 37,736,120
- Total Pageviews: 104,972,033
- Avg Pageviews per Session: 2.0334
- Max Pageviews in one Session: 141,882
Standard Site
- Visits: 51,603,221
- Unique Visitors: 37,723,188
- Pageviews: 104,910,382
- Avg Pageviews per Session: 2.033
Alpha Site
- Visits: 986
- Unique Visitors: 822
- Pageviews: 7,087
- Avg Pageviews per Session: 7.188
Beta Site
- Visits: 19,896
- Unique Visitors: 16,235
- Pageviews: 54,564
- Avg Pageviews per Session: 2.742
Those numbers look sane to you guys?
--
David Schoonover
dsc(a)wikimedia.org
--
Jon Robson
http://jonrobson.me.uk
@rakugojon
I've been playing around with skins a lot recently. The MobileFrontend
extension has had to do various transformations to the content
pre-rendering to ensure certain things do not get rendered on mobile
(for example table of contents) as well as allow us to do collapsible
sections.
When rendering content skins are currently limited to rendering the
'bodytext' key. They cannot retrieve the underlying content. I would
like to remove restrictions to skin designers - for instance currently
a skin designer has no control over where the table of contents should
do. They could not easily put it in a side bar for instance.
I was experimenting with using the onOutputPageParserOutput hook [1]
(running based on the current skin) and think it might be a better
approach to run the transformations on smaller chunks of data. For
instance the table of contents is known to be in the lead section so
it seems like it would be more efficient to look for it there rather
than throughout the entire document. The solution is not complete but
provides an approach that I think would be more efficient on the long
term.
It seems like a good idea to experiment with this approach in
MobileFrontend extension with the goal to upstream it to core.
Would appreciate comments from people who know this area of code
rather well - Max comes to mind :)
Thoughts welcomed!
[1] https://gist.github.com/jdlrobson/5439509
There seem to be a few teething problems but with help from Matt
Walker we managed to set up a banner on test wiki [1] (beta and alpha
versions of the site) today that runs on mobile only and only for
Android devices (albeit it's not 100% reliable yet in that it won't
target Android devices viewing a MediaWiki instance in Opera
Mobile/Mini).
Currently it simply shows a banner suggesting the user downloads the
Wikimedia Commons app with a "yes please" button and "no thanks"
button. The "yes please" button links to the Google Play store, the no
thanks button closes it and sets a cookie so it isn't shown again.
The campaign is configured here:
* http://test.wikipedia.org/w/index.php?title=Special:CentralNotice&method=li…
and the banner configured here:
* http://test.wikipedia.org/w/index.php?title=Special:NoticeTemplate/view&tem…
/
Currently there is no UI on the Special:CentralNotice page to target
certain devices (Matt had to make some changes behind the scenes for
this time) but soon it should allow banners to be targeted to
'desktop', 'iphone' and 'android' so it will be possible to run iphone
only campaigns.
Obviously there are new and big challenges with running banners on a
mobile device - space is even more important so any campaigns that do
run on these devices will need to be very creative!
Thanks to Matt and the Fundraising team for working with us to get
this live! Please feel free to ask any questions or start
conversations around banners on mobile / interesting uses. I'm happy
to provide my knowledge on what is possible and what is not possible
and I cross my fingers that people will start thinking about mobile in
designing their banners rather than pushing a desktop tested banner on
to a mobile phone :-)
[1] https://test.m.wikipedia.org/wiki/Main_Page?mobileaction=beta
I was very happy to just notice www.wikivoyage.org
Who should I speak to about optimizing the wikivoyage landing page for
mobile...?
(Sorry... I can never remember how to find the equivalent wiki page :-))
I just received this mail from a user:
"...I upgraded to Mediawiki 1.20 on me dev machine, and things
seem to work fine.
However using MF for MW 1.20 has no auto-detect, so it's not very
useful to me :)
Is there any chance you fix this on the master version as well (which
has built it auto-detect)?"
It seems various other people are being confused with how mobile
detection works [1], myself included so I have setup this basic stub
of a page [2] to hopefully document this. Please share your knowledge
here!
[1] http://www.mediawiki.org/w/index.php?title=Extension_talk:MobileFrontend&of…
[2] http://www.mediawiki.org/wiki/Extension:MobileFrontend/Configuring_Browser_…
On Wed, Apr 3, 2013 at 8:34 PM, Jon Robson <jrobson(a)wikimedia.org> wrote:
> Eek. I'm afraid getting this to work on an older MediaWiki/MobileFrontend
> extension is extremely problematic. So much has changed both in core and the
> MobileFrontend extension to this....
>
> Are you sure migrating your wiki will be as problematic as you suspect?
>
> Best wishes and sorry to the be the bearer of bad news!!
> Jon
>
>
> On Tue, Apr 2, 2013 at 6:50 AM, ויקיפועל <wikipoel(a)gmail.com> wrote:
>>
>> Hi, thank you for fixing this bug I opened. I appreciate it. However
>> I'm using MW 1.18 and an older version of MobileFrontend without
>> skins. I know it's a lot to ask for, but perhaps you'd be willing to
>> see if such a small fix can also be applied for older versions of the
>> extension?
>> Migrating my entire wiki to 1.20 just for this is problematic
>>
>> Thank you
>>
>>
>> https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/MobileFrontend…
>
>
Hi,
After upgrading my site's MediaWiki install to 1.21rc1, we noticed the
MobileFrontend (at master) was throwing a fatal error. I mentioned it in
#wikimedia-mobile and Ori found that it depends on a core change that was
only merged recently (https://gerrit.wikimedia.org/r/#/c/49071/).
Since MobileFrontend is considered to be stable, it would be nice if there
was a REL1_21 branch so that we could get the features that have been
introduced since REL1_20 was branched, and that don't require 1.22.
Thanks!
-- Legoktm
http://enwp.org/User:Legoktm
The mobile team has been working on some significant performance
improvements. One of the trickier yet hopefully most rewarding improvement
is to remove the need to vary the mobile varnish cache by X-Device header
for regular page HTML. Updated code is already live in production[1],
however until we set $wgMFVaryResources = true, pages in the mobile varnish
cache will still be varied by X-Device header as they have been all along.
We've enabled $wgMFVaryResources on testwiki[2] and have been doing some
sanity checking to ensure major devices are still working correctly. During
our regular weekly MobileFrontend deployment on 16 April, we plan to enable
this on mediawiki.org for further, crowd-sourced testing. More details to
follow. Assuming things go well, we will go ahead and make this change live
everywhere, hopefully during our deployment on 23 April.
We expect this will have a significant increase in the mobile varnish cache
hit ratio.
[1]
https://gerrit.wikimedia.org/r/#/c/55226/https://gerrit.wikimedia.org/r/#/c/58458/https://gerrit.wikimedia.org/r/#/c/56774/
[2]
https://gerrit.wikimedia.org/r/58427
--
Arthur Richards
Software Engineer, Mobile
[[User:Awjrichards]]
IRC: awjr
+1-415-839-6885 x6687
The iOS version of the Commons app is now bumped to 1.0.0 for provisional
App Store release; however we don't know how long review will take or if
there'll be any surprises of course...
Big thanks to our new developer Monte Hurd for lots of little visual &
behavior fixes since earlier versions!
Also available via TestFlightapp.com -- registered beta users should have
an email in their inbox with install directions.
For those who aren't in on the beta yet or aren't getting announcement
emails:
* First you must register and join the beta group at http://tflig.ht/Zl9Ef7
* THEN, you must log into http://testflightapp.com ON YOUR DEVICE and
"Connect" it. This will register your device ID with us, so we can include
you in the next beta push. If you don't do this step, you won't be able to
install the betas!
A couple people who registered in the last few days aren't included in this
release; you'll be in the next one.
Note I created a Wikimedia test account for the reviewers and uploaded a
couple photos to it:
https://commons.wikimedia.org/wiki/User:App_Store_Review_Tester
The Apple store reviewers might upload a couple junk photos on this account.
-- brion
Hello! New beta release of commons app went out today!
Features of v1.0 beta 7 include:
- Added opt out from EventLogging
- Remove {{Uncategorized}} template after adding categories
- Be more consistent and proactive in syncing modifications (adding categories)
- Add a minimal About page
- Add option to send feedback via email from within the app
- i18n updates
There is no major new feature, just a bunch of bug fixes and minor features.
Bug reports welcome at
https://bugzilla.wikimedia.org/enter_bug.cgi?product=Commons%20App
(pick the Android component). The apps team uses Trello to organize
our work, you can look at the board for our current iteration at
https://trello.com/board/sprint-6/5162461b43476994540002ba
Pull requests welcome at https://github.com/wikimedia/android-commons
Thank you!
This mail brought to you by the vague feeling that this list feels
lonely and has abandonment issues.
--
Yuvi Panda T
http://yuvi.in/blog
Hi, the good news is that we are seeing more community efforts to have
new Wikimedia related mobile apps in different platforms. The last
example being the Wikisource android app.
There is this entry in my (sadly long) backlog:
2012-12-19: Criteria to get a mobile app accepted as official.
The motivation to add it was that a contact at Nokia had asked the
requirements to develop a Wikipedia app for Windows Phone accepted as
official. We didn't (and afaik we don't) have an answer.
I believe we need that list of requirements, especially when the
Wikimedia Foundation is applying the "narrow focus" strategy,
encouraging other community developers to take the lead in the platforms
& Wikimedia projects we are not pushing directly.
Does anybody want to take the lead, start the discussion and a draft in
a wiki page? Even if the requirements and process to become an official
application will need the approval of the WMF you don't need to be an
employee to push a proposal. We will help whoever takes task.
--
Quim Gil
Technical Contributor Coordinator @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
Hello! New beta release of commons app went out today!
Features of v1.0 beta 6 include:
## v1.0 beta 6
- Add categorization
- Add a 'Modifications Sync' framework for doing eventual-consistent page edits
- More consistent designb between single and multiple upload
- i18n updates
You can now add categories to your uploads, both single and multiple
uploads! They will be 'eventually consistent' - the categories aren't
saved immediately as you hit the 'save' button but will be saved on
next sync, whenever Android decides that is.
We're getting closer to a full v1.0 release - user account creation
and a bit more polish ought to do it.
Bug reports welcome at
https://bugzilla.wikimedia.org/enter_bug.cgi?product=Commons%20App
(pick the Android component).
Pull requests welcome at https://github.com/wikimedia/android-commons
Thank you!
--
Yuvi Panda T
http://yuvi.in/blog
Do people read this list at all? it's been incredibly super low
traffic, super low noise and super low signal. Should we just abandon
this and go back to wikitech-l?
--
Yuvi Panda T
http://yuvi.in/blog