Hi,
If you look at https://phabricator.wikimedia.org/T104942#1436332 (linked from this thread, before Adam posted his own data) an analysis was done on a file called "per-domain-count" which we previously extracted from sampled 1:1000 logs for approximately 25 days for all kinds of domain-popularity purposes and cleanups that we've been doing as part of the HTTPS project (more background at https://phabricator.wikimedia.org/T102827#1429852 and also see T102826, T102814, T102815).
Those logs above are sampled and aren't as accurate as the Hadoop data Adam used due to other infrastructure faults that have happened in that 25-day period but they are generally okay for extracting those broad conclusions, especially if we look at the relative popularity of e.g. .wap. vs. .m. rather than the absolute numbers.
Finally, note that in any case there is a hard limitation of a look-behind window of 90 days due to our data retention policy, as well as practical considerations for extracting results from unsampled logs for larger periods of time. You're absolutely right, though, that a 1-day sample is usually not enough, especially considering the seasonality of data like e.g. a very different mobile-to-desktop ratio on weekends.
Faidon
On Thu, Jul 16, 2015 at 09:55:14AM -0400, John wrote:
Can we look at a wider sample? using a single day as judgement factor is a bad idea. However if the data supports your position I dont see any serious problems. You might want to take a look at either the UA's or refering sources to see if there is a primary source for the traffic and mitigate that.
On Thu, Jul 16, 2015 at 9:03 AM, Adam Baso abaso@wikimedia.org wrote:
Looks like the user pageviews for wap.wikipedia.org and mobile.wikipedia.org subdomains are approximately 0.02% of the size of pageviews for m.wikipedia.org subdomains based on a recent one day check.
hive> select count(*) from wmf.webrequest where year = 2015 and month = 7 and day = 14 and access_method = 'mobile web' and (uri_host like '%.wap.wikipedia.org' OR uri_host like '%. mobile.wikipedia.org') and is_pageview = true and agent_type = 'user';
35,543
hive> select count(*) from wmf.webrequest where year = 2015 and month = 7 and day = 14 and access_method = 'mobile web' and uri_host like '%.m.wikipedia.org' and is_pageview = true and agent_type = 'user';
202,024,891
On Thu, Jul 16, 2015 at 5:41 AM, John phoenixoverride@gmail.com wrote:
... Have we done any analysis on usage of those subdomains?
On Thu, Jul 16, 2015 at 8:34 AM, Adam Baso abaso@wikimedia.org wrote:
There's a ticket for removing mobile.wikipedia.org and
wap.wikipedia.org
domains/subdomains, which are legacy domain names superceded by m.wikipedia.org and its subdomains.
https://phabricator.wikimedia.org/T104942
The rationale for the removal of these legacy domain names is to help support HSTS preloading in browsers with the existing TLS SAN cert.
After review of the ticket, can anyone think of a compelling reason to
keep
those old domain names?
I'm going to open a separate thread on mobile-l about this given this
is
more mobile-targeted, yet some people only operate on one of wikitech-l
or
mobile-l.
-Adam _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l