Hi all!
This week, ops will be redirecting all mobile web traffic from the mobile caches to the text caches, as part of a larger project to consolidate and simplify our the cache architecture. This means that the data in the webrequest table in the webrequest_source=‘mobile’ partition will now live in the webrequest_source=‘text’ partition.
Joseph and I will work to make sure our prod Hadoop jobs keep working. This is just a heads up in case you have any of your own jobs depending on this partition.
https://phabricator.wikimedia.org/T109286 https://phabricator.wikimedia.org/T122651
-Ao
Thanks Andrew! I will file a task to make sure my team is on top of it :).
Unrelated to this do we have any updates on what's going on with the EventLogging replication lag? It would be good to have an update in the pertinent thread.
On 18 January 2016 at 09:59, Andrew Otto otto@wikimedia.org wrote:
Hi all!
This week, ops will be redirecting all mobile web traffic from the mobile caches to the text caches, as part of a larger project to consolidate and simplify our the cache architecture. This means that the data in the webrequest table in the webrequest_source=‘mobile’ partition will now live in the webrequest_source=‘text’ partition.
Joseph and I will work to make sure our prod Hadoop jobs keep working. This is just a heads up in case you have any of your own jobs depending on this partition.
https://phabricator.wikimedia.org/T109286 https://phabricator.wikimedia.org/T122651
-Ao
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Oliver, EL replication lag is being tracked in this ticket.
https://phabricator.wikimedia.org/T123634
Jaime updated it today.
On Mon, Jan 18, 2016 at 1:23 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Thanks Andrew! I will file a task to make sure my team is on top of it :).
Unrelated to this do we have any updates on what's going on with the EventLogging replication lag? It would be good to have an update in the pertinent thread.
On 18 January 2016 at 09:59, Andrew Otto otto@wikimedia.org wrote:
Hi all!
This week, ops will be redirecting all mobile web traffic from the mobile caches to the text caches, as part of a larger project to consolidate and simplify our the cache architecture. This means that the data in the webrequest table in the webrequest_source=‘mobile’ partition will now
live
in the webrequest_source=‘text’ partition.
Joseph and I will work to make sure our prod Hadoop jobs keep working.
This
is just a heads up in case you have any of your own jobs depending on
this
partition.
https://phabricator.wikimedia.org/T109286 https://phabricator.wikimedia.org/T122651
-Ao
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Count Logula Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
That is most appreciated, but it would probably be good for transparency and making-sure-everyone-hears purposes if the mailing lists got updates too.
I'll take Jaime's commentary there and turn it into a generally-consumable thing for the relevant thread (and include a link to the phab ticket so that anyone hunting in email for information knows where to look for ongoing updates).
On 18 January 2016 at 10:40, Andrew Otto otto@wikimedia.org wrote:
Oliver, EL replication lag is being tracked in this ticket.
https://phabricator.wikimedia.org/T123634
Jaime updated it today.
On Mon, Jan 18, 2016 at 1:23 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Thanks Andrew! I will file a task to make sure my team is on top of it :).
Unrelated to this do we have any updates on what's going on with the EventLogging replication lag? It would be good to have an update in the pertinent thread.
On 18 January 2016 at 09:59, Andrew Otto otto@wikimedia.org wrote:
Hi all!
This week, ops will be redirecting all mobile web traffic from the mobile caches to the text caches, as part of a larger project to consolidate and simplify our the cache architecture. This means that the data in the webrequest table in the webrequest_source=‘mobile’ partition will now live in the webrequest_source=‘text’ partition.
Joseph and I will work to make sure our prod Hadoop jobs keep working. This is just a heads up in case you have any of your own jobs depending on this partition.
https://phabricator.wikimedia.org/T109286 https://phabricator.wikimedia.org/T122651
-Ao
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Count Logula Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
I wanted to add that as a proxy for webrequest_source='mobile', your jobs/queries could now use access_method='mobile app' or access_method='mobile web', as needed.
On Mon, Jan 18, 2016 at 9:59 AM, Andrew Otto otto@wikimedia.org wrote:
Hi all!
This week, ops will be redirecting all mobile web traffic from the mobile caches to the text caches, as part of a larger project to consolidate and simplify our the cache architecture. This means that the data in the webrequest table in the webrequest_source=‘mobile’ partition will now live in the webrequest_source=‘text’ partition.
Joseph and I will work to make sure our prod Hadoop jobs keep working. This is just a heads up in case you have any of your own jobs depending on this partition.
https://phabricator.wikimedia.org/T109286 https://phabricator.wikimedia.org/T122651
-Ao
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
--Madhu Software Engg, Analytics