Just a heads-up:
Analytics-store is seeing several hours of replag on s1, s4, and s6.
s4 is me doing a commonswiki schema change, which should be done shortly. s1 and s6 are lagging due to load from queries like:
create table staging.enwiki_intra select a.pl_from as page_id_from, a.pl_title as page_title_to, b.page_id as page_id_to from enwiki.pagelinks a left join enwiki.page b on a.pl_title=b.page_title where a.pl_namespace=0 and a.pl_from_namespace=0;
Some very large cross joins there. No doubt it's intended activity. And if not, now at least you know what's happening ;-)
BR Sean
Hi Sean,
Thanks for the email. The two create queries are mine. Should I kill one?
Leila On Mar 5, 2015 7:09 AM, "Sean Pringle" springle@wikimedia.org wrote:
Just a heads-up:
Analytics-store is seeing several hours of replag on s1, s4, and s6.
s4 is me doing a commonswiki schema change, which should be done shortly. s1 and s6 are lagging due to load from queries like:
create table staging.enwiki_intra select a.pl_from as page_id_from, a.pl_title as page_title_to, b.page_id as page_id_to from enwiki.pagelinks a left join enwiki.page b on a.pl_title=b.page_title where a.pl_namespace=0 and a.pl_from_namespace=0;
Some very large cross joins there. No doubt it's intended activity. And if not, now at least you know what's happening ;-)
BR Sean
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi Leila
On Fri, Mar 6, 2015 at 1:38 AM, Leila Zia leila@wikimedia.org wrote:
Hi Sean,
Thanks for the email. The two create queries are mine. Should I kill one?
Lag has now reached 24h for s1 and s6, plus I found a few other 'research' user connections apparently fully blocked waiting on staging metadata, so I've killed your queries.
I recommend building tables like that using batched inserts, so both replication and other people can co-exist with you :-) Super large write-transactions running for more than an hour or two are never a good idea.
BR Sean
Hi Sean,
On Thu, Mar 5, 2015 at 9:59 PM, Sean Pringle springle@wikimedia.org wrote:
Hi Leila
On Fri, Mar 6, 2015 at 1:38 AM, Leila Zia leila@wikimedia.org wrote:
Hi Sean,
Thanks for the email. The two create queries are mine. Should I kill
one?
Lag has now reached 24h for s1 and s6, plus I found a few other 'research' user connections apparently fully blocked waiting on staging metadata, so I've killed your queries.
Thanks for letting me know.
I recommend building tables like that using batched inserts, so both replication and other people can co-exist with you :-) Super large write-transactions running for more than an hour or two are never a good idea.
makes sense. will follow your advice.
Leila
BR Sean
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics