The query listed selecting from wikidatawiki.wb_terms was me /
wmde-analytics and runs daily.
I agree that some sort of query limits would make sense.
What limits are currently in place?
On Tue, 3 Jan 2017 at 21:04 Nuria Ruiz <nuria(a)wikimedia.org> wrote:
Sorry, that was me.
I would urge you to setup a per-user/per-service
query resource limits,
otherwise poorly performant queries will affect all users
(and in cases
like this, create downtime). I have >set up query limits for all
research/analytics users temporarily until 3rd January.
Could we make this limits permanent? I do not see a reason not to do that.
Let me know if I am missing something.
Thanks,
Nuria
On Thu, Dec 29, 2016 at 12:56 PM, Dan Andreescu <dandreescu(a)wikimedia.org>
wrote:
Thanks Jaime, we will use the Summit to show people how to use Hadoop /
Druid for some of this analysis, and that should get these types of
problems off your plate.
*From: *Jaime Crespo
*Sent: *Sunday, December 25, 2016 04:57
*To: *A mailing list for the Analytics Team at WMF and everybody who has
an interest in Wikipedia and analytics.
*Reply To: *A mailing list for the Analytics Team at WMF and everybody
who has an interest in Wikipedia and analytics.
*Subject: *[Analytics] analytics-store was down due to excessive load
analytics-store was brought down at 6am, and then again at 9am UTC 25 Dec
due to multiple executions of long running queries (some of them 2 days
long) such as:
SELECT LEFT(timestamp, 8) AS yearmonthday, timestamp, userAgent, clientIp,
webHost, COUNT(*) AS copies FROM log.PageContentSaveComplete ...
SELECT COUNT(*) AS count, term_entity_type, term_type, term_language FROM
wikidatawiki.wb_terms ...
select date('20161218000000') as day, actions, count(*) as repeated from
(select group_concat(event_action order by timestamp, action_order.ord
separator '-') as actions from (select ...
I would urge you to setup a per-user/per-service query resource limits,
otherwise poorly performant queries will affect all users (and in cases
like this, create downtime). I have set up query limits for all
research/analytics users temporarily until 3rd January.
--
Jaime Crespo
<http://wikimedia.org>
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics