Sorry, that was me. 

>I would urge you to setup a per-user/per-service query resource limits, otherwise poorly performant queries will affect all users (and in cases like this, create downtime). I have >set up query limits for all research/analytics users temporarily until 3rd January.

Could we make this limits permanent? I do not see a reason not to do that. Let me know if  I am missing something. 

Thanks, 

Nuria

On Thu, Dec 29, 2016 at 12:56 PM, Dan Andreescu <dandreescu@wikimedia.org> wrote:

Thanks Jaime, we will use the Summit to show people how to use Hadoop / Druid for some of this analysis‎, and that should get these types of problems off your plate.

From: Jaime Crespo Sent: Sunday, December 25, 2016 04:57 To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. Reply To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. Subject: [Analytics] analytics-store was down due to excessive load

analytics-store was brought down at 6am, and then again at 9am UTC 25 Dec due to multiple executions of long running queries (some of them 2 days long) such as:

SELECT LEFT(timestamp, 8) AS yearmonthday, timestamp, userAgent, clientIp, webHost, COUNT(*) AS copies FROM log.PageContentSaveComplete ...

SELECT COUNT(*) AS count, term_entity_type, term_type, term_language FROM wikidatawiki.wb_terms ...

select date('20161218000000') as day, actions, count(*) as repeated from (select group_concat(event_action order by timestamp, action_order.ord separator '-') as actions from (select ...

I would urge you to setup a per-user/per-service query resource limits, otherwise poorly performant queries will affect all users (and in cases like this, create downtime). I have set up query limits for all research/analytics users temporarily until 3rd January.

--

Jaime Crespo

<http://wikimedia.org>

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics