On Tue, Sep 15, 2015 at 6:56 PM, Marko Obrovac <mobrovac@wikimedia.org> wrote:
On 15 September 2015 at 19:37, Dan Andreescu <dandreescu@wikimedia.org> wrote:
I worry a little bit about the performance without having a batch api, but we can certainly try it out and see what happens. Basically we will be requesting the page view information for every NS_MAIN article in every wiki once a week.  A quick sum against our search  cluster suggests this is ~96 million api requests.

96m equals approx 160 req/s which is more than sustainable for RESTBase.

True, if we distributed the load over the whole week, but I think Erik needs the results to be available weekly, as in, probably within a day or so of issuing the request.  Of course, if we were to serve this kind of request from the API, we would make a better batch-query endpoint for his use case.  But I think it might be hard to make that useful generally.  I think for now, let's just collect these one-off pageview querying use cases and slowly build them into the API when we can generalize two or more of them into one endpoint.