On Wed, Oct 2, 2013 at 8:51 AM, Christian Aistleitner <christian@quelltextlich.at> wrote:

* Do we need to have all of the data in MySQL in a trivial schema at
all costs, or
* is it better to have the data at least in MySQL but with a not totally
trivial way to query, or
* would it be even better to have a nice clean interface to the data
that you maybe cannot query by SQL but allows to formulate queries in
a straight forward way?

What would you prefer?

Depending on the absolute value of "all costs", I'd prefer #1, or a combination of #2&#3.

For GLAM (which is what I am mostly involved in), monthly page views would suffice, and those should be easily done in MySQL.

Daily views would be nice-to-have, but do not reed to be in MySQL. They could even be ticket-based: I submit a request for daily views for month X for a few thousand pages, and get a ticket ID back; then I can periodically check if the ticket is finished, and pick up the cached results.

If the issue is not storage (and it shouldn't be!) but rather MySQL database size, how about a new database for each month, covering all projects? Even sqlite might be a good alternative, allowing for server-less SQL.

Cheers,
Magnus