Hello, quarry currently retains query results forever. This proves to be a problem when removed data is retained in quarry. In order to improve this while still leaving results accessible, results older than 90 days will be removed. The queries will not be impacted by this change. And the results can of course be regenerated (with current data) at any time by resubmitting the query.
https://phabricator.wikimedia.org/T360041
The plan is for this to go into effect in later October (2024-10-21), could go in earlier but I won't be here until then.
Thank you!
Hello,
We wanted to provide some context: due to the continued interest in Quarry, we've decided to give it more attention and plan some much-needed maintenance. This will include tasks like upgrading Python versions and possibly making some enhancements to Quarry itself. One thing that caught our attention is the lack of trimming for query results. At the moment, old queries might still display deleted data, which could sometimes include sensitive information that has already been removed upstream. Plus, as query data gets older, it can become outdated and less relevant. Keeping and using this older data could lead to inaccurate insights, which might result in misleading conclusions or poor decision-making. To improve this, we’re proposing a change: we'll keep the queries themselves but remove the results 90 days after the last time the query was run. We’d love to hear your thoughts on this! If you have any concerns, please share them with us under this task https://phabricator.wikimedia.org/T360041
We’ll be collecting feedback until 2024-10-21, as noted by Vivian. If no concerns are raised by that date, we’ll move forward with the query results trim. If any concerns do come up, we’ll pause and work with you to find a solution. Before we kick off the results trim, we’ll send out another notification via cloud-announce to keep everyone informed.
Thanks in advance for your feedback! Joanna