Ok, so the trouble was that a configuration file was exposed publicly by accident. To fix the problem, the following steps were taken:
0. I stopped all wikimetrics services (queue and web)
1. Coren reset the labsdb password for my user, I copied and replaced it in the db_config.yaml file
2. I reset the wikimetrics user db password and replaced it in db_config.yaml
3. I reset the Flask secret key that guards sessions and replaced it in web_config.yaml
4. I reset the Google OAuth consumer credentials and replaced them in web_config.yaml
5. I did not reset the MediaWiki OAuth consumer credentials as these were not leaked
6. I restarted apache and celery, and wikimetrics started serving again
I'm fairly confident that a reset secret key just means all people who were logged in may have to login again. But there may be something unforeseen that went wrong - just let me know.