Our Scholia webapp on Toolforge is struggling with the request load at https://scholia.toolforge.org/
According to toolviews (also displayed at https://people.compute.dtu.dk/faan/scholia-page-view-statistics.html) we had yesterday almost 700,000 daily hit. Monitoring the log (tail -f uwsgi.log) I see quite a lot of various "troubling" hits, e.g., to the static files. I do not know what it is but I imagine it is some kind of GenAI wannabees with a Playwright script crawling Scholia.
In the current Scholia webapp code, we have embarrassments:
1) Serving static files in the Flask webapp.
2) Blocking requests to Wikidata API or the SPARQL endpoint here and there in the code (most Scholia requests are client-side SPARQL requests though).
I have been moving some static file requests to tools-static.wmflabs.org, but still need to do some more that is embedded in Bootstrap.
I am thinking about moving from Flask to an async framework. I am gaining some experience with FastAPI for web services.
Am I correct that async on Toolforge will buy us a bit of extra performance?
Are there other Toolforge users that have struggling webapps and, if yes, then what do you do? If it continue we could do login I suppose.
In the Scholia repo we have this issue: https://github.com/WDscholia/scholia/issues/2727
best regards Finn Årup Nielsen https://people.compute.dtu.dk/faan/