Hi,
For one of my projects, I need to be able to keep the most up to date
version of wikipedia html pages for a few languages like en, zh, de, es, fr
etc. So this is done currently in two steps,
1. Listen to changes on stream API documented here
<https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams> and then
extract the page titles.
2. For each of the titles, get the latest HTML using the Wikipedia REST api
<https://en.wikipedia.org/api/rest_v1/#/Page%20content/get_page_title__title_>
and
persist the HTML.
I understand that in order to avoid the 429 (Too many requests error), we
need to make sure we limit the api request to 1 per second. Just wanted to
check if we can make requests to different languages like
en.wikipedia.org,
fr.wikipedia.org etc in parallel or do those requests also need to be done
in serial manner (1 per second), in order to not hit HTTP 429 error.
Please let me know if you need more information.
--
Regards,
Aadithya
--
Sent from my iPad3