Vinay, could you tell us what you diff URLs and external links for? If it is for regular crawling of Wikimedia projects pages, you may be interested in our OAI feed for search engines: https://meta.wikimedia.org/wiki/Wikimedia_update_feed_service
Nemo