Hello John,<br><br><div class="gmail_quote">On 21 January 2012 02:56, John R. Frank <span dir="ltr">&lt;<a href="mailto:jrf@mit.edu">jrf@mit.edu</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

For example, given a wikipedia.Page() instance for urlname=Takashi_Murakami, call it &quot;central node,&quot; I want to iterate through the Pages returned by getReferences() and only keep those that ref&#39;ed my central node on a date in the past, like November 30, 2011.<br>


<br>
Is the best (only?) way to do this to iterate through previous revisions of those Pages and verify that they ref&#39;ed the central node before that date?  What about pages that did ref and no longer do?  (probly rare?)</blockquote>

<div><br></div><div>Pywikipedia can only help with the &#39;a&#39; side of the &#39;a-&gt;b&#39; link - that is: the page where the link is stored. It should be able to retrieve old page sources, and tell you which pages were linked *from* that page.</div>

<div><br></div><div>The inverse, however, is difficult. To do this efficiently, MediaWiki keeps this data in an extra table, but this table of course only has the most recent data. This data is in the &#39;pagelinks&#39; table, which is dumped, but it is also huge (for enwiki, other wikis are much smaller):</div>

<div><br></div><div>enwiki-latest-pagelinks.sql.gz  4.0G  [1]</div><div><br></div><div>However, this table is dumped monthly (somewhat), with the oldest version in november 2009 [2], so it might be possible to use this.</div>

<div><br></div><div>Best regards,</div><div>Merlijn van Deen</div><div><br></div><div>[1] <a href="http://dumps.wikimedia.org/enwiki/latest/">http://dumps.wikimedia.org/enwiki/latest/</a></div><div>[2] <a href="http://dumps.wikimedia.org/enwiki/">http://dumps.wikimedia.org/enwiki/</a></div>

</div>