Thanks Tomasz for the great work. I think most 2 old backups with the latest are more than enough. Most people use the latest as you have mentioned. <br><br>For huge dump a month window is reasonable as well but if this is pushed to two weeks, that would be perfect but I wonder whether people can really benefit from the dump every two weeks because the download, decompress, and processing of some huge dumps needs a lot of time.<br>
<br>bilal<br><br><br><div class="gmail_quote">On Thu, May 14, 2009 at 9:02 PM, Tomasz Finc <span dir="ltr">&lt;<a href="mailto:tfinc@wikimedia.org">tfinc@wikimedia.org</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Now that we are generating all but the biggest of wiki&#39;s reliably I&#39;d<br>
like to start the discussion of retention for older data base dumps.<br>
<br>
If we can reliably stick to a two week window for each wiki&#39;s dump<br>
iteration, how many dumps would back would it make sense to keep?<br>
<br>
Most clients that I&#39;ve talked to only need the latest and simply look at<br>
the older ones in case the newest dump failed a step.<br>
<br>
If there are other retention cases then I&#39;d love to hear them and figure<br>
what&#39;s feasible to do.<br>
<br>
Operations wise I&#39;d be thinking of keeping somewhere between 1-5 of the<br>
previous dumps and then archiving copies of each dump at 6month windows<br>
for permanent storage. Doing that for all of the current dumps is way<br>
more space then we have currently available but that&#39;s also why were<br>
working on funding for those storage servers.<br>
<br>
Is that overkill or simply not enough? let know.<br>
<br>
--tomasz<br>
<br></blockquote></div><br>