My technology/power of community inspired opinion is that we don't need to worry about that problem right now. We could recreate all the content in short order were all the datacenters simultaneously struck by asteroids, and more feasible long-term storage solutions will present themselves in the next few decades. Anything we do right now is just going to get replaced. On Mon, May 4, 2009 at 6:35 PM, Samuel Klein <meta.sj(a)gmail.com> wrote:

If scanning involves destroying or harming the books, which it does, and future technologies can scan the pages without actually opening the books, then it's clear which solution I would choose. In many cases we have extra books though. On Tue, May 5, 2009 at 3:48 PM, David Goodman <dgoodmanny(a)gmail.com> wrote:

Samuel Klein wrote: I can tell you what the Rosetta folks would say: they would say that they paid $125k to Norsam for 5 prototype discs, and that we are free to do the same. Norsam have developed this technology at great cost and expect a commercial return, regardless of who's paying them. <http://www.internetnews.com/storage/article.php/3771051/Storage+That+Really+Lasts.htm> Personally I think it would be a waste of general funds, since I don't expect we'll see the end of civilisation any time in the next year or two. Maybe if there was a directed grant, it would be appropriate. Or we could have a small investment fund aimed at paying for such an archive in 20 years or so, when the process will be cheaper. By the way, it's FIB etching, not laser etching, and the discs are nickel-coated silicon, not plain nickel. -- Tim Starling

Brian wrote: The obvious reply to that is that the Rosetta project aims to make an archive readable with 17th century technology, which digital information compressed with advanced algorithms is not. They try to make an issue out of the obsolescence of digital technology, which I think is overwrought. Just because I don't have a slot in my computer where I can insert a 1970s era magnetic tape doesn't mean it's unreadable. I don't have a 750x optical microscope lying around either. Both media are readable using extant technology. There have been some problems with restoration of data where the decoding software has been lost. But the popular, well-documented digital formats of the past are as readable as ever: I have a program on my computer called groff which is largely backwards-compatible with runoff, one of the earliest digital typesetting formats, dating back to the 1960s. There is still a great deal of extant text dating to ancient times, despite the fact that copying was fantastically expensive, and that everything was written on flammable materials in a time when flame was the only artificial light source. Maybe the future will be more like Orson Scott Card's Homecoming series than the dark ages: a future with such a weight of carefully recorded and preserved history that studying it, even in overview, becomes the work of a lifetime. Anyone who claims to know what the far future will be like is a charlatan. But I think it would be foolish to assume that it will be anything like the past. -- Tim Starling

On Tue, May 5, 2009 at 12:02 AM, Tim Starling &lt;tstarling(a)wikimedia.org&gt; wrote: If we or anyone were to go this route, wouldn't microfiche in a sealed plastic container be a lot cheaper and more practical to mass-produce? See the section on preservation through moisture-tight containers: http://graphics.kodak.com/docimaging/uploadedFiles/D-31.pdf My personal plan for saving civilization is through intrinsically worthless plastic jewelry, kind of like this idea: http://www.google.com/patents?id=yLk3AAAAEBAJ&dq=4249330 Make these cheap pendants colorful, make them collectible, let people string dozens on a necklace, and soon you'd have thousands of copies of books floating through society that can never be lost. Wow, I can't believe they let us post this stuff on foundation-l :) Thanks, Pharos

On Tue, May 5, 2009 at 12:02 AM, Tim Starling &lt;tstarling(a)wikimedia.org&gt;wrote;wrote: Umm, if civilization ends, we won't be around to see it, and the Wikimedia Foundation mission will have been fulfilled ("every single human being can freely share in the sum of all knowledge"). No one is seriously suggesting using WMF funds for this nonsense, are they?

On Tue, May 5, 2009 at 9:17 AM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt; wrote: I'm pretty sure buying another server or offering a slightly higher salary on the next job offering or just leaving the money to accumulate interest would do considerably more to advance Wikimedia's mission than this. I think there are still going to be much more useful things to spend thousands of dollars on. The utility of this project is virtually zero from any perspective. Of course, since all of Wikimedia's data is freely available, anyone else who'd like to store it in some durable form for any sum of money is absolutely free to do so. Or they could give Wikimedia a directed grant. But it would be a waste of Wikimedia's money.

On Tue, May 5, 2009 at 10:12 AM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt; wrote: Assuming civilization collapses to begin with. And assuming the collapse is so complete that there are literally no computers left, even in the hands of the most powerful (who would likely lead rebuilding efforts). And assuming that civilization doesn't recover in a short enough period of time that most records will be intact anyway. And assuming that people can actually find the handful of disks or whatever that are probably locked up in a vault somewhere. And assuming they still have microscopes, but not computers. And assuming they bother to actually look at the disks in a microscope, instead of melting them down as scrap metal or using them as doorstops or just dumping them in a landfill. And assuming that vast quantities of trivia interspersed with incomplete scraps of poorly-explained science would in fact be useful for rebuilding civilization. (Have you ever tried to learn anything practical from Wikipedia? Textbooks would at least be useful.) Yeah, I'd say virtually zero utility. But if some weirdos want to waste money on it, that's their business. They can also prepare for the end of the world in 2012 as predicted by the Mayans, if they like.

On Tue, May 5, 2009 at 10:32 AM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt; wrote: But if you don't postulate a catastrophic event that we can't plan for, like civilization ending due to an overnight thermonuclear war, then we don't need to plan in advance. If Wikipedia ceases to exist at some point, and if at that point it looks like it would be useful to preserve its contents, we could preserve it more durably at that point. We don't need to preserve it now. In fact, it would be counterproductive: better to preserve it at the last possible moment, when it will contain as much data as possible.

2009/5/5 Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt;om>: As I said, the Internet Archive considers precisely that their job (as an internet library). More examples than the IA would be good, of course. - d.

On Tue, May 5, 2009 at 11:29 AM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt;wrote;wrote: If you really want to spend your time and efforts based on what "people still consider worthwhile"... Oh nevermind. Obviously, Wikipedia has It would be nice if deleted articles were made public, though. I would put a pretty large bet on the fact that someone is going to think they need to keep Wikipedia long past the point where it's worth it to keep it. Wrong decisions will be made to delete or oversight content, but whatever isn't oversighted or deleted will be kept by someone long after its usefulness has dropped below the cost of maintaining it. There are lots of packrats working on Wikipedia.

On Tue, May 5, 2009 at 12:09 PM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt; wrote: In 3000 years, nobody will give a rat's ass about Britney Spears' discography (again, to pick a random example of "pop culture"). That's a bet I'm willing to make. -Chad

On Tue, May 5, 2009 at 12:51 PM, David Gerard &lt;dgerard(a)gmail.com&gt; wrote: Touché. That being said, this is all still purely hypothetical since a societal collapse would mean that the vast majority of us aren't here to observe it anyway. Especially if it's in 3000+ years :) -Chad

Because the rate of change of technological progress has presumably been slower than it will be over the next 3000 years. Then again, given an AI capable of digesting an enormous amount of information (for example, reading the Wikipedia of 2050), it might find Brittany Spears just as interesting as anything else. On Tue, May 5, 2009 at 11:04 AM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt;wrote;wrote:

2009/5/5 Anthony &lt;wikimail(a)inbox.org&gt;rg>: I'm not convinced that discounting to present value applies here. You can't describe all of life in terms of economics (in fact, it seems describing economics in terms of economics isn't entirely wise!). How do you assign a monetary value to future historical knowledge? For that matter, how do you assign a monetary value to present historical knowledge? Or any kind of knowledge? In economics things have value due to scarcity, knowledge is freely reproducible, so the concept of scarcity doesn't really apply - either it exists, or it doesn't. Access to knowledge may have monetary value, but the existence of the knowledge doesn't, the concept just doesn't apply.

2009/5/5 Anthony &lt;wikimail(a)inbox.org&gt;rg>: Perhaps, but I'm still not entirely convinced. It may be an economic question, but that doesn't mean our theories of economics can answer it. I think now we're into the realms of philosophy, rather than economics... It is a difficult question. There is a fairly well known (I believe) question in ethics about whether a certain impact on a specific known person is the same as that impact on an abstract unknown person. It comes up when trying to answer questions like: Is it right to kill one person in order to make a drug that will, somewhere down the line, save thousands of lives? Most people would, I believe, answer "no", which means they put more value on the life of a specific person than a hypothetical person (I think I'm one of those people, although I can't justify it rationally...). I think your viewpoint is similar to that viewpoint (they are different questions, obviously, but I think they are similar enough to draw meaningful comparisons), so I guess it is a widely held one. I'm not really sure what my viewpoint is (so, by extension, I'm not sure how to assign value to future historical knowledge), I'll have to think about it... If value was determined by utility, water would be far more expensive that diamonds. You say "marginal utility" rather than just "utility", but I don't pay a different amount for my first glass of water each day than my second, even though the first is far more useful to me. It clearly has value (otherwise there would be no such thing as academia), but I don't think it has a well defined monetary value. Our theories of economics are an approximation to reality that allow us, in some circumstances, to predict how things will work out (in fact, those predictions are often self-fulfilling, which rather complicates things). I don't think our economics can adequately describe the value of knowledge. We have gone massively off-topic into a discussion of the philosophy of economics and epistemology here. I think we are all agreed that, for now at least, we don't want to pursue these ideas, so let's just shelve them and move on!

On Tue, May 5, 2009 at 6:28 PM, Anthony &lt;wikimail(a)inbox.org&gt; wrote: Except, you asked why we don't pay less... Refer to the second part of my answer :)

On Tue, May 5, 2009 at 6:35 PM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt;wrote;wrote: So if most people had a degree, people with degrees would earn less than people without degrees? I guess you have to explain the "whatever that means". But anyway, I guess you threw me off by mentioning "academia". I go back to my original answer :). The monetary value of the existence of knowledge is the amount one is willing to pay someone to create (or discover) that knowledge. As someone who used to work in "research and development", I can attest to the fact that there are people who are willing to pay for the creation/discovery of knowledge, even outside of academia and government.

On Tue, May 5, 2009 at 08:29, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt; wrote: You seem to be stuck in the "expensive copies" mindset of data preservation: since making copies is difficult, it makes sense to be concerned about who will have custody of those copies, and how they will go about guarding them. But making copies of Wikipedia *isn't* difficult: it costs 39 cents to make a copy of Enwiki's current article text on a hard drive, or 68 cents to burn it to a dual-layer DVD. Instead of making a single copy, protecting it from every conceivable harm, entrusting it to a guardian, and hoping he doesn't get bored with the task, make millions copies and spread them around the world. Most of these will be lost or destroyed, but the sheer number ensures that some will wind up in the hands of people who are interested in preserving them. -- Mark [[en:user:Carnildo]]

On Tue, May 5, 2009 at 4:19 PM, Aryeh Gregor &lt;Simetrical+wikilist(a)gmail.com&gt; wrote: By accident or by some other reason, we have much better optics than computers. So, it is reasonably to suppose that some future civilization will achieve much faster good optics than good computers.

On Wed, May 6, 2009 at 12:50 PM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt; wrote: * For example, having optics for 2700 years and having computers for somewhat more than 50 years. * Optics is able to help humans with disabilities for a long time, while computers are just starting with that task. * Optics is able to reach events around the beginning of the Universe, while computers are starting now to make really big things (for example, really useful Internet for a lot of humans exists a couple of years). * A lot of basic cures invented thanks to optics. * Thanks to optics we are able to make computers. * Generally, everything which differs one simple technology from the complex contemporary technology depends on optics. Of course, it is [still] not possible to compare such things exactly; as well as such comparison may be a good source for parody. And, of course, computers started to help in all of those fields, but, generally, the most of ordinary things around us still depend much more on optics than on computers.

David Goodman wrote: Not that I want to denigrate glass makers, but what is wrong with naturally occurring clear material such as rock crystal and the like? Yours, Jussi-Ville Heiskanen

On Wed, May 6, 2009 at 3:52 PM, Aryeh Gregor &lt;Simetrical+wikilist(a)gmail.com&gt; wrote: There is wide range of possible collapse scenarios. Just the most drastic ones assume that people won't be able to understand written English, to have 17th century optics and similar. However, the most of them assume that such society won't be able to unpack 2TB of data (now and without Commons; probably dozens or hundreds of TBs in a couple of years) and put it in some useful form. In article [1] it is described that ancient Egyptians and Mesopotamians developed basic optics. Note that those are the first civilizations. Dominant languages of particular civilizations were never been a problem: Long after Sumerian civilization had disappeared Assyrians knew their language because it was useful (not to mention ancient Latin, Greek, Chinese, Sanskrit etc.). I really don't expect that written English won't be known long enough (maybe people won't be able to pronounce it well, but they'll be able to read)... Besides that, we have many other useful language editions. At the other side, I prefer reasonably durable "paper" form. If it is possible to make some plastics or so which may last for, let's say, 1000 years, we may renew those editions every 50 or 100 years and distribute them to national libraries or put at various places at/in the Earth. Also, David mentioned copying of databases all around the world. This is a good idea, too. Maybe, we should make a wider project in cooperation with UN and other global organizations. That would be very useful for the best scenarios, like it may be if Internet would be damaged, while electricity and computers would still exist. [1] - http://en.wikipedia.org/wiki/History_of_optics

On Wed, May 6, 2009 at 8:13 PM, Platonides &lt;Platonides(a)gmail.com&gt; wrote: Sell my computer for food, water, guns, and ammunition?

On Wed, May 6, 2009 at 6:54 PM, Chad &lt;innocentkiller(a)gmail.com&gt; wrote: Perhaps a tattoo there is the safest place for Wikipedia then!

On Wed, May 6, 2009 at 8:13 PM, Platonides &lt;Platonides(a)gmail.com&gt; wrote: In such a futuristic world, I would expect that the major sources of power would be things like solar and geothermal that don't require long-distance supply chains. Then even if the world falls into anarchy, some well-stocked parts will still have power for a good long while. So you wouldn't need to actually print it out, you'd have computers running continuously in some places. Even if 95% of humanity was wiped out, you'd still have a few hundred million people. Not one of them is going to be in a position to save some computers? Even militaries, which are prepared for all sorts of disasters -- some of which will have computers in multiple geographically distributed bunkers deep underground with enough fuel on-site to keep them running for days to years? Screw Wikipedia. If I want to preserve useful knowledge, I'll make sure to safeguard my textbooks. In terms of utility for rebuilding society, the value of Wikipedia is zero compared to even a tiny university library. And there are many thousands of university libraries already conveniently scattered around the world, not a few of them in subbasements where they'll be resistant to nasty things happening on the surface. On Thu, May 7, 2009 at 12:16 AM, Tim Starling &lt;tstarling(a)wikimedia.org&gt; wrote: Okay, I can agree with that.

2009/5/10 Michael Peel &lt;email(a)mikepeel.net&gt;et>: The new dumps are progressing very well. Presumably when they're done we can give the Internet Archive and any similar archivists a yell. - d.

2009/5/10 Anthony &lt;wikimail(a)inbox.org&gt;rg>: Why? The private parts are just deleted stuff. The deleted stuff isn't generally very valuable, that's why it was deleted.

On Sun, May 10, 2009 at 5:16 PM, Anthony &lt;wikimail(a)inbox.org&gt; wrote: By the way, the same argument could be made about all the prior versions - it "generally isn't very valuable", and that's why it was edited. The current versions, or something close to them, are widespread enough to do in case of catastrophe. Google cache, answers.com, etc. At least for the English language version, anyway.

2009/5/10 Casey Brown &lt;cbrown1023.ml(a)gmail.com&gt;om>: They are produced, but they aren't distributed (that's what "private" means!). Do they get stored anywhere other than the file server?

The OAI updater is for incremental updates of search indexes using MWSearch. On Sun, May 10, 2009 at 3:34 PM, Anthony &lt;wikimail(a)inbox.org&gt; wrote:

Aryeh Gregor wrote: I wouldn't go quite that far. The idea of doing it (or having done it) makes people feel good, due to the collective sci-fi-like fantasy implicitly promulgated by the project itself -- a future world of poverty and decay, saved by the serendipitous discovery of a time-capsule sent from the past. It's a spectacle, a stunt, and it has PR value. I certainly don't begrudge the Long Now Foundation for having done this with the Rosetta Project, since their primary goal is to encourage long-term thinking, and expensive stunts are obviously a key part of that. But Wikimedia's goals are somewhat different, and we could probably find some stunts which are more relevant to our mission. -- Tim Starling

On Thu, May 7, 2009 at 12:16 AM, Tim Starling &lt;tstarling(a)wikimedia.org&gt; wrote: Producing long-lived snapshots of important projects, and preserving them for posterity, is more than a feel-good effort -- it is good practice. Most of the layout work needed to produce this sort of copy must be done for any sort of print copy, which is certainly a useful class of dump. If this were simply a stunt, then it would be worth doing a few times for impact. In this case I think it's a valid and beautiful way to store data in its own right. And it is getting cheap enough that individuals might want to get offline copies in this format. I checked in with Alexander Rose of the Rosetta project, and here's the latest news: They are working with their etchers to lower the cost of the process. The current amortized cost of making 10 nickel discs (each with 10,000 pages in a 100x100 grid) is around $500 each. They can also make polymer copies for much less that are likely stable for at least a century. As they standardize the process, the price may continue to drop. The specific process they use involves a few steps : material is rendered at 300-600dpi (text and images), and laid out in 11x11" pages. These are saved as separate image files, numbered 1 to 100,000 in 100 directories, and sent to the etcher; which fits each directory of 100 images onto one row. You need a microscope to read the result, but a decent USB microscope could do it. Are the goals so different? It seems to me long-term thinking is part and parcel of comprehensively realizing Wikimedia's goals, from licensing and access to archival revision tracking and multilingualism. We should probably discuss this in the original thread about strategic planning. Rose says they would be glad to include something like a "top 10,000 articles" selection from as many languages as possible in the Rosetta library. (Thanks!) He suggests this would be a sought-after gift for major donors. I think that's probably true, and could pay for such an initiative, but encouraging long-term thinking and long-term valuation of knowledge around Wikimedia (not just in general) is a more important reason to consider it. SJ

A brief update: On Thu, May 7, 2009 at 5:17 AM, Samuel Klein The amortized cost is closer to $600 for nickel disks and $150 for polymer. Back to the near term : can someone tell me the rough cost per GB and expected lifetime of the backups our colo uses? (I would guess $1 and 20 years.) Are there full offsite backups? I would feel better if people would help update http://meta.wikimedia.org/wiki/Contingency_planning ... SJ

On Tue, May 5, 2009 at 10:12 AM, Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt;wrote;wrote: But what about the current present value of the long term utility? How do you measure "long term utility" anyway? Sort out the 1% that might be useful first, maybe...

On Tue, May 5, 2009 at 7:13 AM, David Gerard &lt;dgerard(a)gmail.com&gt; wrote: What David said :) Good dumps being made readily available leaves the door open for others to do whatever they like with the information, including etching, copying by hand, delivering to space, etc. But without that first step of sharing the data we've got, everything else about archival preservation is a little problematic. Only the Foundation is in a position to make the project data available at the moment, and it should be a key priority. Short-term success is making us a bit long-term complacent, I think. -- Phoebe, who had the joy of handling and writing about an original copy of http://en.wikipedia.org/wiki/Cyclopaedia,_or_Universal_Dictionary_of_Arts_a… once, and who believes that there is value in 'old encyclopedias' as well as new.