image backup and info on wikitech wiki

List overview All Threads
Download

newer

older

Future of Javascript and mediaWiki

Re: [Wikitech-l] gmane need not...

grin

16 Dec 2008 16 Dec '08

8:05 a.m.

Hello, I am seeing more and more project on growing the commons media collection which raised the question (in my head, that's where) about its backup. Checking meta kindly redirected me to https://wikitech.leuksman.com/view/Backup_procedures which kind of trying to describe the backup methods, but I have a strange feeling that the page isn't very well maintained. Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date. Now, would some kind soul either update it up to reality, or kindly point me to the actual description, or describe it and give me the right to update it. :-) Or any other way which results in: a) me knowing what's the current state of art, and b) a page where everyone can do the same. Since images cannot be retrieved - AFAIR - the question of their backup is even more important than the database, which can be (and does indeed) backed up to the whole human race. Thanks, grin

Show replies by date

Daniel Kinzler

16 Dec 16 Dec

8:48 a.m.

grin schrieb:

...

Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.

I'm not a server admin and I don't know the exact details, but as far as I know, the information there is correct in so far as there are two storage servers for media files in tampa, one replicating the other. I think the second one is even located in a different data center, though in the same building. What exactly the status of automatic replication between these servers is, I do not know. What I do know is that the german chapter plans an off-size backup for media files in amsterdam. We will order the hardware we need for that this year still. I don't know how long it will take to set up a working replication process, but I hope that it will not be too long. Having an off-site backup seems a good idea, the next hurricane isn't that far away. We should also think about providing reasonable media bundles for download in some way. My current idea is to create bundles of all media files used or contained in a given category (maybe even including two or three levels of subcategories). Such bundles would have to be created by an off-line job, i suppose. And maybe requesting them should be limited to admins, to avoid flooding. -- daniel

Hay (Husky)

9:59 a.m.

I think that any initiative of a new backup or download possibility of Commons files should be welcome. If the whole Commons collection is indeed only located on two servers in the whole world that gives me a little bit of a scary feeling... -- Hay On Tue, Dec 16, 2008 at 11:48 AM, Daniel Kinzler <daniel(a)brightbyte.de> wrote:

...

grin schrieb:

Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.

Gerard Meijssen

10:03 a.m.

Hoi, Those copies are in the same location *that *is scary Not good. Thanks, GerardM 2008/12/16 Hay (Husky) <huskyr(a)gmail.com>

...

grin schrieb:

Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.

I'm not a server admin and I don't know the exact details, but as far as

I know,

the information there is correct in so far as there are two storage

servers for

media files in tampa, one replicating the other. I think the second one

is even

located in a different data center, though in the same building. What

exactly

the status of automatic replication between these servers is, I do not

know.

What I do know is that the german chapter plans an off-size backup for

media

files in amsterdam. We will order the hardware we need for that this year

still.

I don't know how long it will take to set up a working replication

process, but

I hope that it will not be too long. Having an off-site backup seems a

good

idea, the next hurricane isn't that far away. We should also think about providing reasonable media bundles for

download in

some way. My current idea is to create bundles of all media files used or contained in a given category (maybe even including two or three levels

subcategories). Such bundles would have to be created by an off-line job,

suppose. And maybe requesting them should be limited to admins, to avoid

flooding.

-- daniel _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Platonides

11:01 p.m.

Daniel Kinzler wrote:

...

grin schrieb:

Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.

I think it really improved with the new servers.

...

What I do know is that the german chapter plans an off-size backup for media files in amsterdam. We will order the hardware we need for that this year still. I don't know how long it will take to set up a working replication process, but I hope that it will not be too long. Having an off-site backup seems a good idea, the next hurricane isn't that far away. -- daniel

I have been mantaining a commons image copy on gmaxwell's so in case a hurricane hit pmtpa only the latest ones would be lost (those copies have already been useful on the image loss). Although in case of a hurricane I hope you have your db copy up to date at the Verein (do you have all revisions text?). There are also copies on that machine for other languages images, but too old. If you want other wikis added, drop me a line... or move to commons :)

5607

days inactive

5608

days old

wikitech-l@lists.wikimedia.org

Manage subscription

4 comments

5 participants

tags (0)

participants (5)

Daniel Kinzler
Gerard Meijssen
grin
Hay (Husky)
Platonides