Hello,
I am seeing more and more project on growing the commons media collection which raised the question (in my head, that's where) about its backup. Checking meta kindly redirected me to https://wikitech.leuksman.com/view/Backup_procedures which kind of trying to describe the backup methods, but I have a strange feeling that the page isn't very well maintained.
Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.
Now, would some kind soul either update it up to reality, or kindly point me to the actual description, or describe it and give me the right to update it. :-) Or any other way which results in: a) me knowing what's the current state of art, and b) a page where everyone can do the same.
Since images cannot be retrieved - AFAIR - the question of their backup is even more important than the database, which can be (and does indeed) backed up to the whole human race.
Thanks, grin
grin schrieb:
Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.
I'm not a server admin and I don't know the exact details, but as far as I know, the information there is correct in so far as there are two storage servers for media files in tampa, one replicating the other. I think the second one is even located in a different data center, though in the same building. What exactly the status of automatic replication between these servers is, I do not know.
What I do know is that the german chapter plans an off-size backup for media files in amsterdam. We will order the hardware we need for that this year still. I don't know how long it will take to set up a working replication process, but I hope that it will not be too long. Having an off-site backup seems a good idea, the next hurricane isn't that far away.
We should also think about providing reasonable media bundles for download in some way. My current idea is to create bundles of all media files used or contained in a given category (maybe even including two or three levels of subcategories). Such bundles would have to be created by an off-line job, i suppose. And maybe requesting them should be limited to admins, to avoid flooding.
-- daniel
I think that any initiative of a new backup or download possibility of Commons files should be welcome. If the whole Commons collection is indeed only located on two servers in the whole world that gives me a little bit of a scary feeling...
-- Hay
On Tue, Dec 16, 2008 at 11:48 AM, Daniel Kinzler daniel@brightbyte.de wrote:
grin schrieb:
Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.
I'm not a server admin and I don't know the exact details, but as far as I know, the information there is correct in so far as there are two storage servers for media files in tampa, one replicating the other. I think the second one is even located in a different data center, though in the same building. What exactly the status of automatic replication between these servers is, I do not know.
What I do know is that the german chapter plans an off-size backup for media files in amsterdam. We will order the hardware we need for that this year still. I don't know how long it will take to set up a working replication process, but I hope that it will not be too long. Having an off-site backup seems a good idea, the next hurricane isn't that far away.
We should also think about providing reasonable media bundles for download in some way. My current idea is to create bundles of all media files used or contained in a given category (maybe even including two or three levels of subcategories). Such bundles would have to be created by an off-line job, i suppose. And maybe requesting them should be limited to admins, to avoid flooding.
-- daniel
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hoi, Those copies are in the same location *that *is scary Not good. Thanks, GerardM
2008/12/16 Hay (Husky) huskyr@gmail.com
I think that any initiative of a new backup or download possibility of Commons files should be welcome. If the whole Commons collection is indeed only located on two servers in the whole world that gives me a little bit of a scary feeling...
-- Hay
On Tue, Dec 16, 2008 at 11:48 AM, Daniel Kinzler daniel@brightbyte.de wrote:
grin schrieb:
Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.
I'm not a server admin and I don't know the exact details, but as far as
I know,
the information there is correct in so far as there are two storage
servers for
media files in tampa, one replicating the other. I think the second one
is even
located in a different data center, though in the same building. What
exactly
the status of automatic replication between these servers is, I do not
know.
What I do know is that the german chapter plans an off-size backup for
media
files in amsterdam. We will order the hardware we need for that this year
still.
I don't know how long it will take to set up a working replication
process, but
I hope that it will not be too long. Having an off-site backup seems a
good
idea, the next hurricane isn't that far away.
We should also think about providing reasonable media bundles for
download in
some way. My current idea is to create bundles of all media files used or contained in a given category (maybe even including two or three levels
of
subcategories). Such bundles would have to be created by an off-line job,
i
suppose. And maybe requesting them should be limited to admins, to avoid
flooding.
-- daniel
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Daniel Kinzler wrote:
grin schrieb:
Even if it is, the entry on image backups is highly nonspecific and contains a big warning about being seriously out of date.
I'm not a server admin and I don't know the exact details, but as far as I know, the information there is correct in so far as there are two storage servers for media files in tampa, one replicating the other. I think the second one is even located in a different data center, though in the same building. What exactly the status of automatic replication between these servers is, I do not know.
I think it really improved with the new servers.
What I do know is that the german chapter plans an off-size backup for media files in amsterdam. We will order the hardware we need for that this year still. I don't know how long it will take to set up a working replication process, but I hope that it will not be too long. Having an off-site backup seems a good idea, the next hurricane isn't that far away.
-- daniel
I have been mantaining a commons image copy on gmaxwell's so in case a hurricane hit pmtpa only the latest ones would be lost (those copies have already been useful on the image loss). Although in case of a hurricane I hope you have your db copy up to date at the Verein (do you have all revisions text?).
There are also copies on that machine for other languages images, but too old. If you want other wikis added, drop me a line... or move to commons :)
wikitech-l@lists.wikimedia.org