I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
Cheers Bill,
If both webservers are using a single database, then their data should be synchronized. What other things are you looking to synchronize?
You might want to look into setting up a distributed caching system (hint: use memcached [1].) This will share the cache amongst the webservers. You might also want to put the codebase on an NFS drive and mount it on each of the webservers. If you are looking into building a scalable MediaWiki installation, I would strongly advise taking a look into Puppet [2].
\daniel renfro
[1.] https://www.mediawiki.org/wiki/Memcached [2.] http://en.wikipedia.org/wiki/Puppet_(software)
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto:mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 12:09 PM To: MediaWiki announcements and site admin list Subject: [MediaWiki-l] synchronizing two MW servers
I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
_______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
On Thu, Oct 23, 2014 at 1:44 PM, Daniel Renfro drenfro@vistaprint.com wrote:
Cheers Bill,
If both webservers are using a single database, then their data should be synchronized. What other things are you looking to synchronize?
What about the files? Specifically the images directory.
You might want to look into setting up a distributed caching system (hint: use memcached [1].) This will share the cache amongst the webservers. You might also want to put the codebase on an NFS drive and mount it on each of the webservers. If you are looking into building a scalable MediaWiki installation, I would strongly advise taking a look into Puppet [2].
Thanks, this is great info. Unfortunately, I don't have full control over the environment, but I will make these suggestions.
\daniel renfro
[1.] https://www.mediawiki.org/wiki/Memcached [2.] http://en.wikipedia.org/wiki/Puppet_(software)
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto:mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 12:09 PM To: MediaWiki announcements and site admin list Subject: [MediaWiki-l] synchronizing two MW servers
I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
OHYEAH, files!
I suggest mounting an NFS share on both webservers so they will write to the same place. If you're going to do this for images, I think it should be done for the entire MW installation. Then you never have to worry about each webserver being on a different version -- every webserver is serving up the same code.
\daniel renfro
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto:mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 1:51 PM To: MediaWiki announcements and site admin list Subject: Re: [MediaWiki-l] synchronizing two MW servers
On Thu, Oct 23, 2014 at 1:44 PM, Daniel Renfro drenfro@vistaprint.com wrote:
Cheers Bill,
If both webservers are using a single database, then their data should be synchronized. What other things are you looking to synchronize?
What about the files? Specifically the images directory.
You might want to look into setting up a distributed caching system (hint: use memcached [1].) This will share the cache amongst the webservers. You might also want to put the codebase on an NFS drive and mount it on each of the webservers. If you are looking into building a scalable MediaWiki installation, I would strongly advise taking a look into Puppet [2].
Thanks, this is great info. Unfortunately, I don't have full control over the environment, but I will make these suggestions.
\daniel renfro
[1.] https://www.mediawiki.org/wiki/Memcached [2.] http://en.wikipedia.org/wiki/Puppet_(software)
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto:mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 12:09 PM To: MediaWiki announcements and site admin list Subject: [MediaWiki-l] synchronizing two MW servers
I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
If minimizing downtime is a concern, then you might not want to serve up the code from a share. It's common in a load-balanced environment to take one or more web servers out of rotation, upgrade their code, then re-introduce them -- then take the other ones out, upgrade code, and bring them back into the load balancer.
You do have a complication with mediawiki in that sometimes running the update.php maintenance script will make changes to the database. Then you have a split environment where some web servers can talk to the database and others might throw errors on schema differences.
Can be tricky. Hopefully your use case is more simple and you can get away with sharing the codebase.
Larry Silverman Chief Technology Officer TrackAbout, Inc.
On Thu, Oct 23, 2014 at 1:03 PM, Daniel Renfro drenfro@vistaprint.com wrote:
OHYEAH, files!
I suggest mounting an NFS share on both webservers so they will write to the same place. If you're going to do this for images, I think it should be done for the entire MW installation. Then you never have to worry about each webserver being on a different version -- every webserver is serving up the same code.
\daniel renfro
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto: mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 1:51 PM To: MediaWiki announcements and site admin list Subject: Re: [MediaWiki-l] synchronizing two MW servers
On Thu, Oct 23, 2014 at 1:44 PM, Daniel Renfro drenfro@vistaprint.com wrote:
Cheers Bill,
If both webservers are using a single database, then their data should
be synchronized. What other things are you looking to synchronize?
What about the files? Specifically the images directory.
You might want to look into setting up a distributed caching system
(hint: use memcached [1].) This will share the cache amongst the webservers. You might also want to put the codebase on an NFS drive and mount it on each of the webservers. If you are looking into building a scalable MediaWiki installation, I would strongly advise taking a look into Puppet [2].
Thanks, this is great info. Unfortunately, I don't have full control over the environment, but I will make these suggestions.
\daniel renfro
[1.] https://www.mediawiki.org/wiki/Memcached [2.] http://en.wikipedia.org/wiki/Puppet_(software)
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto:mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 12:09 PM To: MediaWiki announcements and site admin list Subject: [MediaWiki-l] synchronizing two MW servers
I have 2 application servers behind a load-balancer that both point to a
3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look
like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Hi, Bill, Our setup involves a server containing the codebase (which is mounted over NFS), two web servers (which again use an NFS mount for the data), and one database server. We handle our sessions and caching with memcached, which is pooled between the two web servers. It's a very stable setup which has worked for awhile. I'd recommend you try this out.
---- Justin Folvarcik
On Thu, Oct 23, 2014 at 2:10 PM, Larry Silverman lsilverman@trackabout.com wrote:
If minimizing downtime is a concern, then you might not want to serve up the code from a share. It's common in a load-balanced environment to take one or more web servers out of rotation, upgrade their code, then re-introduce them -- then take the other ones out, upgrade code, and bring them back into the load balancer.
You do have a complication with mediawiki in that sometimes running the update.php maintenance script will make changes to the database. Then you have a split environment where some web servers can talk to the database and others might throw errors on schema differences.
Can be tricky. Hopefully your use case is more simple and you can get away with sharing the codebase.
Larry Silverman Chief Technology Officer TrackAbout, Inc.
On Thu, Oct 23, 2014 at 1:03 PM, Daniel Renfro drenfro@vistaprint.com wrote:
OHYEAH, files!
I suggest mounting an NFS share on both webservers so they will write to the same place. If you're going to do this for images, I think it should
be
done for the entire MW installation. Then you never have to worry about each webserver being on a different version -- every webserver is serving up the same code.
\daniel renfro
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto: mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 1:51 PM To: MediaWiki announcements and site admin list Subject: Re: [MediaWiki-l] synchronizing two MW servers
On Thu, Oct 23, 2014 at 1:44 PM, Daniel Renfro drenfro@vistaprint.com wrote:
Cheers Bill,
If both webservers are using a single database, then their data should
be synchronized. What other things are you looking to synchronize?
What about the files? Specifically the images directory.
You might want to look into setting up a distributed caching system
(hint: use memcached [1].) This will share the cache amongst the webservers. You might also want to put the codebase on an NFS drive and mount it on each of the webservers. If you are looking into building a scalable MediaWiki installation, I would strongly advise taking a look
into
Puppet [2].
Thanks, this is great info. Unfortunately, I don't have full control
over
the environment, but I will make these suggestions.
\daniel renfro
[1.] https://www.mediawiki.org/wiki/Memcached [2.] http://en.wikipedia.org/wiki/Puppet_(software)
-----Original Message----- From: mediawiki-l-bounces@lists.wikimedia.org [mailto:mediawiki-l-bounces@lists.wikimedia.org] On Behalf Of Bill Traynor Sent: Thursday, October 23, 2014 12:09 PM To: MediaWiki announcements and site admin list Subject: [MediaWiki-l] synchronizing two MW servers
I have 2 application servers behind a load-balancer that both point to
a
3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look
like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
We (www.uesp.net) have a similar MediaWiki setup and here is roughly how we have set it up (whether it is a "good" design is another question):
- Multiple content servers running Apache with identical copies of the MediaWiki PHP files - Squid load balancer in front of the content servers - All content servers use the same memcached server for cache - PHP sessions shared via NFS - Images shared via NFS (./images/ linked to NFS shared directory ) - Local file cache turned off
The files shared via NFS are on a different server. This has worked well for us and there is only very rarely an issue with the cached content of certain pages (usually cleared up with a purge for that page).
On 23 October 2014 12:09, Bill Traynor btraynor@gmail.com wrote:
I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
I've been told that NFS is NOT allowed in the environment where I'm configuring my wiki. Can the /images directory reside on shared SAN mount? Or, am I better off just writing an rsync script to keep the application servers synced?
Thanks Bill
On Thu, Oct 23, 2014 at 2:21 PM, Dave Humphrey dave@uesp.net wrote:
We (www.uesp.net) have a similar MediaWiki setup and here is roughly how we have set it up (whether it is a "good" design is another question):
- Multiple content servers running Apache with identical copies of the
MediaWiki PHP files
- Squid load balancer in front of the content servers
- All content servers use the same memcached server for cache
- PHP sessions shared via NFS
- Images shared via NFS (./images/ linked to NFS shared directory )
- Local file cache turned off
The files shared via NFS are on a different server. This has worked well for us and there is only very rarely an issue with the cached content of certain pages (usually cleared up with a purge for that page).
On 23 October 2014 12:09, Bill Traynor btraynor@gmail.com wrote:
I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
-- Dave Humphrey -- dave@uesp.net Founder/Server Admin of the Unofficial Elder Scrolls Pages -- www.uesp.net www.viud.net - Building the world's toughest USB drive _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Forget the SAN question, apparently that's not shareable either. So I think I'm stuck with implementing an rsync script to keep my app servers synced.
On Fri, Oct 31, 2014 at 10:43 AM, Bill Traynor btraynor@gmail.com wrote:
I've been told that NFS is NOT allowed in the environment where I'm configuring my wiki. Can the /images directory reside on shared SAN mount? Or, am I better off just writing an rsync script to keep the application servers synced?
Thanks Bill
On Thu, Oct 23, 2014 at 2:21 PM, Dave Humphrey dave@uesp.net wrote:
We (www.uesp.net) have a similar MediaWiki setup and here is roughly how we have set it up (whether it is a "good" design is another question):
- Multiple content servers running Apache with identical copies of the
MediaWiki PHP files
- Squid load balancer in front of the content servers
- All content servers use the same memcached server for cache
- PHP sessions shared via NFS
- Images shared via NFS (./images/ linked to NFS shared directory )
- Local file cache turned off
The files shared via NFS are on a different server. This has worked well for us and there is only very rarely an issue with the cached content of certain pages (usually cleared up with a purge for that page).
On 23 October 2014 12:09, Bill Traynor btraynor@gmail.com wrote:
I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
-- Dave Humphrey -- dave@uesp.net Founder/Server Admin of the Unofficial Elder Scrolls Pages -- www.uesp.net www.viud.net - Building the world's toughest USB drive _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
I've seen Unison http://www.cis.upenn.edu/~bcpierce/unison/ mentioned frequently in the same breath as rsync. I'm much more familiar with rsync myself.
Check me on this, but I think rsync is a one-way sync tool. Seems to me that's not going to work out to keep multiple wiki nodes in sync unless only one is writable and the rest are just read-only mirrors. Unison is, I believe, bi-directional. This is going to matter to you if each wiki node has the ability to save uploaded files.
Bi-directional syncing has its issues when merge conflicts arise. Someone uploads a file by the exact same name to two different nodes at exactly the same time. Unlikely? Yes. But if it happens, Unison's going to create some kind of merge conflict.
Larry Silverman Chief Technology Officer TrackAbout, Inc.
On Fri, Oct 31, 2014 at 9:45 AM, Bill Traynor btraynor@gmail.com wrote:
Forget the SAN question, apparently that's not shareable either. So I think I'm stuck with implementing an rsync script to keep my app servers synced.
On Fri, Oct 31, 2014 at 10:43 AM, Bill Traynor btraynor@gmail.com wrote:
I've been told that NFS is NOT allowed in the environment where I'm configuring my wiki. Can the /images directory reside on shared SAN mount? Or, am I better off just writing an rsync script to keep the application servers synced?
Thanks Bill
On Thu, Oct 23, 2014 at 2:21 PM, Dave Humphrey dave@uesp.net wrote:
We (www.uesp.net) have a similar MediaWiki setup and here is roughly
how we
have set it up (whether it is a "good" design is another question):
- Multiple content servers running Apache with identical copies of
the
MediaWiki PHP files
- Squid load balancer in front of the content servers
- All content servers use the same memcached server for cache
- PHP sessions shared via NFS
- Images shared via NFS (./images/ linked to NFS shared directory )
- Local file cache turned off
The files shared via NFS are on a different server. This has worked well for us and there is only very rarely an issue with the cached content of certain pages (usually cleared up with a purge for that page).
On 23 October 2014 12:09, Bill Traynor btraynor@gmail.com wrote:
I have 2 application servers behind a load-balancer that both point to a 3rd server where the database lives. I'm curious if there are any best practices around keeping the two MediaWiki servers synchronized?
I found Extension:WikiSync (http://www.mediawiki.org/wiki/Extension:WikiSync) but it doesn't look like it's still developed.
I assume Wikimedia does this now, anyone know how?
Thanks Bill
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
-- Dave Humphrey -- dave@uesp.net Founder/Server Admin of the Unofficial Elder Scrolls Pages --
www.uesp.net
www.viud.net - Building the world's toughest USB drive _______________________________________________ MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
mediawiki-l@lists.wikimedia.org