Nick Hill wrote:
A specific host name is used for all updates.
For example,
updates.wikipedia.org would receive form data from any
update. Which physical server/ ip address this goes to depends upon the
result of the negotiation system I previously described.
The 'master hostname'.
After each grant, the ip address of the master hostname is compared with
the ip address of the machine which just received a grant.
To put it clearer, if a different machine has just been granted master
status, the ip address which
updates.wikipedia.org resolves to will
change to the ip address of the machine which just been granted master
status. Bind reloads and the change is pushed to the secondary DNS servers.
If it
differs, the ip address of the master hostname is changed. This change
is pushed to the slave DNS servers using ordinary BIND8 protocol. The
DNS TTL for the domain name of the master wiki server would be set to
300 seconds. If a master went down for any reason, everyone should see
the alternative master within 20 minutes.
Each wiki server has a cron job running which periodically runs a shell
script. This shell script first checks the web server and database are
running as expected by performing an http get command and comparing the
result of the get with the expected result. If they match, the database
and web server are considered working. If this fails, the script exits.
If the script keeps exiting, the arbitration server will never receive
notification the machine works. The arbitration server will remove the
IP address of the failing wiki server from the DNS record.
If the http get is successful, the shell script goes on to invokes SSH
to connect to the arbitration server and run a script on the arbitration
server. The return status of the script on the arbitration server is
communicated as the return value of the ssh command to the script
running as a cron job on the wiki server. This return value tells the
wiki server whether it has been granted master status.
The script running on the remote machine updates the hosts file for the
DNS server according to the algorithms previously described.
All this can be achieved using cron, shell scripting, perhaps a little
PERL for editing the hosts file on the arbitration server (although this
can be done in shell also) and SSH with public/private keys for
communication between the machines.
If the arbitration server failed, all wikis would remain available for
read access only.
The arbitration server is the same as the master DNS server. The
secondary DNS servers should be running a recent version of Bind and
configured to accept push updates from the master.
Whenever a machine tries to obtain master ststus and fails, the ip
address of the current master server is compared to the current master
MySQL server ip address. If it differs, the config file is autmatically
updated and mysql reloaded.