Hi,
at the office hour yesterday (cf. http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt):
| [...] | <Coren> multichill: The long story short; replicating | databases is happening soon (Within the month) | Replicating multiple copies of commons and wikidata | isn't going to happen that way; it needs to be built | into application logic or using federated tables. | Almost /all/ of the problems the TS have had with | replication were caused by that redundancy and | trying to keep it synced. | <multichill> Coren: So you're basically saying Toollabs is | useless for me | <Platonides> {{ref needed}} | <Coren> We're all more than happy to help you (and any other | maintainer) with adapting your tools to work in that | setup. | <scfc_de> Coren: In that case, Tools will not be able to | replace Toolserver. | <giftpfla1ze> what are federated tables btw? | <Ryan_Lane> AFAIK toolserver will also have this limitation | at some point | <scfc_de> Ryan_Lane: What do you mean? | [...]
There were never answers to this, so I bring it up here again:
1. How were "almost /all/ of the problems the TS have had with replication" "caused by that redundancy and trying to keep it synced"?
2. What limitation will the Toolserver have at some point?
Tim
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt tim@tim-landscheidt.dewrote:
Hi,
at the office hour yesterday (cf. http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt ):
| [...] | <Coren> multichill: The long story short; replicating | databases is happening soon (Within the month) | Replicating multiple copies of commons and wikidata | isn't going to happen that way; it needs to be built | into application logic or using federated tables. | Almost /all/ of the problems the TS have had with | replication were caused by that redundancy and | trying to keep it synced. | <multichill> Coren: So you're basically saying Toollabs is | useless for me | <Platonides> {{ref needed}} | <Coren> We're all more than happy to help you (and any other | maintainer) with adapting your tools to work in that | setup. | <scfc_de> Coren: In that case, Tools will not be able to | replace Toolserver. | <giftpfla1ze> what are federated tables btw? | <Ryan_Lane> AFAIK toolserver will also have this limitation | at some point | <scfc_de> Ryan_Lane: What do you mean? | [...]
There were never answers to this, so I bring it up here again:
How were "almost /all/ of the problems the TS have had with replication" "caused by that redundancy and trying to keep it synced"?
What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding plans for the databases, and due to a change in how we'll be doing replication. Of course, I've heard this in passing. For answers to both of these questions you'll need to talk to binasher and/or notpeter on IRC, as they are the ones doing the database work.
- Ryan
<a rel="license" href="http://creativecommons.org/licenses/by/3.0/"><img alt="Creative Commons License" style="border-width:0" src=" http://i.creativecommons.org/l/by/3.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href=" http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution 3.0 Unported License</a>. On May 1, 2013 8:49 PM, "Ryan Lane" rlane@wikimedia.org wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt tim@tim-landscheidt.dewrote:
Hi,
at the office hour yesterday (cf. http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt):
| [...] | <Coren> multichill: The long story short; replicating | databases is happening soon (Within the month) | Replicating multiple copies of commons and wikidata | isn't going to happen that way; it needs to be built | into application logic or using federated tables. | Almost /all/ of the problems the TS have had with | replication were caused by that redundancy and | trying to keep it synced. | <multichill> Coren: So you're basically saying Toollabs is | useless for me | <Platonides> {{ref needed}} | <Coren> We're all more than happy to help you (and any other | maintainer) with adapting your tools to work in that | setup. | <scfc_de> Coren: In that case, Tools will not be able to | replace Toolserver. | <giftpfla1ze> what are federated tables btw? | <Ryan_Lane> AFAIK toolserver will also have this limitation | at some point | <scfc_de> Ryan_Lane: What do you mean? | [...]
There were never answers to this, so I bring it up here again:
How were "almost /all/ of the problems the TS have had with replication" "caused by that redundancy and trying to keep it synced"?
What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding plans for the databases, and due to a change in how we'll be doing replication. Of course, I've heard this in passing. For answers to both of these questions you'll need to talk to binasher and/or notpeter on IRC, as they are the ones doing the database work.
- Ryan
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Remove wiki offline Do not allow diffrent languages on same feeds Delete unused accounts Delete dangerous information Acknowledge that the sole creator could be anyone, and expects more order in a hectic database with too much junk being reedited over and over. Once its written correctly do not allow repeaters to do over incorrectly work that was ligit. Figure out how many angles are approaching the database. This toolserver can cause havoc or can contain order depends on proper knowledge used in operating it. Thanks MILASTAR.TS.RO On May 1, 2013 9:17 PM, "Patricia Pintilie" pintilieempire@gmail.com wrote:
<a rel="license" href="http://creativecommons.org/licenses/by/3.0/"><img alt="Creative Commons License" style="border-width:0" src=" http://i.creativecommons.org/l/by/3.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href=" http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution 3.0 Unported License</a>. On May 1, 2013 8:49 PM, "Ryan Lane" rlane@wikimedia.org wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt tim@tim-landscheidt.dewrote:
Hi,
at the office hour yesterday (cf. http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt):
| [...] | <Coren> multichill: The long story short; replicating | databases is happening soon (Within the month) | Replicating multiple copies of commons and wikidata | isn't going to happen that way; it needs to be built | into application logic or using federated tables. | Almost /all/ of the problems the TS have had with | replication were caused by that redundancy and | trying to keep it synced. | <multichill> Coren: So you're basically saying Toollabs is | useless for me | <Platonides> {{ref needed}} | <Coren> We're all more than happy to help you (and any other | maintainer) with adapting your tools to work in that | setup. | <scfc_de> Coren: In that case, Tools will not be able to | replace Toolserver. | <giftpfla1ze> what are federated tables btw? | <Ryan_Lane> AFAIK toolserver will also have this limitation | at some point | <scfc_de> Ryan_Lane: What do you mean? | [...]
There were never answers to this, so I bring it up here again:
How were "almost /all/ of the problems the TS have had with replication" "caused by that redundancy and trying to keep it synced"?
What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding plans for the databases, and due to a change in how we'll be doing replication. Of course, I've heard this in passing. For answers to both of these questions you'll need to talk to binasher and/or notpeter on IRC, as they are the ones doing the database work.
- Ryan
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Hey,
On Wed, 1 May 2013, Ryan Lane wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt tim@tim-landscheidt.dewrote:
| [...]
There were never answers to this, so I bring it up here again:
- How were "almost /all/ of the problems the TS have had with replication" "caused by that redundancy and trying to keep it synced"?
Thanks for this question :) - I also want to know.
From my perspective it does not look like this and even the data inconsistencies appear when we have no commons copy on a mysql instance.
And: DaB experimented with federated tables for commons too and we decided to not do this since it does not perform from the start. Probably nowadays when I planned something new in this area (which does not seem to make sense for TS) I'd really give Galera a try - http://codership.com/content/using-galera-cluster
- What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding plans for the databases, and due to a change in how we'll be doing replication. Of course, I've heard this in passing. For answers to both of these questions you'll need to talk to binasher and/or notpeter on IRC, as they are the ones doing the database work.
Thanks for telling...
Cheers Marlen/nosy
On 02/05/13 10:03, Marlen Caemmerer wrote:
Hey,
On Wed, 1 May 2013, Ryan Lane wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt tim@tim-landscheidt.dewrote:
There were never answers to this, so I bring it up here again:
- How were "almost /all/ of the problems the TS have had with replication" "caused by that redundancy and trying to keep it synced"?
Thanks for this question :) - I also want to know. From my perspective it does not look like this and even the data inconsistencies appear when we have no commons copy on a mysql instance. And: DaB experimented with federated tables for commons too and we decided to not do this since it does not perform from the start. Probably nowadays when I planned something new in this area (which does not seem to make sense for TS) I'd really give Galera a try - http://codership.com/content/using-galera-cluster
FWIW, my {{ref needed}} phrase in the log was also intended to that statement by Coren, not to multichill reply.
Additionally, I rembember you data inconsistencies happen even with native mysql replication (as informed by Nosy earlier).
On May 2, 2013 3:08 AM, "Marlen Caemmerer" marlen.caemmerer@wikimedia.de wrote:
Hey,
On Wed, 1 May 2013, Ryan Lane wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt <tim@tim-landscheidt.de
wrote:
| [...]
There were never answers to this, so I bring it up here again:
- How were "almost /all/ of the problems the TS have had with replication" "caused by that redundancy and trying to keep it synced"?
Ryan, repeaters are from the root of a program inwhich start the initial
setup.
Thanks for this question :) - I also want to know. From my perspective it does not look like this and even the data
inconsistencies appear when we have no commons copy on a mysql instance.
And: DaB experimented with federated tables for commons too and we
decided to not do this since it does not perform from the start.
Probably nowadays when I planned something new in this area (which does
not seem to make sense for TS) I'd really give Galera a try - http://codership.com/content/using-galera-cluster
- What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding plans for the databases, and due to a change in how we'll be doing replication. Of course, I've heard this in passing. For answers to both
of
these questions you'll need to talk to binasher and/or notpeter on IRC,
as
they are the ones doing the database work.
Thanks for telling...
Cheers Marlen/nosy
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list:
toolserver-l@lists.wikimedia.org