Around 2000 (~0.01%) revisions from replicated srwiki_p database seem
to be missing on Toolserver.
For example, revision 5242120 exists in database[1], but doesn't exist
on Toolserver.
Does anybody know the reason for this anomaly?
P.S.
I know that I should open a Jira bug, but because of unknown reason I
can't log in Jira. I have username 'celicni', but I can't log in, and
when try to reset password, I get 'No user with that username exists.'
[1] http://sr.wikipedia.org/w/index.php?title=%D0%9F%D0%B8&diff=5242120&oldid=5…
Cheers,
Goran Obradovic
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello DaB!
Hello to everybody!
Since about 2 days I got errors because my bot script started running
on mayapple.toolserver.org. As you might remember, back last year (I
think) there were [1], [2] and [3].
[1] https://jira.toolserver.org/browse/TS-1449
[2] https://jira.toolserver.org/browse/TS-1360
[3] https://jira.toolserver.org/browse/TS-1452
In the course of this you installed a backport of opencv2 (the "2" is
important) to nightshade - even this seems to be noted in the links.
This was done to '/usr/local/lib/python2.6/' and it looks like this is
missing on mayapple which causes my bot to stop working. Could please
somebody compile, copy, symlink, ... this backport to mayapple too?
That would be great! Thanks.
Then another issue - I think (again - sorry but yes! ;) we have some
problem with cron job execution. When looking at my bots (even thought
this is not COMPLETELY clear beacuse of other issue mentioned before)
but also at the lower left of [4] to be precise [5] and compare "Week
6" with "Week 7" till now ("Week 9") then we clearly have some issues.
The first part of "Week 7" may be until mid "Week 8" looks resonable,
but the rest looks "free floating" to me which may point to some load
or other issue... what do you think? Thanks again.
[4] http://munin.toolserver.org/Login/hawthorn/cron_jobs_sh.html
[5] http://munin.toolserver.org/Login/hawthorn/cron_jobs_sh-month.png
Thanks a lot for your help and time and greetings!
DrTrigon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlE1FlIACgkQAXWvBxzBrDDgPwCg06mWlZ1hzl0O0B//GPsFAee5
CpcAoKecoBwvb7ZOk01JG6DpPoHWsjd9
=YtdD
-----END PGP SIGNATURE-----
Hi,
I am planning to make some maintenace reports for the Danish Wikipedia
at regular intervals, like once a week or once every few days, but the
exact time to run the programs doesn't really matter.
So what time of the day is it best to run such programs?
And is there a way to tell SGE that it may choose the most convenient
time to start to job?
Hi Tim and all,
2013/4/8 Tim Landscheidt <tim(a)tim-landscheidt.de>
> "Marc A. Pelletier" <marc(a)uberbox.org> wrote:
>
> > [...]
>
> > The database replication is also well on its way; you can find the
> > current roadmap at:
>
> > https://wikitech.wikimedia.org/wiki/Tool_Labs/Database_plan
>
> > [...]
>
> To quote from there:
>
> | Overview
>
> | * All public wikis will be replicated to the LabsDB servers,
> | with private user data redacted.
>
> | * First, data will be replicated to a special set of data-
> | base servers (PreLabsDBDBS) that use triggers to rewrite
> | or remove private data. They will write row based bin-
> | logs. Production shards will map 1:1 with mysql in-
> | stances, unlike on toolserver where some are combined via
> | a custom replication engine.
>
> | * Triggers will be created with the help of the redactatron
> | schema review tool.
>
> | * The actual labs databases will replicate from the above
> | mentioned databases. Users will access data via views
> | that only include reviewed tables and columns to ensure
> | that unreviewed tables (such as from a new extension)
> | aren't exposed without prior review.
>
> | * Replicated data will be stored on flash storage, while
> | each system will have a traditional disk array attached to
> | store labs project data. Users will be able to join
> | project tables against wiki tables, but only within the
> | current shard.
>
> | * The labs team will integrate these databases with labs,
> | automating database creation and access on a per-project
> | basis.
>
> This means that JOINs for example between wikis and Commons
> or Wikidata will not be possible. WTF? One of the stated
> goals of Tool Labs is "Provide a location for analytics
> work", so any changes here should /enhance/ the possibili-
> ties the Toolserver offers and not shrink them. This is BTW
> one of the top items on the "Needed Toolserver features"
> list.
>
We will see into this and find out what is possible to do until when.
You'll hear more about it soon.
Cheers, Silke
--
Silke Meyer
Internes IT-Management und Projektmanagement Toolserver
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. (030) 219 158 260
http://wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
<Tron>Greetings, Programs!</Tron>
So, not many updates for a while, as things have been progressing at a
fair clip in the "oh, my god boring gruntwork" front.
The biggest news is the addition of Petr Bena to the tools project
sysadmin team as its first volunteer. Petr has been very involved in
the setup and administration of the Tool Labs' predecessor projects, and
will continue to steer the bots project where the rules are a little
more relaxed to facilitate more experimental development.
He's also joining me on the tools project proper, to help provide
support to maintainers over a wider range of times, and to increase
availability of sysadmins. You can find him hanging around
#wikimedia-labs, often at times where I am not available.
There is some documentation-in-progress that give a lot of information
on how to set up your tools on the Labs architecture at:
https://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/Help
Please don't hesitate to comment if you see missing information, or if
parts of it are less clear than idea.
On the other fronts, the wikitech management interface is now in place
for self-serve of tool account creation by Labs users; this requires
moving already-existing tool accounts to the new scheme, and a brief
outage for that purpose later this week (see note below)
Experiments with a bulletproof replacement for gluster are well on their
way; with NFS from a highly redundant server as the currently favored
option. With a bit of luck, I'll use the opportunity given by the
outage for the tool account switchover to move the shared tools
filesystem to NFS as a trial run.
The database replication is also well on its way; you can find the
current roadmap at:
https://wikitech.wikimedia.org/wiki/Tool_Labs/Database_plan
=== Planned outage ===
In order to move the extant tool accounts to the new, final scheme, and
(progress permitting) move the shared filesystems to a new storage
server, there will be a brief outage of the Tool Labs infrastructure
this Thursday April 11 starting at 16:00 UTC. The outage is expected to
last 20 minutes during which service will be intermittently unavailable.
Announcements will be sent by email, on IRC and on the servers 30
minutes before the start of maintenance, at its start, and upon completion.
Impact:
* Jobs running on the grid engine will be stopped then restarted
automatically at the end of the maintenance window. If you are running
a job that cannot or should not be restarted automatically without
intervention from its maintainers, please make certain that it has been
stopped before the start of the maintenance window;
* The login server will be restarted during the window, ending active
sessions;
* The web service will be intermittently unavailable; and
* Running processes not scheduled through the grid engine will be killed.
Recovery plan:
In case of unplanned failure during the maintenance window,
configuration will be rolled back to the current version and a new
window will be planned after postmortem. Disruption of services will
take place as noted and an announcement will be sent.
-- Marc
Sorry, seems I used the wrong sender address...
---------- Forwarded message ----------
Date: Sat, 6 Apr 2013 23:59:12
From: Marlen Caemmerer <nosy(a)c-base.org>
To: Wikimedia Toolserver <toolserver-l(a)lists.wikimedia.org>
Subject: Re: [Toolserver-l] Ipv6 issues
Hey,
On Sat, 6 Apr 2013, Merlissimo wrote:
>
> I think nosy needs some sleep because she hasn't slept last night.
>
> "getent ipnodes willow" does not return the ipv6 address configured at
> /etc/hostname6.bnx0
Well...
rnosy@hemlock:~$ host willow
willow.toolserver.org has address 185.15.59.202
willow.toolserver.org has IPv6 address 2a02:ec80:101::2:4
rnosy@hemlock:~$ host 2a02:ec80:101::2:4
4.0.0.0.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.0.1.0.0.8.c.e.2.0.a.2.ip6.arpa domain
name pointer willow.toolserver.org.
root@willow:~# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232
index 1
inet 127.0.0.1 netmask ff000000
bnx0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 185.15.59.202 netmask ffffffc0 broadcast 185.15.59.255
ether 0:1d:9:71:74:cb
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252
index 1
inet6 ::1/128
bnx0: flags=2000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2
inet6 fe80::21d:9ff:fe71:74cb/10
ether 0:1d:9:71:74:cb
bnx0:1: flags=2000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2
inet6 2a02:ec80:101::2:4/64
root@willow:~# host willow
willow.toolserver.org has address 185.15.59.202
willow.toolserver.org has IPv6 address 2a02:ec80:101::2:4
root@willow:~# host 2a02:ec80:101::2:4
4.0.0.0.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.0.1.0.0.8.c.e.2.0.a.2.ip6.arpa domain
name pointer willow.toolserver.org.
I dont see any problem there.
Do you?
> So record is missing at dns. If dns cannot be changed ipv6 address must be
> added to /etc/inet/ipnodes (which is always a good idea if dns is not 100%
> reliable). /usr/lib/inet/in.ndpd is running.
>
> ifconfig -a6 shows that the ip address is configured three times as local
> interface for the same physical interface. Is this expected? And physical
> interface bnx0 is associated with a link local address only. This shows that
> the router is not propagating the site prefix. So you must change the router
> config or add the site prefix locally. Maybe interfaces were not plumbed.
>
I'd really like to reboot tomorrow. Everytime I do what is the equivalent to
/etc/init.d/networking restart it adds an interface with the v6 address.
The next try from my side would be to switch back to the old v6 range to see if
the problems all go away. I currently cant reach even from the same network.
root@willow:~# ping 2a02:ec80:101:301::2:2
^C
Solaris is not my home base system but I only changed the prefixes in the
config files so it should work as it did before.
Lets see how this goes tomorrow after some sleep.
Cheers
Marlen/nosy
I know we had maintenance about 12 hours ago, but I am unable to launch any
of a half dozen IRC bots, reba and other toolserver IRC bots seem to be
having similar issues.
I got emails from cron every 5 minutes for a couple hours last night.
----
User:Hersfold
hersfoldwiki(a)gmail.com
Sent from my Windows Phone
------------------------------
From: John
Sent: 4/6/2013 10:06
To: Wikimedia Toolserver
Subject: [Toolserver-l] Issues
I know we had maintenance about 12 hours ago, but I am unable to launch any
of a half dozen IRC bots, reba and other toolserver IRC bots seem to be
having similar issues.
Heya folks :)
I wanted to introduce myself to those of you who don't know me yet. As
Silke mentioned previously from now on I'll be supporting her with
communication for her toolserver work at Wikimedia Deutschland.
For the last year I've been doing community communications for
Wikidata. Now that the project is in a very good shape it needs a bit
less of my attention and I will spend time on other projects as well
like for example the toolserver.
I'm now going to spend some time catching up on all things toolserver
and labs. If there is anything in particular I can help you with
please let me know.
I am on IRC as Lydia_WMDE. My user page on wiki is
http://meta.wikimedia.org/wiki/User:Lydia_Pintscher_(WMDE)
Looking forward to working with you!
Cheers
Lydia
--
Lydia Pintscher - http://about.me/lydia.pintscher
Community Communications for Wikidata
Wikimedia Deutschland e.V.
Obentrautstr. 72
10963 Berlin
www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das
Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
