Fwd - Randy Bush in Tokyo is reporting WMF donation server DNS issues.

---------- Forwarded message ---------- From: Randy Bush randy@psg.com Date: Tue, Nov 26, 2013 at 3:44 PM Subject: wikimedia dns issue To: North American Network Operators' Group nanog@nanog.org

leslie, you out there?

ryuu.psg.com:/Users/randy> host links.email.donate.wikimedia.org. links.email.donate.wikimedia.org is an alias for recp.mkt41.net. recp.mkt41.net has address 74.121.50.40

$ dscacheutil -flushcache

does not clear it

this is all in tokyo. so i ssh into a server in seattle westin.

psg.com:/usr/home/randy> host links.email.donate.wikimedia.org. links.email.donate.wikimedia.org is an alias for recp.mkt41.net. recp.mkt41.net has address 74.121.50.40

the real or unreal CNAME RR is pretty prevalent

pretoria

root@afnog:~ # host links.email.donate.wikimedia.org. links.email.donate.wikimedia.org is an alias for recp.mkt41.net. recp.mkt41.net has address 74.121.50.40

london starts to scare me

drinx.linx.net:/root# host links.email.donate.wikimedia.org. ;; connection timed out; no servers could be reached drinx.linx.net:/root# host links.email.donate.wikimedia.org. ;; connection timed out; no servers could be reached

so i look at the nearest zone i can find

ryuu.psg.com:/Users/randy> doc -p -w donate.wikimedia.org. Doc-2.2.3: doc -p -w donate.wikimedia.org. Doc-2.2.3: Starting test of donate.wikimedia.org. parent is wikimedia.org. Doc-2.2.3: Test date - Tue Nov 26 10:46:48 JST 2013 SYSerr: No servers for donate.wikimedia.org. returned SOAs ... Summary: YIKES: doc aborted while testing donate.wikimedia.org. parent wikimedia.org. Incomplete test for donate.wikimedia.org. (1) Done testing donate.wikimedia.org. Tue Nov 26 10:46:49 JST 2013

at least the parent is ok

ryuu.psg.com:/Users/randy> doc -p -w wikimedia.org. Doc-2.2.3: doc -p -w wikimedia.org. Doc-2.2.3: Starting test of wikimedia.org. parent is org. Doc-2.2.3: Test date - Tue Nov 26 10:47:45 JST 2013 Summary: No errors or warnings issued for wikimedia.org. Done testing wikimedia.org. Tue Nov 26 10:47:48 JST 2013

aha!

ryuu.psg.com:/Users/randy> ns donate.wikimedia.org. donate-lb.eqiad.wikimedia.org.

i suspect a load balancer

yes, i reported this to wikimedia two days ago. a response, then black hole. problem persists.

randy