[Wikitech-l] GSOC-2012 proposal(Convention Extension)

Send Wikitech-l mailing list submissions to wikitech-l(a)lists.wikimedia.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.wikimedia.org/mailman/listinfo/wikitech-l or, via email, send a message with subject or body 'help' to wikitech-l-request(a)lists.wikimedia.org You can reach the person managing the list at wikitech-l-owner(a)lists.wikimedia.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Wikitech-l digest..." Today's Topics: 1. Re: Time to redirect to https by default? (Ryan Lane) 2. Re: Time to redirect to https by default? (Ryan Lane) 3. Re: Time to redirect to https by default? (Ryan Lane) 4. Re: Time to redirect to https by default? (MZMcBride) 5. Re: Committing followups: please no --amend (Chad) 6. Re: Time to redirect to https by default? (Platonides) 7. Re: Time to redirect to https by default? (Ryan Lane) 8. rsync on scap/sync reporting 'no space left on device' for a lot of hosts (Arthur Richards) 9. Re: Time to redirect to https by default? (Antoine Musso) ---------------------------------------------------------------------- Message: 1 Date: Tue, 3 Apr 2012 03:34:13 +0900 From: Ryan Lane <rlane32(a)gmail.com> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Subject: Re: [Wikitech-l] Time to redirect to https by default? Message-ID: <CALKgCA0RLExmwpdJ18ATtPJ_b=h7JBc7AJ=Pj+2zgUYvnRPJ4w(a)mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 On Mon, Apr 2, 2012 at 12:33 PM, Tim Starling <tstarling(a)wikimedia.org> wrote: > On 02/04/12 06:14, Ryan Lane wrote: >> TL;DR: we have no plans for anonymous HTTPS by default, but will >> eventually default to HTTPS for logged-in users. >> >> 1. It would require an ssl terminator on every frontend cache. The ssl >> terminators eat memory, which is also what the frontend caches do. > Once we enable it by default for logged-in users, we will care a lot > more if someone tries to take it down with a DoS attack. Unless the > redirection can be disabled without actually logging in, a DoS attack > on the HTTPS frontend would prevent any authenticated activity. > It suggests a need for a robust, overprovisioned service, with tools > and procedures in place for identifying and blocking or throttling > malicious traffic. Indeed. We're already pretty over provisioned. We have 4 servers per datacenter, each of which is very bored. All they are doing is acting as a transparent proxy, after ssl termination. We're using RC4 by default (due to BEAST), and AES is also available (the processors we are using have AES support). Ideally we'll be using STS for logged in users. This will mean it's impossible to turn off the redirection for users that have already logged in for whatever period of time we have STS headers set. We need to consider blocking a DoS from the SSL proxies, the LVS servers, or the routers. >> 3. Some countries may completely block HTTPS, but allow HTTP to our >> sites so that they can track users. Is it better for us to provide >> them content, or protect their privacy? >> 4. It's still possible for governments to see that people are going to >> wikimedia sites when using HTTPS, so it's still possible to oppress >> people for trying to visit sites that are disallowed. > It's also possible for governments to snoop on HTTPS communications, > by using a private key from a trusted CA to perform a > man-in-the-middle attack. Apparently the government of Iran has done this. We really should publish our certificate fingerprints. An attack like this can be detected. An end-user being attacked can see if the certificate they are being handed is different from the one we advertise. We could also provide a convergence notary service (or one of the other things like convergence). > If we really want to protect the privacy of our users then we should > shut down the regular website and serve our content only via a Tor > hidden service ;) I agree that it's impossible to provide total protection of a user's privacy. We could provide a number of services that would help users, though. That said, I don't feel this should be on the top of our priority list. - Ryan ------------------------------ Message: 2 Date: Tue, 3 Apr 2012 03:58:41 +0900 From: Ryan Lane <rlane32(a)gmail.com> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Subject: Re: [Wikitech-l] Time to redirect to https by default? Message-ID: <CALKgCA1ZXK8TBedG4cgdLeg-LsW7aN=ujf9O2V0QKL33DUYDVA(a)mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 On Mon, Apr 2, 2012 at 4:20 PM, Petr Bena <benapetr(a)gmail.com> wrote: > That's not what I wanted to say, I wanted to say "https may cause > troubles with caching", In fact some caching servers have problems > with https since the header is encrypted as well, so they usually just > forward the encrypted traffic to server. I don't say it's impossible > to cache this, but it's very complicated Using SSL by default means all transparent proxies inbetween aren't hit at all, since they'd be a MITM. I don't necessarily see this as a bad thing, as transparent proxies often break things. Browsers cache things differently from HTTPS sites, but otherwise everything should work as normal. The SSL termination proxies transparently proxy to our frontend caches after termination. Links are sent as protocol-relative so that we don't split our cache, as well. - Ryan ------------------------------ Message: 3 Date: Tue, 3 Apr 2012 04:00:32 +0900 From: Ryan Lane <rlane32(a)gmail.com> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Subject: Re: [Wikitech-l] Time to redirect to https by default? Message-ID: <CALKgCA18yzdDpiqS1RAViaz8O7nw68K2Bz0qZaCp1i0g1TxTNg(a)mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 On Mon, Apr 2, 2012 at 6:34 PM, Tei <oscar.vives(a)gmail.com> wrote: > Perhaps have a black list of countries that are know to break the > privacy of communications, then make https default for logued users in > these countries. > This may help because: > ?- It only affect a subgroup of users (the ones from these countries) > ?- It only affect a subgroup of that subgroup, ?the logued users (not all) > ?- It create a blacklist of "bad countries" where citizens are under > surveillance by the governement > This perhaps is not feasible, if theres not easy way to detect the > country based on the ip. I'd definitely not support doing something like this. This would incredibly complicate things. - Ryan ------------------------------ Message: 4 Date: Mon, 02 Apr 2012 16:26:57 -0400 From: MZMcBride <z(a)mzmcbride.com> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Subject: Re: [Wikitech-l] Time to redirect to https by default? Message-ID: <CB9F83D1.18E9B%z(a)mzmcbride.com> Content-Type: text/plain; charset="ISO-8859-1" Ryan Lane wrote: all) >> ?- It create a blacklist of "bad countries" where citizens are under >> surveillance by the governement >> >> This perhaps is not feasible, if theres not easy way to detect the >> country based on the ip. > I'd definitely not support doing something like this. This would > incredibly complicate things. Someone came into #wikimedia-tech a few days ago and asked about something similar to this. The idea was to use site-wide JavaScript to auto-redirect users to https on one of the Chinese Wikipedias. I believe this was in combination with geolocation functionality, but I'm not sure. Do you have any thoughts on individual wikis doing this, assuming there's local community consensus? MZMcBride ------------------------------ Message: 5 Date: Mon, 2 Apr 2012 17:05:03 -0400 From: Chad <innocentkiller(a)gmail.com> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Subject: Re: [Wikitech-l] Committing followups: please no --amend Message-ID: <CADn73rPYs_H5tR0fQ0bvwAkjrJTnhA3Tk52FmLj1RbsekMp38A(a)mail.gmail.com Content-Type: text/plain; charset=UTF-8 On Tue, Mar 27, 2012 at 6:06 AM, Tim Starling <tstarling(a)wikimedia.org> wrote: wrote: >>> For commits with lots of files, Gerrit's diff interface is too broken >>> to be useful. It does not provide a compact overview of the change >>> which is essential for effective review. >>> >>> Luckily, there are alternatives, specifically local git clients and >>> gitweb. However, these don't work when git's change model is broken by >>> the use of git commit --amend. >>> >> They do; it just wasn't obvious to you how to do it, but that doesn't >> mean it can't be done. >> >> $ git fetch https://gerrit.wikimedia.org/r/p/analytics/udp-filters >> refs/changes/22/3222/3 && git branch foo FETCH_HEAD >> $ git fetch https://gerrit.wikimedia.org/r/p/analytics/udp-filters >> refs/changes/22/3222/4 && git branch bar FETCH_HEAD >> $ git diff foo..bar >> >> The two 'git fetch' commands (or at least the part before the &&) can >> be taken from the change page in Gerrit. > It doesn't work, I'm afraid. Because of the implicit rebase on push, > usually subsequent changesets have a different parent. So when you > diff between the two branches, you get all of the intervening commits > which were merged to the master. > Examples from today: > https://gerrit.wikimedia.org/r/#change,3367 > Patchsets 1 and 2 have different parents. > https://gerrit.wikimedia.org/r/#change,3363 > Patchsets 1, 2 and 3 have different parents. > It's possible to get a diff between them, and I did, but it's tedious. > I figure we should pick a workflow that doesn't waste the reviewer's > time quite so much. The problem here is the implicit rebase. As long as the review backlog isn't long and/or people aren't submitting conflicting changes, rebasing amended changes against master creates more harm than good. For amending commits, you should use `git review -R` so you don't rebase the change (again) against master (see for example [0], difference between patch 2 and 3). I've updated the docs[1], but they are, briefly: git review -d 123 (make changes) git commit -a --amend git review -R If you're not using git-review and have been using the alias, your amended patchsets have not been creating this problem. -Chad [0] https://gerrit.wikimedia.org/r/#change,4020 [1] https://www.mediawiki.org/wiki/Git/Workflow#Amend_your_change ------------------------------ Message: 6 Date: Mon, 02 Apr 2012 23:31:28 +0200 From: Platonides <Platonides(a)gmail.com> To: wikitech-l(a)lists.wikimedia.org Subject: Re: [Wikitech-l] Time to redirect to https by default? Message-ID: <jld5hm$suh$1(a)dough.gmane.org> Content-Type: text/plain; charset=ISO-8859-1 On 02/04/12 20:34, Ryan Lane wrote: this. >> > We really should publish our certificate fingerprints. An attack like > this can be detected. An end-user being attacked can see if the > certificate they are being handed is different from the one we > advertise. We could also provide a convergence notary service (or one > of the other things like convergence). Indeed. Detecting a potential MITM is useless if you can't determine if it's real or not. For instance the switch from RapidSSL to DigiCert certificate was quite suspicious. I don't know how to best publicise it, though. I suppose we would list them somewhere like https://secure.wikimedia.org/servers.html but if nobody knows it's there... ------------------------------ Message: 7 Date: Tue, 3 Apr 2012 06:35:50 +0900 From: Ryan Lane <rlane32(a)gmail.com> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Subject: Re: [Wikitech-l] Time to redirect to https by default? Message-ID: <CALKgCA0xW+wJu5LxzxQrJTaUfAf1ELPFjfucVeHaBF8CbaJjAw(a)mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 > Indeed. Detecting a potential MITM is useless if you can't determine if > it's real or not. For instance the switch from RapidSSL to DigiCert > certificate was quite suspicious. > I don't know how to best publicise it, though. I suppose we would list > them somewhere like https://secure.wikimedia.org/servers.html but if > nobody knows it's there... What's https://secure.wikimedia.org? - Ryan ------------------------------ Message: 8 Date: Mon, 2 Apr 2012 16:19:20 -0700 From: Arthur Richards <arichards(a)wikimedia.org> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Subject: [Wikitech-l] rsync on scap/sync reporting 'no space left on device' for a lot of hosts Message-ID: <CAG5Yvh+pr-=KLLzj2sVucWX5a8PJG1mb=sd+0wc+FK+0KZpybw(a)mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 I just ran scap and saw the following for a lot of hosts: srv285: rsync: write failed on "/usr/local/apache/common-local/php-1.19/cache/l10n/l10n_cache-ab.cdb": No space left on device (28) srv285: rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.7] srv285: rsync: connection unexpectedly closed (2051 bytes received so far) [generator] srv285: rsync error: error in rsync protocol data stream (code 12) at io.c(601) [generator=3.0.7] Also, on configchange: mw21: rsync: write failed on "/apache/common-local/wmf-config/CommonSettings.php": No space left on device (28) mw21: rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.7] mw21: rsync: connection unexpectedly closed (37 bytes received so far) [generator] mw21: rsync error: error in rsync protocol data stream (code 12) at io.c(601) [generator=3.0.7] Not sure if this is a problem and/or if others are aware/working on it, but thought I'd mention it. -- Arthur Richards Software Engineer, Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687 ------------------------------ Message: 9 Date: Tue, 03 Apr 2012 01:22:10 +0200 From: Antoine Musso <hashar+wmf(a)free.fr> To: wikitech-l(a)lists.wikimedia.org Subject: Re: [Wikitech-l] Time to redirect to https by default? Message-ID: <jldcat$bto$1(a)dough.gmane.org> Content-Type: text/plain; charset=ISO-8859-1 On April 2nd, 2012 at 23:35, Ryan Lane wrote: Some old experiment. Nothing to see here :-) -- Antoine "hashar" Musso ------------------------------ _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l End of Wikitech-l Digest, Vol 105, Issue 10 *******************************************