On Wed, Oct 5, 2011 at 7:18 AM, Platonides &lt;Platonides(a)gmail.com&gt; wrote: I'll do some tests; it may be possible to avoid having to do serial connection setup/teardown by using SSH's ControlMaster[1] setting to piggyback the 'new' connections on one that's already open. [1] < http://www.anchor.com.au/blog/2010/02/ssh-controlmaster-the-good-the-bad-th… (Part of what makes svn:externals slow is that you're jumping over to another site entirely -- that's an extra DNS lookup, possibly separate auth, and who knows how well the other server performs.) -- brion

Daniel Friesen wrote: Sorry? Note that only a small subset of those with commit access do have full ssh to run commands. And a good system shouldn't need those hacks (which is why we are discussing in advance).

Brion Vibber wrote: I know about ControlMaster (which we can only those of us with ssh+git can benefit), but just launching a new process and waiting if there's something new will slow-down. OTOH git skips the "recurse everything locking all subfolders" step, so it may be equivalent. Maybe there's some way for fetching updates from aggregate repositories at once and I am just when everything is solved, though.

On Wed, Oct 5, 2011 at 1:59 PM, Brion Vibber &lt;brion(a)pobox.com&gt; wrote: Ok, did some quick tests fetching updates for 16 repos sitting on my Gitorious account. Ping round-trip from my office desktop to Gitorious's server is 173ms, making the theoretical *absolute best* possible time involving a round-trip for each at 2-3 seconds. Running a simple loop of 'git fetch' over each repo (auth'ing with my ssh key, passphrase already provided) takes 53 seconds (about 3 seconds per repo). This does a separate ssh setup & poke into git for each repo. Clearly unacceptable for 600+ extensions. :) Turning on ControlMaster and starting a long-running git clone in the background, then running the same 'git fetch' loop took the time down to about 10 seconds (<1s per repo). ControlMaster lets those looped 'git fetch's piggyback on the existing SSH connection, but still has to start up git and run several round-trips. Better, but still doesn't scale to hundreds of extensions: several minutes for a null update is too frustrating! Checking them out as submodules via 'git submodule add' and then issuing a single 'git submodule update' command takes... 0.15 seconds. Nice! Looks like it does indeed see that there's no changes, so nothing has to be pulled from the upstream repos. Good! The downside is that maintaining submodules means constantly pushing commits to the containing repo so it knows there are updates. :( Probably the most user-friendly way to handle this is with a wrapper script that can do a single query to fetch the current branch head positions of a bunch of repos, then does fetch/pull only on the ones that have changed. This could still end up pulling from 600+ repos -- if there are actually changes in them all! -- but should make typical cases a *lot* faster. We should check in a little more detail how Android & other big projects using multiple git repos are doing their helper tools to see if we can just use something that already does this or if we have to build it ourselves. :) -- brion

Op 5 okt. 2011 om 15:07 heeft Happy Melon &lt;happy.melon.wiki(a)gmail.com&gt; het volgende geschreven: Actually, we plan to start pushing updates automagically every day, so that will be a reality. Is that ssh session share thingy also available for Windows users? Siebrand

On 6 October 2011 00:12, Siebrand Mazeland &lt;s.mazeland(a)xs4all.nl&gt; wrote: As Windows users generally use putty (although I'm not 100% sure about git-mingw, in this respect, but it works at least with tortoisesvn, and probably also with tortoisegit): pageant is what you're looking for. It's available from the putty website [1]. Best, Merlijn [1] http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

On 06.10.2011, 2:12 Conrad Irwin wrote: Actually, it's pretty common, most if not all active core devs do that because every change to core APIs must be accompanied with a grep through extensions to check if it's OK or something will need to be fixed. -- Best regards, Max Semenik ([[User:MaxSem]])

On 11-10-05 03:07 PM, Happy Melon wrote: I wish we could stop pushing TWN updates as code commits. I'm getting tired of trying to look through an extension's log for changes and having to sort through it where 3/4 of the commits are TWN updates. Someone mentioned an idea at one point of having only English in core (as the source for translations) and then pulling localization updates in with some script, part of building the tarballs would be pulling in the localizations and bundling them. -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Ashar Voultoiz wrote: What about commits which add a message? Most i18n commits are irrelevant for normal coding. But there are code commits which carry along i18n changes. And I don't like solutions like keeping English at a different repo, or having to split a commit in two independent repositories. Maybe subrepositories can help here?

On 9 October 2011 21:52, Daniel Friesen &lt;lists(a)nadir-seen-fire.com&gt; wrote: Huge +1. I would sincerely welcome a move away from PHP-based i18n files. Having data in executable format is just stupid imho. I'm not sure I want to add that burden to TWN right now. It's just single vserver with no uptime guarantees. I'm not opposed to the idea though - having efficient l10n update in the core, enabled by default, providing always up-to-date translations and perhaps also loading new languages on demand[1] would soo awesome. But like I said, that would need some serious effort to code and to make stable and secure content distribution channel. Any volunteers? :) [1] This would also satisfy those who think that including all l10n makes the tarball too big -Niklas -- Niklas Laxström

On 9 October 2011 22:33, Niklas Laxström &lt;niklas.laxstrom(a)gmail.com&gt; wrote: I don't really see how changing the format is going to have any impact by itself. Whether PHP, XML, a hand-rolled data format or anything else, it still doesn't play nicely with version control. Fundamentally we want to make changes to content in a version-controlled project, and we want everyone to have the latest versions of those changes; but we don't want the version history *of the i18n content* mixed in with the version history of the rest of the project. The solution to that issue is obvious and has nothing to do with the file format: if you don't want your changes showing up in the version history of your repository, make your changes outside the repository! This would seem something of a slap in the face to anyone running an 'unplugged' wiki (on an intranet or low-connectivity area); *especially* one using a language other than English. I doubt this would play very happily with the Foundation's vision of bringing offline content to, for instance, rural Africa. Not all MediaWiki installations run on a webserver permanently hooked into the internet; some of them run from a USB stick plugged into an OLOC laptop in the middle of the middle of nowhere. --HM

On 10.10.2011, 2:04 Daniel wrote: If you can't trust your on-the-fly .php downloads, you can't trust localisations in other formats either: we have some messages with full HTML allowed, and default contents for JS messages can be used to screw up the wiki and steal everyone's passwords. Of course, it's less lethal than server-side php execution, but still an unacceptably dangerous attack vector. -- Best regards, Max Semenik ([[User:MaxSem]])

Daniel Friesen wrote: Man, that kills all the fun :) Also, that means that we have an excuse for holing those pesty firewalls that don't allow us to "enable" their site [1]. 1- http://bit.ly/qiYUyL

On Fri, Oct 7, 2011 at 3:48 PM, Daniel Friesen &lt;lists(a)nadir-seen-fire.com&gt; wrote: I also use two working copies (one read-only for development and one for commiting). This is mostly due to lack of svn stash and local branches (I have at least two large patches on my local copy I'm forced to manually unmerge by editing svn diffs). --vvv