> > So it sounds like
> respect is also centralized in the wikimedia foundation, please
> include
> me in your email to your underlings Tim, as I would also like to
> have
> respect, maybe it will mean my request for image dumps is taken
> seriously!? It would be nice if respect was earned, but it
> sounds like
> Tim can create/enforce respect in a more simple way, sounds like
> a good
> leadership path to me.. NOT! :)
> >
> Has it occurred to you that maybe the one person in charge of both
> dumps and media storage is a busy person who also happens to be on
> vacation right now?
Hi,
No, but you bring up a good point, having a single person in charge of both of these departments is not working very well, since the "department of image dumps" hasn't actually released any publicly available image dumps in 4+ years, and the "department of xml dumps" has only released a single successful* full history dump of enwiki in 3+ years.
*only partially successful as some of the revision history is missing from the dump
cheers,
Jamie
>
> Roan Kattouw (Catrope)
>
Hi everyone!
My Google Summer of Code is finished, but the code I have written for
the interwiki transclusion is not yet ready to be merged in trunk.
Some optimizations have to be made and some little issues are still to
be discussed.
The main remaining question is: should I use a separate DB or a shared
table to store the GlobalTemplateLinks table? [1]
Following the advice of my mentor, Catrope, my code is currently using
a separate DB (which name is stored in $wgGlobalDatabase) since it is
the mechanism used by CentralAuth and it is very simple for users to
set it up.
Platonides thinks that I should be using a shared table, since my code
is not an extension. However, a disadvantage of shared table is that
currently, adding a shared table in $wgSharedTables will force the
user to share the users table, which is not always wanted.
I have further developments to accomplish on my code concerning
database tables (such as creating a GlobalNamespaces table to store
the distant namespaces names). I would like to solve this issue first,
so that I don't have to rewrite everything later...
Can you please provide your opinion about this?
Thanks in advance
Best regards
[1] http://www.mediawiki.org/wiki/User:Peter17/Reasonably_efficient_interwiki_t…
--
Peter Potrowl
http://www.mediawiki.org/wiki/User:Peter17
Hi,
>
> > I can say that despite being a nobody at Mozilla and having gotten
> > only one (rather trivial) patch accepted, I feel like I'm
> taken more
> > seriously by most of their paid developers than by most of
> ours.
>
> I'm sorry to hear that, and I'd like to know (off list) which paid
> developers are making you feel that way. I've made it pretty
> clear to
> Danese that I think you're one of our best volunteer developers, and
> that you should be given whatever you ask for, which, I think it's
> understood, includes respect. Maybe I can help to get that
> message to
> filter down to the rest of the organisation.
>
So it sounds like respect is also centralized in the wikimedia foundation, please include me in your email to your underlings Tim, as I would also like to have respect, maybe it will mean my request for image dumps is taken seriously! It would be nice if respect was earned, but it sounds like Tim can create/enforce respect in a more simple way, sounds like a good leadership path to me.. NOT! :)
cheers,
Jamie
Hi,
I think it would be a nice gesture if the wikimedia foundation decentralized some of the internal projects that have had little success over the last few years. Two that come to mind are the enwiki pages-meta-history file creation (1 successful dump in ~3 years), and apparently very little internal concern over this, and also the images/thumbnail wiki-specific dumps that have been neglected for several years. If these two projects are opened up to the community I think that would be great!
cheers,
Jamie
Whoops, I meant to address this mail to wikitech-l, not wikitext-l. ;-)
Regards, Jan Paul
> Hello,
>
> In commit 72458 I've added the InlineEditor extension. [1] This extension is a working implementation of the prototype(s) earlier posted on this list. It's not actually for use on live wikis, but more a proof-of-concept and framework to experiment with. I will explain the extension in detail for those of you who might be interested.
>
> == Design overview ==
> The extension exists of several parts, structured in sub-directories like the UsabilityInitiative extension. The InlineEditor extension itself provides a framework for different edit modes to build on. It displays the edit modes, provides an interface to mark editable pieces of wikitext, provides a client-side inline editor which the edit modes *may* use, is configurable with several fallback options to the full/traditional editor, and handles previewing, publishing, undo and redo.
>
> Every other extension provides an edit mode for the InlineEditor extension. They hook into InlineEditorMark and InlineEditorDefineEditors. The first one is called whenever wikitext is passed through the extension, and all edit modes can mark their editable pieces. Once this is done, a few algorithms will combine this with information of previously edited pieces, generate both wikitext to run through the parser, and JSON which is passed to the client, which maps the editable pieces to the original wikitext. The other hook is to include CSS, JS and messages to the page.
>
> == Limitations ==
> There are many things which are sub-optimal right now:
> * The editor is slow. Whenever changing a small element and previewing it, the entire page is reparsed. This will be fixed by parsing only the element if possible (i.e. references have side effects at the bottom of the page).
> * It's for now only possible to use the editor as primary editor, with a link to the full/traditional editor. There will be a configuration option whether to do this, or display a message at the top of the traditional edit page to switch to this editor.
> * I've not tested things in older browsers (or IE at all, for that matter). I only know it runs fine in Firefox and Chrome, but it may have bugs in other browsers.
> * The edit modes are really, really, basic right now. They may or may not screw things up. Most of them have just one or a few regular expressions which do well in general, but may fail at many edge-cases.
> * The editor may not handle all the messages and edge cases of the traditional editor.
> * The extensions is written for MediaWiki 1.16 but may or may not work with other versions.
>
> Also, I'm not sure at all whether the current set of edit modes is the way to go. Currently, they are mutually exclusive. Meaning that text marked by one editor is never included in text marked by another editor. However, maybe it's better to not have edit modes like this, but different granularity of editing. I.e. sentence => paragraph => block. This way the user will get familiar with more wikitext instead of always seeing small portions. The framework currently doesn't allow for overlap in markings, but I will work to make this possible.
>
> == Goals ==
> Goal of this extension is to provide a framework to easily play with different modes of editing in-line. Feel free to write extensions that use this framework, or help with the framework itself. Any usability or technical suggestions are also welcome!
>
> I hope to get some documentation up on mediawiki.org anytime soon, but note that the code is heavily documented inline. Feel free to ask any questions: I'm probably forgetting to mention some things that may not be clear to everyone. Also, there is no public wiki at the moment to test this extension with, will work on that, but if someone else can enable it on a test wiki that would be great too!
>
> To install the extension(s), check the instructions in /trunk/extensions/InlineEditor/InlineEditor.php. Thanks for your time reading this!
>
> Regards,
> Jan Paul
>
> [1] http://www.mediawiki.org/wiki/Special:Code/MediaWiki/72458
Hello folks,
Today I shot the full history en wiki dumps that claimed they would take
three months to complete. I started a new en wiki run which tests out
two new features: running jobs step by step (in arbitrary order,
assuming any dependencies have been run) and breaking up the xml file
creation into chunks that run in parallel.
Because of this, you may notice some funkiness with the status pages
over the next little while; things like the progress line are going to
be out of whack, and I am sure we will find new and exciting bugs
(though hopefully not in the dump file output).
We have some bizarre behavior around the index pages which all seem to
claim that the given date is its own previous dump. I'll be looking
into it, but in the meantime, the old dumps are available at:
http://dumps.wikimedia.org/enwiki/20100817/http://dumps.wikimedia.org/enwiki/20100730http://dumps.wikimedia.org/enwiki/20100130http://dumps.wikimedia.org/enwiki/20100116http://dumps.wikimedia.org/enwiki/20091103
The parallelizing scheme just does dumps of n pages in sequence (where n
is arbitrarily set at 2 million right now), including all revisions or
not, depending on the dump. This shouldn't screw with anyone's code
that depends on the page IDs to be in order.
This is only being tested out on the en wiki dumps at present; all other
jobs will run just as they used to.
Ah, I wonder if anyone out there would be interested in working on
dbzip2 or seeing if it is still needed; this is a parallelizing bzip2
with some features that pbzip2 doesn't have. See
http://www.mediawiki.org/wiki/Dbzip2
This could potentially save us time in the recombine phase of the bzip2
history dumps, if people want to have those available as one file nad
not just as separate pieces.
We don't have even a start on that for 7zip, so that's another thing for
someone to look into... any takers?
Ariel Glenn
Okay, so here's my take...
First of all I want to thank Aryeh for taking the time to write out the
original mail in the thread. You've obviously done quite a bit of
thinking. I realize there has been discontent and even concern with the
way things have been / are between Foundation tech staff / paid
contractors and "everyone else" on the volunteer dev side of this
Project upon which we all toil. Aryeh, your mail was a well articulated
instance of those feelings.
I'm not going to deconstruct your suggestions; I actually agree with
several of them (although I think you'd find the reality of flagship
open source projects somewhat different than the legend). Instead I'm
going to make a more general statement about why and how we're trying to
shape our future efforts...
As I came into WMF, the UX project was still in full swing and we
decided not to interrupt their momentum until the Stanton Grant was
finished. I focused my efforts instead on planning for the new Fiscal
Year's budget and hiring plans and on observing the work patterns of the
existing Tech staff, the expectations and needs of the Programs staff
and the community interaction dynamic. During most of this time I've
been close to silent on public lists because my opinions were still in
formation. I figured there was nothing worse than somebody rushing in
with a lot of uninformed blathering.
Gradually I decided these were the most serious problems (in order of
importance) to tackle with a new design WMF Tech:
1. Eliminate single points of failure / bottlenecks
2. Reconfigure into teams designed to encourage faster (shorter
duration) and more accurate projects / deployments
3. Develop programs to encourage / grow volunteers into paid
staff...recognizing that as a non-profit WMF needs to plan for more
frequent turnover in the Tech department to ensure that we can grow
expertise across the dev community rather than concentrating it in a few
core people.
4. Restore the balance between community-led and foundation-led efforts
so WMF feels again like a partnership rather than concentric circles of
permission / blame
Once I recognized these objectives, I started working out how to reach
them. Notice that they are in order of importance above...starting with
hiring / training / documenting and otherwise creating redundancies to
mitigate the SPOF problem (in Operations to keep the sites functioning
for instance) is a big focus of the hiring plan for this Fiscal Year.
Fully 1/3 of the Tech hires in this fiscal year are SPOF related.
Item 2 is necessary to again make us very productive (as at least one
respondent on the current thread believes we should be given there are
"more of us than ever before"). Experienced developers know that more
hands aren't necessarily better unless you can modularize work to form
small agile teams within the larger structure to get discreet bits of
work done. The Linux Kernel team does this, and Apache projects are
implicitly organized this way. Most of the expense of this re-org hits
in the next fiscal year (hiring engineering program managers to help
teams stay undistracted and productive and to maintain an overview so
discreet bits of work add up to the correct whole), so this year about
1/3 of the Tech hiring will serve this item.
But the crux of Aryeh's original email is not about the first two items
on my list...it really rests in items 3 and 4.
It is my belief that the Foundation can not and should not plan to fund
or achieve most development projects cathedral-style. To do so would
not be consistent with our founding principles (and in any case wouldn't
be possible given our resources). At the same time our Projects have
grown in both complexity and popularity to the point where we can't we
solely expect volunteerism to provide for all needs.
There has to be a partnership...not a detente. To that end it is clear
we need to improve transparency and afford more opportunities for
participation and cooperation between paid Tech staff and the volunteer
developer community. We need to undertake this shift with realistic
expectations. We are never going to be interested in every scrap of
volunteer engineering ever undertaken, but we are going to take steps to
make it easier for volunteer engineers wishing to contribute to figure
out what contributions are sought and how to gain increased
responsibility in our essentially meritocratic community (which is how
Mozilla finds their new paid staff as well). We're going to create more
entry points and to get volunteers more involved. In this fiscal year
1/3 of the new Tech hires are in service of items 3 and 4...
If you're still reading, then you must really care to know what I
think. Thank you for that. Mostly I wanted to share that re-balancing
the volunteer-to-paid staff dynamic is definitely a first-year priority
for me, although its not more urgent than establishing a new primary
Data Center and making sure we have staff to keep the site running...but
its coming. We're going to co-create (or re-create it). Like all work
of digital "intentional communities"[1], its going to take work on all
sides to get it right...and I hope you'll help.
Danese Cooper
CTO, Wikimedia Foundation
danese(a)wikimedia.org
[1] http://en.wikipedia.org/wiki/Intentional_community
Hi Everyone,
I searched the archives of both list and couldn't find any thread
about it. I could miss it, so sorry it it already has been discussed.
We all know that uploading a file on commons, and on Wikimedia, is
kind of tricky nowdays. However, this could be changed thanks to
HTML5.
HTML5 includes the drag and drop thingy that makes the uploads easier
and that can automatically fetch the EXIF datas. If we want we could
even allow the multiple drag and drop.
Anyway this could be a solution to look at, you can get more
information there :
http://hacks.mozilla.org/2009/12/firefox-36-fileapi-demo-reading-exif-data-…
and there : http://hacks.mozilla.org/2009/12/file-drag-and-drop-in-firefox-3-6/
All the best,
--
Christophe