Like most perennial ideas, there is certainly value in this one. There are
two big wikis that do this that you might take a look at to hone your ideas
(you might even suggest changes to them directly):
(english, 10M sites)
On Tue, Oct 27, 2009 at 3:04 PM, Shlomi Fish <shlomif(a)iglu.org.il> wrote:
This is a request-for-comments (RFC) about an idea that had surfaced on
#wikipedia at the time about creating an open web directory similar to
only world-editable and with a more convenient
This was motivated after I was referred to the "Wikipedia is not a web
directory" section of:
History of Web Directories:
I'm not sure how many of the younger folks here are very familiar with the
history and motivation behind web directories, so I'll explain a little to
best of my knowledge.
Back when the Internet and the World Wide Web started to become popular,
search engines were much less accurate than Google, or the search engines
now compete with it, using similar algorithms. As a result, it was often
to find stuff on the Internet using Lycos or different search engines. As a
result, people have actively used web-directories and especially yahoo.com
(which started as a hand-maintained directory by two Stanford students, and
grew into a successful Internet company), as a way to find resources that
considered high-quality by human editors.
Yahoo and similar directories organised the content in a tree of
with some stuff like "symbolic links", etc. Part of the problem with Yahoo
that it was closed for edits only by human editors, which caused it to
grow out-of-date. As a result, it was eventually surpassed in
comprehensiveness and accuracy by dmoz.org
gained some notoriety after Google periodically mirrored it as
Google directory (with some enhancements like sort-by-page-rank and a
load time, and a better search). Eventually, Google removed it from their
front page and search results in favour of Froogle and other stuff which
in my (possibly non-representative) opinion much less useful than their
Directory, and dmoz.org
went into much greater obscurity. Soon afterwards,
English wikipedia and other wikimedia projects started gaining a lot of
momentum, popularity and page rank, which caused it to rank high among many
search engine searches (although to the defence of Google and other search
engines, one should note that they do seem to have a diversification
algorithm, which makes the search results not be dominated by a single
- whether wikipedia.org
Why a Web Directory:
While I enjoy the English wikipedia a lot (and have contributed to it -
), I still think that web
directories have been having (or possibly and unfortunately "had been
their advantages and appeal. The primary reason is because they list any
of interest, including many that would be considered as not "notable"
for inclusion under the relevant "External Links" in the Wikipedia, but
may prove of interest. They also serve a similar purpose to the wikipedias'
category pseudo-trees of allowing to find similar articles of interest.
A lot of techno-geeks are now saying "Category trees are dead! Tags are the
future". It is true that traditionally the filesystems of popular operating
systems such as UNIX (e.g: Linux, Mac OS X, etc.), DOS/Windows, etc. are
organised in a directory tree and not a tag, which inspired a lot of
stuff to be similar (as the protocols mirrored the semantics of the UNIX
system). However, there are many good reasons (besides
why they are organised in a hierarchy, instead of in free-form tags. (You
see the Google Reader feeds-organised-in-tags or the Flock browser huge
based bookmarks menu for why they sometimes fail). Not to mention that like
wikipedia, a certain resource can be tagged with more than one category
Isaac Newton ( http://en.wikipedia.org/wiki/Isaac_Newton
) belongs to
century English people" , "Fellows of the Royal Society" , "English
So I still think the idea of a web directory appeals to me.
The problems with ODP/dmoz.org
As someone who used to be a dmoz.org
editor, I found two main problems
1. Too much red-tape: an editor could only edit the categories he was given
permissions for, and not anything above. There were some meta-editors who
edit anything and can also give permissions for more categories which take
time, but I still have been thinking that the best thing would be a
like "everyone can edit everything unless explicitly forbidden" thing.
Another thing I didn't like about this red-tape and authority was an
where as I edited the Perl "FAQs, Tutorials and Helps" category and added a
sub-category of "Tutorials" where I placed some stuff. Then when an editor
reviewed my work when I asked for another category, they didn't like the
that one of the texts for the mission statement only reflected my thoughts,
and so deleted the category and moved everything I wrote their to the
category. This naturally was a destructive change that made me frustrated,
I would have been happy to change the mission statement or guidelines of
category after the fact.
2. The UI was lacking: there were many forms required to review, submit
edit a single link, the editing server was kinda slow, there was very
AJAX, and editing in general was much less convenient than the wikipedia
link which gives a gigantic textarea with a convenient and concise syntax.
For a long time I felt guilty about not dedicating enough time to edit
, and had reminders to edit it occasionally (which I tended to
but eventually passively stopped editing. I now realise I could not be
for my lack of enthusiasm.
Note that I still feel that dmoz.org
is a useful resource which is often
and useful. As great as the Wikipedia is, I still think there's a place for
high-profile web-directory. Maybe this is one of the trends that will
retro, like push technology which was considered a fad was re-incarnated as
RSS/Atom feeds which seem to have gained a lot of popularity, and even
to have some business potential.
The Challenges of a more open / more free web directory:
I'm not sure that a wikimedia-sponsored web directory is a good idea yet.
here are some thoughts about the challenges:
1. The three S's: Spam, spam, spam. A web directory is likely to be a huge
spam target and will need good anti-spam controls. However, I personally
that while spam should be a factor we take into consideration, it should
prevent us from creating new and exciting user-contributed web sites.
One of the reasons I hate spam is not so much that I am bothered by it
arriving in my inbox, but rather because it makes some people paranoid. My
personal web-site contains an <a href="mailto:firstname.lastname@example.org"
rel="webmaster">shlomif(a)iglu.org.il</a> E-mail at the bottom of each
lately most sites I visited either had it obscured under many ways, or even
just had a contact form. Some people have even told me that I should hide
web address to reduce the amount of spam I receive because "prevention is
better than the cure".
I'm sorry, but I'd rather not destroy paradise just so I can save it. I'd
rather see some spam on blogs and in E-mail than destroy their
usability/accessibility, and by corrollary think that a more open web-
directory should not have fear of spam as the main obstacle in its way.
2. We may wish to build upon the existing data of the ODP which is
as machine-readable data under this licence:
The Free Software Foundation describes the ODL as a non-free license,
the right to redistribute a given version not being permanent, and the
requirement to check for changes to the license.
Whether something is indeed free/open or not is a term of much debate as I
mention here in a somewhat different context:
Whether the interpretation of the FSF to the freeness of the licence is
correct here, and whether it matters much in this case (as RMS himself was
quoted as saying that commercial games can have "non-free" art and plots as
long as their engines are free and it was OK ethically and morally). Still
may prove to be a problem if we want to gain some public acceptance for the
3. Shouldn't we try to convince dmoz.org
to remedy the two problems I've
mentioned, rather than starting our own competing and diverging effort?
Like I said earlier, I'm still very sceptical about whether this idea will
work and be a good one. At the moment, I'm unemployed by choice, but still
have many other endeavours and different priorities and so cannot commit to
dedicating a lot of time to this wiki-directory. I'm already active in the
English wikipedia, the English wiktionary, used to edit the English
and would like to work again, and naturally have my own web-sites and blogs
(not really wikis, though I have comments there), which often take greater
precedence and interest. So my expectation is that if such an effort is
started, it will need to grow organically in a similar way that wikinews or
wikibooks or some of the popular topical Wikia wikis have gained public
Shlomi Fish http://www.shlomifish.org/
"Star Trek: We, the Living Dead" - http://shlom.in/st-wtld
Chuck Norris read the entire English Wikipedia in 24 hours. Twice.
foundation-l mailing list