I'm still wondering how the images are stored currently- on the hd of a
single Server? They don't seem to be in the database.
Once we use a cluster of Apaches the files need to be available to all
servers. The easiest solution might be to mount the media dir from a
fileserver (with a big Raid array) via nfs.
I think the sessions are stored in the database, so this is no problem.
Gabriel Wicke
Hello,
many work is put into images, and a lot of places use thumbnails in the article
and a [[Media:...|larger version]] link to a big version. This requires two
uploads and some complicated code, especialy if the image and its caption
should be right aligned floats.
I'd like to propose to make the software handle this frequent task. The image
syntax would be extended to provide an options field. This can be done
backwards compatible and similar to the options for table cells:
[[Image:Leopard.jpg|options|descriptive caption]]
Options are:
* thumbnail to generate a right aligned thumbnail of the image, using
the caption text as image caption, adding an enlarge icon
to the image and a [more] link to the big image
* thumbnail-left same as above, but left aligned.
* left, right makes the image a left/right aligned float, without a
caption, without resizing it. (e.g. for use on pages like
http://en2.wikipedia.org/wiki/Rudolf_Nebel )
* ___px renders the image inline, just like the classicale [[Image:,
but resizes it to ___ pixels width. Height is calculated
automatically.
[[Image:Leopard.jpg|250px|Leopard liegt auf einem Ast]]
could be used to generate the thumbnail at
http://de.wikipedia.org/wiki/Leopard
The thumbnails are stored on disk and are only generated if no thumbnail for
that size exists already.
Examples can be found at
http://jeluf.mormo.org/testwiki/wiki.phtml?title=Extended_image_syntax
The stylesheets are proof-of-concept, help on layout questions is welcome.
The code implementing the above is attached as a patch.
Best regards,
JeLuF
I like the idea of several small machines better than one big machine.
At least, if the mailing list server is up when the DB or Web server is
down, the developers can COMMUNICATE with each other to figure out how
to fix it.
Wikipedia doesn't seem any faster this month than before Christmas. My
main concern is response time. When I click on a link, I don't want to
wait more than 5 second. Especially when I'm following multiple links: A
links to B, then I want to glance at [[talk:B]] before adding a comment.
At 5 or 10 seconds per user click, three clicks means 15 to 30 seconds
before I can stop typing. It makes me wish I could do all my editing
off-line.
Ed Poor
To whatever extent possible, I'd like to buy a large number of general
purpose boxen that can be reasonably pressed into service for a
variety of configurations, rather than trying to overspecify what we
are going to do and buy machines that are highly specific to those
purposes.
My thinking is that we will be needing N general boxen to do load
balancing/squid/apache and then also email and dns on separate
machines too I think, and 2 specialized boxes to be database servers.
We have one very good machine (when it's fixed!) to be our database
server. We will want another one that's not quite as expensive but
which would be reasonably capable of serving as a backup unit. If the
general profile that we are buying is capable of doing that, then
great. If not, then at least one machine is going to have to be
special.
Does that sound sensible?
--Jimbo
Hi,
considering your responses, I've improved the syntax and the code.
Changes:
* Orthogonal syntax. Reduced keywords: thumb(nail), left, right, ___px.
any combination of these is possible (well, left|right is possible,
but not useful). [[Image:Leopard.jpg|left|200px|thumb|African Leopard]]
will create a 200 pixel wide left aligned thumbnail image with the
caption African Leopard.
* Thumbnail default alignment depends on language's writing direction
(left-to-right ./. right-to-left). Can be overwritten as above.
* Configured it not to stamp the enlarge icon into the image. The image
is shown below the image right-aligned. (can be changed to left using
CSS)
* Nicer icon (magnifying glass, contributed by bumms13, thanks!)
* Fixed image page linking back to image page
* removed mostly all changes to imagepage. Only change kept:
Include the image at the image page. Should this be restricted
to images smaller than 150 KByte?
Examples and TODO-List are at
http://jeluf.mormo.org/testwiki/wiki.phtml?title=Extended_image_syntax
Best regards,
JeLuF
Would it be possible to set it so that uploads are never placed in a
path called /ad/ (as they sometimes are at the moment) - I think it's
pretty common for software to assume that any such image is an advert
and not display it. Certainly Norton Personal Firewall's ad blocking
does this...
--
Allan Crossman - http://dogma.pwp.blueyonder.co.uk
PGP keys - 0x06C4BCCA (new) || 0xCEC9FAE1 (compatible)
I think we should do this ASAP, because the issues are only going
to get harder. But the main issue, Timwi, is social -- there are
certainly cases where two different people have the same username
but in different languages.
Timwi wrote:
>
> Thomas R. Koll wrote:
> >On Mon, Dec 29, 2003 at 11:04:00PM +0100, Erik Moeller wrote:
> >
> >>Stuff like single sign on
> >
> >That is not possible anymore. It would be total chaos.
>
> I'm pretty confused at this assessment of yours.
>
> All it would take to achieve this would be to:
>
> - Add a field to the 'cur' and 'old' tables (or perhaps even just the
> 'cur' table) to specify what language an article is in.
> - Move all the articles from all languages into one database.
> - Adapt code accordingly.
>
> Should be doable (given adequate manpower).
>
> Timwi
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l(a)Wikipedia.org
> http://wikipedia.mormo.org/mailman/listinfo/wikitech-l
>
I throught I would throw some ideas of configurations for new wikipedia
systems:
Assuming we have a fast and reliable infrastructure for wikipedia to
operate on, I would hope and expect many more people to benefit from the
gret project. The new hardware configuration needs to have redundancy
built in and reliability.
The DNS system is, by nature, distributed and designed to have many
systems adding redundancy to the resolution service.
A single ip address will always be a single point of failure as the
routers leading up to the physical ip destination will take some time to
propogate a new physical destination for the ip address.
It therefore makes sense to have redundancy switching at the DNS level
and having systems located at different network locations offering
equivalent service.
This brings issues of updates and authentication into the question. The
slave machine cannot authenticate or take database updates. A mechanism
is needed to automatically nominate a machine as master or slave. The
DNS system and each machine would respond to this.
Each wiki server, when booted, will, by default, be a slave. Each wiki
server will periodically request master status from the arbitration
server. The arbitration server can have an algorithm thus:
a)Each wiki server requests master status every 5 minutes.
b)The arbitration/master DNS server checks whether the wiki server
making the request was the last server to have the request granted. If
it is, the request is granted again.
c)If the server making the request is other than the wiki server who had
the request granted last, then if the last grant was >10 minutes ago,
the request is granted. Otherwise, the request is denied.
d) If the master wiki server makes a request and does not receive a
grant, it demotes itself to slave. (Including if it receives no answer
at all)
A specific host name is used for all updates. The 'master hostname'.
After each grant, the ip address of the master hostname is compared with
the ip address of the machine which just received a grant. If it
differs, the ip address of the master hostname is changed. This change
is pushed to the slave DNS servers using ordinary BIND8 protocol. The
DNS TTL for the domain name of the master wiki server would be set to
300 seconds. If a master went down for any reason, everyone should see
the alternative master within 20 minutes.
If the arbitration server failed, all wikis would remain available for
read access.
Redundancy switching:
Exactly the same system can take care of system redundancy. The master
server takes each request for master as an indication that the wiki
server is ready to handle queries.
To complement a master hostname is a 'slave hostname'. The slave
hostname resolves to the ip address of any machine which is ready to
handle a query. The DNS server may have multiple IP address entries for
the slave hostname. The query load will be spread evenly between all
machines whose ip address is registered against the slave hostname using
round-robin.
If a machine fails to request master status for 10 minutes, it's ip
address entry is de-registered from the slave hostname, all DNS servers
are updated using normal BIND protocol. If a machine goes off-line,
after 15 minutes, no more requests should be sent to it until it is
working again.
This system provides a totally automatic redundancy switching, master
arbitration and master/slave database selection.
--
Server requirements for fine grained databases:
The database containining the text of each article is fine grained. Each
article text being a few K in size. The cost of putting this in memory
would be small. The benefits of avoiding hard disk head movements large.
Hard drive head movements are expensive; both in terms of I/O time (and
therefore server performace) and mechanical wear and tear/ reliability.
As the number of articles increases, the number of seeks across a disk
surface increases. By copying the fine-grained database to ramdisk when
the machine boots, and using the ramdisk image for all queries, much
load is removed from the hard drive. System performance should improve
(at least when first powered on) by an order of magnitude. The ramdisk
database must be replicated to another database located on the physical
hard disk 1) So that when the machine boots, a database image is
available to load into ramdisk and 2) In order to keep a more solid copy
than the ephemeral ramdisk image.
Dual 64 bit Opteron mainboards are available with 8 DIMM sockets capable
of taing 16Gb. eg:
http://www.tyan.com/products/html/thunderk8w_spec.html
If the system will be using >4Gb memory, moving to 64bit is a good idea.
4Gb is an addressable limit for 32 bit.
Mainboard $432:
http://shopper.cnet.com/Tyan_Thunder_K8W_S2885ANRF___mainboard___extended_A…
Memory throughput is probably more important than CPU Mhz.
2xAMD Opteron 240 @$220 each =$440
8x1Gb Registered ECC $300 each
http://www.crucial.com/store/listModule.asp?module=DDR+PC2700&cat=RAM&packa…
Mainboard and CPU solution:
8Gb Dual 64 bit Opteron hypertransport with 8Gb ram:
US$3272
Eric card:
http://us.daxten.com/overview.cfm?prodID=37
$715
plus ancilliaries (good PSU, two HDDs CD rom with Knoppix left in for
remote system repair via Eric. Total around $4500.
Two of these to provide redundancy comes to around $9000.