Thanks for the pointer Michael -- Timed Media Handler seems like a good
example of job queuing use.
The way TMH does job queuing seems like a feasible option for the
post-upload processing I'm doing. Rather than enqueuing them upon the
file's Upload Complete event firing, TMH seems to put video transcoding
jobs into the queue the first time they're requested -- i.e. the first time
a page containing the video is loaded. My initial impression was that it'd
be faster from the user's perspective if the job were enqueued as soon as
possible, which I assume would be onUploadComplete.
Maybe there's a negligible difference here, or maybe I don't understand
something -- any thoughts?
On a separate note, I've found a way to speed up the post-upload processing
needed for my extension. The ray-tracing can be divided among multiple CPU
cores. (I've tried both ray-tracing libraries supported by the molecular
visualization package I'm using, and they only support multicored, not
multithreaded, distribution of the task.) The amount of time needed to do
the post-upload processing seems to decrease proportional to the number of
cores used.
Given that, would it be possible to use multiple cores for this post-upload
processing? If so, how many cores could be used for a given one of these
ray-tracing tasks? If that distribution got the time needed to complete
this processing down to something that was deemed reasonable for the user,
would that make it unnecessary to enqueue jobs for this option?
Thanks,
Eric
On Fri, May 4, 2012 at 7:19 PM, Michael Dale <mdale(a)wikimedia.org> wrote:
You will want to put into a jobQueue you can take a
look at the Timed
Media Handler extension for how post upload processor intensive
transformations can be handled.
--michael
On 05/04/2012 04:58 AM, emw wrote:
Hi all,
For a MediaWiki extension I'm working on (see
http://lists.wikimedia.org/**pipermail/wikitech-l/2012-**
April/060254.html<http://lists.wikimedia.org/pipermail/wikitech-l/2012-A…)l>),
an
effectively plain-text file will need to be converted into a static image.
I've got a set of scripts that does that, but it takes my medium-grade
consumer laptop about 30 seconds to convert the plain-text file into a
ray-traced static image. Since ray-tracing the images being created here
substantially improves their visual quality, my impression is that it's
worth a moderately expensive transformation operation like this, but only
if the operation is done once.
Given that, I assume it'd be best to do this transformation immediately
after the plain-text file has completed uploading. Is that right? If
not,
what's a better time/way to do that processing?
I've looked into MediaWiki's 'UploadComplete' event hook to accomplish
this. That handler gives a way to access information about the upload and
the local file. However, I haven't been able to find a way to get the
uploaded file's path on the local file system, which I would need to do
the
transformation. Looking around related files I see references to
$srcPath,
which seems like what I would need. Am I just missing some getter method
for file system path data in UploadBase.php or LocalFile.php? How can I
get the information about an uploaded file's location on the file system
while in an onUploadComplete-like object method in my extension?
Thanks,
Eric
______________________________**_________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/**mailman/listinfo/wikitech-l<https://lists.…
______________________________**_________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/**mailman/listinfo/wikitech-l<https://lists.…