[Labs-l] Faster hard disks

[Ryan Lane]

On Thu, May 22, 2014 at 4:03 PM, Emilio J. Rodríguez-Posada <
emijrp at gmail.com> wrote:

> Hello;
>
> I'm processing Wikipedia dumps. For now, I'm copying some dumps into the
> tool path (/data/project/tool/dumps) to preserve them for my study, because
> only the last 2 dumps are in /public/dumps. And when I launch the jsub, the
> script read them from there.
>
> But I have a question, is /public/dumps faster than /data/project ? I mean
> in r.pm. or any technical feature. Or all are the same?
>
>
As far as I know they both come from the same system right now.

/mnt is the fastest you're going to get, but I'm almost positive you can't
use that in the tools project


> By the way, when processing dumps, I have found that reading from a 7z
> dump is faster than from a bz2, so I think that the hard disks are playing
> here a important role, more than CPU.
>
>
Yep, that's very likely.

- Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20140522/6fa98761/attachment.html>