Hi,

The hadoop cluster is still having issues this morning (ingestion from kafka fails on one partition).

Please refrain from using it, it needs all possible resources to catch back :)

Thanks

Joseph

On Thu, Jan 28, 2016 at 2:55 PM, Joseph Allemandou <jallemandou@wikimedia.org> wrote:

Hi again,

The data ingestion problem is fixed, and the cluster is starting the race against new data coming in.

For the moment we are about 20 hours behind on the two big partitions (text and upload), so catching up will take some time.

We'll keep you updated as things go along.

Cheers

Joseph

On Thu, Jan 28, 2016 at 11:57 AM, Joseph Allemandou <jallemandou@wikimedia.org> wrote:

Hi Analytics fellows,

We are experiencing issues with loading data into the hadoop cluster, therefore blocking the full job pipeline.

When fixed, the cluster will be heavily loaded trying to catch up, so please, be nice with it and don't run heavy jobs in the next hours.

We'll keep you posted about resolution.

Many thanks, and sorry for the inconvenience.

Joseph

--

Joseph Allemandou

Data Engineer @ Wikimedia Foundation

IRC: joal

--

Joseph Allemandou

Data Engineer @ Wikimedia Foundation

IRC: joal