Try 10 out of 10
Exception:
Cannot execute: ['/usr/lib/spark2/bin/spark-submit', '--master', 'yarn', '--conf', 'spark.dynamicAllocation.maxExecutors=20', '--conf', 'spark.executor.memory=2g', '--conf', 'spark.executor.memoryOverhead=50145', '--conf', 'spark.jars.ivy=/tmp/airflow_ivy2', '--conf', 'spark.jars.ivySettings=/etc/maven/ivysettings.xml', '--conf', 'spark.pyspark.python=mjolnir_venv/bin/python', '--conf', 'spark.sql.shuffle.partitions=1000', '--conf', 'spark.yarn.maxAppAttempts=1', '--archives', 'hdfs://analytics-hadoop/user/ebernhardson/mjolnir_venv.zip#mjolnir_venv', '--jars', 'hdfs://analytics-hadoop/wmf/refinery/current/artifacts/refinery-hive.jar', '--packages', 'org.wikimedia.search:mjolnir:0.5-SNAPSHOT', '--driver-memory', '2g', '--name', 'mjolnir-make_folds-enwiki-dbn-20180215-query_explorer-pruned_mrmr-20200930-spark', '--queue', 'root.default', '--deploy-mode', 'cluster', '/srv/deployment/wikimedia/discovery/analytics/spark/mjolnir-utilities.py', 'make_folds', '--date', '20200930', '--output-path', 'hdfs://analytics-hadoop/wmf/data/discovery/mjolnir/training_files/date=20200930/wikiid=enwiki/labeling_algorithm=dbn/feature_set=20180215-query_explorer-pruned_mrmr', '--feature-set', '20180215-query_explorer-pruned_mrmr', '--feature-vectors-table', 'mjolnir.feature_vectors', '--labeling-algorithm', 'dbn', '--labels-table', 'mjolnir.labeled_query_page', '--num-folds', '5', '--wiki', 'enwiki']. Error code is: 1.
Log: Link
Host: an-airflow1001.eqiad.wmnet
Log file: /var/log/airflow/mjolnir/make_folds-enwiki-dbn-20180215-query_explorer-pruned_mrmr/2020-09-30T00:00:00+00:00.log
Mark success: Link