Try 10 out of 10
Exception:
Cannot execute: ['/usr/lib/spark2/bin/spark-submit', '--master', 'yarn', '--conf', 'spark.dynamicAllocation.maxExecutors=35', '--conf', 'spark.executor.cores=6', '--conf', 'spark.executor.memory=2g', '--conf', 'spark.executor.memoryOverhead=27569', '--conf', 'spark.jars.ivy=/tmp/airflow_ivy2', '--conf', 'spark.jars.ivySettings=/etc/maven/ivysettings.xml', '--conf', 'spark.pyspark.python=mjolnir_venv/bin/python', '--conf', 'spark.sql.shuffle.partitions=1000', '--conf', 'spark.task.cpus=6', '--conf', 'spark.yarn.maxAppAttempts=1', '--archives', 'hdfs://analytics-hadoop/user/ebernhardson/mjolnir_venv.zip#mjolnir_venv', '--jars', 'hdfs://analytics-hadoop/wmf/refinery/current/artifacts/refinery-hive.jar', '--packages', 'org.wikimedia.search:mjolnir:0.5-SNAPSHOT', '--driver-memory', '2g', '--name', 'mjolnir-train-enwiki-dbn-20180215-query_explorer-pruned_mrmr-20200923-spark', '--queue', 'root.default', '--deploy-mode', 'cluster', '/srv/deployment/wikimedia/discovery/analytics/spark/mjolnir-utilities.py', 'train', '--date', '20200923', '--output-path', 'hdfs://analytics-hadoop/wmf/data/discovery/mjolnir/trained_models/date=20200923/wikiid=enwiki/labeling_algorithm=dbn/feature_set=20180215-query_explorer-pruned_mrmr', '--model-parameters-table', 'mjolnir.model_parameters', '--remote-feature-set', '20180215-query_explorer', '--training-files-path', 'hdfs://analytics-hadoop/wmf/data/discovery/mjolnir/training_files/date=20200923/wikiid=enwiki/labeling_algorithm=dbn/feature_set=20180215-query_explorer-pruned_mrmr']. Error code is: 1.
Log: Link
Host: an-airflow1001.eqiad.wmnet
Log file: /var/log/airflow/mjolnir/train-enwiki-dbn-20180215-query_explorer-pruned_mrmr/2020-09-23T00:00:00+00:00.log
Mark success: Link