Try 6 out of 6
Exception:
Cannot execute: ['/usr/lib/spark2/bin/spark-submit', '--master', 'yarn', '--conf', 'spark.dynamicAllocation.maxExecutors=195', '--conf', 'spark.executor.cores=2', '--conf', 'spark.executor.memory=5g', '--conf', 'spark.jars.ivy=/tmp/airflow_ivy2', '--conf', 'spark.jars.ivySettings=/etc/maven/ivysettings.xml', '--conf', 'spark.locality.wait=0', '--conf', 'spark.pyspark.python=mjolnir_venv/bin/python', '--conf', 'spark.sql.shuffle.partitions=1000', '--conf', 'spark.yarn.maxAppAttempts=1', '--archives', '/srv/deployment/wikimedia/discovery/analytics/environments/mjolnir/venv.zip#mjolnir_venv', '--jars', 'hdfs://analytics-hadoop/wmf/refinery/current/artifacts/refinery-hive-shaded.jar', '--packages', 'org.wikimedia.search:mjolnir:0.7,sramirez:spark-infotheoretic-feature-selection:1.4.4,sramirez:spark-MDLP-discretization:1.4.1', '--driver-memory', '6g', '--name', 'mjolnir-feature_selection-20180215-query_explorer-dbn-20180215-query_explorer-pruned_mrmr-20220825-spark', '--queue', 'default', '--deploy-mode', 'cluster', '/srv/deployment/wikimedia/discovery/analytics/spark/mjolnir-utilities.py', 'feature_selection', '--date', '20220825', '--output-path', 'hdfs://analytics-hadoop/wmf/data/discovery/mjolnir/feature_vectors/date=20220825/feature_set=20180215-query_explorer-pruned_mrmr', '--feature-set', '20180215-query_explorer', '--feature-vectors-table', 'mjolnir.feature_vectors', '--labeling-algorithm', 'dbn', '--labels-table', 'mjolnir.labeled_query_page', '--num-features', '50', '--output-feature-set', '20180215-query_explorer-pruned_mrmr', '--output-table', 'mjolnir.feature_vectors', '--wikis', 'arwiki', 'dewiki', 'enwiki', 'fawiki', 'fiwiki', 'frwiki', 'hewiki', 'idwiki', 'itwiki', 'jawiki', 'kowiki', 'nlwiki', 'nowiki', 'plwiki', 'ptwiki', 'ruwiki', 'svwiki', 'viwiki', 'zhwiki']. Error code is: 1.
Log: Link
Host: an-airflow1001.eqiad.wmnet
Log file: /var/log/airflow/mjolnir/feature_selection-20180215-query_explorer-dbn-20180215-query_explorer-pruned_mrmr/2022-08-25T20:24:25.197234+00:00.log
Mark success: Link