I’m having trouble with Apache Airflow DAG BashOperator() executing using /bin/sh instead of what I would prefer /bin/bash and getting an in complete set of bash environment variables on Ubuntu 18.04 .
I have one server that it works correctly using /bin/bash and another that it does not and uses /bin/sh instead and I haven’t been able to figure out why yet or how to fix it. Has anyone encountered this or resolved this same issue?
I verified that the linux user running the BashOperator has /bin/bash as their shell in /etc/passwd
The Airflow DAG important parts:
dag = DAG( 'zz_environment_test', default_args=default_args, description='Ariflow Environment Test', catchup=False, schedule_interval=None) #commands #step 1 : Cut release branch from develop for Stage environment_test_command = """ cd %s;bash zz_environment_test.sh %s; """ % (release_target_directory, release_target_directory) step_1_environment_test = BashOperator( task_id='step_1_environment_test', depends_on_past=False, bash_command=environment_test_command, dag=dag) step_1_environment_test
In zz_environment_test.sh I have
whoami printenv > mytest/bash_from_airflow_printenv.txt
Which on the environment that has the correct environment variables has value
And the one that isn’t working has