I am new in airflow. I created my first dag below, selecting data from google big query table & saving it to a pd dataframe. Need suggestion in below Where should i provide connection id of my big query As pd.read_gbq requires authintication, how to handle same in airflow dags import os import pandas as […]
- Tags -11, 'depends_on_past': False, 'email_on_retry': True, 'retries': 2 } dag = DAG('test_gbq', 'start_date': datetime(2018, 2, 26, 30, dag=dag) testing, default_args=default_args, dialect='standard') print(len(df)) default_args = {'owner': 'Martin james', how to handle same in airflow dags import os import pandas as pd from airflow.contrib.operators import bigquery_operatorfrom #this will, I am new in airflow. I created my first dag below, python_callable=test_python, schedule_interval="0 12 * * *") testing = PythonOperator( task_id="test", selecting data from google big query table & saving it to a pd dataframe. Need suggestion in below Where should i provide connection id of m