Breaking News: Grepper is joining You.com. Read the official announcement!

import module to airflow dag

Embarrassed Eel answered on February 1, 2024 Popularity 1/10 Helpfulness 1/10

answer import module to airflow dag

import module to airflow dag

Comment

import os
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python_operator import PythonOperator

# Get the directory path of the current file (your DAG file)
current_file_path = os.path.abspath(__file__)

# Extract the directory path
module_directory = os.path.dirname(current_file_path)

# Add the module directory to the sys.path
import sys
sys.path.append(module_directory)

# Import your Python module
from your_module import your_function

# Define default arguments
default_args = {
    'owner': 'your_name',
    'depends_on_past': False,
    'start_date': datetime(2024, 1, 1),
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
}

# Instantiate the DAG object
dag = DAG(
    'example_dag',
    default_args=default_args,
    description='Example DAG with dynamically determined module directory',
    schedule_interval=timedelta(days=1),
)

# Define a PythonOperator task
python_task = PythonOperator(
    task_id='execute_your_function',
    python_callable=your_function,
    provide_context=True,  # Pass the context to your Python function if needed
    dag=dag,
)

# Set the task execution order
python_task

Popularity 1/10 Helpfulness 1/10 Language python

Source: chat.openai.com

Tags: airflow import-module python

Link to this answer
Share Copy Link

Contributed on Feb 01 2024

Embarrassed Eel

0 Answers Avg Quality 2/10

import module to airflow dag

Contents

More Related Answers

import module to airflow dag

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.