README

Each sub-dag runs for a specific core rates month. The orchestrator executes each sub-dag then combines the outputs to identify the best rate.

---

Orchestrator

Airflow Link to Orchestrator

Params Example:

{
    "orchestrator_sql_loc": "/usr/local/airflow/dags/core_licensable_data_orchestrator/sql",
    "sql_loc": "/usr/local/airflow/dags/core_licensable_data_orchestrator/sql",
    "s3_trino_bucket": "turquoise-health-payer-export-main",
    "db_conn_name": "trino_default",
    "payer_schemas_list": [
        "hive.public_2025_03",
        "hive.public_2025_02",
        "hive.public_2025_01",
        "hive.public_2024_12",
        "hive.public_2024_11"
    ],
    "hospital_schemas_list": [
        "glue.hospital_data",
        "glue.hospital_historical_2025_02",
        "glue.hospital_historical_2025_01",
        "glue.hospital_historical_2024_12",
        "glue.hospital_historical_2024_11"
    ],
    "sub_version_list": [
        "2025_03",
        "2025_02",
        "2025_01",
        "2024_12",
        "2024_11"
    ],
    "version": "v1_1",
    "schema_name": "tq_dev.internal_dev_csong_cld_",
    "provider_types": [
        "Childrens Hospital",
        "Critical Access Hospital",
        "Short Term Acute Care Hospital",
        "ASC",
        "Physician Group"
    ]
}

---

Sub-DAGs

Airflow Link to Sub-DAG

Params Example:

{
    "s3_trino_bucket": "turquoise-health-payer-export-main",
    "db_conn_name": "trino_default",
    "payer_schema": "hive.public_2025_03",
    "hospital_schema": "glue.hospital_data",
    "sub_version": "2025_03",
    "schema_name": "tq_dev.internal_dev_csong_cld_",
    "version": "v1_1",
    "provider_types": [
        "Childrens Hospital",
        "Critical Access Hospital",
        "Short Term Acute Care Hospital",
        "ASC",
        "Physician Group"
    ]
}

Test Params

Orchestrator

To generate files in tq_dev.internal_dev_csong_cld_test:

{
    "orchestrator_sql_loc": "/usr/local/airflow/dags/core_licensable_data_orchestrator/sql",
    "sql_loc": "/usr/local/airflow/dags/core_licensable_data_orchestrator/sql",
    "s3_trino_bucket": "turquoise-health-payer-export-main",
    "db_conn_name": "trino_default",
    "payer_schemas_list": [
        "hive.cld_utils"
    ],
    "hospital_schemas_list": [
        "hive.cld_utils"
    ],
    "sub_version_list": [
        "test"
    ],
    "version": "test",
    "schema_name": "tq_dev.internal_dev_csong_cld_",
    "provider_types": [
        "Childrens Hospital",
        "Critical Access Hospital",
        "Short Term Acute Care Hospital",
        "ASC"
    ]
}

To generate files in tq_dev.internal_dev_csong_cld_test:

Sub-DAG

{
    "sql_loc": "/usr/local/airflow/dags/core_licensable_data_sub_dag/sql",
    "s3_trino_bucket": "turquoise-health-payer-export-main",
    "db_conn_name": "trino_default",
    "payer_schema": "hive.cld_utils",
    "hospital_schema": "hive.cld_utils",
    "sub_version": "test",
    "version": "test",
    "schema_name": "tq_dev.internal_dev_csong_cld_",
    "provider_types": [
        "Childrens Hospital",
        "Critical Access Hospital",
        "Short Term Acute Care Hospital",
        "ASC"
    ]
}