records_mover.airflow.hooks package

Module contents

class records_mover.airflow.hooks.RecordsHook(s3_temp_base_url=None, aws_conn_id='aws_default')

Bases: BaseHook

Airflow Hook which provides Records object

Parameters

  • s3_temp_base_url (Optional[str]) –

  • aws_conn_id (str) –

__init__(s3_temp_base_url=None, aws_conn_id='aws_default')

Create a new RecordsHook

Parameters

  • s3_temp_base_url (Optional[str]) – If provided, use this URL for any temporary storage which requires use of S3. If not provided, any operations (e.g., Redshift load/unloads) which are not directly provided an S3 bucket will raise.

  • aws_conn_id (str) – Airflow connection ID to use for AWS credentials.

get_conn()
Returns

Records object which can be used for moving records data

Return type

records_mover.records.Records

class records_mover.airflow.hooks.SqlAlchemyDbHook(db_conn_id)

Bases: BaseHook

Airflow hook to provide a SQLAlchemy engine from an Airflow database connection ID

Parameters

db_conn_id (str) –

__init__(db_conn_id)
Parameters

db_conn_id (str) – Airflow connection ID for the database which should be connected to.

Return type

None

get_conn()
Returns

SQLAlchemy engine corresponding to this Airflow database connection ID.

Return type

Engine