phenomedb.pipelines

The pipelines module contains tasks that are pipeline generators. These tasks are executed to create dynamically generated pipelines.

PhenomeB Pipeline Example

Example of using a task to create a Pipeline, using the PipelineFactory to chain tasks together and register it with Airflow

class phenomedb.pipelines.BasicSetup(username=None, task_run_id=None, db_env=None, db_session=None, execution_date=None, add_pipelines=True, pipeline_run_id=None)
class phenomedb.pipelines.BatchCorrectionAssessmentPipelineGenerator(**kwargs)
class phenomedb.pipelines.DynamicPipelineTask(pipeline_name=None, username=None, pipeline_run_id=None, task_run_id=None, db_env=None, db_session=None, debug=False, execution_date=None, upstream_task_run_id=None)
class phenomedb.pipelines.GenerateSingleTaskPipelines(task_run_id=None, pipeline_run_id=None, username=None, db_env=None, db_session=None, execution_date=None, validate=True, upstream_task_run_id=None, debug=False)

This pipeline is used for generating the single task pipelines. These are typically used for API methods or administrative methods such as generating the caches or running reports.

class phenomedb.pipelines.ImportAllMetabolightsPipelineGenerator(task_run_id=None, username=None, upstream_task_run_id=None, execution_date=None, db_session=None, db_env=None, debug=False, pipeline_run_id=None)
class phenomedb.pipelines.PipelineGeneratorTask(**kwargs)
class phenomedb.pipelines.RebuildPipelinesFromDB(db_env=None, type='all', username=None, db_session=None, task_run_id=None, pipeline_run_id=None, execution_date=None)
class phenomedb.pipelines.RunBatchCorrectionAssessmentPipeline(saved_query_id=None, correction_type=None, run_combat_and_norm_mixedresiduals=False, variable_of_interest=None, metadata_covariates=None, reload_cache=False, task_run_id=None, username=None, upstream_task_run_id=None, model_Y_ci=None, model_Y_min=None, model_Y_max=None, execution_date=None, db_session=None, db_env=None, wait_for_completion=False, debug=False, pipeline_run_id=None)
class phenomedb.pipelines.RunMWASMulti(saved_query_ids=None, method='pearson', correction_type=None, variable_of_interest=None, reload_cache=False, task_run_id=None, username=None, upstream_task_run_id=None, model_Y_ci=None, model_Y_min=None, model_Y_max=None, multiple_correction=None, scaling=None, transform=None, execution_date=None, db_session=None, db_env=None, debug=False, pipeline_run_id=None)