Control Pipelines (Start, Stop, Drop, or Reset a Pipeline)
After you create and test a pipeline, you control a pipeline by starting, stopping, or dropping the pipeline. You can also reset a pipeline.
- Start a Pipeline
After you create a pipeline, you can start the pipeline. - Stop a Pipeline
UseSTOP_PIPELINE
to stop a pipeline. When a pipeline is stopped, no future jobs are scheduled for the pipeline. - Drop a Pipeline
The procedureDROP_PIPELINE
drops an existing pipeline. - Reset a Pipeline
Use the reset pipeline operation to clear the record of the pipeline to the initial state.
Parent topic: Using Data Pipelines for Continuous Load and Export
Start a Pipeline
When a pipeline is started the pipeline runs continuously in a scheduled
job. The pipeline's scheduled job repeats, either by default every 15 minutes or at the
interval you set with the interval
attribute.
Stop a Pipeline
STOP_PIPELINE
to stop a
pipeline. When a pipeline is stopped, no future jobs are scheduled for the
pipeline.
By default currently running jobs complete when you stop a pipeline. Set the
force
parameter to TRUE
to terminate any running
jobs and stop the pipeline immediately.
See STOP_PIPELINE Procedure for more information.
Drop a Pipeline
DROP_PIPELINE
drops
an existing pipeline.
If a pipeline has been started, it must be stopped before the pipeline can be dropped. See STOP_PIPELINE Procedure for more information.
In order to drop a pipeline that is started, set the force
parameter
to TRUE
to terminate any running jobs and drop the pipeline
immediately
See DROP_PIPELINE Procedure for more information.
Reset a Pipeline
You can optionally use reset pipeline to purge data in the database table associated with a load pipeline or to remove files in object store for an export pipeline. Usually this option is used when you are testing a pipeline during pipeline development.
Reset pipeline operates as follows:
-
Load Pipeline: For a load pipeline, resetting the pipeline clears the record of the files being loaded by the pipeline. When you call either
START_PIPELINE
orRUN_PIPELINE_ONCE
after resetting a load pipeline, the pipeline repeats the data load and includes all the files present in the object store location.When
purge_data
is set toTRUE
, the procedure truncates the data in the database table. -
Export Pipeline: For an export pipeline, resetting the pipeline clears the last tracked data in the database table. When you call either
START_PIPELINE
orRUN_PIPELINE_ONCE
after resetting an export pipeline, the pipeline repeats exporting data from the table or query.When
purge_data
set toTRUE
, the procedure deletes existing files in the object store location specified with thelocation
attribute.
To reset a pipeline: