To configure Snowpipe, you need to use the
ALTER PIPE statement. The syntax for the
ALTER PIPE statement is as follows:
ALTER PIPE pipe_name OPTIONS ( file_format = file_format_name, staging_parallelism = number_of_threads, ingestion_parallelism = number_of_threads );
pipe_name parameter is the name of the Snowpipe that you want to configure.
file_format_name parameter is the name of the file format that the data files are in.
staging_parallelism parameter is the number of threads that Snowpipe will use to stage the data files.
ingestion_parallelism parameter is the number of threads that Snowpipe will use to ingest the data files into the table.
Here is an example of how to configure a Snowpipe:
ALTER PIPE my_pipe OPTIONS ( file_format = csv, staging_parallelism = 8, ingestion_parallelism = 16 );
This will configure the
my_pipe Snowpipe to use CSV format for the data files, 8 threads for staging the data files, and 16 threads for ingesting the data files into the table.
You can also configure Snowpipe to load data automatically. To do this, you need to specify the
AUTO_INGEST option in the
CREATE PIPE or
ALTER PIPE statement.
AUTO_INGEST option can be set to either
FALSE. If it is set to
TRUE, Snowpipe will automatically load data as it arrives in the cloud storage location. If it is set to
FALSE, you will need to manually trigger Snowpipe to load the data.
Here is an example of how to configure Snowpipe to load data automatically:
CREATE PIPE my_pipe ( COPY INTO my_table FROM my_stage OPTIONS ( file_format = csv, staging_parallelism = 4, ingestion_parallelism = 8, AUTO_INGEST = TRUE ) );
This will create a Snowpipe called
my_pipe that will load data from the
my_stage stage into the
my_table table automatically. The data files in the
my_stage stage must be in CSV format. Snowpipe will use 4 threads to stage the data files and 8 threads to ingest the data files into the table.