To configure Snowpipe, you need to use the ALTER PIPE statement. The syntax for the ALTER PIPE statement is as follows:
ALTER PIPE pipe_name OPTIONS ( file_format = file_format_name, staging_parallelism = number_of_threads, ingestion_parallelism = number_of_threads );
The pipe_name parameter is the name of the Snowpipe that you want to configure.
The file_format_name parameter is the name of the file format that the data files are in.
The staging_parallelism parameter is the number of threads that Snowpipe will use to stage the data files.
The ingestion_parallelism parameter is the number of threads that Snowpipe will use to ingest the data files into the table.
Here is an example of how to configure a Snowpipe:
ALTER PIPE my_pipe OPTIONS ( file_format = csv, staging_parallelism = 8, ingestion_parallelism = 16 );
This will configure the my_pipe Snowpipe to use CSV format for the data files, 8 threads for staging the data files, and 16 threads for ingesting the data files into the table.
You can also configure Snowpipe to load data automatically. To do this, you need to specify the AUTO_INGEST option in the CREATE PIPE or ALTER PIPE statement.
The AUTO_INGEST option can be set to either TRUE or FALSE. If it is set to TRUE, Snowpipe will automatically load data as it arrives in the cloud storage location. If it is set to FALSE, you will need to manually trigger Snowpipe to load the data.
Here is an example of how to configure Snowpipe to load data automatically:
CREATE PIPE my_pipe ( COPY INTO my_table FROM my_stage OPTIONS ( file_format = csv, staging_parallelism = 4, ingestion_parallelism = 8, AUTO_INGEST = TRUE ) );
This will create a Snowpipe called my_pipe that will load data from the my_stage stage into the my_table table automatically. The data files in the my_stage stage must be in CSV format. Snowpipe will use 4 threads to stage the data files and 8 threads to ingest the data files into the table.
