A clustering key is a subset of columns in a table (or expressions on a table) that are explicitly designated to co-locate the data in the table in the same micro-partitions. This is useful for very large tables where the ordering was not ideal (at the time the data was inserted/loaded) or extensive DML has caused the table’s natural clustering to degrade.

Some general indicators that can help determine whether to define a clustering key for a table include:

Queries on the table are running slower than expected or have noticeably degraded over time.

The clustering depth for the table is large.

A clustering key can be defined at table creation (using the CREATE TABLE command) or afterward (using the ALTER TABLE command). The clustering key for a table can also be altered or dropped at any time.

All ingested data stored in Snowflake tables is encrypted using AES-256 strong encryption.

All files stored in internal stages for data loading and unloading automatically encrypted using AES-256 strong encryption.

Periodic rekeying of encrypted data.

Support for encrypting data using customer-managed keys.

Partition pruning is a term used in reference to table partitioning. It refers to the ability of the optimizer to disregard, or "prune," those table partitions that are not needed to satisfy a query. Instead of scanning an entire (huge) table, just a small portion is scanned.

When Snowflake warehouse cannot fit an operation in memory, it starts spilling (storing) data first to the local disk of a warehouse node, and then to remote storage.

In such a case, Snowflake first tries to temporarily store the data on the warehouse local disk. As this means extra IO operations, any query that requires spilling will take longer than a similar query running on similar data that is capable to fit the operations in memory.

Also, if the local disk is not sufficient to fit the spilled data, Snowflake further tries to write to the remote cloud storage, which will be shown in the query profile as "Bytes spilled to remote storage".

Submit a request to the Support team at Snowflake

Using Sigma Computing's tool can be a great alternative to Excel for data analysis and visualization. By incorporating Sigma Computing's tool into our workflow, we can take advantage of its powerful features and capabilities to optimize our data analysis process.

Sigma Computing's tool is easy to use and provides a wealth of powerful features that can help make our data analysis process more efficient. With Sigma Computing's tool, we can quickly analyze and visualize our data with minimal effort. Furthermore, Sigma Computing's tool also provides the ability to easily share and collaborate on our analyses with other users.

Finally, Sigma Computing's tool also provides a wide range of data connectors that allow us to quickly and easily connect to a variety of data sources. This makes it easy for us to access and analyze data from virtually any source, giving us the flexibility to work with data from multiple sources. Additionally, Sigma Computing's tool also offers a comprehensive set of security features, so we can be sure that our data is kept safe and secure.

Fivetran is multi-cloud, and that can orchestrate data between clouds. Fivetran components may be deployed across different clouds while managed from one console. You can deploy Fivetran on either AWS or Google Cloud, and plans for other providers are underway.

‍Fivetran is a low-code ETL solution that automates the ETL processes and offers a multitude of pre-built connectors for well-known data sources.

Fivetran is the automated data movement platform moving data out of, into and across your cloud data platforms.

SnowSQL offers a quick way to export data into your local system (your own computer). However, this feature is also its sticking point if you’re working with a large amount of data as you’ll end up burdening your local option.

In contrast, a COPY command lets you utilize cloud storage (and, by proxy, better performance under large data sizes). Plus, this cloud storage, regardless of AWS, GCP, or Azure, is usually affordable and can be a better option if you need to export large data.

It depends from user to user but some of the easiest ETL Tools that you can learn are Hevo, Dataddo, Talend, Apache Nifi because of their simple-to-understand UI and as they don’t require too much technical knowledge.

#1) Integrate.io
#2) Skyvia
#3) Altova MapForce
#4) IRI Voracity
#5) Astera Centerprise
#6) Dataddo
#7) Dextrus
#8) DBConvert Studio By SLOTIX s.r.o.
#9) Informatica – PowerCenter
#10) IBM – Infosphere Information Server
#11) Oracle Data Integrator
#12) Microsoft – SQL Server Integrated Services (SSIS)
#13) Ab Initio
#14) Talend – Talend Open Studio for Data Integration
#15) CloverDX Data Integration Software
#16) Pentaho Data Integration
#17) Apache Nifi
#18) SAS – Data Integration Studio
#19) SAP – BusinessObjects Data Integrator
#20) Oracle Warehouse Builder
#21) Sybase ETL
#22) DBSoftlab
#23) Jasper

Cloning a database or schema affects tags in that database or schema as follows:

- Tag associations in the source object (e.g. table) are maintained in the cloned objects.

- For a database or a schema:

The tags stored in that database or schema are also cloned.

When a database or schema is cloned, tags that reside in that schema or database are also cloned.

If a table or view exists in the source schema/database and has references to tags in the same schema or database, the cloned table or view is mapped to the corresponding cloned tag (in the target schema/database) instead of the tag in the source schema or database.

Snowflake provides sample data sets, such as the industry-standard TPC-DS and TPC-H benchmarks, for evaluating and testing a broad range of Snowflake’s SQL support.

Sample data sets are provided in a database named SNOWFLAKE_SAMPLE_DATA that has been shared with your account from the Snowflake SFC_SAMPLES account.

The database contains a schema for each data set, with the sample data stored in the tables in each schema. The database and schemas do not utilize any data storage so they do not incur storage charges for your account. You can execute queries on the tables in these databases just as you would with any other databases in your account; however, just as with other databases, executing queries requires a running, current warehouse for your session, which consumes credits.

Fivetran offers 160+ data source connectors and they regularly add new connectors.

There are several ways to schedule a data load:

- If you are using any ETL tools, you can use available schedulers in respective tools.

- If you create ETL's using python , SnowSQL scripts, either you can use bash scripts & cron jobs or batch files & windows scheduler.

Private Sharing on Snowflake is an area which allows:

- View and work with data that others have shared with you.

- View data that your account has shared with others.

- Review requests for data.

- Manage your data exchanges.

- Manage reader accounts.

To access the Private Sharing pages.

- Login to Snowsight, the Snowflake web interface.

- Select Data » Private Sharing.

- Select a specific Private Sharing tab.

They are two different commands:

LET - Assigns an expression to a Snowflake Scripting variable, cursor, or RESULTSET.

SET - Initializes the value of a session variable to the result of a SQL expression.

Generally, the terms “cold” and “hot” mean where the data was located earlier (traditional file storage).

Hot data is for analysts who need their data to be subsecond/ high concurrency to help them answer real-time questions.

Cold data commonly sits in your data warehouse and is used for reporting and planning.

Streamlit is an open source app framework in Python language. It helps create web apps for data science and machine learning in a short time. It is compatible with major Python libraries such as scikit-learn, Keras, PyTorch, SymPy(latex), NumPy, pandas, Matplotlib etc.

Streamlit was acquired by Snowflake in March 2022 and it enables data scientists and other developers to easily build data applications with Python using its open source framework. Following the acquisition, Snowflake is now advancing its Streamlit integration (in development), so developers can bring their data and machine learning (ML) models to life as secure, interactive applications — all within Snowflake.

Streamlit is an open source app framework in Python language. It helps create web apps for data science and machine learning in a short time. It is compatible with major Python libraries such as scikit-learn, Keras, PyTorch, SymPy(latex), NumPy, pandas, Matplotlib etc.

Streamlit was acquired by Snowflake in March 2022 and it enables data scientists and other developers to easily build data applications with Python using its open source framework. Following the acquisition, Snowflake is now advancing its Streamlit integration (in development), so developers can bring their data and machine learning (ML) models to life as secure, interactive applications — all within Snowflake.

If you want to ADD an IDENTIFY COLUMN in a Snowflake table, you just need to either:\n\n1. CREATE the TABLE with the IDENTITY VALUE LIKE THIS:\n\nCREATE OR REPLACE TABLE EXAMPLE_TABLE(\n YOUR_ID IDENTITY START 1 INCREMENT 1, \n YOUR_NAME VARCHAR\n);\n*Note: you can replace IDENTITY with AUTOINCREMENT - it works exactly the same way.\n\n2. ALTER a TABLE and add the IDENTIFY VALUE COLUMN [REMEMBER - you can only do this if there is NO DATA within the TABLE!]\n\nALTER TABLE EXAMPLE_TABLE ADD COLUMN YOUR_ID INT IDENTITY(1,1);\n\n*Note: All of my examples show IDENTITY(1,1) where we start from the number 1 and increment by 1 for each row. You can use different values here if you want. You can start from any number and you can increment any amount as well. IF you do not specify a START VALUE or an INCREMENT VALUE then the default values for both of them are equal to 1.\n\nYou also have 1 additional method to add auto-numbering. This is by using the SEQUENCE object. This object is separate from the table itself so its great for if you want a unique value ACROSS multiple tables. The syntax to CREATE a SEQUENCE is below. Again, a sequence is a separate object from the table itself but its very similar to an IDENTITY or AUTOINCREMENT functionality but its more versatile I would say.\n\nCREATE SEQUENCE EXAMPLE_SEQUENCE_FOR_YOU START 1 INCREMENT 1;\n// this creates a sequence the same as before with a START VALUE of 1 and an INCREMENT VALUE of 1.\n\nCREATE OR REPLACE TABLE EXAMPLE_TABLE(\n YOUR_AUTO_ID SET DEFAULT EXAMPLE_SEQUENCE_FOR_YOU.nextvalue;\n YOUR_NAME VARCHAR\n);\n//or\nALTER TABLE ALTER COLUMN YOUR_AUTO_ID SET DEFAULT EXAMPLE_SEQUENCE_FOR_YOU.nextvalue;\n\nFYI: The AUTOINCREMENT AND IDENTITY actually use sequences behind the scenes!\n