Scaling up vs. Scaling out Warehouses

210 viewsConnecting to SnowflakeVirtualwarehouses

Scaling up vs. Scaling out Warehouses

Alejandro Penzini Answered question December 19, 2023

Scaling Warehouse Resources:

Snowflake offers two approaches to scaling warehouse resources:

Vertical Scaling (Resizing): Increase compute resources within a single warehouse. Improves performance, especially for large complex queries, and reduces queuing due to resource limitations.

Note: Resizing is not intended for handling high concurrency; consider additional warehouses or multi-cluster options.

Horizontal Scaling (Adding Clusters): Requires Snowflake Enterprise Edition and expands a multi-cluster warehouse with additional compute units.

Resizing Considerations:

Limited benefit for small, basic queries. Larger size doesn’t guarantee faster execution.
Running queries unaffected. Resizing only impacts queued and new queries after additional resources are provisioned.

5XL/6XL to 4XL or smaller: Brief dual billing period while the old warehouse shuts down.

Multi-cluster Warehouses Improve Concurrency

Leveraging Multi-Cluster Warehouses:

Snowflake Enterprise Edition users can opt for multi-cluster warehouses specifically designed to handle:

High concurrency: Ideal for scenarios with numerous concurrent users or queries.
Dynamic workloads: Adapts automatically to fluctuating user/query demands.

Considerations for Multi-cluster Warehouses:

Enterprise Edition users: Configure all warehouses as multi-cluster.
Mode: Prefer Auto-scale for automatic cluster adjustments (unless Maximized mode is vital).

Number of Clusters:
Minimum: Keep default (1) for on-demand scaling. Increase for high-availability needs.
Maximum: Set based on warehouse size and anticipated costs. Consider potential credit consumption with all clusters running concurrently.

Alejandro Penzini Answered question December 19, 2023