Introduction:

Today’s article discusses the cost risks associated with using the Snowflake Data Cloud. It emphasizes the importance of proper administration and resource monitoring to mitigate these risks. We will also mention our service, Snoptimizer, which can help automate cost optimization and risk minimization related to Snowflake accounts.

Part 1: My Experience with Snowflake’s Data Cloud

As a Snowflake Data Superhero for the past 4 years, I can’t deny, I love using Snowflake. I fell in love with it at the beginning of 2018 when I realized how easily I could execute all of our Big Data Consulting Practice Solutions we had been doing for 18+ years. In the past, we would often run into scale challenges as the data’s size grew but Snowflake brought both ease of use and amazing scale to almost all of my big data consulting projects.

Over the last three years, my team and I have worked on hundreds of Snowflake accounts. I’ve come to realize that if Snowflake Anti-patterns occur or if poor compute security practices are used, Snowflake accounts are exposed to large cost risks, particularly with regard to computing costs. While Snowflake is an amazingly scalable cloud database and is the best cloud data warehouse I’ve used in the last 3+ years, the deployment of a Snowflake Account without proper settings and administration exposes a company to these cost risks.

Part 2: Examples of Data Cloud Cost Risks

Let’s say you actually used the Classic Console to create a new warehouse and used all the default settings…  Even if you didn’t run any query, the cost for the standard settings would be 10 minutes * XL Warehouse (16 credits/hour) @ $3/credit.  It is only $8 for those 10 minutes but it was $8 spent on nothing.  Let’s say you had a rogue (or curious) trainee on a Snowflake Account that didn’t understand what they were doing and does the same thing they ONLY change the size to a 6XL. Your 10-minute run-for-nothing cost exposure is 10 min * 6XL Warehouse (512 credits/hour) @ $3/credit. Your account just spent $256 for 10 minutes of nothing.

Snowflake Cost Risk Use Case 6XL – 1 cluster:

• Cost per hour @ $3/credit = 3 * 512 = $1536Cost per day @ $3/credit = $36,864

Snowflake Cost Risk Use Case 6XL – 5 clusters:

*[we know this is a worst-case scenario on AWS/Snowflake and this one would be rare BUT without resource monitors and correct permissions it is possible]*Cost per hour @ $3/credit = 3 * 512 = $7680Cost per day @ $3/credit = $184,320

Snowflake Cost Risk Use Case 6XL – 10 clusters:

*[we know this is a worst-case scenario on AWS/Snowflake and this one would be rare BUT without resource monitors and correct permissions it is possible]*Cost per hour @ $3/credit = 3 * 512 = $15,360Cost per day @ $3/credit = $368,640

As you can see, this is unreasonable exposure to cost risks.  If you are a Snowflake administrator, make sure you make appropriate changes to control costs and cost risk.  If you want an automated approach to Cost Risk Management that can be set up easily in a few hours then try our Snoptimizer Cost Risk Solution.

Snowflake Cost Risk Mitigation – Administration – ACCOUNT ADMIN – MUST DO

The most important way to minimize Snowflake Cost Risk is to create Resource Monitors for every warehouse that has to suspend actions.  Here is the code to do that: [replace 150 with your daily credit limit].

CREATE RESOURCE MONITOR "REMOVE_SNOWFLAKE_COST_RISK_EXAMPLE_RM" WITH CREDIT_QUOTA = 50
FREQUENCY = 'DAILY', START_TIMESTAMP = 'IMMEDIATELY', END_TIMESTAMP = NULL
TRIGGERS
ON 95 PERCENT DO SUSPEND
ON 100 PERCENT DO SUSPEND_IMMEDIATE
ON 80 PERCENT DO NOTIFY;

ALTER WAREHOUSE “TEST_WH” SET RESOURCE_MONITOR = “REMOVE_SNOWFLAKE_COST_RISK_EXAMPLE_RM”;

Part 3: Use Snoptimizer

Snoptimizer offers Snowflake users a unique solution to enhance the cost, performance, and security of their Snowflake accounts.

It provides you with a consistent and reliable daily analysis of how your Snowflake account is being optimized and rightsized to achieve maximum efficiency.

Try Snoptimizer today. Sign up and schedule a personal demo with us!

Conclusion:

To conclude, we now understand that Snowflake's Data Cloud Cost Risk is a real issue that needs proper administration. While the Snowflake Data Cloud offers immense scale and power for any analytical data processing need, it must be optimized and continuously monitored by a service like Snoptimizer. Remember, with great data processing power comes great cost management responsibility. If an administrator mistakenly grants access to an untrained user to create a 6XL instance that they don't need and isn't within the business budget, it can result in significant costs.

Try Snoptimizer today to avoid a data-driven cost catastrophe.  If a new warehouse appears, we have you covered!

Introduction:

 

The Snowflake Data Cloud provides impressive scalability and processing power for analytical data. It's an incredible advancement that now allows for launching T-shirt warehouse sizes ranging from XS (1 virtual instance) up to 6XL with 512 EC2 instances when running on AWS on 1 cluster. However, it's important to set up your Snowflake account with resource monitors to control costs. Snowflake Resource Monitors serve as your primary way to control costs within the Snowflake Data Cloud.

Let’s show you how easy it is to set up your Snowflake cost guardrails so your costs don’t go beyond what you expect.

We recommend either hiring a full or part-time Snowflake administrator focused on cost optimization and database organization or using our Snowflake Cost Optimization Tool – Snoptimizer.  Snoptimizer automates setting up resource monitors on your Snowflake account for each warehouse and tons of other cost optimizations and controls on your Snowflake Account. Let’s dig into the only true Snowflake Cost Risk Guardrails you have had for a while, Resource Monitors.

 

Resource Monitors – Your Snowflake Data Cloud Cost Guardrails

Resource Monitors are technically relatively easy to set up from the Snowflake Web GUI or command line. Even though setting up one Resource Monitor is relatively easy, it’s still easy to make incorrect assumptions and not have enough effective monitoring and suspending in place. It is like having your guardrails not installed yet if you do not do the following which is too easy to do:

Finding out that Snowflake consumption-based pricing was so reasonable was game-changing for me and my consulting company.  We could finally provide scale to any analytical challenge and solution we needed to create. This was never possible before. I remember building predictive marketing tools and we often had to crunch large data sets we would often run into scaling challenges and have to spend tons of time and engineering effort to engineer for scale.

 

Try Snoptimizer:

 

 

Try Snoptimizer today. Sign up and schedule a personal demo with us!

 

To Recap - How Snoptimizer Helps You

Snoptimizer quickly and automatically optimizes your Snowflake account for security, cost, and performance. It eliminates headaches and concerns about security risks and cost overruns across your Snowflake account.

In short, Snoptimizer makes managing your Snowflake cost, performance, and security much easier and more automated.

 

Conclusion:

 

Having properly set up Snowflake Guardrails in the form of Resource Monitors is extremely important. If you're unsure whether or not you have these in place, it's time to take action. Activate Snoptimizer today to optimize your system in just a few hours, and ensure continuous and regular cost optimization monitoring. If a new warehouse appears, we've got you covered!

In conclusion, setting up Snowflake resource monitors is crucial for controlling costs in the Snowflake Data Cloud.

Introduction:

 

I have been working with Snowflake since the beginning of 2018 and it has been one of the most enjoyable and scalable data solutions I have encountered in my 27+ year career as a data engineer, data architect, data entrepreneur, and data thought leader. It is an extremely powerful platform (with nearly unlimited scalability, limited only by Snowflake's allocation of Compute within an Availability Zone) that requires responsible usage.

In the past 3 years, I've analyzed more than 100 Snowflake accounts and found that about 90% of them were not completely optimized for cloud data costs. That's why my team and I are thrilled to introduce Snoptimizer, the first automated Snowflake cost optimization service.

One of the reasons why 90% of those accounts did not have resource monitors or regular optimizations is that Snowflake is initially cost-effective and typically provides significant savings, especially for on-prem migrations that we have completed. However, companies that do not optimize their Data Cloud Costs are missing out on big opportunities! That's why we created Snoptimizer, and I'm also sharing my top 6 Snowflake cost and risk optimizations below. Hope you find them helpful!

 

Part 1: My Best Practices for Optimizing Snowflake Costs and Reducing Cost Risks:

 

Best Practice #1 – Resource Monitors.

One of the initial features of Snoptimizer is the automation of daily Resource Monitors at a warehouse level, which is based on the Snowflake Metadata database history and warehouse and Resource Monitor settings. This is set up immediately following the purchase of Snoptimizer.

By doing this, both cost risk and guardrails for all warehouse computing are reduced. Snoptimizer utilizes Snowflake Metadata and warehouse/Resource Monitor settings to automatically monitor resources daily at the warehouse level. This helps limit risks and ensures that constraints are not exceeded.

 

Best Practice #2 – Auto Suspend Setting Optimization.

Snoptimizer automates another optimization by analyzing the workloads in the Warehouse and making changes to the Auto Suspend settings. Depending on the workload, Snoptimizer can also automate additional cost savings for you.

 

Best Practice #3 – Monitoring Cloud Services Consumption Optimization

Snoptimizer analyzes your Snowflake Account's Cloud Services consumption to quickly identify opportunities for cost savings. We thoroughly review usage and billing details for each service to ensure that only what is necessary is provisioned, reducing waste and minimizing costs. Optimizing Cloud Services is one of the most effective ways to lower your Snowflake spending while still meeting your data and compute demands.

 

Best Practice #4 – Regular Monitoring of Storage Usage Across your Entire Snowflake Account

At Snoptimizer, our goal is to help you save on Storage costs. We start by reviewing your Storage History for the past 60 days to identify any settings that may be causing you to overpay. We commonly find that charges related to Time Travel and/or Snowflake Stages are unnecessary and can be avoided.

At Snoptimizer, we can help you make the most of your storage space. Our service optimizes your settings based on your actual usage, so you're only paying for what you need. By analyzing 60 days of storage history, we're able to find ways to reduce costs by up to 25%, all without sacrificing performance or features.

 

Best Practice #5 – Daily Monitoring of Warehouse Compute.

Besides just adding Resource Monitors that suspend Warehouses we also provide daily monitoring of Snowflake Warehouse consumption reporting daily spikes and anomalies or changes in rolling averages. Most accounts we come across do not have regular monitoring of warehouse usage on regular proactive settings.

 

Best Practice #6 – Regular Monitoring of New Snowflake Services.

Besides monitoring compute warehouses, Snoptimizer also immediately starts monitoring consumption on all existing and new Cloud Services (private preview and after) incurring costs from Automatic Clustering to Search Optimization to Materialized Views and all existing and new costs. This huge benefit automates Snoptimizer. We are ALWAYS optimizing your cost consumption and reducing cost risk! We are always there for you!

 

Part 2: Try Snoptimizer today

 

At Snoptimizer, we offer Snowflake users a one-of-a-kind tool to enhance their Snowflake accounts' cost efficiency, performance, and security.

 

Our tool offers a dependable and regular daily analysis of your Snowflake account optimization and rightsizing, ensuring maximum efficiency.

Try Snoptimizer today. Sign up and schedule a personal demo with us!

 

Conclusion:

 

After reading this article, we hope you have a better understanding of the best practices for optimizing Snowflake costs and reducing cost risks. These practices include using resource monitors, optimizing auto-suspend settings, monitoring cloud service consumption, regularly monitoring storage usage, daily monitoring of warehouse computing, and keeping track of new Snowflake services.

We encourage you to try out our automated Snowflake optimization tool for better cost, security, and performance efficiency.

Try Snoptimizer today! Sign up and schedule a personal demo with us!

Introduction:

 

Snowflake pricing is determined by how much you use compute resources such as warehouses (virtual compute instances) and storage, as well as other costs like cloud services, cloud storage, and data transfer. Most of your Snowflake costs will be for computing resources, which typically account for 90% or more of your monthly costs. With Snowflake, you don't have to make any upfront commitments or sign any long-term contracts.

You can start with a free trial account, and you won't be charged if you don't use any billable services. You only pay for what you use. Snowflake pricing may vary depending on the platform and region you're using.

 

Which tools can help optimize your Snowflake costs?

 

There are a multitude of tools that you can leverage to confidently optimize and minimize your Snowflake costs. One such tool is Snoptimizer, which can be a game-changer for your organization.

Snoptimizer is the first automated Snowflake Cost Optimization Service that ensures significant cost savings (up to 50% on Snowflake compute) without sacrificing performance.

We built Snoptimizer because we saw a significant need in the marketplace. We were often called in by Snowflake customers for Snowflake Health checks, and 98% of the time, their accounts were not fully optimized.

Snoptimizer runs regularly and scours your Snowflake Operations Account Meta Data (over 40 views) continuously looking for Snowflake storage and computing anti-patterns and inefficiencies related to cost.

 

Usage-Based Pricing:

 

Usage Based Pricing in cloud services, especially in Snowflake, can be incredibly awesome sometimes. The fact that we can even start an account off with 400 credits for 30 days for a Proof of Concept (POC) is just amazing to me. Before this, our consulting company hesitated to introduce these more expensive solutions to our consulting clients which were small or medium size businesses because these solutions were out of their pricing comfort zone (especially when working with analytical databases that could scale like Exadata, Teradata, and Netezza).

 

What is the pricing on Snowflake?

 

For those of you who are new to Snowflake, let's start with Snowflake consumption pricing basics. Snowflake overall is usage or consumption-based pricing.  This means you only pay for what you use. Technically, you could set up a free Snowflake Trial Account and never pay anything because you never used any of the services that have a cost. T

For most Snowflake Accounts, Snowflake Compute or the Snowflake Warehouses (which are virtual compute engines) is where 90% or more of your costs are. The other four cost areas of Storage Costs, Cloud Computing Costs, Cloud Services Costs, and Data Transfer costs are typically easily 10% or less of the Snowflake SaaS costs per month.  Often the others can even be 1% or less unless you have certain use cases or end up mistakenly using Snowflake Cost Anti-patterns.

Please keep in mind that as soon as your Snowflake Account is provisioned, you the administrator, or a person with their credit card associated with the account have extreme cost risk by default.  Our best practice is to always enable Snowflake Cost Optimization with Snoptimizer immediately after provisioning a Snowflake Account. If you decide against that then at the very least you should limit access or set up standard Snowflake Cost Minimization Guardrails and Snowflake Cost Optimization and Cost Minimization Best Practices.

For those of you who are more Snowflake savvy and already know the basics then let's cover more advanced Snowflake pricing details.

Snowflake Compute Pricing - Advanced

 

One of the first things that Snoptimizer does is automate daily Resource Monitors at a warehouse level based on all the Snowflake Metadata Database history and warehouse and Resource Monitor settings. This gets set almost immediately after you purchase Snoptimizer. This has both huge cost risk reduction limits and guardrails for all of your warehouse compute.

One cool thing you can do is reduce your default query time out to 4 hours or less instead of 2 days by default with the following code.

ALTER WAREHOUSE SET STATEMENT_TIMEOUT_IN_SECONDS = 14400;

[/signinlocker]

 

How to Optimize Your Costs?

 

Over the last 3 years, my teams and I have analyzed over 100 Snowflake accounts, and about 95% of them were not fully optimized for both Cloud data costs and Cloud cost risk minimization. This is why my team and I are so excited to have created Snoptimizer (the first automated Snowflake Cost Optimization Service) - Easily optimize your Snowflake Data Cloud Account here in a few hours.

I think the reason why 90% of those accounts didn't have resource monitors or regular optimizations in place was initially Snowflake is incredibly cost-effective and typically had massive savings, especially from on-prem migrations that we have done.  However, companies that do not optimize their Data Cloud Costs are missing out big time!

 

Try Snoptimizer today:

 

Snoptimizer quickly and automatically optimizes your Snowflake account for security, cost, and performance. It eliminates headaches and concerns about security risks and cost overruns across your Snowflake account.

 

 

Try Snoptimizer today. Sign up and schedule a personal demo with us!

Optimization in a few hours, hassle-free!

 

Conclusion:

 

I hope the Snowflake Basic and Advanced Pricing information above is useful to you on your Snowflake Journey. For me, finding out that Snowflake consumption-based pricing was so reasonable was game-changing for both myself and my consulting company. Before Snowflake, we couldn't provide compute scale with enough speed to many of the largest big analytical challenges and solutions our clients needed.

I remember building predictive marketing tools and we often had to crunch large data sets we would often run into scaling challenges and have to spend tons of time and engineering effort to engineer for scale. Keep in mind that if you don't use Snowflake's Services smartly, you can end up spending a lot of money. Therefore, we recommend using Snoptimizer to help you reduce your costs.

 

If you're looking to optimize your Snowflake account costs, try Snoptimizer today!

Sign up and schedule a personal demo with us!

 

Snowflake is still my favorite Analytical Database since the beginning of 2018 but as I often present in my live training sessions and webinars, WITH GREAT POWER (practically unlimited computing scale) comes GREAT RESPONSIBILITY.

In this article, I'll cover the TOP 3 Snowflake Cost Anti-patterns my Snowflake Cost Optimization team and I have come across after 3 years of analyzing hundreds of Snowflake Accounts.  I cannot begin to state how you should either invest in a PART or FULL-TIME Snowflake DBA focused on cost and organization or if you do not have that financial luxury then use our automated Snowflake Cost Optimization Service - Snoptimizer.  It's incredibly easy and low-cost to set up Snoptimizer compared to having these anti-patterns manifest (which I know happens on too many Snowflake Accounts based on our review of hundreds of them).  If you do not have cost guardrails like Resource Monitors enabled your Snowflake Compute Consumption Risk is high and it's honestly gross negligence as a data administration professional to allow this.

Let's go through the TOP 3 Snowflake Cost Anti-patterns.

Top 3 Anti-Patterns

The first Snowflake anti-pattern is by far the worst and happens all too often.

Snowflake Cost Anti-pattern #1

Sadly, we all too often see that Resource Monitors are not set up correctly.  Some Snowflake accounts have them set up but do not have them set up at an effective grain.  One anti-pattern is that the administrator sets a large credit-sized resource manager for the overall account and no other resources managers.  It is okay to have some Resource Monitors cover the account or multiple warehouses but we highly recommend having 1 Resource Monitor set for Daily monitoring for each warehouse with auto-suspend enabled once a credit limit is reached.  This is currently the only real solution to having guardrails on your Snowflake consumption.  Without doing this you are exposing your company and Snowflake account to significant cost risk.

An additional anti-pattern related to Resource Monitors that we see too frequently is Administrators do not want to be responsible for stopping the computer so they set up Resource Monitors with ONLY notifications.  The problem with this is that notifications are just that...ONLY something to notify you.  What if you only have 1-2 Snowflake Account Administrators and they are not monitoring the emails or web notifications frequently enough and a Large to 6XL warehouse comes online without auto suspend enabled?

Another problem as well is that Snowflake Administrators set up Resource monitors BUT do not attach them to a warehouse. This is the same as having a guardrail but it's not activated.  Ugh!

Finally, we also see Resource Monitors get set up but then Account Administrators who do not enable their email or notifications correctly.

Snowflake Cost Anti-pattern #2

Another major Snowflake Cost anti-pattern is related to storage.  We do not see this nearly as often as #1 but it can also be a cost risk danger if you do not understand the impacts of enabling longer Time-Travel settings on Snowflake.  If you have many of your tables with time-travel set to 30,60, or 90 days, but you don't need that much time travel and will never use it, then you should change those configurations to lower time-travel settings.

There are similar potential problems with any table that is frequently updating and changing data.  These types of data tables will challenge Snowflake's architecture because every data change requires recreating micro-partitions.  So if you have a 90-day time-travel set and you are changing a table with large amounts of rows/sizes every few minutes or hours then it's going to add up as all of those immutable micro-partitions for every change are saved for 90 days.  Also, remember by default Snowflake forces a 7-day fail-safe of storage.  So if you have Time-Travel set to 90 days then it's 97 days of storage you will pay for.

Snowflake Cost Anti-pattern #3

Setting a Warehouse to "Never" auto suspend or high auto suspend settings.  If you set a Warehouse to never suspend then you are creating a never-ending spend on a warehouse until you manually or through code suspend it.  If the warehouse size is only XS then this isn't incredibly horrible but if it's a larger size the costs can grow very fast and you lose all of the value of Snowflake's consumption-based pricing.

 

Conclusion for Snowflake Cost Anti-patterns:

These are the top 3 most dangerous Snowflake Cost Anti-patterns we have come across.  There are many others but they are typically not as severe as these. These Snowflake Cost Anti-patterns are real and introduce your company and yourself to sizable cost risks.  This is why we recommend using Snoptimizer or enabling Snowflake Best Practice Cost Optimization by your team. Especially set up Resource Monitors IMMEDIATELY or at least in the same data as your Snowflake Account is provisioned.