Let me first state, I love using Snowflake, the technology itself. I fell in love with it at the beginning of 2018 when I realized how easily I could execute all of our Big Data Consulting Practice Solutions we had been doing for 18+ years so much easier than we had before. In the past with regular RDBMS solutions, Hadoop complex messes, and expensive MPP systems like Teradata, Netezza, and Exadata we often would run into scale challenges as the data grew.  Snowflake brought both ease of use and amazing scale to almost all of my big data consulting projects.

After the last three years of working on hundreds of Snowflake accounts though my team and I realized that if Snowflake Anti-patterns occur or if poor compute security practices are used that Snowflake Accounts are exposed to large cost risks, especially with compute costs. While Snowflake is an amazingly scalable cloud database and is the best cloud data warehouse I’ve used in the last 3+ years, the deployment of a Snowflake Account without proper settings and administration exposes a company to these cost risks.  We cover the Snowflake CREATE WAREHOUSE Default Settings in that other article but let’s say you actually used the Classic Console to create a new warehouse and used all the default settings and just created it.  Even if you didn’t run any query the cost for the standard settings would be 10 minutes * XL Warehouse (16 credits/hour) @ $3/credit.  It is only $8 for that 10 minutes but it was an $8 spent for nothing.  Let’s say though you had a rogue (or curious) trainee on a Snowflake Account that didn’t understand what they were doing and does the same thing they ONLY change the size to a 6XL.  Your 10-minute run-for-nothing cost exposure is 10 min * 6XL Warehouse (512 credits/hour) @ $3/credit.  Your account just spent $256 for 10 minutes of nothing.

Let’s take this to the extreme cost risk that many Snowflake Accounts are exposed to if they are not using automated tools like our Snowflake Cost Optimization and Cost Risk Minimization Service – Snoptimizer.

Snowflake Cost Risk Use Case 6XL – 1 cluster:

*Cost per hour @ $3/credit = 3 * 512 = $1536
*Cost per day @ $3/credit = $36,864

Snowflake Cost Risk Use Case 6XL – 5 clusters:

[we know this is a worst-case scenario on AWS/Snowflake and this one would be rare BUT without resource monitors and correct permissions it is possible]
*Cost per hour @ $3/credit = 3 * 512 = $7680
*Cost per day @ $3/credit = $184,320

Snowflake Cost Risk Use Case 6XL – 10 clusters:

[we know this is a worst-case scenario on AWS/Snowflake and this one would be rare BUT without resource monitors and correct permissions it is possible]
*Cost per hour @ $3/credit = 3 * 512 = $15,360
*Cost per day @ $3/credit = $368,640

As you can see, this is unreasonable exposure to cost risks.  If you are a Snowflake administrator, make sure you make appropriate changes to control costs and cost risk.  If you want an automated approach to Cost Risk Management that can be set up easily in a few hours then try our Snoptimizer Cost Risk Solution.

Snowflake Cost Risk Mitigation – Administration – ACCOUNT ADMIN – MUST DO

CREATE RESOURCE MONITOR "REMOVE_SNOWFLAKE_COST_RISK_EXAMPLE_RM" WITH CREDIT_QUOTA = 50 
FREQUENCY = 'DAILY', START_TIMESTAMP = 'IMMEDIATELY', END_TIMESTAMP = NULL
TRIGGERS 
ON 95 PERCENT DO SUSPEND 
ON 100 PERCENT DO SUSPEND_IMMEDIATE 
ON 80 PERCENT DO NOTIFY;

Community Conclusion:

Snowflake Data Cloud cost risk is a real issue that needs to be properly administrated.  While the Snowflake Data Cloud brings incredible scale and power to any analytical data processing need, the Data Cloud needs to be properly optimized and continuously monitored by a service like Snoptimizer.  Again, with great data processing power, comes great cost management responsibility.  If an administrator messes up somehow and gives access to a non-trained user to create that 6XL who doesn’t need it and doesn’t have the business budget for it then it can cost you BIG TIME.

Get Snoptimized now to avoid a data-driven cost catastrophe.  If a new warehouse appears, we have you covered!

The Snowflake Data Cloud brings incredible scale and power to any analytical data processing.  It is an amazing revolution to now be able to launch their T-shirt warehouse sizes from XS (1 virtual instance) to 6XL of raw 512 EC2 instances if you are running on AWS on 1 cluster.  While this is awesome computing power, you must set up your Snowflake account with resource monitors to control costs. Snowflake Resource Monitors are your main guardrails on controlling costs with the Snowflake Data Cloud.

Let’s show you how easy it is to set up your Snowflake cost guardrails so your costs don’t go beyond what you expect.

I recommend either hiring a full or part-time Snowflake administrator focused on cost optimization and database organization or using our  Snowflake Cost Optimization Tool – Snoptimizer.  Snoptimizer automates setting up resource monitors on your Snowflake account for each warehouse and tons of other cost optimizations and controls on your Snowflake Account.   Let’s dig into the ONLY true Snowflake Cost Risk Guardrails you have had for a while, Resource Monitors.

Resource Monitors – Your Snowflake Data Cloud Cost Guardrails

Resource Monitors are technically relatively easy to set up from the Snowflake Web GUI or command line. Even though setting up one Resource Monitor is relatively easy, it’s still easy to make incorrect assumptions and not have enough effective monitoring and suspending in place.  It is like having your guardrails not installed yet if you do not do the following which is too easy to do:

aaa

Finding out that Snowflake consumption-based pricing was so reasonable was game-changing for me and my consulting company.  We could finally provide scale to any analytical challenge and solution we needed to create.  This was never possible before.  I remember building predictive marketing tools and we often had to crunch large data sets we would often run into scaling challenges and have to spend tons of time and engineering effort to engineer for scale.

aaa

Conclusion:

Snowflake Guardrails in the form of Resource Monitors correctly set up are EXTREMELY IMPORTANT.  If you don’t. know if you have these in place, then what are you waiting for?  Activate your Snoptimizer now and be optimized in a few hours as well as continuous regular cost optimization monitoring. If a new warehouse appears, we have you covered!

I have been working with Snowflake since the beginning of 2018 and it has been one of the most fun and scaleable data solutions I have worked with in my 27++ year data engineer, data architect, entrepreneur, and data thought leader career. With great power (almost unlimited scale – only limited by Snowflakes’ allocation of Compute within an Availability Zone) also comes great responsibility.

Over the last 3 years, I have analyzed over 100 Snowflake accounts and about 90% of them were not fully optimized for Cloud data costs. This is why my team and I are so excited to have created Snoptimizer (the first AUTOMATED Snowflake Cost Optimization Service). Easily optimize your Snowflake Data Cloud Account here in a few hours. I think the reason why 90% of those accounts didn’t have resource monitors or regular optimizations in place was initially Snowflake is incredibly cost-effective and typically had massive savings, especially from on-prem migrations that we have done. Companies that do not optimize their Data Cloud Costs are missing out big time! This is why we designed Snoptimizer and I’m also SHARING my top 9 Snowflake Cost and Risk Optimizations below. Enjoy!

My Snowflake Cost Optimization and Cost Risk Reduction Best Practices

Best Practice #1 – Resource Monitors. One of the first things that Snoptimizer does is automate daily Resource Monitors at a warehouse level based on all the Snowflake Metadata database history and warehouse and Resource Monitor settings. This gets set almost immediately after you purchase Snoptimizer. BOOM. This has both huge cost risk reduction limits and guardrails for all of your warehouse compute.

You can join us to learn more about Cost Optimization at our webinar here!

Sethe e cost optimization automation code at the bottom of this post. Check out ResouMonitors’tors best practices

Best Practice #2 – Auto Suspend Setting Optimization.

Another optimization automated by Snoptimizer is looking at the Warehouse workloads and automating changes to the Auto Suspend settings. BOOM. Depending on workloads can automate additional cost savings with this automation for you.

Sethe e cost optimization automation code at the bottom of this post that helps you optimize auto_suspend. If you like videos, check out our Auto Suspend best practices

Best Practice #3 – Monitoring Cloud Services Consumption Optimization

Snoptimizer immediately looks to see if you have any large costs associated with Cloud Services on your Snowflake Account.

Sethe e cost optimization automation code at the bottom of this post. Check out our Cloud Services best practices.

Best Practice #4 – Regular Monitoring of Storage Usage Across your entire Snowflake Account.

Snoptimizer immediately reviews your Storage History over the past 60 days and looks at if you have any errant settings that have to overpay for Storage. Too often we find cossetting stings related to Time Travel and/or Snowflake Stages.

See the cost optimization automation code at the bottom of this post that helps you optimize Storage Costs. Al, so check out our videos on Monitoring Storage, Compute and Services, and Usage!

Best Practice #5 – Daily Monitoring of Warehouse Compute.

Besides just adding Resource Monitors that suspend Warehouses we also provide daily monitoring of Snowflake Warehouse consumption reporting daily spikes and anomalies or changes in rolling averages. Most accounts we come across do not have regular monitoring of warehouse usage on regular proactive settings. Check out WarehouseWarehouses’Warehouses best practices!

Best Practice #6 – Regular Monitoring of New Snowflake Services.

Besides monitoring compute warehouses, Snoptimizer also immediately starts monitoring consumption on all the existing and new (private preview and after) Cloud Services that incur costs from Automatic Clustering to Search Optimization to Materialized Views and all existing and new costs. This is a huge benefit to the automation of Snoptimizer. We are ALWAYS looking out for your cost consumption optimization and cost risk reduction! We are always there for you!

Conclusion:

Snowflake Data Cloud Cost Optimization and Snowflake

Data Cloud RISK Cost Reduction is so so important to do immediately on your Snowflake Account. What are you waiting for? Buy Snoptimizer now and be optimized in a few hours along with regular cost optimization monitoring. If a new warehouse appears, we have you covered!

Usage Based Pricing in the cloud in some ways is incredibly awesome.  For those of us who lived through the days of scaling Data Centers and waiting for hardware it still is pretty mind blowing.  When I first came across Snowflake at the beginning of 2018 I was amazed at how reasonable the cost was for any type of business, big or small.

The fact that we could even start an account off with 400 credits for 30 days for a Proof of Concept (POC) was just amazing to me.  Prior to this, when working with analytical databases that could scale like Exadata, Teradata, and Netezza our consulting company hesitated to introduce these more expensive solutions to our consulting clients which were small or medium size businesses because these solutions were out of their pricing comfort zone.

Snowflake Pricing Details – Basic

For those of you who are new to Snowflake, let’s start with Snowflake consumption pricing basics.  Snowflake overall is usage or consumption based pricing.  This means you ONLY pay for what you use.  Technically, you could actually setup a free Snowflake Trial Account and never pay anything because you never used any of the services that have a cost.  This is really awesome.

At the same time, as soon as your Snowflake Account is provisioned, you the administrator or person with their credit card associated with the account have extreme cost risk by default.  Our best practice is to ALWAYS enable Snowflake Cost Optimization with Snoptimizer immediately after provisioning a Snowflake Account.  If you decide against that then at the very least you should limit access or setup standard Snowflake Cost Minimization Guardrails and Snowflake Cost Optimization and Cost Minimization Best Practices.

Snowflake Compute Pricing – Basics

For most Snowflake Accounts, Snowflake Compute or the Snowflake Warehouses (which are virtual compute engines) is where 90% or more of your costs are.  The other four cost areas of Storage Costs, Cloud Computing Costs, Cloud Services Costs, and Data Transfer costs are typically easily 10% or less of the Snowflake SaaS costs per month.  Often the others can even be 1% or less unless you have certain use cases or end up mistakenly using Snowflake Cost Anti-patterns.

Snowflake Pricing Details – Advanced

For those of you who are more Snowflake savvy and already know the basics then let’s cover more advanced Snowflake pricing details.

Snowflake Compute Pricing – Advanced

*The prices stated above were standard prices taken from Snowflake’s pricing pages and pdf.  If you are a capacity customer with Snowflake I’m sure your Snowflake sales representative (also named Major Account Executive, Account Director, or Sales Director, etc.) will have worked with you to commit to some type of term of 1 to 3 years in their form of a Capacity Contract Customer.  The pricing typically varies dependent upon your spend and your commitment length pretty much like any SaaS vendor.

Over the last 3 years my teams and I have analyzed over 100 Snowflake accounts and about 95% of them were not fully optimized for both Cloud data costs and Cloud cost risk minimization.  This is why my team and I are so excited to have created Snoptimizer (the first AUTOMATED Snowflake Cost Optimization Service) – Easily optimize your Snowflake Data Cloud Account here in a few hours.

I think the reason why 90% of those accounts didn’t have resource monitors or regular optimizations in place was initially Snowflake is incredibly cost effective and typically had massive savings savings especially from on-prem migrations that we have done.  Companies though that do not optimize their Data Cloud Costs are missing out big time! This is why we designed Snoptimizer and I’m also SHARING my top 9 Snowflake Cost and Risk Optimizations below.

One of the first things that Snoptimizer does is automate daily Resource Monitors at a warehouse level based on all the Snowflake Metadata Database past history and Warehouses and Resource Monitor settings. This gets set almost immediately after you purchase Snoptimizer. BOOM. This has both huge cost risk reduction limits and guardrails for all of your warehouse compute.

See cost optimization best practices and automation code at Snowflake Solutions.

Conclusion:

I hope the Snowflake Basic and Advanced Pricing information above is useful to you on your Snowflake Journey.  For me, finding out that Snowflake consumption based pricing was so reasonable was game changing for both myself and my consulting company.  Prior to Snowflake, we couldn’t really provide compute scale with enough speed to many of the largest big analytical challenges and solutions our clients needed.  I remember building predictive marketing tools and we often had to crunch large data sets and we would often run into scaling challenges and have to spend tons of time and engineering effort to engineer for scale.

Snowflake is still my favorite Analytical Database since the beginning of 2018 but as I often present in my live training sessions and webinars, WITH GREAT POWER (practically unlimited computing scale) comes GREAT RESPONSIBILITY.

In this article, I’ll cover the TOP 3 Snowflake Cost Anti-patterns my Snowflake Cost Optimization team and I have come across after 3 years of analyzing hundreds of Snowflake Accounts.  I cannot begin to state how you should either invest in a PART or FULL-TIME Snowflake DBA focused on cost and organization or if you do not have that financial luxury then use our automated Snowflake Cost Optimization Service – Snoptimizer.  It’s incredibly easy and low-cost to set up Snoptimizer compared to having these anti-patterns manifest (which I know happens on too many Snowflake Accounts based on our review of hundreds of them).  If you do not have cost guardrails like Resource Monitors enabled your Snowflake Compute Consumption Risk is high and it’s honestly gross negligence as a data administration professional to allow this.

Let’s go through the TOP 3 Snowflake Cost Anti-patterns.

Top 3 Anti-Patterns

The first Snowflake anti-pattern is by far the worst and happens all too often.

Snowflake Cost Anti-pattern #1

Sadly, we all too often see that Resource Monitors are not set up correctly.  Some Snowflake accounts have them set up but do not have them set up at an effective grain.  One anti-pattern is that the administrator sets a large credit-sized resource manager for the overall account and no other resources managers.  It is okay to have some Resource Monitors cover the account or multiple warehouses but we highly recommend having 1 Resource Monitor set for Daily monitoring for each warehouse with auto-suspend enabled once a credit limit is reached.  This is currently the only real solution to having guardrails on your Snowflake consumption.  Without doing this you are exposing your company and Snowflake account to significant cost risk.

An additional anti-pattern related to Resource Monitors that we see too frequently is Administrators do not want to be responsible for stopping the computer so they set up Resource Monitors with ONLY notifications.  The problem with this is that notifications are just that…ONLY something to notify you.  What if you only have 1-2 Snowflake Account Administrators and they are not monitoring the emails or web notifications frequently enough and a Large to 6XL warehouse comes online without auto suspend enabled?

Another problem as well is that Snowflake Administrators set up Resource monitors BUT do not attach them to a warehouse. This is the same as having a guardrail but it’s not activated.  Ugh!

Finally, we also see Resource Monitors get set up but then Account Administrators who do not enable their email or notifications correctly.

Snowflake Cost Anti-pattern #2

Another major Snowflake Cost anti-pattern is related to storage.  We do not see this nearly as often as #1 but it can also be a cost risk danger if you do not understand the impacts of enabling longer Time-Travel settings on Snowflake.  If you have many of your tables with time-travel set to 30,60, or 90 days but you don’t need that much time travel and will never use it then you should change those configurations to lower time-travel settings.

There are similar potential problems with any table that is frequently updating and changing data.  These types of data tables will challenge Snowflake’s architecture because every data change requires recreating micro-partitions.  So if you have a 90-day time-travel set and you are changing a table with large amounts of rows/sizes every few minutes or hours then it’s going to add up as all of those immutable micro-partitions for every change are saved for 90 days.  Also, remember by default Snowflake forces a 7-day fail-safe of storage.  So if you have Time-Travel set to 90 days then it’s 97 days of storage you will pay for.

Snowflake Cost Anti-pattern #3

Setting a Warehouse to “Never” auto suspend or high auto suspend settings.  If you set a Warehouse to never suspend then you are creating a never-ending spend on a warehouse until you manually or through code suspend it.  If the warehouse size is only XS then this isn’t incredibly horrible but if it’s a larger size the costs can grow very fast and you lose all of the value of Snowflake’s consumption-based pricing.

Conclusion for Snowflake Cost Anti-patterns:

These are the top 3 most dangerous Snowflake Cost Anti-patterns we have come across.  There are many others but they are typically not as severe as these. These Snowflake Cost Anti-patterns are real and introduce your company and yourself to sizable cost risks.  This is why we recommend using Snoptimizer or enabling Snowflake Best Practice Cost Optimizations by your team or a consulting provider especially setting up Resource Monitors IMMEDIATELY or at least in the same data as your Snowflake Account is provisioned.