Cost Governance on Snowflake

Cost Governance on Snowflake.  In this article I will break down what you can do for Cost Governance on Snowflake as of July 2022.  Snowflake is making decent strides in this area, even though I still think you need to use a full Snowflake Cost Optimization service like Snoptimizer™ or Nadalytics.  The reality is that Snowflake still derives the bulk of its revenue from consumption based services (their compute warehouses….) and while they have amazing NPS scores there are still many many pitfalls around costs on Snowflake.  Quick history of what was available before Summit 2022 announcements.

Before Snowflake Summit 2022, Cost Governance in Snowflake was honestly pretty weak.  It only had the following GUI and optimization tools:

  1. Snowflake’s Standard Classic Console.  Daily Summary – Very limited and ONLY available to very limited ROLES!
  2. Snowsight – Usage Views – More granularity of costs but there are problems with some default views and bugs.  Again, by default it is locked down to certain roles.  Personally I do not understand why costs at an organization should only be viewable by ACCOUNTADMIN Role by default.
  3. Third Party Optimization Tools
    1. Nadlytics
    2. Snoptimizer™
  4. Third Party “Reactive” Reporting Tools (from all the Snowflake Health Check Consulting Engagements I’ve done, this was the most common set of tools for Cost Governance on Snowflake).
    1. Sigma Computing Cost and Usage
    2. Looker Snowflake Cost and Usage
    3. Tableau Snowflake Cost and Usage
    4. Many other smaller fragmented brands with “reactive” reporting around costs.  The problem with reactive reporting is … if something REALLY goes bad like a long running query where there is NO Resource monitor OR the resource monitor is ONLY set to kick in when the query ends which by default could actually be 48 hours….  If this happens $1000s or $10,000+++ of dollars can be spent within a day easily with no true Data to Value provided!

***Also, I have documented for years how compute default settings all over your normal Snowflake Account are at odds with Cost Optimization such as these [most of which have been there since 2018 but some were introduced with our friend Snowsight…]

* Snowflake Warehouse Default Settings (Why is the default set to an XLARGE?)
* Similarly, why is Auto Suspend set by default to 10 minutes?
* Default Usage View on Snowsight – Ugh..

*And those are only some of the major ones.  There are all sorts of cost pitfalls…  My fellow Data Superhero Slim outlines them here:

After Snowflake Summit 2022, these major Cost Governance announcements were provided..

For full recap – Snowflake Summit Recap] – This has been a great advance especially in Cost Governance on Snowflake Capabilities as well as flexibility.  [we just have to wait until it all gets into GA !!]

#1. New Resource Groups concept announced where you can combine all sorts of snowflake data objects to monitor their resource usage. [this is huge since Resource Monitors were pretty primitive]

#2. Concept of Budgets that you can track against. [both Resource Groups and Budgets coming into Private Preview in the next few weeks]

*FRANK:  this is huge and in hindsight you kind of ask yourself… why wasn’t this in the core product to begin with?

#3. More Usage Metrics are being made available as well for SnowPros like us to use or Monitoring tools. This is important since many enterprise businesses were looking for this.

*This is pretty cool that “finally” you can actually have Organization Level Reporting.  (Even though as of this date, I’m not aware of any of the Sigma, Looker, or Tableau interfaces adding this.). Again though while this is better than nothing it has real latency reactive vs. proactive.

Do you want to optimize your Snowflake Account, Try Snoptimizer™ Assessment for Free.