Day: March 12, 2024

Dataops Role in the Automation of Automation

Posted on by Frank Bell
Dataops Role in the Automation of Automation

We provided in depth coverage of the concept of the Automation of Automation which we see accelerating Data Driven Business and Business Solutions to an entirely new level.  Besides Generative AI (GenAI) it really is the culmination of automations of software development over the last 20-30 years.  When I started this Data and Software game back in the nineties you had to develop pretty much everything.  Now I would say that 80-90% of software that we had to develop back then is now within libraries.  Most of which are open source.  

In addition to that coding has progressed and become incredibly easier by stable reliable software libraries and package managers, there has been the last 20 or so years of I hate to say it …. “Zapier” or “IFTTT” – If this then that solutions upon solutions.  There are hundreds of them now.  

I mean in the Snowflake ecosystem I have watched as far back as 2018 there were maybe 3-5 ELT type tools Stitch (Acquired by Talend), Fivetran (now Fivetran/HVR), and Matillion where really Stitch and Fivetran sort of pioneered the “Connect this to That … NO CODE solution”.  Now, there are at least 30 if not 50++ endless “Connect this to That” data solutions.  (actually I feel like there is 100+ but its a super fragmented market beyond Fivetran with Big Bucks Andresson Horowitz fueling the endless marketing on it.  We have even seen new upstarts like Omnata Data Superheroes come out with incredibly easy and low cost “Connect this to That Data Solutions”

Then to top off that, we had the Hightouches nad the Census AND many others come out with the “Data Activation” concepts which were actually I’m pretty sure built off of my original article around how to build a CDP on top of Snowflake.  When I wrote that it was really not thought to be possible and the tech wasn’t really there.  Now its totally possible and we see significant traction around “Don’t let your friends buy PURE CDPs” marketing and messaging.  (Which I typically agree with but hey – you have to evaluate what is the best business/technical solution for you.  What is even funnier though with this statement is that MANY of the CDP vendors I know have actually moved to having Snowflake as their backend.  

So with that fun start, let’s talk about Dataops Role in the Automation of Automation.  Quite a few years ago, Guy and Kent Graziano were telling me to check out this TrueDataOps stuff and I was like, wtf is this new term.  Like do we have to have 1 more new term.  I mean I still don’t think we need Data Mesh but that’s another story or “thought leadership argument”.

While I’m kind of slow to take on new concepts “sometimes” I now truly see how Dataops facilitates better Data Product Overall Solutions.  Without it, or without dataops.live at this point for our solutions, you can only go so far with your Automation of Automation of Quality and repeatable solutions 

Snowflake Solutions Center by Dataops.live

Posted on by Frank Bell

We wanted to cover one of the largest advances in overall Snowflake Dataops Technology by our partner, Dataops.live.

We have worked for many years with the Dataops.live Team and are incredibly excited to see them help Snowflake deploy by creating the overall Snowflake Solutions Center for powering hundreds and eventually thousands of Snowflake Industry and Solution demonstrations and proof of concepts.  This is huge step forward in leveraging innate Snowflake functionality of Zero-Copy Clone capabilities along with github repo type practices to truly enable Data Product building and testing automation.

Take a quick look at what Dataops is enabled through the new Snowflake Solutions Center.

Snowflake Solutions Center - It is really a Catalog of Solutions for Snowflake Sales Engineers

This is honestly amazing to see how much Snowflake Solutions and Features have grown over the years.  We believe this level of new Dataops automation really is what Snowflake needs to take it to another level of growth in the Data Cloud with quality and automation BUILT INTO Data Products.

Let's take a look at the Solutions Center (Powered by Dataops.live)

Snowflake Solutions Center Catalog of Demonstrations and Proof of Concepts

(This aligns with our viewpoint of the Automation of Automation that we have written extensively about)

The image of the Solutions Center below illustrates now how easily Snowflake Sales Engineers can select existing Demos or Proof of Concept Industry or Vertical based existing solutions.  What is REALLY important here is that this allows Snowflake Sales Engineers to have consistent deployment of solutions to prospects and customers.  It really "accelerates" the art of the possible and the automation of automation.

  • Search and Discover industry solutions - Within the image you will notice that now Sales Engineers can easily search on a catalog of Solutions
  • Deploy tailored solution instances for many different customers
  • HUGE!  Actually continuously monitor and validate solutions (Instead of the issues we would have with live version of the DCR 5, 5.5, 6.  Now you know what you are getting!)
  • In a way this is the Meta of Meta Data.
Snowflake Solutions Center
Snowflake Solutions Center

 

 

Solution Center Itemized "SOLUTION" Homepage and Description

The image below illustrates all the meta data related to a Solution.  This allows for repeatability with BOTH Code and Data for a quality tested and VALIDATED solution.

  • Each solution has both CUSTOMIZED and VALIDATED customer solution scripts
  • The SOLUTION itself can easily be created and tailored to the Snowflake Sales Engineers specific prospect or customer audience
  • This allows better understanding of the technical solutions available to BOTH Account Executives and Sales Engineers.
  • Each Solution is Repeatable and known and includes explanations and Step by Step Detailed Instructions so every level of Snowflake Sales Engineer can learn and demonstrate the unique solution.
Snowflake Solutions Center Homepage
Snowflake Solutions Center Homepage

 

Conclusion:

The Snowflake Solutions Center is honestly a Game Changer to us and we deploy the SAME TYPE of solutions for our customers with Dataops.live.
Get Started with a 15 minute review of how you can change your solutions to become truly Dataops Based Data Products!

We can show you how this new way of thinking and deploying data products will change your business and lives in certain industries!

  • Automate CICD predictably
  • Orchestrate your Data Solutions Data Products