Unlock hidden savings in your Snowflake environment. This guide reveals proven strategies to reduce compute and storage costs. Optimise your warehouses, monitor usage, and gain full control over your Snowflake costs.
Pay as you go models are ever growing, ultimately giving you more control over your costs. Snowflake operates an easy-to-understand costing model, with any additional Snowflake managed features easy to grasp. There are two main groups of cost; storage, and compute.
When considering the storage cost model in Snowflake, it's important to note that the pricing is based on the average number of terabytes stored throughout the month. For example, at the time of publishing this blog, for Snowflake running on Amazon Web Services (AWS) in London, Snowflake will charge you $42.00 per TB for on demand pricing, however, you can cut this down to $24 for any upfront payments. This can be a real advantage if you know how much data you must store or are looking to store in the near future.
To provide context, let us compare the storage costs of the three major cloud providers: Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure (Azure):
All the above exclude any free tiers of storage. If the required storage capacity is known in advance, it is generally more cost-effective to pay upfront, as this aligns with the pricing structure of the main hyperscalers. It is also worth noting that Snowflake offers varying rates depending on the chosen cloud provider and the specific region where the Snowflake environment is hosted
Snowflake have their own compute currency called ‘credits’. A credit is worth a value based on the Snowflake tier you require. If we look at AWS in London, then for the Standard version of Snowflake it will cost you $2.70 per credit. If you want the Enterprise level features it jumps to $4.00 per credit, then up to $5.40 per credit for the Business Critical version that includes key features for data compliance. The main question here is, what version do I really need?
To answer that question, you really need to know the difference between the versions of Snowflake. Enterprise provides some great additional features such as tagging, materialised views, row and column level data obfuscation, up to 90 days’ time travel on tables, multi-cluster ability, query acceleration and much more. But, do you really need these for your use case? The full edition feature matrix can be found here.
The next question is, what uses compute credits? It's simple as it is based on the size and time you are running Snowflake compute warehouses (also known as virtual warehouses). There are various levels of warehouses, from XS (extra small), through to 6XL.
Figure 1 - A Cost table of warehouses size to credits per hours in Snowflake.
The pricing table for the size of warehouses you can spin up is consistent. Warehouses consume double the credits for the next size up starting at one credit per hour (just $2.70 per hour) but warehouses won’t be on 24/7 like an on-premises database, they are only on as and when you need them. (Note these are charged per cluster within the warehouse, by default it's only one cluster for standard accounts). Snowflake will charge you sixty seconds minimum for spinning up a warehouse no matter how much is calculated within that time. Check here for full details.
There are some additional costs for extra services that Snowflake manages via their compute resources. There are things like materialised views (Enterprise only) and Snowpipe to name a couple. They are charged at their own credit per hour rate, but only when being used.
It's as simple as that. You don’t have to worry about any management of resources. This is all Snowflake's responsibility. No need to have database administrators looking after on-premises servers, worrying about keeping it updated and secure, backups, maintenance etc as all of this is done for you, empowering you to focus on getting your data to insight as quickly as possible.
It can be tricky to remain on top of costs when processes get added, so keeping track of those and monitoring them is essential to making sure you don’t foot a heavy bill at the end of the month. Snowflake has some great in-house tools to prevent overspend by enabling the set up of resource monitors at both warehouse and account level to make sure you never spend more than your budget.
There is limited visual insights given by Snowflake, but a whole load of data within the account monitoring schemas that Snowflake provides for users to keep tabs on their account usage. There are various views that will tell you how many credits you are consuming, by query, tasks, pipelines but they will need some analytics skills to pluck out the views that will be useful to you. Something we have done at Ometis is take these data points and built them into an easy to consume Qlik SaaS dashboard to help with our analysis.
Spending too much? Well luckily there are some quick wins available to start reducing your costs in just a matter of minutes.
These are just some of the high-level things you can do to keep cost down and prevent it from getting out of hand. There are many more activities you can do to get your costs down to a minimum. Our Snowflake monitoring dashboard can give you spend on your processes. Request a free demo to see how it works!
At Ometis we currently run all eight of our production systems through Snowflake via various pipeline methods and they cost in total less than $2.00 per day on a standard Snowflake account. We have implemented all the above methods as well as others to keep our costs as low as possible.
Monitoring your costs is not a one-time activity, but rather an ongoing activity. For example a file that has landed might be incorrect and you don’t have the right error handling. That could cause a warehouse to be on much longer than you anticipated and being able to spot errors and cost changing over time gives your more understanding of costs to be able to make impact.
So, here’s a myth for you, “using a larger warehouse will always cost me more”. Incorrect! In fact, there are cases where using larger warehouses could reduce cost by saving on time. A simple example is as follows.
Being able to process the data in a quicker time than consuming the higher rate of credits can really boost your savings.
This is not always the case, there are instances where you might need a large amount of parallel processing (file ingestion is a good example), whereas a single large and small warehouse will likely process the files in a similar time. So, using a multi cluster small warehouse to get the job done quickly with the smaller amount of credit usage is key.
This blog has delved into the world of snowflake pricing, empowering you to manage your costs effectively. Some of the key takeaways are: