Have you been thinking about migrating your reporting to Microsoft Fabric or Snowflake but you have no idea what it would actually cost?
That’s the scariest thing, and the general thinking is that cloud reporting is expensive. But the good news is that it doesn’t have to be. I’ve built Fabric and Snowflake based solutions for multiple clients and for some of them the recurring monthly cost has been less than £100! That’s the Fabric/Snowflake running cost and the cost of using Azure Data Factory to extract the data from source and load it into Fabric/Snowflake.
Obviously, the more data you process, and the more complexity, the bigger the cost in a usage-based model, but these aren’t super small businesses with next to no data. These are businesses with millions of records being processed daily from multiple systems. So there’s every chance that your infrastructure costs won’t be as frightening as you might have anticipated.
In terms of keeping costs down there are a few key things to consider:
Only have the system running when it’s needed
Snowflake switches itself off and on automatically so that’s all handled out of the box, but for Fabric you’ll have to do it yourself. You’ll also want to chain your processes so you need the engine running for as short a period as possible. I use GitHub actions that orchestrate the whole end-to-end process of switching on Fabric, extracting the source data, transforming it, and refreshing the Power BI models, before switching Fabric off again at the end.
Go for the smallest (cheapest) capacity possible and only scale up if needed
For Snowflake that would be an X-SMALL warehouse and for Fabric it’s an F2 capacity. Sometimes you can work around the limitations of the smaller capacities to keep costs low as well. For example, the one restriction I’ve found with the workload I’m putting through the Fabric F2 capacity is that I can’t run notebooks in parallel. However, I’ve just switched those processes to run sequentially for now and they’re working fine, albeit the process just takes a bit longer. I could also look at running a higher capacity that would allow me to run in parallel and therefore take a shorter period of time, but I’ve not got to that point yet.
Only process what you need to process
It sounds obvious, but as you’re paying for usage you don’t want to be carrying any unnecessary weight. So only retain the amount of historical data you want to report on. Don’t build models that aren’t being used, and don’t extract data from a source system that isn’t part of the picture yet either.
I typically use Power BI as the visualisation tool, and the switching off of capacities mentioned above works really well when you only need to refresh your dashboards once or periodically throughout the day. Because you only need Fabric/Snowflake active until the Power BI model has refreshed, once that’s done all the data is available via Power BI, and the cloud engine can be switched off until the next scheduled refresh.
If you were to use Power BI then that also comes with a cost if you wanted users to access dashboards via the Power BI service. In this case a Power BI Pro license will set you back £10.80+VAT per user per month RRP. Although you can get your licenses at a 25% discount if you go via the right reseller.
The post Want to look at cloud reporting but not sure what the costs will be? appeared first on BI Design.
