Strategies to Optimize Snowflake Cost

Reducing Cost and Improving Performance in Snowflake

Introduction

  • Overview of the importance of optimizing Snowflake cost
  • Introduction to the speakers and their expertise
  • Agenda for the presentation
  • Highlighting key accomplishments in reducing cost

Snowflake Deployment at Insta

  • Overview of Snowflake deployment at Insta
  • Number of users and warehouses
  • Queries per month and data usage
  • Access control settings for data security

Accomplishments and Cost Reduction

  • Summary of cost reduction achievements
  • Reducing storage cost by 33%
  • Optimizing query runtime and performance
  • Example of query runtime reduction

Observe, Analyze, and Optimize

  • Importance of observation and analysis for cost optimization
  • Query profiling and metrics
  • Query fingerprinting and tagging
  • Accountability and ownership

Analyzing Query Profile

  • Understanding query profile for optimization
  • Identifying expensive nodes and aggregates
  • Optimizing table scans through filtering and clustering
  • Monitoring caching and spilling

Optimizing Warehouse Size

  • Determining optimal warehouse size
  • Factors to consider for scaling
  • Importance of minimizing byte scan
  • Balance between vertical and horizontal scaling

Storage and Cloning Optimization

  • Maximizing storage efficiency
  • Using transient and temporary tables
  • Optimizing time travel and cloning
  • Monitoring and managing retained by clone bytes

Conclusion

  • Summarizing key learnings and takeaways
  • Importance of early cost detection and optimization
  • Role of accountable individuals in driving optimization
  • Appreciation for the audience's time and attention