Snowflake's support for external tables has seen significant updates since our last blog post on how to extend your Snowflake implementation with MinIO. External tables allow users of Snowflake to treat data in object storage like MinIO as a read-only table in Snowflake without migration. Snowflake's ongoing enhancements to their external table functionality clearly demonstrate the growing popularity of this feature. This makes sense given how critical it is for organizations to connect to and leverage their data wherever it is, whether it is on-prem, in the public clouds, or at the edge.

These enhancements make external tables in Snowflake even more performant and are designed to both simplify data management and enhance security. This blog post aims to outline some of the key enhancements that can help you build out your own external table functionality.

Vectorized Scanners for Parquet Files

One of the most notable improvements is the introduction of vectorized scanners for Parquet files. This new feature enhances scan performance by 8x and query performance by 2x compared to the previous non-vectorized scanner. By taking full advantage of Parquet’s columnar format, this scanner significantly boosts the efficiency of processing large datasets stored in object storage. This means faster data retrieval and processing, making your analytics tasks smoother and more efficient in Snowflake.

Simplified Masking Policies

Security and data protection are paramount to every data infrastructure. Acknowledging this, Snowflake has made strides in simplifying how sensitive data is managed. Administrators can now directly associate masking policies with virtual columns in external tables. Previously, this required creating views to mask sensitive columns, adding complexity to the process of managing sensitive information in external tables. This update streamlines data governance, ensuring that sensitive information is protected without the need for additional configurations.

Secure Data Sharing for Iceberg Tables

Snowflake now supports secure data sharing for Iceberg tables using shares. This allows users to share Iceberg tables directly, without having to create secure views first. This enhancement not only simplifies the sharing process but also ensures that data sharing is secure and compliant with organizational policies. It’s a significant step forward for organizations looking to leverage Iceberg tables in object storage for collaborative environments. It may be the push that many need to start building out their open table format data lakes using Iceberg and MinIO alongside Snowflake.

Hybrid Tables

The introduction of hybrid tables in Snowflake combines the best of both transactional and analytical databases. This new feature provides more flexibility in handling and integrating data, a use case that is particularly useful for external tables. Hybrid tables allow for more dynamic and versatile data management, supporting a broader range of use cases and improving overall data strategy.

Check for Updates

These updates make Snowflake's external tables more powerful and easier to use, which in turn helps organizations manage their data more effectively and securely. Whether you're dealing with large datasets, sensitive information, or complex data-sharing needs, these new features provide the tools necessary to enhance your data strategy.

For more detailed information on these and other updates, check out Snowflake's official documentation and release notes. By staying on top of these updates, you can ensure that your use of external tables in Snowflake remains optimized and secure, making the most of what this powerful data platform has to offer by leveraging your data wherever it is. Please reach out to as at hello@min.io or on our Slack channel with any questions.