Deploy MinIO and Trino with Kubernetes

AJ AJ on AI/ML 8 May 2024

Trino (formerly Presto) is a SQL query engine - not a SQL database. Trino has eschewed the storage component of the SQL database to focus on only one thing - ultra-fast SQL querying. Trino is just a query engine and does not store data. Instead, Trino interacts with various databases or directly on object storage. Trino parses and analyzes the SQL query you pass in, creates and optimizes a query execution plan that includes the data sources, and then schedules worker nodes that are able to intelligently query the underlying databases they connect to.

MinIO is frequently used to store data from AI/ML Workloads, Datalakes to lake houses whether it be Dremio, Hive, Hudi, StarRocks or any of the other dozen or so great AI/ML tools solutions. MinIO is more efficient when used as the primary storage layer, which decreases total cost of ownership for the data stored, plus you get the added benefits of writing data to MinIO that is immutable, versioned and protected by erasure coding. In addition, saving data to MinIO object storage makes it available to other cloud native machine learning and analytics applications.

In this tutorial, we'll deploy a cohesive system that allows distributed SQL querying across large datasets stored in Minio, with Trino leveraging metadata from Hive Metastore and table schemas from Redis.

Components

Here are the different components and what they do in our setup process we’ll go through next.

Minio: Minio can be used to store large datasets, like the ones typically analyzed by Trino.
Hive Metastore: Hive Metastore is a service that stores metadata for Hive tables (like table schema). Trino can use Hive Metastore to determine the schema of tables when querying datasets.
PostgreSQL for Hive Metastore: This is the database backend for the Hive Metastore. It's where the metadata is actually stored.
Redis: In this setup, Redis for storing table schemas for Trino.
Trino: Trino (formerly known as Presto) is a high-performance, distributed SQL query engine. It allows querying data across various data sources like SQL databases, NoSQL databases, and even object storage like Minio.

Prerequisites

Before starting, ensure you have the necessary tools installed for managing your Kubernetes cluster:

kubectl: The primary command-line tool for managing Kubernetes clusters. You can use it to inspect, manipulate, and administer cluster resources.
helm: A package manager for Kubernetes. Helm allows you to deploy, upgrade, and manage applications within your cluster using pre-defined charts.

Repository Cloning

To access the resources needed for deploying Trino on Kubernetes, clone the specific GitHub repository and navigate to the appropriate directory:

git clone https://github.com/minio/blog-assets.git

cd blog-assets/trino-on-kubernetes

Kubernetes Namespace Creation

Namespaces in Kubernetes provide isolated environments for applications. Create a new namespace for Trino to encapsulate its deployment:

kubectl create namespace trino --dry-run=client -o yaml | kubectl apply -f -

Redis Table Definition Secret

Redis will store table schemas used by Trino. Secure these schemas with a Kubernetes Secret. The following command creates a generic secret, sourcing data from a JSON file:

kubectl create secret generic redis-table-definition --from-file=redis/test.json -n trino || true

Add Helm Repositories

Helm repositories provide pre-packaged charts that simplify application deployment. Add the Bitnami and Trino repositories to your Helm configuration:

helm repo add bitnami https://charts.bitnami.com/bitnami || true

helm repo add trino https://trinodb.github.io/charts/ || true

Deploy MinIO for Data Storage

Initialize MinIO

Prepare MinIO within the Trino namespace.

kubectl minio init -n trino

Create MinIO Tenant

Set up a multi-tenant architecture for data storage. The example below creates a tenant named “tenant-1” with four servers, four storage volumes, and a capacity of 4 GiB:

kubectl minio tenant create tenant-1 --servers 4 --volumes 4 --capacity 4Gi -n trino

Set Up Hive Metastore

Trino utilizes Hive Metastore to store table metadata. Deploy PostgreSQL to manage the metadata, then set up the Hive Metastore:

Install PostgreSQL

helm upgrade --install hive-metastore-postgresql bitnami/postgresql -n trino -f hive-metastore-postgresql/values.yaml

Deploy Hive Metastore

Use a preconfigured Helm chart to deploy Hive Metastore within the Trino namespace:

helm upgrade --install my-hive-metastore -n trino -f hive-metastore/values.yaml ./charts/hive-metastore

Deploying MinIO and Trino with Kubernetes

Trino and MinIO create a powerful combination for distributed SQL querying across large datasets. Follow these steps to deploy and configure the system.

Deploy Redis to Store Table Schemas

Redis is a high-speed, in-memory data store used to hold Trino table schemas for enhanced query performance. Deploy it in the Trino namespace using a Helm chart:

helm upgrade --install my-redis bitnami/redis -n trino -f redis/values.yaml

Deploy Trino

Deploy Trino as the distributed SQL query engine that will connect to MinIO and other data sources:

helm upgrade --install my-trino trino/trino --version 0.7.0 --namespace trino -f trino/values.yaml

Verify Deployment

Confirm that all components are running correctly by listing the pods in the Trino namespace:

kubectl get pods -n trino

Security Review and Adjustments

Review and adjust security settings as needed. To disable SSL certificate validation for S3 connections, update the additionalCatalogs section of the values.yaml file with the following property:

hive.s3.ssl.enabled=false

Testing

Port Forward to MinIO Tenant Service

Port forward to the MinIO service of the tenant, enabling local access:

kubectl port-forward svc/minio -n trino 9443:443

Create Alias and Bucket for Trino

1. Create Alias: Establish an alias for the tenant using the credentials from the MinIO deployment:

mc alias set my-minio https://localhost:9443/ minio_access_key minio_secret_key --insecure

2. Create Bucket: Create a new bucket that Trino will use

mc mb my-minio/tiny --insecure

Access Trino UI via Port Forward

1. Obtain Pod Name: Retrieve the name of the Trino coordinator pod:

export POD_NAME=$(kubectl get pods --namespace trino -l "app=trino,release=my-trino,component=coordinator" -o jsonpath="{.items[0].metadata.name}")

2. Port Forward: Forward local port 8080 to the coordinator pod:

kubectl port-forward $POD_NAME 8080:8080

3. Access UI: Use the Trino UI in your browser by visiting http://127.0.0.1:8080.

Query Trino via CLI

Access the Trino coordinator pod and start querying via the command line:

kubectl exec -it deploy/my-trino-coordinator -n trino -- trino

SHOW CATALOGS;

SHOW SCHEMAS IN minio;

Schema

--------------------

default

information_schema

CREATE SCHEMA minio.tiny

WITH (location = 's3a://tiny/');

CREATE TABLE minio.tiny.customer

WITH (

format = 'ORC',

external_location = 's3a://tiny/customer/'

)

AS SELECT * FROM tpch.tiny.customer;

SELECT * FROM minio.tiny.customer LIMIT 50;

SHOW SCHEMAS IN minio;

Schema

--------------------

default

information_schema

tiny

(3 rows)

Confirm Data in MinIO Bucket

After creating the bucket, confirm that the data is stored in MinIO by listing the contents with the mc command-line tool. Use the following command:

mc ls my-minio/tiny --insecure

It's as simple as that!

Final Thoughts

When troubleshooting configuration issues, especially those concerning security, thoroughly review the values.yaml files for each component to ensure proper settings.

Trino stands out for its ability to optimize queries across various data layers, whether specialized databases or object storage. It aims to minimize data transfer by pushing down queries to retrieve only the essential data required. This enables Trino to join datasets from different sources, perform further processing, or return precise results efficiently.

MinIO pairs exceptionally well with Trino due to its industry-leading scalability and performance. With the ability to handle significant workloads across AI/ML and analytics, MinIO effortlessly supports Trino queries and beyond. In recent benchmarks, MinIO achieved an impressive 325 GiB/s (349 GB/s) for GET operations and 165 GiB/s (177 GB/s) for PUT operations across just 32 nodes. This remarkable performance ensures that data stored in MinIO remains readily accessible, making MinIO a reliable and high-performing choice for Trino without becoming a bottleneck.

If you have any questions on MinIO and Trino be sure to reach out to us on Slack!