Skip to main content

IOMETE Community Edition Deployment on AWS

You're about to install the IOMETE Lakehouse Platform Community Edition on AWS. The IOMETE Community Edition is the most generous Data Lakehouse Platform on the market. Enjoy the benefits of a data lakehouse platform with no restrictions on data volume, users, or queries.

IOMETE Community Edition is supported by the community. Feel free to join the IOMETE Community Discord Server for support and discussions.

Installing the IOMETE platform on AWS should take approximately 25 minutes
  • 15 minutes to create the infrastructure (EKS cluster, VPC, etc.) using Terraform
  • 5 minutes to install the IOMETE platform using Helm
Prerequisites:

Installation Steps​

Please clone the IOMETE Community Edition Deployment on AWS and follow the instructions below.

1. Terraform​

Reference: https://registry.terraform.io/modules/iomete/iomete-data-plane/aws/1.9.0

The AWS user that will be used to run Terraform needs to have the following permissions
  • Create IAM roles
  • Create S3 buckets
  • Create EKS clusters
  • EC2 full access

See Required Permissions to Deploy IOMETE on AWS for more details.

1. Check terraform/main.tf file, and update the values accordingly.​

Please also check the following specific settings

2. Run terraform​

cd terraform
terraform init --upgrade
terraform apply

Once terraform is done, get the EKS cluster config using the following command:

# Update AWS region and EKS cluster name accordingly.
aws eks update-kubeconfig --region <region> --name <cluster_name>

# Example
aws eks update-kubeconfig --region us-east-1 --name lakehouse

2. Prepare Database​

You can bring your own database, or use the one deployed by IOMETE.

info

This postgresql database is for testing purpose only. It is not recommended to use it in production. For production, please use your own database that is optimized for production use.

Add bitnami helm repo if you haven't done so.

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

Deploy postgresql database, and wait for it to be ready.

helm upgrade --install -n iomete-system \
  -f postgresql/postgresql-values.yaml \
   postgresql bitnami/postgresql

Wait for postgresql pod to be ready. It takes about ~1 minute

kubectl get pods -n iomete-system -l app.kubernetes.io/name=postgresql

3. Deploy IOMETE Data Plane​

The data-plane-values.yaml file houses the values for the IOMETE Data Plane helm chart.

tip

You don't need to alter anything in this file for a default installation. However, if you want to tailor the installation to your needs (perhaps you're using your own database and distinct credentials), then you can modify the values within this file.

Add, IOMETE helm repo:

helm repo add iomete https://chartmuseum.iomete.com
helm repo update

Deploy IOMETE Data Plane:

helm upgrade --install -n iomete-system iomete-data-plane \
  iomete/iomete-data-plane-community-aws \
  -f data-plane-values.yaml --version 1.9.3

Wait for IOMETE Data Plane pods to be ready. It takes about ~6 minutes to get everything ready in the first time installation.

kubectl get pods -n iomete-system

4. Configure ISTIO Ingress Gateway​

Apply the following configuration for an HTTP gateway:

kubectl -n istio-system apply -f istio-ingress/resources/gateway-http.yaml

How to use IOMETE Data Plane​

Once, IOMETE Data Plane is deployed, you can access the IOMETE Data Plane UI using the following command:

kubectl get svc istio-ingress -n istio-system

From the output, copy the EXTERNAL-IP value, and open it in your browser http://EXTERNAL-IP

info

For the first time use username and password from data-plane-values.yaml file adminUser section. Default values are admin and admin. On your first login, you will be asked to change the temporary password.

That's it! You've successfully set up IOMETE using Terraform. If you encounter any issues or have any questions please join the IOMETE Community Discord Server for support and discussions.

note

It's important to store your Terraform state file in a secure location to avoid losing it. If you lose the state file, you won't be able to manage the cluster with Terraform anymore. You can store the state file in a git repository, S3 bucket, or local folder. For more information on Terraform backends, please refer the FAQ below about saving terraform state in an external location.

Quickstart Resources​

Start using IOMETE with the following guides

Sync data from JDBC Databases

Read our guide on how to sync data from JDBC sources, like MySQL, PostgreSQL, and Oracle.

Querying Files in AWS S3

If you have data files in AWS S3, you can directly query them using the S3 connector.

Getting Started with Spark Jobs

Learn how to run Spark jobs on IOMETE.

Getting started with DBT

Learn how to use DBT with IOMETE.

ON THIS PAGE