Skip to main content

Lakehouse statuses

This page describes the IOMETE lakehouse cluster statuses

note

We need to understand the cluster components to understand the lakehouse cluster's statuses. Lakehouse cluster comprises of driver and executors.

  • Driver: is the gateway to accept and keep connections, plan executions, and orchestrate executors.
  • Executors: are the components that do the actual processing.

Statuses‚Äč

A lakehouse cluster can be one of the following statuses:

StatusDescription
StoppedCluster is completely turned off.

Driver: Not running
Executors: No executors running
Accepting connections: No
Cost charging: No
PendingCluster is newly started manually and waiting for resources for the driver

Driver: Not running. Waiting for the resources
Executors: No executors running
Accepting connections: No
Cost charging: No
SuspendedThis status happens when auto-scale is enabled on the cluster. When the cluster stays without any workload, it scales down and turns off the executors to prevent charging costs. Only the driver is running. When the driver gets a query, it starts executors to handle the processing

Driver: Running
Executors: No executors running
Accepting connections: Yes
Cost charging: Only for driver
Scaling-upThis status happens when auto-scale is enabled on the cluster. The cluster decides to scale up executors based on the workload needs up to the maximum of the cluster size.

Driver: Running
Executors: 0 or some already running, and new executors are being started
Accepting connections: Yes
Cost charging: For driver and already running executors
RunningCluster is running state

Driver: Running
Executors: Running
Accepting connections: Yes
Cost charging: For driver and running executors