Skip to main content

IOMETE AWS Changelog

Release notes for IOMETE installation module.

We hope these improvements will make your experience with IOMETE even better. As always, please let us know if you have any feedback or suggestions for further enhancements.

info

IOMETE Community Edition is supported by the community. Feel free to join the IOMETE Community Discord Server for support and discussions.

Release 1.11.2:​

  • Pod Security Context: Added securityContext settings to all pods, ensuring that all resources are running as non-root users.
    • Note that Spark images requires root user only if using default hostPath volumes, which can be adjusted from Volume Settings in the IOMETE Console by choosing Dynamic PVC.
    • Currently, only Apache Ranger requires root user permissions, which will be addressed in a future release.
  • Eliminated Assets Bucket: We have eliminated additional bucket created/used for assets, which was previously required for storing assets like 'sql query results', 'spark history data', etc. Now, assets are stored in the same bucket as the data lake.
  • Role/Permissions Settings: Redesigned and improved role/permissions settings for better user management and access control.
    • Updated permissions page desing for better user experience.
    • IAM Settings now controlled by two permissions: View and Manage. Previously each component of IAM settings (Users / Groups / Roles) managed by separate permissions.
    • Data Security / Data Catalog: List and View actions combined into a single View permission. Create and Manage actions combined into a single Manage permission.
    • Added new permission for Spark Settings.
    • Added new permission for Data Plane Settings.
  • Logging Source: Users now have the flexibility to choose their logging source. Previously, only Loki was supported. Now, users can choose between Kubernetes, Loki, and Elasticsearch.
    • Kubernetes: Displays logs only for active/current resources and does not include historical data (e.g., logs for terminated resources are not available).
    • Elasticsearch: Requires an SSL connection and an API key for authentication.
  • Iceberg upgraded to 1.5.
  • Ranger now operates as non-root with securityContext configuration.
  • Added Requests and Limits to all IOMETE Kubernetes resources.
  • Iceberg: Fixed the issue with reading parquet time_micros fields defined as bigint in the schema by adding BigIntVector allocation in VectorizedArrowReader.

Release 1.9.0:​

  • LDAP Configuration: The IOMETE Console now supports LDAP configuration, offering a more streamlined and secure way to manage user access and authentication.
  • Advanced Spark Catalog Options: Additional configuration capabilities have been introduced for Spark Catalog registration, providing more flexibility and control to users.
  • Custom Node Pool Configuration: Users can now tailor node pool settings to their specific needs, allowing for optimized resource allocation and usage.
  • Resource Tagging: The ability to add tags to various resources within the IOMETE Console improves resource management, making it easier to categorize and track Lakehouse, Spark Connect, Spark Jobs, etc.
  • Spark Connect Clusters - Connection Stability: Issues affecting Spark Connect Cluster connections, particularly from Jupyter notebooks and Windows environments, have been addressed.
  • Spark Operator Webhook Configuration: This can now be deployed separately. This also enabled removing ClusterRole requirements from the base Helm chart.
  • Removed Cluster Role: The Cluster Role previously required (to control MutatingAdmissionWebhookConfiguration) for the spark-operator is removed from the base Helm chart.
  • User Permissions: Spark clusters previously running under root user permissions are now configured to operate under non-root user permissions.
  • Grafana Metrics: The hardcoded namespace issue in the Grafana monitoring dashboard has been resolved.
  • Bug Fixes: Various minor issues from the last release, including Spark URL problems, have been resolved.
  • Volume Settings: Users can now adjust volume settings directly from the IOMETE Console. This includes selecting volume configurations for each Spark resource, with options for hostpath, dynamic PVC, and size specifications on a per-resource basis.

Release 1.6.0: The first Community Edition for AWS 💰​

🎉 Announcement

We're excited to announce the first community edition of IOMETE for AWS. This version is free to use and includes all the major features of the Enterprise version. Community Edition is self-hosted and self-supported. For fully-managed support, please contact us at support@iomete.com

  • IOMETE Data Lakehouse Platform running on your AWS account
  • No limits on the number of users, data size, queries, or data sources
  • Deploy and enjoy the modern data lakehouse platform withing just 20 minutes

ON THIS PAGE