Skip to main content

MySQL to IOMETE using Airbyte


This is an end-to-end guide about how to move MySQL data to IOMETE using Airbyte.

Destination Configuration​

Basically, the destination configuration form has two separate parts, IOMETE and Staging area configurations.

Fill in the configuration parameters with your own data according to the table below.

ParameterTypeRequiredNotes
Lakehouse Connection URLStringRequiredExample: airbyte://frankfurt.iomete.com/12312421312/default.
This is a combination of hostname, account number and lakehouse name. Check the lakehouse's connection details from the IOMETE console.
UsernameStringRequiredUsername to use to access IOMETE.
PasswordStringRequiredPassword associated with username.

IOMETE destination configuration first part

Fill the Staging area informations.

ParameterTypeRequiredNotes
Bucket NamestringName of the bucket to sync data into.
Bucket PathstringSubdirectory under the above bucket to sync the data into.
RegionstringSee documentation for all region codes.
Access Key IDstringAWS/Minio credential.
Secret Access KeystringAWS/Minio credential.
S3 Filename patternstringThe pattern allows you to set the file-name format for the S3 staging file(s), next placeholders combinations are currently supported: {date}, {date:yyyy_MM}, {timestamp}, {timestamp:millis}, {timestamp:micros}, {part_number}, {sync_id}, {format_extension}. Please, don't use empty space and not supportable placeholders, as they won't recognized.
Purge Staging DatabooleanThe connector creates staging files and tables on S3. By default, they will be purged when the data sync is complete. Set it to false for debugging purposes.

Staging area configuration

Source Configuration​

Let's configurate MySQL database source.

πŸ’‘ We will be using a publicly accessible employees database instance that contains the Employees Sample Database.

Here are the details of employees public database:

Host: iomete-tutorial.cetmtjnompsh.eu-central-1.rds.amazonaws.com
Port: 3306
Username: tutorial_user
Password: 9tVDVEKp

πŸ’‘ For connection to Amazon RDS MySQL or MariaDB, add enabledTLSProtocols=TLSv1.2 in the JDBC parameters.

MySQL source configuration

Synchronization​

  • Set replication parameters.
    Connection replication configuration

  • Start sync and monitor synchronization status from history.
    Successful sync result

Synchronization completed successful. Let’s check migrated data in the IOMETE console.

Editor​

The employees database has been created and the current_dept_emp table has been successfully migrated.

Migration result in the IOMETE SQL editor