MariaDB

MariaDB is an open-source relational database management system based on SQL – Structured Query Language. It is widely used for various purposes including web applications, data warehousing, and e-commerce due to its reliability, performance, and rich feature set.

Connection Properties

In a YAML file, the config section contains the following properties:

Connectorname: MariaDB
host: Hostname or IP address of the server
port: Server running port
username: Username
password: Password
database: Database
drivername: mysql+pymysql

Example Configuration

version: 1.0.1
encrypt_credentials: false
direct_target_import: false
union_all_tables: true
add_dbname_column: false
direct_load_to_destination: true
plugins:
  extractors:
    - name: MariaDB
      connectorname: MariaDB
      schemaname: 
      config:
        host: hostname
        port: 3306
        username: user
        database: database
        password: password
        drivername: mysql+pymysql
      properties: 
      metadata: 
      select:
        - tablename

Configure the Bold Data Hub to connect MariaDB

Click the Data Hub icon on the Navigation Pane.

MariaDB Hub- BoldBI

Click Add Pipeline and provide the new pipeline’s name.

MariaDB Hub- BoldBI

Select the newly created pipeline and add the MySQL template.

MariaDB Hub- BoldBI

Configuration Parameters

Parameters	Description
Host:	Specify the hostname of the MariaDB server.
Port:	Specify the port number of the MariaDB server (default is 3306).
Username:	Provide the username to authenticate with the MariaDB server.
Password:	Provide the password to authenticate with the MariaDB server.
Database:	Specify the name of the MariaDB database from which data will be extracted.
Driver Name:	Specify the driver name for connecting to MariaDB (e.g., mysql+pymysql).
Select:	Tablename(s): Specify the table name list to load tables from the MariaDB server.
Metadata: (Optional)	Replication Method: Specify the replication method for the table(s). Options include FULL or INCREMENTAL.
	Replication Key: Specify the replication key for incremental replication. This key helps in identifying new or updated records.
	Replication Value: Specify the replication value to start the incremental replication from a particular point.

Update the details required in the template and Click Save, choose the desired destination to save the pipeline.

MariaDB Hub- BoldBI

Creating a Pipeline in Bold Data Hub automatically creates a Data Source in Bold BI. The Bold BI Data Source is a live data source to the destination database used in Bold Data Hub. For more information on the relationship between Bold Data Hub Pipeline and the associated Data Sources in Bold BI , please refer to Relationship between Bold Data Hub Pipeline and Associated Data Sources in Bold BI

Schedule Bold Data Hub Job

To configure interval-based scheduling, click on the schedules tab and select the created pipeline and click on the schedule icon and configure it.

MariaDB - BoldBI

For on-demand refresh, click Run Now button.

MariaDB - BoldBI .

The Schedule history can be checked using the history option as well as logs.

MariaDB - BoldBI

Click on Logs to see if the run is completed and data source is created in Bold BI.

MariaDB - BoldBI

Click Edit DataSource Option to view the created tables.

Metadata Properties

In the metadata section, define the mode of data refresh. There are two modes: INCREMENTAL and FULL_TABLE. It only supports DateTime datatype columns.

INCREMENTAL

metadata:
  TableName:
    replication_method: INCREMENTAL
    replication_key: Column name
    replication_value: column value that data starts from

FULL_TABLE

This mode fetches data from the date column mentioned in the replication key from the start date as mentioned in the replication value. Once it is scheduled, the replication value is updated based on the interval_type and interval_value from the imported data. For ex set interval_type as ‘year’ and intervalue value as ‘1’.In first schedule, will fetch the record from Jan 1, 2000 to Dec 31, 2000. In next schedule, will fetch the record from Jan 1, 2001 to Dec 31, 2001 and so on.

metadata:
  TableName:
    replication_method: FULL_TABLE
    replication_key: Column name
    replication_value: column value that data starts from
    interval_type: days/hours/minutes/year/month
    interval_value: integer value to add in interval type

Example Configuration

INCREMENTAL

version: 1
encrypt_credentials: false
plugins:
  extractors:
    - name: tap_postgres
      connectorname: MariaDB
      config:
        host: Hostname or IP address of the server
        port: Server running port
        username: Username
        password: Password
        database: Database
        drivername: mysql+pymysql
      select:
        - TABLE1
        - TABLE2
      metadata:
        TABLE1:
          replication_method: INCREMENTAL
          replication_key: last_modified_on
          replication_value: 2023-07-19 00:00:00
        TABLE2:
          replication_method: INCREMENTAL
          replication_key: last_modified_on
          replication_value: 2023-07-19 00:00:00

FULL_TABLE

version: 1
encrypt_credentials: false
plugins:
  extractors:
    - name: tap_postgres
      connectorname: MariaDB
      config:
        host: Hostname or IP address of the server
        port: Server running port
        username: Username
        password: Password
        database: Database
        drivername: mysql+pymysql
      select:
        - TABLE1
        - TABLE2
      metadata:
        TABLE1:
          replication_method: FULL_TABLE
          replication_key: last_modified_on
          replication_value: 2023-07-19 00:00:00
          interval_type: days
          interval_value: 6
        TABLE2:
          replication_method: FULL_TABLE
          replication_key: last_modified_on
          replication_value: 2023-07-19 00:00:00
          interval_type: days
          interval_value: 6