Web

The web connector fetches data from a REST API and supports nested JSON fields.

Limitations

  1. Currently supports Json response.

Connection Properties

The config section in a YAML file includes the following properties:

url: URL
username: Username
password: Password
type: HHTP method either GET/POST
headers: Header parameters
params: Query Paramaeter value
data: Data value

Configure the Bold Data Hub to connect Web

  1. Click the Data Hub icon on the Navigation Pane.

Web Data Hub - BoldBI

  1. Click Add Pipeline and provide the new pipeline’s name.

Web Data Hub - BoldBI

  1. Select the newly created pipeline and add the Web template.

Web Data Hub - BoldBI

Configuration Parameters

Parameter Description
URL: Specify the URL of the web service from which data will be extracted. This URL should point to the API endpoint providing the required data.
Authentication (Optional): Username: If the web service requires basic authentication, provide the username here. Password: Corresponding password for basic authentication.
Request Type: Specify whether the connector should use the GET or POST method for making requests to the API.
Parameters (Optional): Provide parameters required for the API call in JSON format - key-value pairs. These parameters are typically used for filtering or customizing the data returned by the API.
Data (Optional): If the API call requires any additional data to be sent, specify it here in JSON format - key-value pairs. This data might include payload or request body parameters.
Headers (Optional): Include any custom headers required for the API call. These headers should be provided in JSON format, specifying key-value pairs such as content type or authentication tokens.
  1. Update the details required in the template and Click Save, choose the desired destination to save the pipeline.

Web Data Hub - BoldBI

  1. Creating a Pipeline in Bold Data Hub automatically creates a Data Source in Bold BI. The Bold BI Data Source is a live data source to the destination database used in Bold Data Hub. For more information on the relationship between Bold Data Hub Pipeline and the associated Data Sources in Bold BI , please refer to Relationship between Bold Data Hub Pipeline and Associated Data Sources in Bold BI

Schedule Bold Data Hub Job

  1. To configure interval-based scheduling, click on the schedules tab and select the created pipeline and click on the schedule icon and configure it.

Web - BoldBI

Web - BoldBI

  1. For on-demand refresh, click Run Now button.

Web - BoldBI.

  1. The Schedule history can be checked using the history option as well as logs.

Web - BoldBI

  1. Click on Logs to see if the run is completed and data source is created in Bold BI.

Web - BoldBI

  1. Click Edit DataSource Option to view the created tables.

Example Configuration

version: 1
encrypt_credentials: false
plugins:
  extractors:
    - name: web
      #Json Response only supported
      connectorname: Web
      config:
        url: url
        # Basic Authentication
        username: username 
        password: password
        type: get 
        headers: { 'ContentType': 'application/json' }