The web connector fetches data from a REST API and supports nested JSON fields.
The config
section in a YAML file includes the following properties:
url: URL
username: Username
password: Password
type: HHTP method either GET/POST
headers: Header parameters
params: Query Paramaeter value
data: Data value
Bold ETL
icon on the Navigation Pane.Add Project
and provide the new project’s name.Web
template.Parameter | Description |
---|---|
URL: | Specify the URL of the web service from which data will be extracted. This URL should point to the API endpoint providing the required data. |
Authentication (Optional): | Username: If the web service requires basic authentication, provide the username here. Password: Corresponding password for basic authentication. |
Request Type: | Specify whether the connector should use the GET or POST method for making requests to the API. |
Parameters (Optional): | Provide parameters required for the API call in JSON format - key-value pairs. These parameters are typically used for filtering or customizing the data returned by the API. |
Data (Optional): | If the API call requires any additional data to be sent, specify it here in JSON format - key-value pairs. This data might include payload or request body parameters. |
Headers (Optional): | Include any custom headers required for the API call. These headers should be provided in JSON format, specifying key-value pairs such as content type or authentication tokens. |
Schedules
and select the created web
project.Run Now
.Schedule
option to schedule the refresh hourly.Edit DataSource
Option to view the created tables, such as the ‘web_result’ table.version: 1
encrypt_credentials: false
plugins:
extractors:
- name: web
#Json Response only supported
connectorname: Web
config:
url: url
# Basic Authentication
username: username
password: password
type: get
headers: { 'ContentType': 'application/json' }