witboost Configurations
A typical witboost configuration file is a YAML file containing settings for both frontend and backend, and of different modules altogether (Builder and Marketplace). This file can be stored inside the values file for the helm chart described above, or can be passed directly to the process as an argument.
In this section we will report an overall description of each configuration entry.
N.B.: All private values that are critical from a security standpoint must be stored in an external secret manager (e.g. Hashicorp Vault). All this kind of entries will be highlighted by the SECRET notation.
app:
# Frontend base URL. This is the public endpoint that users will connect to. In all our deployments this must
# be the same as "backend.baseUrl"
baseUrl: https://my.witboost.deployment.com
# Application title. Usually you can just set it to "witboost"
title: witboost
backend:
# Backend base URL. This is the public endpoint that will expose the APIs. In all our deployments this must
# be the same as "app.baseUrl"
baseUrl: https://my.witboost.deployment.com
listen:
# HTTP port that the backend will be listening to. The default value is 7007
port: 7007
auth:
keys:
# [SECRET] Authorization secret for backend-to-backend communications. The key can be any base64 encoded
# secret. The following command can be used to generate such a key in a terminal:
# > node -p 'require("crypto").randomBytes(24).toString("base64")'
- secret: my_backend_plugins_auth_secret
cors:
# Backend base URL used to prevent Cross Origins errors. This is usually the same as "app.baseUrl"
origin: https://my.witboost.deployment.com
# This is the database configuration used by both Builder and Marketplace modules.
# Each will have its separate database configuration, but they can be the same in case you are relying on a
# single external database
database:
plugin:
# Database configuration for the Marketplace module
marketplace:
# The only supported RDBMS right now is postgres, so this will be fixed to "pg"
client: pg
connection:
# Database name for the Marketplace module, usually can be set to "marketplace"
database: marketplace
# Database remote hostname
host: witboost.database.com
# Database user
user: meshdev
# [SECRET] Database password
password: my_postgresql_password
# SSL configuration for the database. It is enough to set rejectUnauthorized flag to false
ssl:
rejectUnauthorized: false
# Database configuration for the Builder module. Also in this case the only supported RDBMS is postgres
client: pg
connection:
# Database name for the Builder module, usually can be set to "builder"
database: builder
# Database remote hostname
host: witboost.database.com
# Database user
user: meshdev
# [SECRET] Database password
password: my_postgresql_password
# SSL configuration for the database. It is enough to set rejectUnauthorized flag to false
ssl:
rejectUnauthorized: false
# Configurations for the management of user configurations that can be set in the settings panel.
# Among these configurations an user can set his/her own tokens for integrations with third-party services.
userConfig:
secrets:
# [SECRET] Symmetric key for user configurations encryption. The key can be any 32 bytes hexadecimal key.
# The following command can be used to generate such a key in a terminal:
# > node -p 'require("crypto").randomBytes(32).toString("hex")'
key: my_config_key
# [SECRET] Input vector for user configurations encryption. The key can be any 16 bytes hexadecimal key.
# The following command can be used to generate such a key in a terminal:
# > node -p 'require("crypto").randomBytes(16).toString("hex")'
iv: my_config_iv
# This is the section to configure action handlers associated to different hooks inside witboost.
# To have a better understanding of this feature, please refer to the "Action Handlers" chapter.
# Here is reported the default behavior for access requests
actionHandlers:
# Selector for the action handler of the "access request" operation
accessRequest:
# Public Hasura endpoint that will be used to handle hooks
graphQLUrl: https://witboost.hasura.com/v1/graphql
# Update ACL URL that is invoked when an access request operation is successfully completed.
# The URL should contain the remote Provisioning Coordinator URL with the correct operation's path
updateAclUrl: http://datamesh-provisioningcoordinator.datamesh-provisioningcoordinator:8088/datamesh.provisioningcoordinator/1.0/updateacl
# List of hooks that will be invoked to accept or reject the operation
hooks:
# The default hook is notification-based: a notification is sent to the owner for approval
- type: notificationAccessControl
# Public Hasura endpoint that will be used to validate the hook's answer
graphQLUrl: https://witboost.hasura.com/v1/graphql
# Auditing configurations
audit:
# By setting this to false, auditing will not be performed neither for REST nor for GraphQL APIs
enabled: true
# For REST calls, only the methods listed here will be audited
methods:
- GET
- POST
- PUT
- PATCH
- DELETE
# Authentication configurations, to select a remote identity provider
auth:
# Autologout feature configuration
autologout:
# Enable/disable the autologout feature
enabled: true
# (optional) Number of minutes after which the inactive user is logged out automatically.
# Default is 60 minutes (1 hour)
idleTimeoutMinutes: 60
# (optional) Number of seconds before the idle timeout where the user will be asked if it's still active.
# A small window will be shown.
# Default is 10 seconds.
# Set to 0 seconds to disable the prompt.
promptBeforeIdleSeconds: 10
# (optional) Enable/disable the usage of worker thread timers instead of main thread timers.
# Default is true.
# If you experience some browser incompatibility, you may try to set this to false.
useWorkerTimers: true
# (optional) List of events that the autologout event listener will catch to detect if the user is active
# These events are standard DOM events. Take a look at the documentation to see supported values.
events:
- eventNameA
- eventNameB
environment: development
providers:
# By setting a provider, witboost will use it to authenticate the user during the login operation.
# In here an example of definition for a Microsoft Active Directory provider is displayed.
# You can check the different providers requirements directly from the Backstage official documentation
microsoft:
development:
clientId: 11111111-ffff-aaaa-8888-999999999999
clientSecret: my_azure_ad_client_secret
tenantId: 22222222-eeee-4444-cccc-000000000000
# Permission configurations for the RBAC plugin
permission:
# By setting this to false, no permission rules are enforced. This means that each user enabled to perform
# operations on the target systems will not be blocked by witboost in case they do not have the right permissions
enabled: true
# Integration configurations with repository managers.
# You can check the different providers requirements directly from the Backstage official documentation
integrations:
# Set to true to tell Builder to use users tokens when authorizing write operations on the target Git provider
usePersonalToken: true
# Here an example of how a Gitlab integration can be configured
gitlab:
# Remote host of the repository manager
- host: gitlab.com
# [SECRET] Token that witboost will use to authenticate on the target repository manager. This token is used by
# witboost backend to read the entities stored in the remote repositories, so it just needs to have read scopes.
token: my_gitlab_token
# Configuration related to in-app and email notifications, customizable by event
notifications:
# email configuration
emailConfig:
# name of the email sender that will be shown to the recipients
senderName: Witboost
# configuration related to the html email template
html:
# the app name that will be displayed inside the email
appName: Witboost
# path of the logo that will be displayed at the top of the email
# the file must be under the wb-notification-backend plugin folder
# the default witboost logo is available using the path shown below
logoPath: 'src/resources/logo.svg'
# Optional field about the footer information that will be displayed at the bottom of the email, inside the footer
# you can use html tags inside the string, as shown below for the default value
footer: '<strong>witboost</strong> is a product by <strong>Agile Lab s.r.l.</strong> <a href="https://www.agilelab.it" style="color: inherit;">www.agilelab.it</a>'
# theme configuration
color:
# primary color that will be used as background color for the footer
primary: '#00acc1'
# secondary color that will be used for the notification title and for the link
secondary: '#00acc1'
# text color for the body of the notification
text: '#4e6980'
# color for the text inside the footer
footerText: '#ffffff'
# configuration related to email account information and authentication
auth:
# hostname of the smtp server related to the provider used
host: smtp-mail.provider.com
# port of the smtp server related to the provider used
port: 587
# authentication type: the value can be 'login' or 'oauth2'
type: login
# email account to use
user: witboost@agilelab.it
# optional field to specify the password for the login authentication type
password: passwordValue
# optional field to specify the client id for the 3-legged oauth2 authentication type
clientId: clientIdValue
# optional field to specify the client secret for the 3-legged oauth2 authentication type
clientSecret: clientSecretValue
# optional field to specify the access token for the 3-legged oauth2 authentication type
accessToken: accessTokenValue
# optional field to specify the refresh token for the 3-legged oauth2 authentication type
refreshToken: refreshTokenValue
# optional field to specify the service client for the 2LO oauth2 authentication type
serviceClient: serviceClientValue
# optional field to specify the private key for the 2LO oauth2 authentication type
privateKey: privateKeyValue
# configuration filters to enable/disable in-app/email notification by event
byEvent:
# Event name. This list contains all the supported ones
- event: NewVersion
# Flag to set the in-app notification for this event
sendNotification: true
# Flag to set the email notification for this event
sendMail: false
# Array containing the recipients that you want to notify.
# Supported values: 'dataProductConsumersOwners', 'dataProductOwners', 'none'
recipients: ['dataProductConsumersOwners']
- event: NewSystem
sendNotification: true
sendMail: true
recipients: ['dataProductOwners']
- event: NewTemplate
sendNotification: true
sendMail: false
recipients: ['dataProductOwners']
- event: NewDomain
sendNotification: true
sendMail: false
recipients: ['dataProductOwners']
- event: NewPolicy
sendNotification: true
sendMail: false
recipients: ['dataProductOwners']
# Catalog configurations, regarding all the entities managed by the Builder backend
catalog:
# Enables additional preliminary validity checks on publish phase when creating a new data product or component (possible values are true or false)
enablePreliminaryChecks: true
# Enables or disable the async validation (possible values are true or false)
enableAsyncValidation: true
rules:
# List of all the entities kinds that will be handled by the Builder backend
- allow:
- Component
- API
- Resource
- Template
- System
- Domain
- Group
- User
- Location
- Release
# List of all the entities kinds that will be hidden in the software catalog
hiddenKinds:
- Hierarchy
- HierarchyClass
# Additional processors, in this case used to read users and groups from a remote organization manager
processors:
# By setting a provider, witboost will use it to retrieve users and groups. A user needs to be retrieved by this
# processor in order to enable the login.
# In here an example of definition for a Microsoft Active Directory provider is displayed.
# You can check the different providers requirements directly from the Backstage official documentation
microsoftGraphOrg:
providers:
- target: https://graph.microsoft.com/v1.0
authority: https://login.microsoftonline.com
clientId: 11111111-ffff-aaaa-8888-999999999999
clientSecret: my_azure_ad_client_secret
tenantId: 22222222-eeee-4444-cccc-000000000000
# In here we set some filters to avoid importing ALL the users and groups of the entire Active Directory
# In this example we will fetch only users and groups that are contained in the root group "witboost"
userGroupMemberFilter: "displayName eq 'witboost'"
groupFilter: "displayName eq 'witboost'"
# With this configuration you can change how entity names are pre-processed before storing them in teh database
# You can completely ignore this configuration if you are fine with the default behavior
entityNameNormalization:
# This regex selects the values to be replaced
regex: '[^a-zA-Z0-9_\\-\\.]'
# This is the character that will be used to replace them
replaceWith: '_'
# Configurations for the witboost modules
mesh:
# Marketplace module configurations
marketplace:
# Public Hasura endpoint that will be used to handle hooks
baseUrl: https://witboost.hasura.com/v1/graphql
# [SECRET] Hasura admin token, used only for backend read operations (e.g. search index creation)
hasuraToken: my_hasura_admin_secret
# If you are querying hasura without the right certificate manager on your development environment
# you might need to set this variable to TRUE in order to send http query instead of https
# By DEFAULT this is set to FALSE and this is the recommended behaviour in general
useHttpsSearchUnauthorized: false
# (optional) Hasura claims to be included in the JWT.
hasuraClaims:
# lists all allowed roles for the user, in this case the `user` role is an allowed role
x-hasura-allowed-roles: ['user']
# if you specific a list of allowed roles (even just one), it is mandatory to specify a default role to be picked during requests (e.g. in this example `user` role will be the default picked role)
x-hasura-default-role: user
# UI configurations
ui:
# Show Observability Tab in Dataproduct/OutputPort Page
showObservability: true
# DataProduct page configurations
dataProductPage:
# Optional field used to display the technical information card when the Data Product descriptor contains the specified fields
technicalInformation:
# Adding these elements will enable the visualization of the technical information contained inside the corresponding fields of the Data Product
# descriptor, if present. Invalid data inside the descriptor is ignored.
- buildInfo
- deployInfo
# Optional field used to add custom fields to be shown inside the data product General Information card.
additionalGeneralInfo:
# It contains a list of custom fields, specifying for each additional field:
# The string label to be displayed for the field.
- label: My custom field
# The string value: the values in double brackets will be resolved with the values from the data product descriptor using Nunjucks.
# It is important to enforce the string type for this field inside the yaml by using the single quotes as shown to avoid errors.
value: 'the values are {{ specific.dbName }} and {{ specific.viewName }}'
# The field type: string | date. Changing the field type will change the way the field will be rendered in the UI.
type: string
# Optional field to specify an external URL. If present, the field will be automatically rendered as a link (you don't need to change the type).
# As shown for the value field, it can contain static string and/or values to be resolved from the descriptor, by using the same notation.
href: '{{ specific.myCustomLink }}'
# Output Port page configurations
outputPortPage:
# The field used to store the data sharing agreements in the descriptor. The marketplace will use this field to display the relative widget in the UI.
dataSharingAgreementFieldName: dataSharingAgreements
# Links displayed in this page
links:
# By changing the following values you can decide which output port field will be used to display the related link.
# e.g. in this case the live data link will be displayed only if the deployInfo.liveData field is set, and will point to its value
endpoint: dataContract.endpoint
liveData: deployInfo.liveData
dataCatalog: deployInfo.dataCatalog
# Optional field used to display the technical information card when the Output Port descriptor contains the specified fields
technicalInformation:
# Adding these elements will enable the visualization of the technical information contained inside the corresponding fields of the Output port
# descriptor, if present. Invalid data inside the descriptor is ignored.
- buildInfo
- deployInfo
# Optional field used to add custom fields to be shown inside the output port General Information card.
additionalGeneralInfo:
# It contains a list of custom fields, specifying for each additional field:
# The string label to be displayed for the field.
- label: My custom field
# The string value: the values in double brackets will be resolved with the values from the output port descriptor using Nunjucks.
# It is important to enforce the string type for this field inside the yaml by using the single quotes as shown to avoid errors.
value: 'the values are {{ specific.dbName }} and {{ specific.viewName }}'
# The field type: string | date. Changing the field type will change the way the field will be rendered in the UI.
type: string
# Optional field to specify an external URL. If present, the field will be automatically rendered as a link (you don't need to change the type).
# As shown for the value field, it can contain static string and/or values to be resolved from the descriptor, by using the same notation.
href: '{{ specific.myCustomLink }}'
# Mesh Supervision page configurations
meshSupervisionPage:
# Show page
enabled: true
# Summary Kpi Card configurations
summaryKpiCard:
# Show card
enabled: true
# Usage Kpi Card configurations
usageKpiCard:
# Show card
enabled: true
# Ratings Kpi Card configurations
ratingsKpiCard:
# Show card
enabled: true
# Env Alignment Kpi Card configurations
envAlignmentKpiCard:
# Show card
enabled: true
# Network Effect Kpi Card configurations
networkEffectKpiCard:
# Show card
enabled: true
# Average questions Kpi Card configurations
averageQuestionsKpiCard:
# Show card
enabled: true
# Provisioner module configurations
provisioner:
# The URL must contain the remote Provisioning Coordinator URL
baseUrl: http://datamesh-provisioningcoordinator.datamesh-provisioningcoordinator:8088/datamesh.provisioningcoordinator/1.0
deployStep:
# Optional field used to specified which field of the deploy step result we want to display.
# Supports nested fields with the dot notation (e.g. 'info.result')
# If not specified, the complete raw deploy step result will be displayed.
resultField: result
# Computational Governance Platform (CGP) configuration
governance:
# base URL where the CGP is listening to requests
baseUrl: http://localhost:8088/datamesh.provisioningcoordinator
metrics:
# enable/disable metrics management on CGP
enabled: true
# CGP endpoints related to metrics
createEndpoint: /v1/computational-governance/metrics
testEndpoint: /v1/computational-governance/metrics/{{metricId}}/test
updateEndpoint: /v1/computational-governance/metrics/{{metricId}}
bumpVersionEndpoint: /v1/computational-governance/metrics/{{metricId}}
changeStatusEndpoint: /v1/computational-governance/metrics/{{metricId}}/status
policies:
# CGP endpoints related to policies
createEndpoint: /v1/computational-governance/policies
testEndpoint: /v1/computational-governance/policies/{{policyId}}/test
updateEndpoint: /v1/computational-governance/policies/{{policyId}}
bumpVersionEndpoint: /v1/computational-governance/policies/{{policyId}}
changeStatusEndpoint: /v1/computational-governance/policies/{{policyId}}/status
stats:
# Label to identify evaluation reports generated during a data product deployment operation
deploymentEvaluationReportsLabel: deployment
# Endpoints for CGP stats
topFailedPoliciesEndpoint: /v1/computational-governance/policies/stats/top-failed
policyExecutionStatsEndpoint: /v1/computational-governance/policies/stats/executions
failedPolicyEvaluationStatsEndpoint: /v1/computational-governance/policies/stats/executions/failures
evaluation:
# CGP endpoints related to governance entity evaluations
latestResourceEvaluationsEndpoint: /v1/computational-governance/resources/latest-evaluations
# Builder module configurations
builder:
# List of environments handled by witboost.
# Please note that this list must contain the same environments as they are defined in the Marketplace DB
environments: [development, qa, production]
# (Optional) The size limit of the payload for request sent to the builder.
# If this property is not specified the default limit is set to 10mb. The value of the limit can be specified according to the syntax of the package `bytes` (https://www.npmjs.com/package/bytes).
payloadSizeLimit: 10mb
# Scaffolder module configurations
scaffolder:
# Blueprints configurations
blueprints:
# Whether blueprints should be enabled or not (blueprints are enabled by default)
# Remember that to use blueprints, you have also to list them among the entities in catalog.rules.allow
enabled: true
# User headers configurations
userHeaders:
# Whether user headers should be enabled or not
enabled: false
# Name prefix of each user-defined header, separated from the rest of the name by a dash.
# For example, if prefix is `wb-user`, header `my-header` becomes `wb-user-my-header`
prefix: wb-user
# Reserved fields consistency validator configurations
reservedFields:
# Global configuration of the consistency validator. It has two fields: `name` and `kind`.
global:
name: true
kind: false
# Local configuration of the consistency validator. You can specify the kind and write an array of paths you want
# to enable the feature for.
local:
# Paths for the `System` entity kind.
system:
- 'spec.mesh.email'
- 'spec.domain'
# Paths for the `Component` entity kind.
component:
- 'spec.mesh.email'
- 'spec.domain'
# Git repository integration configurations
repositoryManager:
# Whether to prioritize a native Git workflow (i.e. Git CLI commands) over API usage when both integrations are available.
# E.g. git push by cloning, fetching, committing, pushing vs git push through Gitlab/Bitbucket/... APIs
# Please note: not all functionalities feature a native git integration; in this case APIs will be used despite this flag.
# The opposite is true as well (i.e. some tasks are always completed by using native git commands)
prioritizeGit: true
# Configurations for the additional custom pages rendered leveraging microfrontends
customPages:
marketplace:
# Here are listed all the microfrontend pages that will be displayed in the marketplace's menu
page:
# For each page we need a title, which will be the one displayed in the tab label
- title: 'Microfrontend test'
# The URL where the microfrontend is running
url: 'http://localhost:4200'
# The route where the page will be reachable inside witboost (must start with "/" and be a valid URL segment)
route: '/test-microfrontend'
# The unique identifier of the microfrontend component (must be alphanumerical and include only "-" and "_")
identifier: 'test-marketplace'
# Name of the mount function exposed by the microfrontend
mountFunction: 'renderApplication'
# Name of the unmount function exposed by the microfrontend
unmountFunction: 'unmountApplication'
# A list of all parameters that will be sent to the microfrontend (for this page no parameters will be extracted)
parameters: []
# Here are listed all the microfrontend pages that will be displayed in the marketplace's data product page
dataProductPage:
# For each page we need a title, which will be the one displayed in the tab label
- title: 'Microfrontend OP test'
# The URL where the microfrontend is running
url: 'http://localhost:4200'
# The route where the page will be reachable inside witboost (must start with "/" and be a valid URL segment)
route: '/test-dp-microfrontend'
# The unique identifier of the microfrontend component (must be alphanumerical and include only "-" and "_")
identifier: 'test-marketplace-dp'
# Name of the mount function exposed by the microfrontend
mountFunction: 'renderApplication'
# Name of the unmount function exposed by the microfrontend
unmountFunction: 'unmountApplication'
# A list of all parameters that will be sent to the microfrontend
parameters:
# The name of the parameter
- name: 'name'
# The path of the parameter value extracted from the input object (take a look at the documentation for more details)
valuePath: 'name'
# Here are listed all the microfrontend pages that will be displayed in the marketplace's output port page
outputPortPage:
# For each page we need a title, which will be the one displayed in the tab label
- title: 'Microfrontend OP test'
# The URL where the microfrontend is running
url: 'http://localhost:4200'
# The route where the page will be reachable inside witboost (must start with "/" and be a valid URL segment)
route: '/test-op-microfrontend'
# The unique identifier of the microfrontend component (must be alphanumerical and include only "-" and "_")
identifier: 'test-marketplace-op'
# Name of the mount function exposed by the microfrontend
mountFunction: 'renderApplication'
# Name of the unmount function exposed by the microfrontend
unmountFunction: 'unmountApplication'
# A list of all parameters that will be sent to the microfrontend
parameters:
# The name of the parameter
- name: 'name'
# The path of the parameter value extracted from the input object (take a look at the documentation for more details)
valuePath: 'name'
# Technical documentation configuration, defining how documentation for the entities is generated and displayed
techdocs:
generator:
# This is the generator configuration. By setting it to 'local', the remote repositories will have the raw markdown files
# interpreted at runtime. This is the default behavior, and works for up to hundreds of documentation repositories
runIn: 'local'
# Tracer configuration
tracer:
# Respect request header flag (default: false). If set to true, the middleware/plugin tracer will always use a value from the specified header (if the value is present).
useHeader: false
# Request/response header name, case insensitive (default: 'X-Request-Id'). Used if useHeader/echoHeader is set to true.
headerName: 'X-Request-Id'
# Ontology configuration
ontology:
# Enable/Disable ontology feature
enabled: false
# List of entity kinds that could be imported from the import hierarchy page
kinds:
- Hierarchy
- HierarchyClass
In the catalog.processors.microsoftGraphOrg.providers.(userGroupMemberFilter | groupFilter) part of the YAML file
shown above, you can specify things other than "displayName eq 'witboost'" - this will filter out all the
groups that are not strictly equal to "witboost".
An example usage of this would be "startsWith(displayName, 'witboost')" - this is useful in the situation where you
want to
bulk import all the Microsoft AD groups that are related to the witboost(You just need to prefix the names of all the
groups
that you would like to bulk import)
Catalog
Data Product Unregistration
By default, when you unregister a data product from the catalog, all of its releases are not deleted. That means, witboost will continue to track changes from the repositories where the releases entities are located. To change this behaviour, you can add the following configuration properties:
catalog:
onDataProductUnregister:
unregisterReleases: true
After unregistering the data product and all of its releases, if in the future you want to register that data product back to the catalog, you will need to manually register each release one by one.
Async validation section
To enable the async validation in witboost you must define the following lines in your app-config.yaml
catalog:
enableAsyncValidation: true
The above lines will substitute the current "Launch test" method with the asychronous one ( see Async validation)
Audit
To enable/disable auditing of the operation performed by the users, just edit the following configuration section:
backend:
audit:
enabled: true
methods:
- GET
- POST
- PUT
- PATCH
- DELETE
By setting enabled to false, auditing is disabled for all operations. In the platform there are two kinds of operations for which audit is collected:
- REST APIs invocations. In this case you can configure which HTTP methods you want to audit by changing the value of
the whitelist field
methods; usually you want to audit all of them apart from GET operations, since the table could grow in dimensions very easily. - Hasura GraphQL mutations. Regular Hasura queries are not audited, since they will not affect the database directly, and will end up in increasing the table dimensions quickly.
Integrations section GitLab
An example GitLab config block to public GitLab:
integrations:
gitlab:
- host: gitlab.com
token: GITLAB_TOKEN
An example GitLab config block to private GitLab:
integrations:
gitlab:
- host: gitlab.my-company.com
apiBaseUrl: https://gitlab.my-company.com/api/v4
baseUrl: https://gitlab.my-company.com
token: OUR_GITLAB_TOKEN
In this case, the only optional field is baseUrl which is formed from the host if needed.
Integrations section Bitbucket server:
integrations:
bitbucketServer:
- host: bitbucket.my-company.org
token: NTUxMjMzNTEyMTQ4OiT9DxGxngtXIFIzXJEI4Vca7IUm
username: admin
password: xhdegwvitlzy
isOAuthToken: false
It is not possible to push on a bitbucket repo if you don't have an oauth2 token. Therefore it is necessary to configure
username and password in this case. Otherwise if you use a oauth2 token set only the property isOAuthToken a true
When importing a template from bitbucket always make sure the branch is present in the url. The default branch is never
present in the url, in this case add ?at=<branchname>. So if you are importing an URL like
this https://<hostname>/projects/data_mesh/repos/test/browse/catalog-info.yaml from the master branch, you should use
instead https://<hostname>/projects/data_mesh/repos/test/browse/catalog-info.yaml?at=master.
Environments check
Environments list for every module(Governance,Builder and Marketplace) should be the same. To make sure this is true, you can enable this control in your app-config.yaml with these lines:
environments:
enabledCheck: true
In this way witboost takes environents available for Governance module and for Marketplace module from Hasura graphql, instead for Builder module environments come from app-config.local.yaml. It then compares them to check that they are the same. If it's not true an exception is throw and process is stopped.
Permissions section
To enable the Role-Based Access Control plugin in witboost you must have these lines in your app-config.yaml
permission:
enabled: true
This will tell witboost to start authorizing requests according to a RBAC policy.
By default, all authorization requests will be denied unless you have already configured some roles and users inside the database
To see how to configure Grant Mechanisms, that enable automatic granting based on particular events, head to the RBAC section
Notifications
This section describes the configuration for customizing informative notification events offered by witboost.
The configuration is specified in YAML format and includes a list of events (byEvent) with corresponding settings for
each event. The settings for each event include the event name (event), whether to send a notification (
sendNotification), whether to send an email (sendMail), and who should receive the notification (recipients).
The following events are currently supported:
NewVersion: Triggered when a new version of a data product is created.
NewSystem: Triggered when a new data product is registered.
NewTemplate: Triggered when a new data product template is registered.
NewDomain: Triggered when a new domain is registered.
NewPolicy: Triggered when a new policy is registered.
For each event, the sendNotification and sendMail settings can be set to true or false depending on whether a
notification or email should be sent. The recipients setting is an array of strings indicating who should receive the
notification.
The platform currently supports the following recipient retrieving functions:
dataProductConsumersOwners: This function retrieves the owners of components that read from the subject data product of the event.
dataProductOwners: This function retrieves all users who own at least one data product.
none: This function retrieves no recipients.
Example configuration:
notifications:
byEvent:
- event: NewVersion
sendNotification: true
sendMail: false
recipients: ['dataProductConsumersOwners']
- event: NewSystem
sendNotification: true
sendMail: false
recipients: ['dataProductOwners']
- event: NewTemplate
sendNotification: true
sendMail: false
recipients: ['dataProductOwners']
- event: NewDomain
sendNotification: true
sendMail: false
recipients: ['dataProductOwners']
- event: NewPolicy
sendNotification: true
sendMail: false
recipients: ['dataProductOwners']
Email notifications
The email notification feature requires an additional section called emailConfig under notifications, related to the
email configuration.
You can customize the sender name by using the dedicated field. In addition, inside the html section you can customize
and the app name, the logo, the footer and the theme used inside the html email. This part of the configuration is
mandatory.
The default values that you can use for the email template are the following ones:
notifications:
# email configuration
emailConfig:
# name of the email sender that will be shown to the recipients
senderName: Witboost
# configuration related to the html email template
html:
# the app name that will be displayed inside the email
appName: Witboost
# path of the logo that will be displayed at the top of the email
# the file must be under the wb-notification-backend plugin folder
# the default witboost logo is available using the path shown below
logoPath: 'src/resources/logo.svg'
# Optional field about the footer information that will be displayed at the bottom of the email, inside the footer
# you can use html tags inside the string, as shown below for the default value
footer: '<strong>witboost</strong> is a product by <strong>Agile Lab s.r.l.</strong> <a href="https://www.agilelab.it" style="color: inherit;">www.agilelab.it</a>'
# theme configuration
color:
# primary color that will be used as background color for the footer
primary: '#00acc1'
# secondary color that will be used for the notification title and for the link
secondary: '#00acc1'
# text color for the body of the notification
text: '#4e6980'
# color for the text inside the footer
footerText: '#ffffff'
You can enable the email notification for each supported event by setting the related sendMail flag to true, as
explained above.
In this case, you will need to specify an additional configuration section called auth under emailConfig, related to
the email account and authentication method to use in order to send the emails.
In particular, you will need to specify the information related to the Smtp server of the provider of the email account,
and then an authentication method.
The supported authentication types are login (based on username and password) and Oauth2.
Example of the configuration using the login authentication:
notifications:
emailConfig:
# configuration related to email account information and authentication
auth:
# hostname of the smtp server related to the provider used
host: smtp-mail.provider.com
# port of the smtp server related to the provider used
port: 587
# authentication type: the value can be 'login' or 'oauth2'
type: login
# email account to use
user: witboost@agilelab.it
# password for the specified email account
password: passwordValue
Oauth2 authentication
Regarding the Oauth2 authentication, you can find further information about the supported parameters in the related section of the documentation of the used email library: Nodemailer documentation - Oauth2. We are supporting both 2LO and 3-legged authentication.
Example of the configuration using the 2LO Oauth2 authentication (service accounts):
notifications:
emailConfig:
# configuration related to email account information and authentication
auth:
# hostname of the smtp server related to the provider used
host: smtp.gmail.com
# port of the smtp server related to the provider used
port: 465
# authentication type: the value can be 'login' or 'oauth2'
type: oauth2
# email account to use
user: witboost@agilelab.it
# Oauth service client
serviceClient: serviceClientValue
# private key used to perform the authentication
privateKey: privateKeyValue
Example of the configuration using the 3-legged Oauth2 authentication:
notifications:
emailConfig:
# configuration related to email account information and authentication
auth:
# hostname of the smtp server related to the provider used
host: smtp.gmail.com
# port of the smtp server related to the provider used
port: 465
# authentication type: the value can be 'login' or 'oauth2'
type: oauth2
# email account to use
user: witboost@agilelab.it
# Oauth client id
clientId: clientIdValue
# Oauth client secret
clientSecret: clientSecretValue
# token used to perform the authentication
accessToken: accessTokenValue
# token used to refresh the access token
refreshToken: refreshTokenValue
In order to use the Oauth2 authentication you will need to register the Witboost application and perform additional operations depending on your provider. The Oauth2 flow has been tested using the 3-legged authentication and Gmail provider. If you are going to configure Oauth2 with Gmail, you can follow the steps explained in this tutorial.
User headers
As described here, witboost enables users to define a set of HTTP headers that will be attached to each HTTP request exchanged between services within the witboost ecosystem while fulfilling provisioning operations (deploy, undeploy, update acl, validate).
User-defined HTTP headers are sent to the Provisioning Coordinator and, depending on the Coordinator's headers forwarding configuration settings , they can be further propagated to other provisioning services (specific provisioners, data catalog proxies, marketplace-compliant services).
The following configurations can be defined under mesh.userHeaders in the app-config*.yaml files:
| Key | Type | Default (if not defined) | Description |
|---|---|---|---|
| enabled | boolean | false | Whether user headers should be enabled or not. When false the headers section will not be visibile on the UI and no user headers will be sent to the coordinator |
| prefix | string | wb-user | Name prefix of each user-defined header, separated from the rest of the name by a dash. For instance, if prefix is wb-user, header my-header becomes wb-user-my-header |
When changing the user headers prefix, remember to update the Coordinator's headers forwarding configuration settings accordingly
Autologout
Autologout is a configurable mechanism that allows Witboost to logout inactive users. Inactive users are the ones that don't perform any action on the Witboost app or that are logged in but no Witboost tab is open in the browser. When enabled, the mechanism will track user actions (mouse movement, mouse click, key pressing, taps, etc.) in order to determine if they are active or not. After a certain amount of inactivity, also called idle time, the user will be logged out and he/she will need to sign in again into the app.
To enable the autologout, just place this configuration in your app-config:
auth:
autologout:
enabled: true
You will see that after a while the user will be automatically logged out, but only after a prompt dialog will show up to inform the user that Witboost is about to log out.
Users will be logged out after the configured inactivity timeout even if they have all Witboost browser tabs closed. Anyway, this behaviour is configurable. Take a look at the Autologout configuration section
You can configure both the inactivity timeout and the number of seconds before idle when to show the inactivity prompt. You can do so by adjusting these values in the configuration:
auth:
autologout:
enabled: true
idleTimeoutMinutes: 60 # number of minutes after which the user will be logged out
promptBeforeIdleSeconds: 10 # number of seconds before logout where the user will be prompted. set to 0 to disable the prompt.
Witboost will track user actions made by a pointer device (e.g. a mouse) and/or a keyboard. However, you can decide to change the list of actions tracked by supplying a list of events in the configuration. By default, the user actions, i.e. events, that Witboost will track to determine if the user is still using the platform will be:
mousemovekeydownwheelDOMMouseScrollmousewheelmousedowntouchstarttouchmoveMSPointerDownMSPointerMovevisibilitychange
If for any reason, you would like to change the list of events that the autologout tracks, just add the events
property in configuration and place your list of events:
auth:
autologout:
enabled: true
# track only keyboard events
events:
- keydown
You can find a complete list of supported events in the Autologout Supported Events section of this page.
Autologout configuration
We did not mention all autologout configuration properties, for this reason here you can find a full list of autologout
configurations, with allowed values and defaults.
All configurations keys listed in this table must be placed under auth.autologoutin your app-config.
| configuration key | description | allowed values | default value |
|---|---|---|---|
enabled | Enable/Disable the Autologout feature | true/false | false |
idleTimeoutMinutes | Number of minutes after which the user will be logged out | [0.5 minutes, > 0.5 minutes) | 60 minutes |
promptBeforeIdleSeconds | Number of seconds before idle when a prompt will be shown. Set to 0 to disable the prompt. | >= 0 seconds. must be smaller than idleTimeoutMinutes | 10 seconds |
events | List of events used to detect user activity on Witboost | Refer to events section | Refer to events section |
useWorkerTimers | Enable/disable the usage of Node's worker thread timers instead of main thread timers. This is helpful if you notice that the Browser is killing inactive tab's timers, like the one used by Autologout. If you experience some browser incompatibility, you may try to set this to false. | true/false | true |
logoutIfDisconnected | Enable/disable the autologout for disconnected users. disconnected users are the ones that are logged in but have no Witboost tab open in their browsers. If enabled, disconnected users will be automatically logged out after idleTimeoutMinutes | true/false | true |
Autologout Supported Events
Here we provide a list of supported DOM events that you can use to fill the configuration, remember that this is an optional step, the Autologout feature is already configured to work with standard user events like mouse movements and keyboard press.
| Event | Description |
|---|---|
abort | Fired when the loading of a resource has been aborted. |
afterprint | Fired after the associated document has started printing or the print preview has been closed. |
beforeprint | Fired just before the associated document begins printing or the print preview opens. |
blur | Fired when an element has lost focus. |
change | Fired for <input>, <select>, and <textarea> elements when a change to the element's value is committed by the user. |
click | Fired when a pointing device button (usually a mouse's primary button) is pressed and released on an element. |
contextmenu | Fired when the right button of the mouse is clicked (before the context menu is displayed), or when the contextmenu key is pressed (in Windows). |
copy | Fired when the user initiates a copy action through the browser's user interface. |
cut | Fired when the user initiates a cut action through the browser's user interface. |
dblclick | Fired when a pointing device button is clicked twice on an element. |
DOMMouseScroll | Fired when the mouse wheel is rolled up or down over an element. |
drag | Fired when an element or text selection is being dragged. |
dragend | Fired when a drag operation is being ended (by releasing a mouse button or hitting the escape key). |
dragenter | Fired when a dragged element or text selection enters a valid drop target. |
dragleave | Fired when a dragged element or text selection leaves a valid drop target. |
dragover | Fired when an element or text selection is being dragged over a valid drop target (every few hundred milliseconds). |
dragstart | Fired when the user starts dragging an element or text selection. |
drop | Fired when an element is dropped on a valid drop target. |
focus | Fired when an element has received focus. |
focusin | Fired when an element is about to receive focus. |
focusout | Fired when an element is about to lose focus. |
fullscreenchange | Fired when the document goes into or out of full-screen mode. |
fullscreenerror | Fired when the browser cannot switch to full-screen mode. |
gotpointercapture | Fired after a pointer event got captured. |
hashchange | Fired when the fragment identifier of the URL has changed (the part of the URL that follows the # symbol). |
input | Fired when an <input> or <textarea> value changes. |
invalid | Fired when a submittable element has been checked and does not satisfy its constraints. |
keydown | Fired when a key is pressed down. |
keypress | Fired when a key that produces a character value is pressed down. |
keyup | Fired when a key is released. |
load | Fired when progress has stopped (used for images, scripts, and CSS files). |
loadeddata | Fired when the first frame of the media has finished loading. |
loadedmetadata | Fired when the metadata has been loaded. |
loadstart | Fired when the browser has started to load a new audio/video. |
lostpointercapture | Fired after a pointer event was released. |
message | Fired when a message is received through a WebSocket. |
mousedown | Fired when a pointing device button (usually a mouse's primary button) is pressed on an element. |
mouseenter | Fired when a pointing device is moved onto the element that has the listener attached. |
mouseleave | Fired when a pointing device is moved off the element that has the listener attached. |
mousemove | Fired when a pointing device is moved over an element. |
mouseover | Fired when a pointing device is moved onto the element that has the listener attached or onto one of its children. |
mouseout | Fired when a pointing device is moved off the element that has the listener attached or off one of its children. |
mouseup | Fired when a pointing device button is released over an element. |
mousewheel | Non-standard. This feature is non-standard and is not on a standards track. Use WheelEvent instead if possible. |
MSPointerDown | Fired when contact is made with the touch surface and the button is pressed. |
MSPointerMove | Fired when the contact or a pen or the cursor of a mouse is dragged along the touch surface. |
offline | Fired when the browser has lost access to the network. |
online | Fired when the browser has gained access to the network. |
open | Fired when a connection with a WebSocket has been opened. |
paste | Fired when the user initiates a paste action through the browser's user interface. |
pause | Fired when the audio/video has been paused. |
play | Fired when the audio/video has been started or is no longer paused. |
playing | Fired when the audio/video is ready to start playing. |
pointercancel | Fired when the browser decides to cancel the tracking of pointer movement. |
pointerdown | Fired when the pointer is activated. |
pointerenter | Fired when a pointer is moved into the hit test boundaries of an element. |
pointerleave | Fired when a pointer is moved out of the hit test boundaries of an element. |
pointermove | Fired when a pointer changes coordinates. |
pointerout | Fired for several reasons including: pointing device is moved out of the hit test boundaries of an element. |
pointerover | Fired when a pointer is moved into the hit test boundaries of an element or one of its children. |
pointerup | Fired when a pointer is deactivated. |
popstate | Fired when the active history entry changes. |
progress | Fired to indicate that the browser is fetching the resource(s). |
resize | Fired when the document view has been resized. |
reset | Fired when a form is reset. |
scroll | Fired when the document view or an element has been scrolled. |
search | Fired when a user presses enter in a search <input>. |
seeked | Fired when a seek operation completed. |
seeking | Fired when a seek operation began. |
select | Fired after some text has been selected in an input element. |
show | Fired when a <menu> element is shown as a context menu. |
storage | Fired when a storage area (localStorage) has been changed in the context of another document. |
submit | Fired when a form is submitted. |
toggle | Fired when the open attribute of a <details> element is toggled. |
touchcancel | Fired when a touch point has been disrupted. |
touchend | Fired when a touch point is removed from the touch surface. |
touchmove | Fired when a touch point is moved along the touch surface. |
touchstart | Fired when a touch point is placed on the touch surface. |
transitionend | Fired when a CSS transition has completed. |
unload | Fired once a page unload (or 'navigate') has occurred. |
wheel | Fired when a wheel button of a pointing device is rotated in any direction. |
visibilitychange | Fired when the content of a tab has become visible or has been hidden. |
Please note that the descriptions are generalized, and the actual behavior may depend on the specific circumstances and context in which the event is used. Be sure to check the Mozilla Network Documentation for a more comprehensive understanding of each event.