Skip to main content

Version 2.2.0+

These are the release notes for the v2.2.0 release of Witboost.

Please refer to the official documentation for a more in-depth overview of the released features.

Features​

Data Contracts​

The new Data Contracts feature enables the definition and monitoring of clear, enforceable agreements between data producers and consumers, ensuring more reliable and predictable data exchanges.

This release delivers comprehensive capabilities to:

  • Define, validate, and seamlessly manage the lifecycle of data contracts and guardians (Builder module)
  • Deploy data contracts and their guardians to the target infrastructure (Provisioning module)
  • Gather monitoring insights and alerts from data contract guardians, fully integrating with the Computational Governance Platform
  • Instantly notify data producers and consumers of any contract violations (Notification system)
  • Track data contract status and issues within the Witboost Marketplace

Attaching a data contract to a Witboost component is as easy as adding a dataContract section within the component's descriptor.

Marketplace New Features​

In this release, we restyled the Marketplace to make it more user-friendly and intuitive, while adding visual support for the Data Contracts feature.

First of all, the "Relations" card was taking a lot of space, and was not conveying the right information to the user. We decided to remove it, and add a new tab in the system detail page, Dependencies, where the user can see all the dependencies of the system. This tab shows the systems that the selected system depends on (the ones that are read by the selected system), and the systems that depend on the selected system (the ones that read from the selected one). The user can also navigate to the system detail page of the dependencies by clicking on them. In this page, we will also have more focus on the data assets: instead of just linking the systems, we are now displaying the data assets that are read in the lineage, and displaying also groups of assets.

The second main change is the introduction of the Data Contracts tab. In this tab, the user can see all the data contracts of the system, and how these data contracts are consumed by downstream systems. The user can see also the data contracts that are consumed by the selected system, and explore the whole lineage by expanding the data contracts. The user can also navigate to the system detail page of the systems that consume the data contracts by clicking on them. This page also shows the errors related to a broken data contract, and all the systems that are directly affected by it.

While we kept the two main entry points of the marketplace (Search and Visual Discovery), we redesigned the Visual Discovery page. Now, the user can hover on systems to see their details and click on them to access the system detail page. From here, the user can also directly navigate to the new specific system detail's tabs, "Dependencies" and "Data Contracts". We kept the regular search panel, and the heatmap concept, but we introduced also the possibility to highlight the systems for which there is at least one runtime error.

As usual, we added the possibility for customers to customize the components that display the metadata information: all the hover popups and the components list can be customized by the platform administrators.

Github Integration​

With this release of Witboost, we have introduced a new integration with GitHub. With this new integration, you can now connect your GitHub provider to Witboost and use it to manage your projects and repositories directly from the platform.

This integration contains a way to define the new GitHub action and all the facilities needed to store the user's personal token, define a mono repository and multiple repositories, fetch branches, and perform all the Builder operations (snapshot, commit, release, and new version). All the platform-level entities (like templates, blueprints, and domains) can be stored in the GitHub repositories as well.

The integration with GitHub works in the same way as the existing integrations. The platform administrators can connect their GitHub provider to Witboost by providing the necessary credentials, and each user can specify their tokens to perform actions on the repositories.

Okta Integration​

With this release, we also introduced the support for Okta as an authentication provider. This new integration allows you to connect your Okta provider to Witboost and use it to manage your users and groups directly from the platform.

The provider will use the Okta endpoint to authenticate users; users and groups must exist, and can be fetched using one of the other integrations proposed. After configuring the Okta provider, the users can log in using their Okta credentials. As per the other existing providers, the Okta endpoint will be used to authenticate the user, while the user's groups will be fetched from the configured endpoint.

Since Okta can also handle user associations to groups, we also introduced a new Okta organization provider. This organization provider will be used to fetch the users and groups information from the Okta endpoint, and periodically update the platform with the latest information.

The logged user will be associated with the groups fetched from the Okta endpoint, and Witboost will use this information to apply the correct permissions to the user.

Environments centralization​

Since the environments are used in many modules of the platform, we decided to centralize them in a single place. This way, the environments can be fetched from the backend and frontend components, and can be used by all the modules: Builder, Marketplace, Governance, and Provisioning.

The configuration of the environments is now centralized in the mesh.builder.environments configuration. Every module will then fetch the environments from the core module, and use them to perform the operations that require the environment information.

For this reason, priorities are no longer needed in the databases of the external modules (e.g. the Marketplace). The priorities are now set in the core module, and the external modules will fetch the environments from the core module. If you want to attach a priority to an environment, you can set it in the core configurations; otherwise, leave it as it was before, but then priorities will not be explicitly defined (the first one in the list will be taken as the one with the highest priority)

Helm chart improvements​

We drastically improved the structure of the Witboost's Helm Chart. This will make the installation and upgrade process more straightforward and less error-prone. The main changes are:

  • Refactor of the witboost-secret. We have factorized the secrets used by the various components of Witboost. For example, all the components and modules now reference WITBOOST_DB_* as the main database. We have kept the flexibility of specifying custom databases, but it should be easier now to understand which secrets are used by which component. Most of the secrets have also being renamed in order to better clarify the target component.

  • Refactor of the values.yaml file. Witboost's values.yaml has been refactored to be cleaner and to hide unneeded complexity. A dependency mechanism between components has been created in order to automatically compute values that are required by dependant components. This will remove from the user the task of manually filling those variables, by automatically computing them based on the values of other components.

  • Witboost secret-waiter-job. We added a gate job that checks the availability of witboost-secrets before starting the installation of the other components. In case the secrets are not available, the job will fail until they are created. This will prevent the installation of the other components to fail because of missing secrets.

Installation improvements​

Since the installation experience was not always easy, we introduced new automation for the platform, facilitating even more the installation of Witboost for new and existing clients.

Now the Hasura management is automatically handled by the installation and upgrade process, so no manual processes like registration, tracking are required anymore. This is achieved through a job that automatically tracks tables in Hasura, and a new configuration that allows the user to specify custom tracking rules.

Witboost now also comes with a default Data Landscape already registered for now installations based on our Starter Kit, speeding up the time to start creating your projects. Of course, for customers that have already their own Data Landscapes set up, this automatic task will check if any Data Landscape is already registered, and will not create new ones.

In this new version it is now also possible to automate via configuration several installation steps that were previously manual, like the automatic registration of a new default RBAC preset including the main roles your platform could need, and the automatic registration of the appropriate EULA document for the platform.

Finally, the registration of infrastructure templates can be now placed in the configuration, to speed up the installation process.

Builder Microfrontends​

Now you can add microfrontends to the Builder pages.

You can add microfrontends to the system detail page and the component detail page, in a similar way to the existing one for the marketplace pages.

Issues fixed and minor improvements​

  • The IdentifierPicker can now be used without the validation, by setting the relative flag. This is useful when the picker is used to generate an identifier used by another picker
  • Updated the internal plugin authentication for all the cross-plugins actions to use the service one instead of the logged user's one. This prevents users with low permissions to see errors related to background actions they are not allowed to perform
  • Fixed the routes for the Blueprint and Template pages: now the URL are more aligned with the content of the pages
  • Fixed an error that corrupted images when loaded from a repository in the template editor. All images in the template editor are now properly rendered, and the binary files as well are not corrupted anymore
  • Added a key for every microfrontend loaded on a single page, to avoid the bug that loaded the same for every tab
  • Fixed the Catalog Graph page to display all the Witboost relations, and not only the ones related to the entities' structure
  • Fixed an error that prevented the title of the IdentitiesPicker from being displayed
  • Resolved an issue where the "entity not found" error could occasionally appear after saving changes via the Editor Wizard
  • Added the ui:disabled support for EntityRelationsPicker and EntitySearchPicker
  • Added the description field to EntitySearchPicker as helper text
  • Fixed the Governance Table's Name and Description column on small-size screens
  • Fixed an issue on the descriptor preview panel (Edit and Test tab) that prevented correctly reading catalog info files from multi-level git branches (e.g., level1/level2/level3)
  • Improved the merge of different files containing array value. Now during the descriptor build phase, if an environment configuration is set, and an array merge must be done, the target value will replace the source value entirely
  • Tags as strings are correctly displayed in marketplace's search table view
  • Improve the error shown when the token does not have enough permissions
  • In the Builder, all the tables' search now correctly filters entities frontend side for spec.mesh.name values with dots
  • Sub-components are now correctly collated when the search scheduler triggers the indexing
  • Made some relations not mandatory: by this, we mean that processing of the source entity is successful even if the target entity is not present in the catalog (e.g. the owner of a project)
  • Indexing is now fixed during stitching when duplicates are present and also inside arrays. This prevents the same entity from being indexed multiple times
  • Add only the unique keys to the output during the stitching phase to avoid duplicates
  • Fixed the kind/type filter in the Data Access panel accessible from the user settings
  • Fixed visibility rules for the Request Access button on non-shoppable systems in the Marketplace
  • Fixed an error that prevented a failed login from showing the reason for the failure to the user (e.g. when the authentication provider is not reachable)
  • The Practice Shaper graph was hard to read, and usually the user would change the position of all the elements manually. Each node position is now saved in the local storage and used to initialize positions when available
  • In the Custom Views, now the path is being honored by the Data Preview component
  • Also, the "no data icon" is shown again in the Data Preview component
  • The payload size limit in the marketplace is now aligned with the platform payload limit size
  • Fix nested yaml substitution in the descriptor, to avoid weird replacements
  • Reworded the error that was thrown in case no personal git token was set during editing
  • Updated the documentation, explaining the "required" property for the dependent picker in an "allOf"
  • href in tags now correctly works when setting the display value
  • The Editor Wizard does not crash anymore if a required field is left blank
  • Fixed a bug that prevented the releases from being shown in the UI when created by a technical user
  • The SelectWidget is not disabled whenever there's only one available option (unless required)
  • Fixed a problem for which sub-components were displayed in the search result page
  • Sub-components can now be included in the readsFrom list
  • Fixed an error that did not show all the custom views for download in their dedicated pages
  • The notifications.systemOpsImpact.triggerEnvironments configuration of the systemOpsImpact notification was not being read from the expected configuration path
  • Fixed an error for which if an entity name had a dot in its display name, the search would now work
  • The IdentifierPicker doesn't crash anymore when the domain is an object
  • Resolved the custom validation of an array field with RegexPicker
  • The SelectWidget now sets the default value when the dependent picker value changes
  • The user can now navigate using the left menu (table of contents) in the tech docs drawer
  • Fixed a bug that prevented LDAP entity provider to take vendor specific configurations set under mesh.ldapOrg
  • Fixed the "select all" action, which was selecting not shoppable components
  • The EntityPicker default value is now loaded correctly
  • In the Custom Views, the "noDataCheck" on the card and sub-card components did not visualize the card if no data condition was met
  • LDAP configuration now correctly reads the environment set in the configuration
  • Fixed a bug where filters would break if a parent domain had no matching results but one of its subdomains did
  • The empty component lists are no longer removed from the descriptor
  • Timed-out governance entity evaluations are no longer marked as "not started" but as failed with the outcome depending on the governance entity severity
  • Updating or creating the default template metadata now is sequential to avoid race conditions on Coordinator startup
  • Affected resources in a "Runtime policy failed" events now include a wasAlreadyNonCompliant field to indicate whether the previous status of the resource in relation to the evaluated policy was already in error
  • Affected derived resources in a "Runtime policy failed" events now include a origin field with a reference to their origin resource