Skip to main content

Changelog v1.2.0

UI​

Changelog v1.3.0​

[#1358] Components icon do not update in Launch Test​

New features and improvements

Changed the condition for polling on the Control Panel page in the test section. The global status of the test is running as long as at least one task is running

Related issue

Closes #1358

[#1376] Fix missing roles in the panel​

Bug fixes

Fixed:

  • a visual bug in the DP graph for word-breaking
  • the spacing between elements in the role/permission table headers
  • the retrieval of roles and permissions when there is a non-existing entity

Related issue

Closes #1376

[#1370] Fix behavior when searching while a data product is already selected​

Bug fixes

  • Fix bug on dp graph highlight
  • Fix bug on dp searching and filtering

Related issue

Closes #1370

[#1368] Refactor authenticate method in the repository manager​

New features and improvements

  • merged authenticate inside CommonRepositoryManager

Related issue

Closes #1368

[#1366] Different kinds of Ok Icon in test tab​

Bug fixes

  • OK icons inside parent test are now filled
  • the number of failed metrics is now correctly displayed
  • in policies/metrics, added tooltip

Related issue

Closes #1366

[#1115] Improve usePermission hook chain​

New features and improvements

  • Added usePermissions hook that takes in input an array of permissions and returns
export type AsyncPermissionResults = {
  loading: boolean;
  allAllowed: boolean;
  anyAllowed: boolean;
  error?: Error;
};

Related issue

Closes #1115

[#1357] ontology legenda improvements​

New features and improvements

Some sentences are rephrased and some grammatical mistakes are corrected.

Closes #1357

[#1350] Define in wb-scaffolder a new picker: CustomUrlPicker​

New features and improvements

  • Added CustomUrlPicker React component in the scaffolder. This component will take some values from a Microservice and enable users to select them while creating a data product or a component using a template.
  • Implemented search with debounce with 2 seconds of delay

Related issue

Closes #1350, #1364

[#1067-FollowUP] Fix apollo import in backend​

Bug fixes

Use in the backend the import @apollo/client/core instead @apollo/client that contains react module.

Related issue

Closes #1067

[#1067] Fix apollo import in backend​

Bug Fix

Use in the backend the import @apollo/client/core instead @apollo/client that contains react module.

Related issue

Closes #1067

[#1067] No-env errors on fresh witboost installation​

New features and improvements

Check if the environments configured in DB of builder, marketplace and coordinator are aligned

Migration

Environments list for every module(Governance,Builder and Marketplace) should be the same. To make sure this is true, you can enable this control in your app-config.yaml with these lines:

environments:
  enabledCheck: true

Breaking changes

if enabledCheck is true and environments are not the same, the application builder will not start and will throw an error.

Related issue

Closes #1067

[#1363] add cache to retrieve git token​

New features and improvements

  • Add cache mechanism in retrieving git token if usePersonalToken is true in the app-config
  • expiresAt is set to 60 seconds (so after 60 seconds a new http call is performed to refresh the git token)

Related issue

Closes #1363

[#1335] Ontology suit automation​

New features and improvements

Added new test suit Ontology, which contains two tests:

  • 11-1 general verification of ontology (panels, labels and ontology page completion)
  • 11-2 a positive user scenario that tests the import and deletion of a new hierarchy

Update 05-05 test: some checks improved

Related issue

Close #1335

[#1359] Create new versions of DP Graph​

New features and improvements

  • Integrate new 2.1.0 DP Graph version to allow data products search.

Related issue

Closes #1359

[#1355] Hierarcy import: class duplication react badly​

New features and improvements

hierarchyStatus table now inserts rows like hierarchy_name.hierarchyclass_name instead of only hierarchyclass_name in order to avoid collision.

Migration

In order to have everything aligned, it is advised to delete all the existing hierarchies and re-insert them. If there are already changes in the table and you want to preserve them, it is recommended to prepend all the hierarchyclass_name rows with the "hierarchy_name."

Bug fixes

There will be no more 502/503 error when inserting new hierarchy with the same hierarchyclass_name.

Related issue

Closes #1355

[#1343] Change the icon for NOT STARTED steps during deployment​

Bug fixes

Added a proper icon for "not started" tasks.

Related issue

Closes #1343

[#1297] Convert user settings page to new WB design​

New features and improvements

Update user section style

Related issue

Closes #1297

[#1341] Add new custom page in the marketplace menu​

New features and improvements

  • Improved the configurations for custom microfrontend pages
  • Added a new configuration to display custom pages in the marketplace menu
  • Defined a new customFrontendApi that can be used everywhere needed to render custom pages

Related issue

Closes #1341

[#1334] Blueprint not visible on different domains​

New features and improvements

  • You can now restrict templates and blueprints to be visible just to a specific domain. by adding the spec.domain property inside the template or blueprint descriptor.

Bug fixes

  • Fixed a bug that prevented blueprints not to be visible for a specific domain

Related issue

Closes #1334

[#1340] Integrate microfrontends into the data product and the output port pages​

New features and improvements

Added a new configuration that lets users load microfrontends in the marketplace output port page and in the marketplace data product page.

The configuration lets developers load microfrontend by setting in the configurations something like:

dataProductPage:
  customPages:
    - title: 'Microfrontend DP test'
      url: 'http://localhost:4200'
      route: '/test-microfrontend'
      identifier: 'test-mf'
      mountFunction: 'renderApplication'
      unmountFunction: 'unmountApplication'
      parameters:
        - name: 'name'
          valuePath: 'DataProduct.name'
        - name: 'descriptor'
          valuePath: 'descriptor'

The documentation has been updated accordingly to better explain to developers how to start developing a new microfrontend application from scratch.

Related issue

Closes #1340

[#1339] Change default values for technical info​

New features and improvements

  • Added disclaimer for publicInfo and privateInfo in docs

Related issue

Closes #1339

[#1133] Relations and Permissions tables​

New features and improvements

Add roles and permissions on the user settings section

Related issue

Closes #1133

[#1339] Change default values for technical info​

New features and improvements

  • Changed from deployInfo to info in app-config.yaml

Related issue

Closes #1339

[#1337] Remove question mark in Gitlab relative path​

Bug fixes

  • add removeQuestionMark function inside the GitlabRepositoryManager that handles file links with "?"

Related issue

Closes #1337

[#1313] Update the test system​

New features and improvements

The updates for the test system. Redesigned selectors and paths for them in most cases. Improved the mechanism of data product creation and deletion, and optimized the process of policy creation. Cleaned up a large amount of unusable code part Refactoring work has been done.

Related issue

Closes #1313

[#1332] Update DP graph version​

Breaking changes

Update dependency graph version to 2.0.7

Related issue

Closes #1332

[#1302] Transition from APIs to Native Git​

New features and improvements

  • native git implementation of branchExists method
  • native git implementation of newBranch method
  • native git implementation of getDefaultBranch method
  • native git implementation of createBranch method
  • native git implementation of listBranches method
  • native git implementation of listFiles method
  • unit tests for helpers.ts
  • refactoring of NativeGit and helpers functions
  • documented the state persistence task displayed among the provisioning plan steps
  • Native push method can now handle multiple files in a single push operation
  • commit.service optimized for pushing multiple files at once
  • snapshot.service optimized for pushing multiple files at once
  • newversion.service optimized for pushing multiple files at once
  • push method now handles also removal of files/directories
  • push method (APIs and Native implementations) now creates first the branch if the one provided does not exists yet
  • push method now takes in input fromBranch: it optionally creates a new branch, if not exists, from it. Otherwise it creates the new branch from the default branch
  • removed tree git method, it is not used anywhere
  • Release, Commit, and New Snapshot services now push changes in their root method in builder.client
  • pushFiles method inside repository.service handles the push of file changes in builder.client

Bug Fixes

  • delete folder 'releases' in the new branch, when creating a new version of a data product
  • call safeRefreshEntity after reverse provisioning task is completed and the new catalog-info is saved
  • fix tooltip in the New Snapshot button

Related issue

Closes #1302, #1226, #578

[#1317] Create Blueprint import Page​

New features and improvements

Create Blueprint import Page

Related issue

Closes #1317

[#1329] Add unit labels to supervision graphs​

New features and improvements

Added the unit to the axis for the graphs (days).

Related issue

Closes #1329

[#1270] Refactoring Control Panel Test Table​

New features and improvements

Refactoring test table

Added component name on test results

Related issue

Closes #1327 #1270

[#1325] Configurable deploy step result parsing​

New features and improvements

  • Add deployStep.resultField optional string configuration field under mesh.provisioner to specify a field name that will be the field extracted from the deploy step result, displayed in the deploy step detail drawer (instead of the complete result).

Related issue

Closes #1325

[#1316] Update the catalog table error badge to include reserved field errors​

New features and improvements

  • Update CatalogTable to show also eventual entity reservedFieldErrors with the existent error badge component in the corresponding row.

Related issue

Closes #1316

[#1326] Platform settings charts​

New features and improvements

Defined a new page "Supervision" in the platform admin section, that displays the events data in terms of lead time and ROI with two graphs.

Related issue

Closes #1326

[#1321] Improve error handling for subdomains​

New features and improvements

Mention the possibility of adding a subdomain when registering a domain.

Bug fixes

  • Make DomainProcessor and MarketplaceDomainProcessor accept the following formats as `subDomainOf` name:
    • <domain_name>
    • domain:<domain_name>
    • domain:default/<domain_name>

Example: subDomainOf: domain:default/myDomain

Related issue

Closes #1321

[#1323] Events Plugin - Add identifier columns​

New features and improvements

  • Added an identifier column on tables event_types and phase_types (backstage db, events schema)
  • New script to populate the events schema tables with mock data

Related issue

Closes #1323

Bug fixes

Fix the wrong link in the new marketplace question notification

Related issue

Closes #1322

[#1239] Make metadata.name and kind unchangeable​

New features and improvements

  • Extends ReservedFieldsProcessor to work with 'metadata.name' and 'kind'.

Related issue

Closes #1239

[#1324] Plans with NOT_STARTED status do not render some steps​

Bug fixes

  • Fixed not rendered tasks for NOT_STARTED plans

Related issue

Closes #1324

[#1319] Remove infrastructureTemplateId as required​

New features and improvements

We have removed infrastructureTemplateId as a required field for systems

Related Issue

Closes #1319

[#1281] Align alert details dialog with the new design​

New features and improvements

  • Aligned the CustomAlertDetails component with the new design.

Related issue

Closes #1281

[#1320] Bugfix Policy Wizard for changing Context on edit policy breaks the stepper​

Bug fixes

Fixed a bug that prevented the change of context for passive policies

Related issue

Closes #1320

[#1315] Events Plugin​

New features and improvements

  • New wb-events-backend backend plugin with db migration
  • New config events.enabled (false by default) to enable the events plugin functionalities

Installations

Config events.enabled is false by default. To enable the (experimental) events functionality and db migrations it must be set to true.

Related issue

Closes #1315

[#1085] Improve scaffolder error when registering invalid component​

Bug fixes

  • Better format the error coming to the analyze step

Related issue

Closes #1085

[#1223] Review characters limit​

New features and improvements

Updated max review length to 5000 characters and improved error message.

Related issue

Closes #1223

[#WIT-89] Ontology Doc details update​

New features and improvements

Update ontology documentation

Related issue

Closes WIT-89

[#1314] Resolve "Spelling mistake in Blueprints Documentation"​

Bug fixes

Corrected multiple typos and mistakes in the documentation.

Related issue

Closes #1314

[#WIT-31] Ontology documentation​

New features and improvements

Ontology documentation

Related issue

Closes WIT-31, WIT-32

[#1253] Outdated documentation for Specific Provisioner section​

Bug fixes

  • Improved the documentation regarding the invocations of Specific Provisioners since it was outdated.
  • Added a section in the documentation that references our open-source starter kit

Related issue

Closes #1253

[#1267] Incomplete doc in Configuring Git credentials​

New features and improvements

Added a description of the scopes required for Gitlab tokens.

Related issue

Closes #1267

[#1263] DP's versions are not displayed during OP creation​

New features and improvements

  • Renamed the menu items "Create" to "Templates" and "Blueprint" to "Blueprints"
  • Added the version to the entity picker when selecting systems
  • Added the version to the data products displayed in the DP Graph
  • Refactored the MarketplaceProductsGraph page by improving a lot the readability and the performance of the code
  • Updated the DP Graph to version 2.0.6:
    • better handling of spaces
    • removed a bug with three dots appearing on the page
    • display of empty domains
    • better management of labels, especially with long names

Related issue

Closes #1263

[#1283] Proper handling of GraphQL errors​

New features and improvements

  • Updated the FE components to keep the visualization consistent in case of GraphQL errors, and to keep showing the entire pages and all the related elements. These changes affect mainly the wb-governance and wb-marketplace plugins.
  • Generalized at the application level the component used to show GraphQL errors. This component is the usual expandable error panel that will be displayed only inside the affected part of the page, without altering the rest of it.

Bug fixes

  • Fixed wb-marketplace CatalogTable FE bug due to missing error handling on loadLatestGovernanceEntityEvals call.

Related issue

Closes #1283

[#1311] Fix Policy Overview top failed table showing empty tag column​

New features and improvements

Improved top failed policy table view

Related issue

Closes #1311

[#1278] Reverse provisioning suit​

New features and improvements

A new test suit that focuses on the Reverse provisioning feature has been added. The suit includes:

  • test that will check the general requirements for RP operation
  • test that will check the positive workflow of the reverse provisioning operation
  • test that will check the return options at each RP step
  • test that will check the operation of saving a yaml file content to a repository

Related issue

Closes #1278

[#943] Add usePersonalToken to docs​

New features and improvements

Updated documentation to include "usePersonalToken" among configurations

Related issue

Closes #943

[#WIT-64] Update ontology page style​

Bug fixes

Update ontology page with new style components

Related issue

Closes WIT-64

[#1310] Bug Fix Policy Overview distribution not adding to 100%​

Bug fixes

Fixed % not adding to 100% in case of both distributions being .5 rounding to 1 excess

Related issue

Closes #1310

[#1268] The policy cron field has an invalid error message​

Bug fixes

Fix bug on required cron expression inside policy/metric wizard

Related issue

Closes #1268

[#1271] Add instructions to run witboost on Windows​

New features and improvements

  • Modified steps on GETTINGSTARTED.md to run the UI on WSL
  • Added scripts to route requests between Windows, WSL, and Docker network

Related issue

Closes #1271

[#1248] Improve Show More/Show Less in description​

New features and improvements

Refactoring expandable items to improve responsiveness

Related issue

Closes #1248

[#1250] Change the default displayed rows​

Bug fixes

Fix for the multiple re-renders on the marketplace dp catalog table

Related issue

Closes #1250

[WIT-38] Handle hierarchy state in OntologyApi​

New features and improvements

  • Added a call to ontology api backend to remove hierarchy status when a hierarchy is deleted

Related issue

Closes WIT-38

[WIT-53] WIT-53 Wire status management to hierarchy processor​

New features and improvements

Fix on query that checks if a hierarchy has already a set status

Related issue

Closes WIT-53

[WIT-88] Add JSON schema validation for hierarchy kind​

New features and improvements

  • Added JSON schema for the Hierarchy kind

Bug fixes

  • Removed duplicated relations in relations.ts

Related issue

Closes WIT-88

[WIT-81] Add error detail to hierarchy​

New features and improvements

Add error detail to hierarchy list, and banner in page

Related issue

Closes WIT-81

[WIT-87] Possible initialization error in hierarchy processor​

New features and improvements

The block that calls ontology api now is surrounded by a try-catch block to avoid initialization errors.

Related issue

Closes WIT-87

[#1250] Change the default displayed rows​

New features and improvements

Move useStorageParams hooks to wb-platform to share among plugins.

Add query params on the marketplace DP catalog table to save

  • Pagination limit
  • Search text

Related issue

Closes #1250

[WIT-53] Status management to hierarchy processor​

New features and improvements

When a new hierarchy is created, it automatically set the status to enabled.

Related issue

Closes WIT-53

[#1306] Revamp the Governance test page​

New features and improvements

  • Update the governance test policy/metric pages using the drawer.
  • Move the test panel to the right
  • Remove the X icon on the selected test

Related issue

Closes #1306

[#1287] Convert marketing output port details page to new WB design​

New features and improvements

Refactoring output port page

Related issue

Closes #1287

[WIT-82] Add a getter for Hierachy Validation Errors​

New features and improvements

  • Added getHierarchiesProcessingErrors inside the OntologyClient that retrieves all processing errors happened during the processing of hierarchies

Related issue

Closes WIT-82

[WIT-64] Update ontology style​

New features and improvements

Update ontology style

Related issue

Closes WIT-64

[#1288] Convert control panel page to new WB design​

New features and improvements

Convert the control panel page to the new WB design.

Related issue

Closes #1288

[#1224] Show warning message when no data is returned on entity picker to give users a hint they might be missing permission​

New features and improvements

  • Add a warning message for entity picker when no data is present, telling the user to check permissions. This will become an error message for required fields.

Related issue

Closes #1224

Bug fixes

Removed the inconsistent link in the documentation.

Related issue

Closes #1307

[#1304] Add sorting on columns of the DP catalog page​

New features and improvements

Add sorting to WbTable Add sorting on the Catalog table for the following columns:

  • Name
  • Domain
  • Published On

Related issue

Closes #1304

[WIT-59] Ontology fsm implementation​

New features and improvements

Now we can call DatabaseOntologyService methods with the parameter that is HierarchyEntity. This allows us to CRUD hierarchies as a whole, meaning we can for example update a hierarchy state and at the same time we can update the state of all the corresponding HierarchyClasses.

Related issue

Closes WIT-59

[WIT-80] Inverse system relationships should be checked to have "wb" prefix​

New features and improvements

technical relationships and the respective "inverse" should have the same "wb" prefix.

Related issue

Closes WIT-80

[#1303] Improve git push without permissions error message​

Bug fixes

  • Improved git push permissions error message
  • Restored fixed height for reverse provisioning logs viewer
  • Prevented horizontal scrollbar to show in the reverse provisioning save dialog

Related issue

Closes #1303

[WIT-52] Define system wide prefix for ontology relations​

New features and improvements

  • Now system relations need to be prefixed with wb-
  • User defined relations instead will be prefixed by processors with wb-u-. at configuration time, user defined relations do not need any prefix.

Related issue

Closes WIT-52

[WIT-27] Add hierarchy delete button​

New features and improvements

Add hierarchy delete button

Related issue

Closes WIT-27

[#1299] Redesign Marketplace specific DP page​

New features and improvements

Refactoring to accommodate new page design

Added breadcrumbs missing on the DP page

Updated docs

Related issue

Closes #1299, #1286

[#1282] Update the deploy step details drawer to show errors if present​

New features and improvements

  • Updated design of LogsDrawer and DeployStepDetailDrawer components.
  • Added a new utility to extract errors from stringified results coming from the deployment steps.
  • Updated LogsCard to use the CodeSnippet component.

Bug fixes

  • Removed unused/unnecessary FE components.

Related issue

Closes #1282

[#1274] Implement git saving mechanism on reverse provisioning​

New features and improvements

  • Two new endpoints (GET and PUT) /components/<namespace>/<name>/catalog-info are now exposed by the builder backend to read and update the catalog-info.yaml of a witboost component
  • The Reverse Provisioning's comparison page features a new Save button (next to the Copy and Download buttons) to push updates to the catalog info file without leaving witboost
  • The wb-repository-manager plugin and the utilities it exposes have been extended with methods and procedures to interact with a remote git repository by issuing native git commands, instead of consuming git providers' (Bitbucket, Gitlab) REST APIs
  • A new configuration mesh.repositoryManager.prioritizeGit (true by default) allows to prioritize a native Git workflow (i.e. Git CLI commands) over API usage when both integrations are available
  • The Reverse Provisioning user manual has been updated with references to the save mechanism

Bug fixes

  • Restored NotFoundError handling on user config backend
  • Fixed the tooltip on the New Snapshot button (the disabled-button message was being displayed when the button was enabled and vice-versa)

Related issue

Closes #1274

[#1090] Tags metadata model expected by the builder differs from Marketplace​

New features and improvements

Get tags from the entity using the entity.spec.mesh.tags instead of entity.metadata.tags. They must have the following format:

tags:
      - tagFQN: HR -> **Required**
        source: Tag
        labelType: Manual
        state: Confirmed

otherwise "No tags" label is displayed.

Related issue

Closes #1090

[WIT-65] Fix domain relations​

Bug fixes

Fix domain relations in the ontology graph

Related issue

Closes WIT-65

[#1279] Add labels to the buttons at card/widget level​

New features and improvements

Update buttons with color and icons in GCP and marketplace

Bug Fix

Fixed env selector not updating correctly the state

Related issue

Closes #1279

[#1284] Fix migration script for ontology backend​

Bug fixes

  • Added migrations to the files property of ontology-backend package

Related issue

Closes #1284

[WIT-60] Create new method in the facility to unregister hierarchies​

New features and improvements

  • Added a new method deleteHierarchyByName in OntologyApi

Related issue

Closes WIT-60

[WIT-58] Self references not allowed in hierarchy​

New features and improvements

Self references are now allowed when defined by users. Self reference defined by technical relations (prepended by "wb-") are checked according to the rules defined in the HLD.

Related issue

Closes WIT-58

[WIT-18] Handle hiearchy instance as a FSM​

New features and improvements

Ontology is now a plugin. This includes 4 new APIs to perform CRUD on top of ontology state(we will need it to make it FSM)

Migration

New table ontology introduced with columns id, hierarchy, status

Related issue

Closes WIT-18

[#1179] Template entity errors​

New features and improvements

  • Show a different style and an error badge on the template card when there are entity errors on the template. Same for Blueprints.

Related issue

Closes #1179

[#1275] Update marketplace DP Catalog page to new design​

New features and improvements

Replace the marketplace DP Catalog table with the new one.

Related issue

Closes #1275

[#1276] Redesign the DP graph according to the new design​

New features and improvements

  • Refactoring the DP Graph page
  • Updated docs

Related issue

Closes #1276

[#1280] Fix env context not aligned between marketplace and cgp​

New features and improvements

Improved governance registry tables

Bug fixes

Fix environments' context between cgp and marketplace

Related issue

Closes #1280

[#1231] Fix deploy steps panel style​

New features and improvements

  • Show No data available label where there are no logs on the deployment step, to inform users that there are no data.

Bug fixes

  • Fixed DeployStepDetailDrawer style.

Related issue

Closes #1231

[WIT-16] Add hierarchy class instances processor​

New features and improvements

  • Added HierarchyClassInstanceProcessor which enables entities to inherit relations as defined by their parent class

Migration notes

Entities are matched by their kind and type field. If there is a entity of type HierarchyClass for that entity's kind and field, then all class configurations will be applied to the instance. This means that this feature is retroactive, i.e. it applies also to already registered entities, if ontology is enabled.

Bug fixes

  • Fixed typo in the Ontology settings page

Related issue

Closes WIT-16

[#1277] Error on the "Docs" tab of the data product​

Bug fixes

Moved the logic to hide the entities from EntityPage to CatalogEntityPage, in this way, we don't change the outlet and avoid the doc plugin routing problem

Related issue

Closes #1277

[#1255] Incorrect hierarchy in Software Catalog​

New features and improvements

  • Add Software Catalog in the breadcrumb, linked to /catalog

Bug fixes

  • Breadcrumb now displays the correct kind of the entity

Related issue

Closes #1255

[#1264] Refactoring Mesh Supervision Page to new design​

New features and improvements

Refactoring mesh supervision page and charts

Updated relative image in docs

Related issue

Closes #1264

[#1160] Rework of the UI Alert component​

New features and improvements

  • Implemented the new customAlertDisplay component to show notification messages (all severities) and generalized at platform level.
  • Implemented the new customAlertDetails component in case of backend errors coming from the Coordinator or CGP, to show additional information about the error occurred: problems, possible solutions, the input descriptor (optional) and the field that generated the error (optional).
  • Added a badge to the customAlertDisplay component to notify the user in case of multiple notifications stacked.
  • Generalized handleFailedResponse handler at platform level and added all the required error cases.
  • Generalized getErrorMessage utility at platform level and added all the required cases.
  • Replaced all the occurrences of alertApiRef with the new customAlertApiRef that is supporting all the new custom error classes for better error handling.
  • Replaced some of the errorApiRef occurrences with the new customAlertApiRef to correctly display the error messages where needed.
  • Added new generateError function at platform level to correctly generate all the possible errors BE side (then properly handled FE side).
  • Updated the sendRequestToCoordinator and sendRequestToCgp functions to raise the correct new CoordinatorError in case the response received is containing an error compliant to the new error structure.
  • Added new custom error classes BackendError and CoordinatorError to handle and display correctly these error cases.

Breaking changes

  • None, since the new customAlertDisplay component will be retrocompatible with the old error structure for backend errors. In addition, it is also listening to the standard alertApiRef, so it is also compatible with usages of this api coming from the Backstage codebase. However, for our codebase, it is highly recommended to use only the new customAlertApiRef that is supporting the new CustomAlertMessage type when posting a message, which includes additional parameters and the possibility to directly (and correctly) post an error with the new error: Error field.

Bug fixes

  • Minor changes to fix some console FE errors.
  • Updated the "Entity not found" message to show a more meaningful message.
  • Fixed some wrong imports.
  • Minor changes to the parseYamlFromUrlIntoObject utility.
  • Fixed wrong usage of the response.text() method in the wb-governance plugin that was causing an incorrect parse of the response received and replaced with response.json().

Related issue

Closes #1160, #1097, #475

[WIT-43] Fetch hierarchy from facility api​

New features and improvements

Fetch hierarchy from facility api instead the catalog.

Related issue

Closes WIT-43

[#1272] Updating the autotests​

New features and improvements

  • A chromedriver updated
  • The "Access control" tests fixed

Related issue

Close #1272

[WIT-34] Create import ontology page​

New features and improvements

Add new hierarchy import page in the ontology section

Related issue

Closes WIT-34 WIT-42

[WIT-17] Hiearchy catalog info entities validation​

New features and improvements

Hierarchy instance is validated before being registered into the backstage catalog.

Related issue

Closes WIT-17

[#1257] Implement reverse provisioning logging mechanism and techdocs​

New features and improvements

  • Reverse Provisioning logging mechanism in step 3 of the Import Wizard
  • Added visual references to reverse provisioning template techdocs
  • Updated reverse provisioning doc for Platform Teams with a section about the log and techdocs mechanisms

Related issue

Closes #1257, #1273

New features and improvements

Add the possibility to hide the catalog entities by config

Related issue

Closes WIT-36

[#1260] Fix input ports error for multiple environments​

Bug fixes

  • Added a migration script for removing InputPort -> DataProduct foreign key and adding an InputPort -> DataProductInstance foreign key.
  • Updated all the Hasura queries

Related issue

Closes #1260

[#1258] Convert policy overview to new design​

New features and improvements

Refactoring the policy overview page

Related issue

Closes #1258

[#1261] Make the logs viewer a reusable common component​

New features and improvements

  • Replace logs card with wb-card to ensure having same shared style
  • Move the Logs card inside wb-platform plugin to reuse it inside the whole application
  • Add task inside log rows

Related issue

Closes #1261

[#1262] Fix import statement​

Bug fixes

  • Changed publishConfig property in wb-ontology-common's package.json

Related issue

Closes #1262

[#1241] Fix the values.yaml in the UI project that contains wrong values​

Bug fixes

Updated helm/values.yaml file

Related issue

Closes #1241

[#1259] Can't use marketplace search with unsafe HTTPS​

New features and improvements

Introduced new config variable useHttpsSearchUnauthorized that is used to query through https ignoring the certificate along the way (specific behaviour only introduced because of poste development environment)

Related issue

Closes #1259

[WIT-19] WIT-19 "Hiearchy retrieval facility"​

New features and improvements

  • Added OntologyApi with its related frontend client

Related issue

Closes WIT-19

[#1237] New test added to the CGP suit​

New features and improvements

  • Added a new test (04-18) to check policy enforcement after deploying multiple DPs. The runner creates and deploys two DPs with different descriptors. Three policies are then created and tested.
  • Fixing the existing testing system to match the new UI changes

Related issue

Closes #1237

[WIT-25] Add Platform Settings Permission​

New features and improvements

  • Added new permission to view and use the platform settings page

Migrations notes

  • For dev and demo environments, we should add the platform.settings.edit permission to users

Related issue

Closes WIT-25 WIT-20

[#1256] Update the CGP result page with the new theme​

New features and improvements

  • Add Widget component
  • Add Tabs component
  • Replace old components with new ones inside Overview, Test, and Control Panel page
  • Refactoring code
  • Add Dockable between policy cue script and descriptor when available

Related issue

Closes #1256

[WIT-41] Fix entities validation​

Bug fixes

  • Fixed Blueprint processor throwing error to entities of kind Hierarchy
  • Fixed Reverse Provisioning Template processor throwing error to entities of kind Hierarchy

Related issue

Closes WIT-41

[#1254] Convert Search Result & Metrics to single tabbed page​

New features and improvements

Refactoring search metrics and policy page to a single tabbed page

Related issue

Closes #1254

[#1249] Cannot view linked DP in the same Domain​

Bug fixes

Fixed a bug that showed in some environments input port relations taken from other environments, resulting in broken links on the graph and dependency pages.

Related issue

Closes #1249

[#1242] Convert CGP Policy and Metric List to new WB Design​

New features and improvements

  • Convert policies/metrics list to new WB Design
  • Add global selector for environment
  • Add new WBTabs components
  • Delete unused code

Related issue

Closes #1242

[#1246] Array Data Type inside descriptor's sample field​

New features and improvements

Update formatting for arrays

Related issue

Closes #1246

[#1252] Resolve [object Object] displayed in search page title​

New features and improvements

Added pageTitleOverride just to display better thing in the tab name while everything else stays the same

Bug fixes

[object object] is now displayed as "Search"

Related issue

Closes #1252

[WIT-21] Feature/create ontology page​

New features and improvements

Create ontology page in the platform settings section

Related issue

Closes WIT-21 WIT-23 WIT-24

[#1247] Load monaco editor loader locally​

Bug fixes

The monaco editor library is not anymore dependant on an external CDN

Related issue

Closes #1247

[#1235] Add documentation for Reverse Provisioning​

New features and improvements

Documentation for Reverse Provisioning

Related issue

Closes #1235

[#1199] Reverse Provisioning​

New features and improvements

  • app-config.yaml: add ReverseProvisioningTemplate in the allowed entities. Add reverseProvisioningEndpoint: /v1/reverse-provisioning and executionPlansEndpoint: /v1/execution-plans under mesh.provisioner
  • Add route in wb-scaffolder for the reverse provisioning
  • Add ReverseProvisioningTemplateEntity with its processor and validator
  • Add automatic registration of ReverseProvisioningTemplate when registering a new Template, if available.
  • Add reverseProvisoning.service in the UI backend
  • Add routes GET /cataloginfo, POST /reverse-provisioning, GET /reverse-provisioning/:reverseProvisioningId/status, DELETE /reverse-provisioning/:reverseProvisioningId/terminate, POST /reverse-provisioning/:reverseProvisioningId/plans
  • Add parsers backend side for the new requests (start reverse provisioning, status, cataloginfo url, plans)
  • Add unit tests for both reverse provisioning services and utils
  • Add Import button in AboutCard only for entities with kind = component
  • Add ImportDialog that opens when clicking on the Import button
  • Add new APIs frontend side to reflect the new backend routes
  • Add ReverseProvisioningStatus and ReverseProvisioningPlansAndMerged objects
  • Add ReverseProvisioningContext frontend side, to hold all the required information for the entire process
  • Add ComparisonWizard that holds monaco editor, added as a dependency in package.json
  • Add ConfirmationDialog that opens if the user wants to terminate the processing step
  • Add InputForm that renders the ReverseProvisioningTemplateEntity (1st step)
  • Add ReviewInformation that renders the previously input fields to send to the coordinator (2nd step)
  • Add Processing component that renders logs fetched from the provisioning coordinator (3rd step)
  • Add ViewDescriptorEditor component that renders the sparse catalog-info received from the coordinator (4th step)
  • Add ReverseProvisioningWizard that holds the 4 steps of the reverse provisioning task
  • Add ReverseProvisioningPage that holds all the previously components and the context
  • Add utils frontend side to download and copy the new catalog-info and to extract errors to display in processing step

Migration

  • ReverseProvisioningTemplate has to be appended to the list of allowed entities in all the config files under catalog.rules.allow

Bug fixes

  • EntityNamePicker: While testing the ReverseProvisioningTemplate, we found a bug by using a field as required with this specific entity (Name field). If the user writes the name and removes it, leaving it blank, the Form won't throw a validation error and continues to the next step. We want to prevent this behaviour by throwing a validation error if the string name is left empty AFTER it was previously populated.

Related issue

Closes #1199

[#1244] Convert CGP Metric Search to new WB Design​

New features and improvements

Redesigned page

Related issue

Closes #1244

[WIT-37] Add hierarchy and hierarchyclass to allowed kinds in rbac​

New features and improvements

  • Added Hierarchy and HierarchyClass to always allowed kinds in RBAC

Related issue

Closes WIT-37

[#1240] Convert CGP Policy Search to new WB Design​

New features and improvements

Refactoring CGP Search Policy Page to new design and wb common components.

Related issue

Closes #1240

[#1206] Combo box style convention​

New features and improvements

Edit snake case on wizards

Related issue

Closes #1206

[WIT-15] Resolve "Add relations to hierarchy processor"​

New features and improvements

  • Added hierarchy class processor that emits class relations

Related issue

Closes WIT-15

[WIT-13] Add hierarchy processor​

New features and improvements

  • Added a simple HierarchyProcessor that is just capable of transforming a hierarchy configuration in a set of classes. No validation yet. No relations yet.

Breaking Changes

  • Added new required property under app-config that is found at ontology.enabled. By default this flag is set to false

Bug fixes

  • Fixed a bug on the BlueprintProcessor that prevented new entity kinds with a different apiVersion to be added to the catalog (they were not passing the BlueprintProcessor validator)

Related issue

Closes WIT-13

[#1233] Resolve "80's scroll bar style on windows"​

Bug fixes

Show scrollbar on blueprint cards on mouse hover only.

Related issue

Closes #1233

[#1120] Return a list of errors for entity in the catalog​

New features and improvements

  • Create custom endpoint to get the list of errors for entities.
  • Show an error badge for data products that have any errors.
  • Show error details on hover on error badge.

Related issue

Closes #1120

[#1220] Define Card at App level​

New features and improvements

  • Add dockable components

Related issue

Closes #1220

[#1234] Resolve UTC vs Local Time check​

Breaking changes

Fixed check on end date BE

Related issue

Closes #1234

[#1216] Improve CGP module test suit​

New features and improvements

  • Added a positive test that will check policy mapping (valid cue script + deployment timing) in the DP control panel
  • Added a negative test that will check policy mapping (invalid cue script + runtime timing) in the Marketplace (flags).
  • Added a positive test that will check policy mapping (valid cue script + runtime timing) in the Marketplace (flags).
  • Added a positive test to check the creation of a Passive policy and that options (timing, engine, resource pre-processing) are disabled during creation.
  • Added a test to check the creation of the "Previous and current" policy

Related issue

Closes #1216

[#1227] Centralize version script​

New features and improvements

Removed get-version.sh script and added a job from common CI library

Related issue

Closes #1227

[#1141] Rbac domain registration​

New features and improvements

  • Add a check in the register operation that checks the domain location already registered
  • Update the documentation regarding same name domain registering

Related issue

Closes #1141

[#1187] Add authorization in notifications plugin​

New features and improvements

Add authorization in notifications router

Related issue

Closes #1187

[#1091] Add a new panel to display logs​

New features and improvements

  • Add new drawer to show deploy logs
  • Poll logs if the deploy is in running otherwise fetch just one time
  • Add logs card to show logs inside deploy step drawer
  • Add copy button to allow user to copy logs
  • Add BE proxy to fetch data from coordinator
  • Add new endpoint inside app-config.yaml to get logs

Related issue

Closes #1091

[#1219] Define Table Wrapper at App level​

New features and improvements

  • Added table wrapper
  • Added witboost card container

Related issue

Closes #1219

[#1221] The deploy pipeline for a data product is displaying a wrong name for dp similarity​

Bug fixes

  • Fixed display name for marketplace compliant tasks
  • Fixed not showing details for metrics and policies in the control panel

Related issue

Closes #1221

[#1207] New Generic App Table​

New features and improvements

Added generic Witboost Table and its components

Related issue

Closes #1207

[WIT-9] Create new plugin platform settings​

New features and improvements

Add new plugin platform settings. To show it add enabled:true under mesh->platformSettings inside the app-config.yaml

Related issue

Closes WIT-9

[#1212] Improving the policy test suite​

New features and improvements

Added new test scenario:

  • create three cue policies with different severity
  • each policy has an invalid cue script = domain:number
  • severity = info, warning, error
  • every time is checking the displaying these policies in the control panel of the created dp.
  • the policy goes through all stages: Draft-Disable, Grace-Deprecated, Enable

Related issue

Closes #1212

[#1215] Hide stopTime field when dag status is running​

Bug fixes

stopTime field is now hidden when a provisioning task status is running

Related issue

Closes #1215

Changelog v1.3.1​

[#1385] Unsafe Advanced Search toggle not working​

New features and improvements

The old feature of useHttpsSearchUnauthorized seems to be removed. I am bringing back the configuration option.

Related issue

Closes #1385

[#1381] Modify EntitySelectionPicker in order to handle data types different from strings​

New features and improvements

  • EntitySelectionPicker can resolve array and object values inside of it.

Related issue

Closes #1381

Changelog v1.3.2​

[WIT-132] Fix task status after execution plan timeout​

Bug fixes

  • The provisioning plan panel now supports the TERMINATED status
  • The provisioning plan task visualization order is now aligned with that communicated by the Coordinator

Related issue

WIT-132

[#1384] Resolve "RBAC tables not loaded in Settings"​

Bug fixes

  • Fixed the huge filters list that were sent to RBAC backend from UI
  • Moved RBAC tables pagination to backend
  • Moved RBAC tables filtering to backend

Related issue

Closes #1384

[#1396] Bad formatted JWT makes backend crash​

Bug fixes

  • audit middleware now correctly handles bad formatted jwt token

Related issue

Closes #1396

[#1395] Alphabetical order inside Catalog​

Bug fixes

  • Catalog page now displays entities in alphabetical order based on the Name column

Related issue

Closes #1395

[#1382] Edit Metadata and different branches in bitbucket​

Bug fixes

  • bitbucket server integration now correctly resolves edit url

Related issue

Closes #1382

[#1398] Data product description​

Bug fixes

  • description in AboutCard now handles /n

catalog-info.yaml

description: |
    This is a multiline


    string using the literal


    block scalar.

Related issue

Closes #1398

[#1400] Access request status field does not display the correct status​

Bug fixes

  • Fixed circle display error

Related issue

Closes #1400

[#1401] Successful test should say success not completed​

New features and improvements

  • The test panel now informs you about a successful test with "Success" instead of "Test completed"

Related issue

Closes #1401

[#1404] Reverse provisioning templates require domain​

Bug fixes

  • ReverseProvisioningTemplate does not require the domain key anymore for RBAC.

Related issue

Closes #1404

[WS-514] Optimize provisioning plan retrieval​

New features and improvements

We are now preventing the Coordinator to send data product descriptors when getting provisioning plans, if they are not needed. This way we're saving memory both on the Coordinator and the Builder.

Changelog v1.3.3​

[#1406] "Tags on Domain/Subdomain"​

Bug fixes

  • Metadata tags are added to "About" content along with the fqn tags

Related issue

Closes #1406

Changelog v1.3.4​

[#1397] Resolve the pagination issue in RBAC table​

New features and Improvements

  • Add warnings to both permissions and roles table
  • Add rbac view service to handle frontend requests

Bug fixes

  • Permission table in settings now correctly displays values

Related issue

Closes #1397

[#1413] Cannot change page in Marketplace > DP Catalog​

Bug fixes

  • Table now uses the correct count returned by the graphql query

Related issue

Closes #1413

Changelog v1.3.5​

[#1416] Governance fix for View or Edit entity permission​

New features and improvements

  • Add loading in CGP page
  • Backend now handles CGP view permissions

Related issue

Closes #1416

[#1417] [WS-322] Domain sorting in the navigation panel​

Bug fixes

  • Domains in the navigation panel are now alphabetically sorted

Related issue

Closes #1417

Changelog v1.3.6​

[WIT-526] Add token in catalog call and skip annotations if config not enabled​

Bug fixes

During entity processor execution:

  • Add token in catalog call
  • Skip annotations if config not enabled
  • Improve docs

Related issue

Closes WIT-526

[WIT-525] ErrorBox now handles new lines and tabs​

  • The error box now handles new lines and tab characters

Closes WIT-525

[#1408] Default Environment sorting​

  • Set the environments order in the marketplace by priority, and alphabetically within those who have no priority

Closes #1408

[#1419] Pass token to the micro frontend​

New features and improvements

  • The token is now passed to the micro frontend

Breaking changes

  • Field microfrontendParameters is now called parameters as seen in the Microfrontend Example

Related issue

Closes #1419

[#1328] RBAC entity_ref and Backstage urn​

New features and improvements

  • Imported catalogRules and modified methods hasSpec and hasMetadata to be case-insensitive
  • witboostRules are now case insensitive in the methods hasSpec and hasMetadata
  • Entries in rbac.roles_subjects are sanitized before being passed into the permission middleware: all values in entity_ref are converted to lowercase and then duplicates are removed
  • hasAnnotation, hasLabel, are left case-sensitive
  • DP Graph does not duplicates domain values
  • infrastructure_template_id is leaved case sensitive due to coordinator requirements

Related issue

Closes #1328

[#1420] Introduce tags in the Markeplace​

New features and improvements

The witboost search is now able to search for all DPs that have a certain tag, both in the marketplace and in the builder. To search in the tags you need to put the exact value you would like to search for.

Closes #1420

Changelog v1.3.7​

[WIT-571] Deactivate ReservedFieldsProcessor if not in config​

Bug fixes

  • Deactivate ReservedFieldsProcessor if not in config

Related issue

Closes WIT-571

[WIT-491] Fix Error Log When Decoding Server To Server Token​

New Features And Improvements

  • You can now optionally decide to include server-to-server requests inside the audit registry by setting backend.audit.skipServerToServerRequests to false

Bug fixes

  • Fixed a bug that caused the audit plugin to throw an error when calling parseEntityRef on a server-to-server token

Related issue

Closes WIT-491

[WIT-318] Fix microfrontend rendering problems​

Bug fixes

Better explanation of how microfrontends should be implemented to avoid rendering problems. In particular, microfrontends should always define a seed for the generated class names so they don't collide with existing ones.

Related issue

Closes WIT-318

[WIT-557] Add error messages to unauthorized responses​

Bug fixes

  • Fixed a bug that caused a JSON parsing error to show when the request was unauthorized.

Related issue

Closes WIT-557

[WIT-319] cannot test a cue engine policy​

Bug fixes

  • The field errors in EvaluationResult is now optional

Related issue

Closes WIT-319

Changelog v1.3.8​

[WIT-643] Resources are not shown​

Bug fixes

  • Added a new RBAC rule to show resources when no domain is associated with them to all users, and only to users belonging to that domain when it is defined.

Related issue

Closes WIT-643

[WIT-641] Add a string conversion in the indexing mechanism​

New features and improvements

  • Fixed Marketplace indexing for some Data Products

Related issue

Closes WIT-641

[WIT-637] Do not display empty values for custom fields​

New features and improvements

  • Do not display empty values for additional general info custom fields when they are empty.

Related issue

Closes WIT-637

[WIT-591] Fix roles table in settings​

Bug fixes

  • Both tables in settings Permission Table and Role Table along with their drawers now handle the case in which entity_ref is null in the table.

Related issue

Closes WIT-591, WS-339

[WIT-339] Fix the hint message on the cron field​

Bug fixes

  • Fixed the hint message under cron fields to reflect that the Day of Week value has a range of 0-6 where 0 is Monday
  • Fixed cron preset labels

Related issue

Closes WIT-339

Changelog v1.3.9​

[WIT-657] Tags not correctly displayed in the output port marketplace page​

Bug fixes

  • In the marketplace output port page, the data is displayed even if tags are not defined.

Related issue

Closes WIT-657

[WIT-648] Make data product descriptor and version available for microfrontends in the output port page​

New features and improvements

  • Version and descriptor of the data product are available in the marketplace for output port microfrontends.

Related issue

Closes WIT-648

Changelog v1.3.10​

[WIT-732] Entity picker for groups and users filtered based on current user​

New features and improvements

Filter entities by current user ownership on:

  • Access request dialog, by using:
mesh:
    marketplace:
        ui:
        accessControl:
            showOnlyUserOwnGroups: true
  • Entity picker on schema template (using showOnlyUserOwnGroups as ui:option)

Related issue

Closes WIT-732

[WIT-756] Close the unregister entity dialog​

Bug fixes

  • After clicking on the “Unregister” button in the window of the domain deleting, if the unregistration fails, the window will be closed and the message about the error will be displayed.

Related issue

Closes WIT-756

[WIT-588] Add snapshot version on deployments​

Bug fixes

  • In the data product deployments, the "version" property has been added to show the right version of the snapshot.

Related issue

Closes WIT-588

[WIT-486] Add feature to block a domain unregistering when the domain is not empty​

New features and improvements

  • Add a feature that blocks domain unregistering if the domain is not empty. A domain is not empty if the domain has a data product registered.
  • In the marketplace, if the data product has a null value in the domain column (for example if the domain was deleted) the string "Unassigned" will be displayed.

Upon attempting to unregister a location that contains a domain, if the domain contains a system and you have the configuration enalbed, you will be shown an error message.

Migration note

  • The feature is disabled by default, so there are no breaking changes. To enable it, set in app-config.yaml the following line:
catalog:
    disableNonEmptyDomainUnregister: true

as written in the documentation.

Related issue

Closes WIT-486

Provisioning Coordinator​

Changelog v1.2.0​

[#501] Clear error message when component validation fails​

New features and improvements

  • Http errors are now caught and processed by the coordinator using the CoordinatorError and message formatting
  • Errors coming from external witboost services are now marked appropriately. Added MARKT_ERROR and DATA_CAT_ERROR for Marketplace Plugin and Data Catalog Plugin errors respectively.
  • Improved testing at API application level

Bug fixes

Fixes a bug where contacting an inactive service would cause an error to be thrown and shown in the UI as-is.

The message formatting is constrained to the TaskResult interface, which only receives a string as a parameter. The coordinator returns a formatted message with line breaks between problems and solution items, but this seems to be overridden at the UI level.

Related issue

Closes #501, #495

[#507] Update specific provisioner API docs​

New features and improvements

  • Added disclaimer for publicInfo and privateInfo fields in docs

Related issue

Closes #507

[#499] feature: Interface specification versioning​

New features and improvements

  • Removes the creation of the buildInfo class from the sbt tasks
  • Modifies the SP interface specification version with a fixed value starting on 2.1.0 (2 for the v2 endpoints, and .1 for the new error structure)
  • Adds the new error structure to the SP interface specification respecting the retrocompatiblity

Migration

Specific Provisioner developers can now start migrating to the new error structure, while maintaining retrocompatibility in the meantime.

Breaking changes

Specification will now have a set version that needs to be modified manually every time the interface-specification is worked on. The endpoint versioning should continue as it has been done so far, since this is the contract with the other services.

Related issue

Closes #499

[#WIT-124] Fix provisioning coordinator multi deploy​

Closes WIT-124

[#469] CI build optimization​

New features and improvements

CI build job refactoring

Related issue

Closes #469

[#WIT-96] Mock provisioner in coordinator project​

New features and improvements

Mock specific provisioner is inside the coordinator codebase. Integration tests are run using the same mock provisioner that is run in dev environment.

Related issue

Closes WIT-96

[#505] Show component name in validation result​

New features and improvements

  • Added componentName field to Task model returned by /v1/provisioningplan

Related issue

Closes #505

[#503] /resource/dataproduct API does not include the operation​

New features and improvements

  • /resource/dataproducts API now returns the operation performed for each descriptor

Related issue

Closes #503

[#502] CGP - default environments​

New features and improvements

New default cgp environments

Related issue

Closes #502

[#480] get data products api​

New features and improvements

  • Handled the case of lastDeploymentDate as null

Related issue

Closes #480

[#492] Add instructions to run coordinator on Windows to Readme​

New features and improvements

Modifies the readme.md to include information about running the Provisioning Coordinator on windows.

Related issue

Closes #492

[#480] /get data products api​

Bug fixes

  • Now the /v1/resources/dataproducts API is returning the last deploy time parameter
  • Modified LocalDateTime to OffsetDateTime for selectByLastDeploymentDateFragment method

Related issue

Closes #480

[#496] failed specific provisioner validations return status null on the response body​

New features and improvements

Fixed parsing of validation result in application layer

Bug fixes

Fixes a bug where the UI would show OK on a failed validation since status was returned null

Related issue

Closes #496

[#435] Resolve "Propagate error message from external call responses"​

Bug fixes

  • Errors are now correctly propagated from external services.

Related issue

Closes #435

[#494] Fix documentation​

Bug fixes

  • Fixed documentation and moved Coordinator and CGP Errors sections to the tech folder

Related issue

Closes #494

[#493] CGP - bug on cue file process creation​

Bug fixes

Cue process files with unique name.

Related issue

Closes #493

[#487] Scheduler hangs-up during sync validation with a lot of components​

Bug fixes

The scheduler now uses a chached thread pool for handling operation instead of an configurable fixed thread pool

Related issue

Closes #487

[#491] hotfix: CGP and Coordinator - error logs that logs exceptions does not contain valuable informations​

New features and improvements

Errors now override toString to include the relevant information, plus the WithNoStackTrace trait is removed to allow stack trace logging

Bug fixes

Fixes a bug where console logging wouldn't show any information about the error that was logged.

Related issue

Closes #491

[#487] revert #487​

New features and improvements

Reverted (only for cgp) issue 487.

Related issue

Closes #487

[#480] retrieve dataproduct deployed​

New features and improvements

  • Added /v1/resources/dataproduct API used by DataProductSimilarity in order to populate its database with the existing data product descriptors. This API accepts in input a kind parameter that admits one value (for now): pre-deployed. This last is saying to the coordinator that the descriptors that we want to take are the ones without deploy info and build info.

Related issue

Closes #480

[#472] Modify interface specification with new errors​

New features and improvements

  • Modifies the open-api specification to the new error structure and now the error information is returned following this structure.

Breaking changes

  • None, as the structure is retrocompatible.

Migration

  • Callers to the CGP should now migrate to receive the new error structure and stop using the error and errors fields of the error response body as these are now deprecated.

Related issue

Closes #472, #473, #449

[#490] Docs is not containing any info about where to take secret info​

Bug fixes

  • Added the location for the secret field in the CGP configuration docs.

Related issue

Closes #490

[#487] Scheduler hangs-up during sync validation with a lot of components​

Bug fixes

The scheduler now uses a cached thread pool for handling operation instead of an configurable fixed thread pool

Related issue

Closes #487

[#489] Jvm parameters in dev environment​

New features and improvements

Jvm parameters for dev environment

Related issue

Closes #489

[#487] Further optimization for the scheduler - related to the original issue​

Bug fixes

The scheduler now uses a chached thread pool for handling operation instead of an configurable fixed thread pool

Related issue

Closes #487

[#488] hotfix: Fix build step on pipeline​

Bug fixes

  • Fixed failing Scala docs

Related issue

Closes #488

[#487] Scheduler hangs-up during sync validation with a lot of components​

Bug fixes

The scheduler now uses a chached thread pool for handling blocking operation instead of a configurable fixed thread pool

Related issue

Closes #487

[#483-FollowUp] change the v1 resolve api logic​

Bug fixes

  • Fixed a bug related to the returned descriptor in control panel

Related issue

Closes #483

[#474] Resolve "Reverse Provisioning"​

New features and improvements

  • New Reverse Provisioning endpoints both on Coordinator and Specific Provisioners
  • New Reverse Provisioning domain and infrastructural components
  • The scheduler and the execution plans are now general purpose and not strictly bound to provisioning workflows
  • Execution plan components are now under the scheduler package
  • Endpoint v1/provisioningplan/{token}/status has been deprecated (but still available) and replaced by v1/execution-plans/{token}/status
  • Endpoint v1/provisioningplan/{token}/terminate has been deprecated (but still available) and replaced by v1/execution-plans/{token}/terminate

Bug fixes

  • The execution plan's stopTime field is now correctly updated

Migration

  • Endpoint v1/provisioningplan/{token}/status has been deprecated (but still available) and replaced by v1/execution-plans/{token}/status
  • Endpoint v1/provisioningplan/{token}/terminate has been deprecated (but still available) and replaced by v1/execution-plans/{token}/terminate

Consumers of these endpoints are strongly advised to switch to the new (equivalent) ones.

Related issue

Closes #474, #479

[#483] Change the Perimeter Resolver logic​

New features and improvements

This MR change the resource retrieved from perimeter resolver. The resources to be retrieved by the perimeter resolver are all successfully deployed data products.

Related issue

Closes #483

[#471] feature: Migrate CGP errors to new structure​

New features and improvements

  • Modifies the base error trait to include the new error fields
  • Defines the new errors, their codes and their categories.
  • Persists the default messages for each error code

Breaking changes

For the CGP callers: None, as this merge request doesn't modify the interface specifications.

For the CGP devs:

  • Now the errors are tied to a specific error so now the scala error used should be well chosen or created.
  • Error messages are now fixed, so the technical error information and further info should be passed in the problems array. It is also strongly encouraged to put proposed solutions in the solutions array when possible.
  • Error messages for problems or solutions should be centralized, avoiding to write custom scripts when instantiating the errors. This helps to modify them more easily in the future and to have the reflected changes everywhere.

Related issue

Closes #471

[#482] Added foreign key from provisioning_operation_state to execution_plan​

New features and improvements

In this MR we insert a foreign key from provisioning_operation_state to execution_plan. In this way it is possible both retrieve the status of infrastructure provisioning and the overall status of deploy

Related issue

Closes #482

[#414] feature: Create the documentation of the existing thrown errors based on their codes​

New features and improvements

Adds documentation about the coordinator error categories and the error structure

Related issue

Closes #414

[#481] Migration script uncorrectly migrates 'status' column​

Bug fixes

  • Fixes a bug in the migration where the 'status' column of execution_plan is incorrectly carried out in the table provisioning_operation_state.
  • Fixes a bug where the column 'environment' is not carried out correctly into provisioning_operation_sate

Related issue

Closes #481

[#478] Solutions to parse descriptor errors are duplicated​

Bug fixes

  • The default solution for "mergeable" errors is now shown only once instead of duplicated for each error

Related issue

Closes #478

[#475] Fix SP logs API​

Bug fixes

  • Fixed migration script
  • Changed the openapi specification

Related issue

Closes #475

[#476] Policy deleted information is lost when migrating​

Bug fixes

Property 'status' of coordinator policy is carried out into 'status' of CGP policy in the migration script

Related issue

Closes #476

[#468-FollowUp] problem with applying multiple policies at the same time​

New features and improvements

  • Updated application.conf in the new helm chart

Bug fixes

  • Fixed not displayed error when calling an external service in CGP

Related issue

Closes #468

[#421] Migrate coordinator logs to new structure​

New features and improvements

  • Changed the specific provisioner logs structure
  • Changed the openapi specifications to fit the new structure
  • Persisting new specific provisioner logs to the database

Related issue

Closes #421 #420 #423 #424

[#411] Modify the openapi spec with new error structure​

New features and improvements

  • Adds the new fields to the errors returned by the coordinator
  • Migrates the existing errors to the new structure
  • Created error structure for coordinator (see #413)
  • Persists the errors and makes the coordinator fail if the db and coordinator are not aligned in the defined error codes
  • Information is now returned as user-friendly messages, including the necessary information in the problems, and including in solutions the instances where the platform team shall be contacted
  • Parsing errors now accumulate different field errors instead of returning them one by one.

Breaking changes

For the coordinator callers: None, as the fields will be optional. However, the migration should be performed to stop using the error and errors field of the error response body.

For the coordinator devs:

  • Now the errors are tied to a specific error (see https://gitlab.com/AgileFactory/Witboost.Mesh/witboost.mesh.ui/-/issues/1048) so now the scala error used should be well chosen or created.
  • Error messages are now fixed, so the technical error information and further info should be passed in the problems array. It is also strongly encouraged to put proposed solutions in the solutions array when possible.
  • Error messages for problems or solutions should be centralized, avoiding to write custom scripts when instantiating the errors. This helps to modify them more easily in the future and to have the reflected changes everywhere.

Migration

Callers to the coordinator can now retrieve the new fields to get more information about the error

Related issue

Closes #411, #412, #413, #415

[#467] Stop time field displays 1970-01-01 for failed component deployments​

Bug fixes

Scheduler's failed tasks don't display anymore date 1970-01-01

Related issue

Closes #467

[#468] Problem with applying multiple policies at the same time​

Bug fixes

  • Fixed remote engine errors during metric and policy evaluation (now it returns an InternalErrorResult)

Related issue

Closes #468

[#463] Expose endpoints to retrieve metric and policy flags​

New features and improvements

  • New CGP endpoint /resources/latest-evaluations
  • Domain and Infra components to retrieve the latest resource evaluations

Related issue

Closes #463

Changelog v1.2.1​

[WIT-178] Fix validation tasks status​

Bug fixes

  • Specific Provisioner API: The /v2/validate (async validation) request body is now aligned with that of the /v1/validate (sync validation)
  • The default value of the provisioning-coordinator.specific-provisioner-path-prefix config, is now consistent with that declared in the documentation (i.e. empty value)
  • The different validation tasks (CGP validation, Specific Provisioner validation) now behave in the same way when communicating their final status. This solves a visual inconsistency in the Control Panel when reading validation results.

Related issue

Closes WIT-178, #510, WS-298

Changelog v1.2.2​

[WS-305] Follow-up: fix scala doc​

[WS-305] Optimize provisioning state endpoint responses​

New features and improvements

Introduced a flag on endpoints used to retrieve provisioning plans that can be used to prevent returning data product descriptors. When descriptors are not needed this configuration could optimize the Coordinator memory usage and the response size.

Related issue #514

[WIT-242] Define execution context for policy evaluations​

Bug fixes

Defined a Cached Thread Pool for policy evaluations in order to avoid thread starvation

Related issue

Closes WIT-242

[WIT-262] Optimize cgp policy stats query​

New features and improvements

Optimized the query used to retrieve the top failed policies

Related issue

Closes WIT-262

[WIT-242] Optimize CGP db transactor config​

New features and improvements

  • The CGP db transactor now uses three different execution contexts for acquiring db connections, executing db transactions and handling blocking operations

Related issue

Closes WIT-242

[#512] Fix validation plan completion condition​

Bug fixes

Fixed sync validation plan completion condition

Related issue

Closes #512

[WIT-131][WIT-132] Fix task status after execution plan termination​

New features and improvements

  • Is now possible to configure the scheduler timeout (30 minutes by default) through config provisioning-coordinator.scheduler-execution-timeout

Bug fixes

  • When an execution plan hits the timeout, its status and the status of all the tasks that are not completed (or failed) is set to Terminated
  • Fixed scheduler that does not stop the execution plan when /terminate API is called
  • The zoned date time of logs received by specific provisioners is now converted to UTC before being persisted

Related issue

Closes WIT-132, WIT-131

Changelog v1.2.3​

[WS-301] Follow-up: Improve test execution​

New features and improvements

  • Improved test execution after the fix to the execution plan status.

Related issue

WS-301

[WS-301] Prevent plan transition to final status before its tasks​

Bug fixes

  • Prevented the plan transition to a final status (e.g. FAILED) while some of its tasks are still running.

Related issue

WS-301

[WIT-297] Update the data catalog openapi specification​

New features and improvements

  • Added ADR for OpenAPI versioning (specific provisioners and data catalog)
  • Aligned the Data Catalog OpenAPI specification to error handling and provisioning logs

Related issue

Closes WIT-297

Changelog v1.2.4​

[WIT-304] Prevent task descriptors retrieval when not necessary​

New features and improvements

  • Optimize memory usage and response times when fetching provisioning plans by not loading task descriptors from the database when not strictly necessary

Related issue

Closes WIT-304

[#517][WS-310] Fixed execution plan tasks order​

Bug fixes

  • Added fixed order for independent tasks

Related issue

Closes #517

Changelog v1.2.5​

[WS-317] Make descriptor kinds configurable​

New features and improvements

  • Added possibility to set the admitted kinds by using configurations (supported-kinds)

Related issue

Closes #519

Changelog v1.2.6​

[WIT-319] Cannot test a cue engine policy​

Bug fixes

  • Fixed a bug where coordinator tried to unmarshal a json as yaml

Related issue

Closes WIT-319

[WIT-370] Validation errors at deploy time even validation itself is ok​

New features and improvements

  • Defined an index on the status field of the execution_plan table to optimize the dangling execution plan cleanup routine
  • Optimized the workflow engine execution logic

Bug fixes

  • Fixed overlapping task executions on the workflow engine
  • Fixed the misalignement between runtime task status and persisted task status

Related issue

Closes WIT-370

[WIT-317] Fix execution plan status after app restart​

New features and improvements

  • When the sync validation operation, run before the execution of a deployment plan, exceeds the timeout, it throws a meaningful error
  • Restored configuration provisioning-coordinator.synchronous-validation-timeout
  • Removed some deprecated and unused configurations
  • Improved logging and errors for debugging purposes

Bug fixes

  • Dangling execution plans are now terminated during application startup. This cleanup routine is activated by the new provisioning-coordinator.terminate-dangling-plans-at-startup configuration (default: true)

Related issue

Closes WIT-317

Changelog v1.2.7​

[WIT-319] Cannot test a cue engine policy​

Bug fixes

  • Fixed a bug where coordinator tried to unmarshal a json as yaml

Related issue

Closes WIT-319

Changelog v1.2.8​

[WIT-623] Fix randomly failing unit tests on CGP​

Bug fixes

  • Fixed some unit tests on CGP that were causing CI/CD pipelines to randomly fail

Related issue

Closes WIT-623

[WIT-340] Vulnerabilities Fix​

Bug fixes

  • Fixed vulnerability affecting org.apache-commons.text by upgrading it to 0.10.0
  • Fixed vulnerability affecting cue by upgrading it to 0.6.0

Related issue

Closes WIT-340

[WIT-522] Handle API errors on external policy evaluation​

Bug fixes

  • API errors on external policy evaluations are now handled instead of throwing an exception

Related issue

Closes WIT-522

[WIT-339] Fix monday day of week value on cron expressions​

Bug fixes

  • Updated the CGP interface specification to reflect that the Day of Week value has a range of 0-6 where 0 is Monday
  • Updated the cron expression validator to reflect that the Day of Week value has a range of 0-6 where 0 is Monday

Related issue

Closes WIT-339

Changelog v1.2.9​

[WIT-777] Improve scheduler task errors handling​

New features and improvements

  • Improved the error handling in case of failed or timed out execution plan tasks

[WIT-774] Improve data catalog plugin configuration​

New features and improvements

  • When the data-catalog-provisioning configuration is enabled, is no longer mandatory to configure the data catalog plugin for all the environments

[WIT-676] Ignore task state modifications after execution plan termination​

Bug fixes

  • Modifications to a task state, after the parent execution plan is terminated, are now ignored

Related issue Closes WIT-676

[WIT-583] Normalize descriptor before cue eval​

Bug fixes

  • cue now correctly evaluates descriptors that have accented letters, by normalizing it before evaluating

Related issue

Closes [WIT-583] Closes [WS-327]

Improve mock in unit tests​

New features and improvements

  • Improved the implementation of a mock method used in the execution of unit tests

[WIT-146] Receive Additional Filters For A Governance Entity Test​

New features and improvements

  • You can now supply additional filters when testing a policy or a metric. Fill in a body in the request as follows:
{
    "filters": [
        {
            "description": "whatever",
            "path": "domain",
            "values": ["finance"]
        }
    ]
}

Breaking changes

  • The body for any request flowing through POST /metrics/{id}/test or POST /policies/{id}/test is mandatory due to some technical limitations we have with our API generator. Hence, also the UI that contacts the test endpoint should be updated accordingly (even by sending an empty body just for letting it work)

Related issue

Closes WIT-146

Marketplace Plugin​

Changelog v1.2.0​

[#76] Field names with dashes make the marketplace plugin fails​

Bug fixes

  • Added modifyFieldNames method for replacing dashes separator with _
  • Fixed a bug for private_descriptor that overwrites public fields with null

Related issue

Closes #76

[#73] Fix InputPort error with multiple environments​

Bug fixes

  • Fixed input port upsert for the same descriptor and multiple environments. Now, the input ports are independent among environments.

Related issue

Closes #73