Skip to main content

Changelog v1.1.0+

Core​

Changelog v1.1.0​

[#1189] fix query graphql​

Bug fixes

Fix query graphQL, rename policy in governance_entity

Related issue

Closes #1189

[#1189] Fix query graphQL​

Bug fixes

Fix query graphQL, rename policy in governance_entity

Related issue

Closes #1189

[#1172] Control panel is not displaying the correct label​

New features and improvements

  • Fix bug on threshold control panel component
  • Add label field on control panel drawer
  • Add tooltip to show value entirely on control panel drawer
  • Fix bug on default color for threshold result component
  • Fix bug on metric/policy wizard review to display resource preprocessing

Related issue

Closes #1172

[#1183] Hidden field resource preprocessing in metrics wizard​

Bug fixes

Set to hidden strategy field and set default values based on form

Related issue

Closes #1183

[#1186] No label metrics are not visualized in the marketplace​

Bug fixes

  • Fixed "No label" metrics that are not displayed in the Marketplace
  • Added the first error in the toast when creating/updating a metric/policy

Related issue

Closes #1186

Bug fixes

  • Fixed broken links in docs pointing to CGP docs

Related issue

Closes #1184

[#1169] In overview of metric and policy insert all field​

New features and improvements

Added missing fields to policy and metric overview page

Related issue

Closes #1169

[#1174] Multiple metrics displayed in the marketplace​

Bug fixes

  • Fixed bug

Related issues

Closes #1174

[#1176] the name in the url does not match the menu item​

Bug fixes

Edit wrong url root label

Related issue

Closes #1176

[#1180] Resolve "Change breakingChange to previousVsCurrent"​

New features and improvements

Renamed the strategy breaking_change to previous_vs_current. This allows the user to understand what this preprocessing does, i.e. place the current version side by side with the latest one in the system

Related issue

Closes #1180

Bug fixes

  • Fixed incorrect index referred to links

Related issue

Closes #1178

[#1175] Fix UI docs broken​

Bug fixes

Removed <br> from docs.

Related issue

Closes #1175

[#1168] Ordering field in wizard metric and policy​

New features and improvements

  • Change order of fields inside metric/policy wizard
  • Rename strategy field to Resource preprocessing

Related issue

Closes #1168

[#1171] More precise CTA in CGP create button​

New features and improvements

  • Made call to action for creating policy/metric in CGP clearer and matched the verbs used in documentation ("add" vs "create")

Related issue

Closes #1171

[#1170] Fix top failed policies time range​

Bug fixes

Top failed policies time range

Related issue

Closes #1170

[#1167] Fix wizard metric​

Bug fixes

Fix bug on metric wizard engine field.

Related issue

Closes #1167

[#1161] Resolve blueprint usage for component adding​

Bug fixes

fix entity picker check on fetched entities

Related issue

Closes #1161

[#1148] improve metric wizard​

Bug fixes

  1. Hidden field engine
  2. Hidden field interaction type, we put always 'User to Platform'

Related issue

Closes #1148

[#1140] Search Results Filters visibility​

Bug fixes

Added filters on navigation from policies overview to search page

Related issue

Closes #1140

[#1166] Create a generic component to render the result of metric​

New features and improvements

  • Create new ThresholdResult component
  • Share default colors and types for thresholds
  • Delete previous useless components
  • Add new component on each of specified pages

Related issue

Closes #1166

[#1161] Resolve blueprint usage bug when adding a component​

Bug fixes

Add the domain value after selecting a data product in the blueprint

Related issue

Closes #1161

[#1158] Switch Policy Overview page endpoints​

New features and improvements

  • Extended the GovernanceApi with stats endpoints
  • Top Failed and Inspector statistics (Policy Overview page) are now computed by calling the new CGP endpoints
  • Added tooltips on the Inspector Table in the Policy Overview page

Related issue

Closes #1158, #982

[#1164] Insert a wrapper when visualize a metric/policy​

Bug fixes

  • Fix metric/policy overview
  • Fix descriptor visualisation
  • Fix long names

Related issue

Closes #1164 #1165

[#1163] Rename graphQL query with 'governance_entity'​

New features and improvements

Rename graphQL query, table and field with 'governance_entity' concept.

Related issue

Closes #1163

[#1123] Add metric result in marketplace CGP results dialog​

New features and improvements

Added metrics result in marketplace flag modal

Bug

Fixed bug on policies overview page when clicking on tags ellipsis on top 10 failed table, after opening the modal there was a wrong navigation on click event.

Related issue

Closes #1123

[#1153] Add metric test page/result​

New features and improvements

  • Add metric test page
  • Add metric result page and related components

Related issue

Closes #1153

[#1162] Add event for new metric​

New features and improvements

Add new event when we create new metric in CGP component

Related issue

Closes #1162

[#1116] Resolve bad rendering on Safari​

Bug fixes

fix header logo rendering on Safari

Related issue

Closes #1116

[#1126] Threshold definition​

New features and improvements

  • Transform environment input on editing from multiselect to select

  • Add new custom component to visualize thresholds and domain

  • Add resize observer to calculate the dimension of the thresholds line dynamically

  • Thresholds and domain inputs have following validation:

  1. Domain max and min value are required
  2. Domain max value cannot be higher than min
  3. Label is required
  4. Color is required
  5. Value is required
  6. Is not possible to insert two equal values
  7. If expression is (Above/Below)OrEqual
  8. If expression is Above/Below
  9. Label cannot have more than 64 chars
  • Update app-config.yaml to include custom colors on
metrics:
  thresholds:
    colors:
      - label: << String >>
        value: << HexColor or StringColor >>
  • Handle different type of threshold expression and update accordingly the chart based on following values:
  1. Above
  2. AboveOrEqual
  3. Below
  4. BelowOrEqual
  • Add number formatter library to allow user to insert decimal and negative numbers

Breaking changes

  • Update types to handle new thresholds schema

Related issue #1126

[#1099] Scaffolding errors more visible​

New features and improvements

  • Fixed the DismissablePanel component style to match the one of the error AlertDisplay
  • Fixed the nested Backstage error visualization for the affected actions of the scaffolder (e.g. the Backstage register action)
  • Refactor the publish actions.
  • Removed unused configuration variable already_initialized_repository_enabled. Updated documentation accordingly.
  • Fixed BitbucketServerRepositoryManager getProjectSlug, isRepoEmpty methods.

Related issue

Closes #1099

[#1157] Restore governance config.d.ts​

Bug fixes

  • Restored config.d.ts in wb-governance

Related issue

Closes #1157

[#1144] Governance section breadcrumbs​

Bug fixes

Fix Wrong name in breadcrumb in specific result page

Related issue

Closes #1144

[#1143] Menu and pages naming​

New features and improvements

Change the name of menu item in Governance

Related issue

Closes #1143

[#1156] Add a configuration to disable blueprints​

New features and improvements

Added a boolean configuration mesh.builder.scaffolder.blueprints.enabled to enable/disable the blueprints.

Related issue

Closes #1156

[#1149] Adjust CGP policy and metric testing endpoints​

New features and improvements

  • Added test endpoint for metrics
  • Improved configuration reading for the governance proxy plugin

Breaking changes

  • mesh.governance section in app-config has been restructured

Migration

We must update the helm charts accordingly with the new app-config structure for the mesh.governance section

Bug fixes

  • Fixed a bug that made metrics endpoints to be mandatory even if CGP metrics are disabled

Related issue

Closes #1149

[Closes #1147] feature: CI Simplification

New features and improvements

Removed build_vars job

Related issue

Closes #1147

[#1128] Strengthening the test suite for CGP​

New features and improvements

  1. Added a new test to check the policy life circle. Here is a brief description of the steps of the test:
  • Create a new cue policy
  • Check "Draft" stance. Open the policy menu (three dots) and check available buttons, check that policy is editable
  • Check "Grace" stance. Go from "Draft" to "Grace", again check the available buttons and check that policy is not editable
  • Check "Disable" stance after "Grace", "New Version" function in that position. Go from "Grace" to "Disable". Check available buttons, create a new version, and check that it is not editable ("Disable"), but the new version has "Draft" status.
  • Check "Enable" stance, "Publish" function. Go from "Disable" to "Enable", check available buttons, try to publish the policy, check the published policy button again, check that it is not editable.
  • Check "Disable" stance after "Enable", "New Version" function in that position. Check available buttons, check that policy is not editable.
  • Check "Deprecated" stance, and available buttons, and that policy is not editable.
  • Clean all steps and results.
  1. Added fixes to existing tests
  2. Update chromedriver to version 113

Related issue

Closes #1128

[#1109] Refactor page Overview and Results​

New features and improvements

Refactor page Overview|Results|Control Panel

Related issue

Closes #1109

[#1078] Auto-logout​

New features and improvements

  • Added Autologout to automatically logout users on idle after a certain amount of time

Related issue

Closes #1078

[#1145] Change the metric model​

Bug fixes

  • Fixed metric creation

Related issue

Closes #1145

[#1076] User headers​

New features and improvements

  • Introduced the sensitive flag on user configurations
  • Now the value of sensitive configurations can be redacted on-demand
  • New wb-user-headers plugin (api and client to interact with the new user headers functionality)
  • User headers section in the Advanced Settings page
  • User headers configurations under mesh.userHeaders
  • User headers forwarding functionality (toward the Coordinator)
  • User headers documentation
  • JWT token forwarding documentation
  • Refactoring of the user settings section (Advanced Settings page) in order to make it aligned with the user headers one

Impacts​

The user headers functionality is disabled by default (config mesh.userHeaders.enabled to false). The headers section will not be visible to the user and no headers will be forwarded to the coordinator until that configuration is enabled.

Related issue

Closes #1076, #1077, #1084

New features and improvements

  • Added SelectEnvironmentRadio component to select just one policy

Bug fixes

  • Fixed the fact that updating the policy environment has not a consistent visualization

Related issue

Closes #1135

Default value of the cueScript field must be an empty string

Bug fixes

  • Made the cue script to assume empty string as default value

Related issue

Closes #1136

[#590] Improve control panel features discovery​

New features and improvements

Introduced some tooltips to better explain the functionalities of the control panel buttons.

Related issue

Closes #590

[Closes #1131] feature: [CGP] Runtime breaking change policies don't make sense

New features and improvements

Both in the new/update policy/metric forms is not possibile anymore to select strategy Breaking Change when the selected timing is Runtime

Related issue

Closes #1131

[#1139] Resolve the test policy that is not working​

Bug fixes

  • Added a missing token in the request for the Test policy

Related issue

Closes #1139

[#1137] Selectors are not accepted in the create and update requests​

Bug fixes

  • Fixed a bug that prevented the creation/update of a policy or metric with selectors

Related issue

Closes #1137

[#1134] Policies Overview top 10 table is not fetching all the failed policies​

Bug fixes

  • Added warning, info, bot_blocking_error to the graphql query that fetches the top 10 fail policies.
  • Changed Type into Strategy in the top 10 table header

Related issue

Closes #1134

[#1108] Restore html and json fields in informative notification​

New features and improvements

  • Restore html and json fields in informative notifications and add parameters in notifications/by-event
  • Make entity query parameter optional in notifications/by-event

Related issue

Closes #1108

[#979] New CGP policies results' search page​

New features and improvements

Add the search policy page with filters and results' table.

Related issues

Closes #979

[#1129] Improve definition of done for issues​

New features and improvements

Introduced the Definition of Done for all the Merge Requests.

This is a checklist of things to always check before putting the MR in review.

Since there is an open MR in Gitlab which is 5 years old to add a variable to reference the issue in the MR template, and no one seems to solve it, I decided to make the MR template the default, so the "Closes #{issue}" is automatically added.

Related issue

Closes #1129

[#1132] Resolve a missing validation entity in BlueprintProcessor​

Bug fixes

  • Added validateEntityKind method to BlueprintProcessor to validate blueprint templates.
  • Added validation schema for blueprints.

Related issue

Closes #1132

[#1124] View, Test, and Delete buttons visibility in the metrics menu​

Bug fixes

  • With cgp.entity.view permission only set, the Test and Delete buttons are now not visible in each menu of the metrics.
  • With cgp.entity.edit permission set, the View button is now visible in each menu of the metrics.

Related issue

Closes #1124

[#1095] Documentation of change management helper center is very limited​

New features and improvements

  • Added 2 tables in Notifications:
    • Table to summarize all the events that trigger a notification.
    • Table to describe recipients of each event.
  • Fixed section in Witboost configuration.

Related issue

Closes #1095

[#1110] Introduce field trigger​

New features and improvements

  • Add new trigger field on policies/metrics
  • Update queries to retrieve new field
  • Update add/edit wizards to handle new constraints

Breaking changes

  • Update types adding new Trigger enum ("Active", "Passive")
  • Remove "Passive" from timing
  • Update proxy requests to handle new endpoints about metrics
  • Update proxy requests to adapt change on new version endpoint (from POST to PATCH)

Related issue

Closes #1110

[#1102] Autotests for CGP module​

New features and improvements

  • policy life circle up to the "grace"/"disable"/"publish" steps
  • test to verify that the edit policy is only in the draft step
  • got optimized way of tests (while creating dp or op) to save a LOT of time pipeline running.
  • reworked test with negative deploy check (3-5)
  • updated tests that check notifications (suit 6)
  • fixed existing tests to check CGP module according to the current position

Related issue

Closes #1102

[#1013] Fix EntityPickers that show the internal ids of the entities​

New features and improvements

Introduced a lot of classes that define how entities inside witboost are structured (as extensions of the basic Backstage ones). To enforce the definition, we added also the JSON schema for each of them, so methods that need to make validations can leverage the associated validators.

The EntityPicker, EntitySelectionPicker, and EntityComponentsPicker classes have been updated, in order to display the human-friendly names of entities instead of the entities' identifiers.

Breaking changes

The allowArbitraryValues flag was removed from all the aforementioned pickers since it would increase the possibility for users to make mistakes.

Related issue

Closes #1013

[#1118] Test and Delete buttons not visible in PoliciesRowMenu​

New features and improvements

Made buttons visible when the user has the right permissions.

Related issue

Closes #1118

[#1117] hotfix: Errors in policy creation​

Bug fixes

  • Fixed resultType
  • Fixed graphQL query

Related issue

Closes #1117

[#1113] Fix Policies Refactor​

Bug fixes

  • Fix policies filter ui
  • Fix Cron field in create/edit

Related issue

Closes #1113

[#1101] CGP backend must send a request for each environment​

New features and improvements

  • Modified message toast when creating and updating a policy
  • Modified the way the request is sent to the coordinator for policy creation and update

Bug fixes

  • Fixed a mismatching between the coordinator response and PolicyValidateResult

Related issue

Closes #1101

[#1114] Update the EvaluationReport DTO​

New features and improvements

  • Aligned Evaluation Result DTO with the one contained in the coordinator
  • Fixed some bugs

Related issue

Closes #1114

[#1083] Update the images in the official doc​

New features and improvements

Changed the following images for the documentation:

  • blueprint selecting
  • selecting new DP while using a blueprint
  • DP graph ("getting starting" and "dp details" pages)
  • Updated the information about blueprint usage

Related issue

Closes #1083

[#1072] CGP permissions review​

Breaking changes

  • The governance menu is now visible only if the user has either cgp.entity.view or cgp.entity.edit permissions.
  • with cgp.entity.view the user can only view the policies/metrics from the Registry.
  • with cgp.entity.edit the user can do all on policies/metrics from the Registry (view, create, edit, delete).
  • in the Flag and Score dialog, results are clickable only if either one of the 2 new permissions are set.
  • Backend blocks actions on policies/metrics (create, edit, delete) if user do not have cgp.entity.edit permission.
  • Frontend shows an ErrorPanel if the user does not have cgp.entity.view or cgp.entity.edit permissions and tries to navigate to a forbidden route

Migrations

  • backstage db -> rbac.permissions:
    • changed row id from builder.policy.template.edit to cgp.entity.view
    • changed row id from builder.policy.global.edit to cgp.entity.edit

Related Issue

Closes #1072

[#989] Restyle the output port marketplace page​

New features and improvements

Align Output port page UI with new mock

Related issue

Closes #989 #1059

[#1112] Query params filters get lost when moving between policies and metrics​

New features and improvements

  • Create custom hook to handle query params using session storage
  • Remove useless HOC
  • Improve code reusability

Bug fixes

Fix bug on policies/metrics query params

Related issue

Closes #1112

[#1098] Update name field in provisioningPlan response​

Bug fixes

Update provisioning plan api response type

Related issue

Closes #1098

[#1096] Refactor metric​

New features and improvements

  • Adapt metrics to have same features as policies
  1. Add multi select filters
  2. Add timing column
  3. Display badge on filters button
  4. Get applied filters from url params
  • Adapt metrics to new BE changes
  • Update metrics add/edit wizard to new constraints
  • Update types

Related issue

Closes #1096

[#1103] Resolve "Refactor policies"​

Breaking changes

Align policies views with the new BE structure

Related issue

Closes #1103

[#1073] Improving the policy suit​

New features and improvements

  • Split a bunch of tests that were in the same scenario into different tests
  • Added a new test scenario when the user can add valid or invalid cron expression to the runtime policy
  • Added new test scenario when the user tries to input invalid cue policy
  • Corrected and improved existing autotests according to the current situation

Related issue

Closes #1073

[#967] Resolve "Edit Metric page"​

New features and improvements

Add edit metrics page

Related issue

Closes #967

[#934] The Data Product gets added to the Builder even though the creation failed​

New features and improvements

  • Add a line in the documentation explaining to not create a data product repository/branch with the same name of a repository/branch deleted from Gitlab but not from the builder.

Related issue

Closes #934

[#980] Improve marketplace CGP results dialog​

New features and improvements

  • Added i icon and tooltip to show the description of every warning/error policy in the FlagAndScoreDialog
  • Added authorization to view the link of the policy. The permission is set to builder.policy.global.edit
  • Increased width of the Flag Dialog

Related issue

Closes #980

[#1060] Show data product description in the marketplace main table​

New features and improvements

Add the description column inside the catalog table.

Related issue

Closes #1060

[#971] Update the Marketplace db schema to handle versions in the right way​

Breaking changes

  • Modified Hasura queries for the marketplace in order to fetch version, descriptor, and private_descriptor from the DataProductInstance table.

Migration

  • Added a migration to move descriptor, version, and private_descriptor field from DataProduct to DataProductInstance and to modify the content of the private_descriptor field of DataProductInstance, OutputPort, and Component tables

Related issue

Closes #971

[#1082] Fix helm package job​

New features and improvements

Fixed a misconfiguration for the helm package job.

Related issue

Closes #1082

[#1070] Fix version consistency between the Docker image and the Helm chart​

Bug fixes

CI configure Docker image correctly in Helm chart values file

Related issue

Closes #1070

[#1065] Policy is not deprecated as instead stated​

Bug fixes

  • Fix the detail icon tooltip
  • Fix policy status inside the detail drawer
  • Hide descriptor when empty in detail test

Related issue

Closes #1065 #1066

[#1071] Add the "view" action also when the user has permission to edit policies​

New features and improvements

  • Add view action for all policy status
  • handle policy overview registry breadcrumb to retrieve preselected filters
  • change marketplace graph sidebar filters button badge

Related issue

Closes #1071

[#1038] Multiselection on policy filters​

New features and improvements

  • Add multi-select on policy filters
  • Update policies list query to handle arrays
  • Build dynamic breadcrumb for Add/Edit policy page with HOC
  • Add a filters badge on the button
  • Set filters accordion opened if there are selected filters

Related issue

Closes #1038

[#1035] Add a new delimiter for array pickers​

Bug fixes

  • Added a bottom margin for every element in the ArrayFieldTemplate in order to space them with a blank line

Related Issue

Closes #1035

[#986] Tests for the policies module​

New features and improvements

  • Update tests 4-1 for general checking CGP module (bookmarks, search and filter functions, environments selection)
  • Added tests 4-2 to check policy creation/deleting
  • Added tests 4-3 to check policy modification (name, description, strategy, severity, interaction type, timing, and context)

Related issue

Closes #986

[#1069] Fix CI failure on Package Job​

Bug fixes

restore ci functionality

Related issue

Closes #1069

### [#1068] Fix consistency between helm and docker versions

New features and improvements

Updated the ci to set the correct version of the docker image in the values.

Related issue

Closes #1068

[#1064] HasuraClaims default breaks deployments​

New features and improvements

This MR changes the default role injected by witboost in the requests througout hasura, from user to admin.

Related issue

Closes #1064

[#1022] Resolve "Policy registry is not keeping the selection of the environment"​

New features and improvements

  • Sync registry filters with url query params
  • Add validation to url query params (if user changes url manually with a wrong value, a default one is set)
  • User can share url with applied filters
  • Pagination limit can be set on query params

TBD Breadcrumbs. Currently, we don't have a way to build dynamic breadcrumbs. Now, clicking on it reset filters. A possible solution could be to use session storage but we would lose the shareable url functionality. I don't advice to mix both because it would be hard to debug and maintain.

Related issue

Closes #1022 #1023

[#1047] Policy result page is not adapting to the context​

Bug fixes

Hide CUE script in the result page when the policy doesn't require it

Related issue

Closes #1047

[#1045] the flags section does not display the results in warnings and info​

Bug fixes

Fix graphQL query to show flags

Related issue

Closes #1045

[#1044] Add field additionalMetadata when create/edit a policy​

New features and improvements

  • add additionalMetadata to create/edit payload

Bug fixes

  • Fx preprocessing input field enable condition

Related issue

Closes #1044

[#1046] Add format 'json' when display Details section​

Bug fixes

Fix json formatting

Fix tags spacing and wrap

Related issue

Closes #1046

[#1043] Remove policies docs under platform customizations​

Bug fixes

Removed policies docs under platform customizations

Related issue

Closes #1043

[#978] New policies specific results page​

New features and improvements

Add the new specific result page

Related issue

Closes #978

[#1042] Adjust blueprint docs title​

Bug fixes

  • Adjusted title in blueprint docs

Related issue

Closes #1042

[#1040] Blueprint Template dependencies error​

Bug fixes

Fix template dependencies inside the blueprint.

Related issue

Closes #1040

[#1036] Resolve the new version feature that sometimes fails silently​

New features and improvements

If the branch already exists we are throwing a new Error instead of only returning so that we stop failing silently and have better error message/handling

Related issue

Closes #1036

[#1027] Fix policy error not scrolling anymore to highlight selected error​

Bug fixes

  • Fixed policy cue editor error highlight
  • Fixed auto scrolling on the first error

Related issue

Closes #1027

[#1033] Remove policies docs from ui repo​

New features and improvements

  • Removed old docs on managing policies

Bug fixes

  • Old screenshots for policy tests are now updated

Related issue

Closes #1033

[#1032] Add blueprint documentation​

New features and improvements

Add public documentation to explain the following to the user:

  • what a blueprint is
  • how it works
  • how the user can use it to create a new DP
  • how the user can use it to add a component to the DP
  • what else the user can do with the blueprint

Related issue

Closes #1032

[#1024] Policy launch results not consistent​

Bug fixes

Fix policy launch results with the right error status

Related issue

Closes #1024

[#1028] Change the default "rows per page" in pagination​

Bug fixes

  • Fix bug on builder breadcrumbs
  • Change default rows per page in pagination (default 25) with 25/50/100

Related issue

Closes #1028 #1031

[#1037] Make the data sharing agreement field displayed in the marketplace configurable​

New features and improvements

Introduced a new configuration mesh.marketplace.ui.outputPortPage.dataSharingAgreementField that can be used to define which field of the descriptor will be used to display the data sharing agreement details.

Related issue

Closes #1037

[#1025] Fix blueprint components order that is not preserved​

Bug fixes

  • Fix blueprint template order
  • Fix condition to show other components

Related issue

Closes #1025 #1026

[#1034] Fix the list of data products in the blueprint that is not consistent with my data products​

New features and improvements

Updated the blueprint documentation to explain better which data products are loaded when selecting one to confront with a blueprint.

Related issue

Closes #1034

[#937] FIx the wrong percentage in the DP Usage card​

Bug fixes

Solved an error with the percentage displayed for the marketplace supervision panel.

Related issue

Closes #937

[#1029] Resolve the wrong type displayed in the templates page​

Bug fixes

Fixed a bug that displayed the wrong template group name on the "Templates" page.

Related issue

Closes #1029

Changelog v1.1.1​

[#1204] Fix regex to handle undefined values​

Bug fixes

  • Handle cases where input is not set and is undefined so it will fail to match the regex even when regex allows empty values
  • Remove the hard limit of length 1 for the input

Related issue

Closes #1204

Changelog v1.2.0​

[#1205] Strange label in control panel test​

Bug fixes

Fix bug on wrong label in control panel test.

Related issue

Closes #1205

[#1214] Fix breadcrumb in Metric Results​

New features and improvements

Update navigation on metrics results

Related issue

Closes #1214

[#1182] Page Search Metrics​

New features and improvements

  • Add metrics search page
  • Refactoring common components

Related issue

Closes #1182

[#1213] Fix flags and scores loading​

Bug fixes

Fixed flags and scores loading error

Related issue

Closes #1213

[#1197] Resolve the environment reset across different tabs of CGP​

New features and improvements

  • Share environment fields across different tabs of cgp

Related issue

Closes #1197

[#1211] Introduce the scroll bar in the error component​

New features and improvements

Add scroll bar inside error box component

Related issue

Closes #1211

[#1190] Resolve "Retrieve metric and policy flags through the CGP endpoint"​

New features and improvements

  • Extended the GovernanceApi with latest resource evaluation retrieval endpoint
  • Flags and Scores in the marketplace data product catalog and in the data product page are now computed by calling the new CGP endpoint

Related issue

Closes #1190

[#981] Add duplicate name entities check when registering location​

Bug fixes

  • Backstage registering process does not throw an error when trying to import a location that spawns an entity with a duplicate name. This MR patches backstage so that when analyzing a location before registering, checks the location will generate entities with names already existing in the catalog. This works only in register dryRun operations, as normally the register operation does not process the location to discover entities. If the location already exists, then the patch has no effect and the workflow proceeds as normal, prompting the user to refresh the location.

Related issues

Closes #981, #584

[#1151] CGP test suite extension​

New features and improvements

  • Added a new test that will check the ability to change context, trigger, time, engine, strategy, severity
  • Fix AT system

Related issue

Close #1151

[#1209] Output Port datacontract is optional​

Bug fixes

Handle optional datacontract/schema/sla

Related issue

Closes #1209

[#1100] Reserved field processor​

New features and improvements

  • ReservedFieldsProcessor is a processor that checks the ‘reserved fields’ and emits an error in processing phase if they were modified
  • Ability to specify the path of the reserved fields in configuration

Related issue

Closes #1100

[#1202] Improvements Test Result section​

New features and improvements

  • Add fixed dimension to test results panel
  • Add scrolling to test results

Related issue

Closes #1202

[#1105] Add validity checks in publish action​

New features and improvements

  • Add additional checks before performing the publish action on GitLab and BitBucket
  • Current additional checks are: YAML parsing, allowed kinds, system/component/domain schema check

Related issue

Closes #1105

[#1204] Fix regex to handle undefined values​

Bug fixes

  • Handle cases where input is not set and is undefined so it will fail to match the regex even when regex allows empty values
  • Remove the hard limit of length 1 for the input

Related issue

Closes #1204

[#1198] Add policy title click action​

New features and improvements

  • Add cursor pointer on policy/metrics page
  • Handle click on title to navigate on view page
  • Add underline decoration on hover for uniformity with the rest of the application

Related issue

Closes #1198

[#1191] Schema Editor Discovery Week​

New features and improvements

Add schema editor section to OP page.

Related issue

Closes #1191

Provisioning Coordinator​

Changelog v1.1.0​

[#456 and #418] Resolve "Update CGP documentation"​

New features and improvements

  • Updated CGP docs to latest changes

Related issue

Closes #456, #418

[#466-FollowUp] Fix CGP Policy migration script​

New features and improvements

Modified table names based on new CGP schema

Bug fixes

Fixes the migration script that wouldn't work with the latest versions of CGP

Related issue

Closes #466

[#466] Fix CGP Policy migration script​

New features and improvements

Modified table names based on new CGP schema

Bug fixes

Fixes the migration script that wouldn't work with the latest versions of CGP

Related issue

Closes #466

[#465] Change breakingChange to previousVsCurrent​

New features and improvements

Renamed the strategy breaking_change to previous_vs_current. This allows the user to understand what this preprocessing does, i.e. place the current version side by side with the latest one in the system

Related issue

Closes #465

[#464] Fix conversion for MetricExecutorResultError​

Bug fixes

  • Handled MetricEvaluationResultError cas in toEvaluationResult method

Related issue

Closes #464

[#461] Metric's values should be validated against a domain​

New features and improvements

  • Added domain check for metric values

Related issue

Closes #461

[#462] performance improvement during evaluation​

New features and improvements

  • created specific query for upsert operations
  • Added index on table evaluation result
  • configured memory sizing on kubernetes
  • configured jvm heap

Related issue

Closes #462

[#450] Define policy execution stats endpoints​

New features and improvements

  • New CGP stats endpoints
  • New CGP stats components (repository, adapter, port)
  • Modified the provisioning flow to attach a configurable label to the CGP evaluation report it generates

Related issue

Closes #450

[#462] Performance improvement during evaluation​

New features and improvements

  • created specific query for upsert operations
  • Added index on table evaluation result
  • configured memory sizing on kubernetes
  • configured jvm heap

Related issue

Closes #462

[#462] Performance improvement during evaluation​

New features and improvements

  • created specific query for upsert operations
  • Added index on table evaluation result
  • configured memory sizing on kubernetes
  • configured jvm heap

Related issue

Closes #462

[#458] Rename table and field with 'governance entity' concept​

New features and improvements

This PR contains a renaming of table, fields, column name, logs with governanceEntity concept

Breaking changes

Some queries in graphQL need to be renamed

Related issue

Closes #458

[#457] Fix status result of report for metric​

Bug fixes

This PR contains some improvements on report status for metric evaluation.

Until all the metrics have been evaluated, the status of the report remains pending. When a response has been received for all the metrics (and therefore associated with the relative threshold) the status of the report is calculated only on the policies.This is because metrics are values ​​and do not have the concept of failure as in policies

We added the environment parameter to the internal evaluate method.

Related issue

Closes #457

[#376] Notification on new policy enabled​

New features and improvements

  • notification on failed scheduled governance entity
  • notification on new governance entity creation
  • updated documentation about CGP configuration (now it includes the notification part)
  • updated helm chart for inserting two new env variables

Related issue

Closes #376

[#446] Update API to new version of threshold​

New features and improvements

  • Changed all the APIs that include Threshold as request parameter
  • Changed the threshold computation

Related issue

Closes #446

[#455] Split CGP env variables names for config​

New features and improvements

  • Added CGP_DATABASE_URL env variable
  • Improved logging in policy scheduler

Related issue

Closes #455

[#455] split cgp env variables names for config​

New features and improvements

  • Added CGP_DATABASE_URL env variable
  • Improved logging in policy scheduler

Related issue

Closes #455

[#453] Timeout on cgp evaluation tasks​

New features and improvements

  • Added timeout configuration for CGP evaluation

Related issue

Closes #453

[#454] Remove from log the descriptor​

New features and improvements

to improve the readability of the logs we have removed, in this MR, the print of the descriptor.

Related issue

Closes #454

[#443] Add metrics testing​

New features and improvements

  • Added new endpoint /metrics/:id/test for testing a metric entity
  • Changed endpoint /policies/:id/evaluate into /policies/:id/test
  • Improved error codes on endpoints. 404 when entities are not found. 422 when request is valid but entity cannot accept it due to its internal state. Error handler has been centralized

Bug Fixes

  • Renaming of most of the references to policies inside the Evaluator. From policy to governanceEntity still more to be renamed

Related issue

Closes #443

[#452] Configurable specific provisioner endpoint version​

New features and improvements

Added a configuration used to specify the specific provisioner endpoint version

Related issue

Closes #452

[#448] Coordinator async task does not handle sync replies from specific provisioner​

Bug fixes

SpecificProvisionerAsyncHttpTask manages specific provisioner sync reply too.

Related issue

Closes #448

Add domain field to governance entity

New features and improvements

  • Added a new domain field for metrics, representing the range of allowed values accepted by the metric

Related issue

Closes #433

[#447] CI Simplification​

New features and improvements

Removed build_vars job

Related issue

Closes #447

[#444] Cron jobs must be executed in isolation​

Bug fixes

Unhandled cron job exceptions are not anymore blocking for other scheduler's tasks

Related issue

Closes #444

[#441] Fix policy migration script​

New features and improvements

Adds to the SQL migration script from old policies to CGP policies the new fields.

Migration

The script can now be used to migrate from old policies to CGP policies correctly

Bug fixes

Fixes a bug where the script was not aligned with the latest version of CGP so the migration would fail

Related issue

Closes #441

[#440] Fix documentation formatting​

New features and improvements

Better formatting on the config documentation for Docusaurus compatibility

Related issue

Closes #440

[#438] Restore the cron expression converter​

New features and improvements

  • New: CronExpression domain object that embeds by-design validation and conversion
  • New: The original cron expression (that provided by the UI) is now retained along with the equivalent expression in cron4s format. For instance, if the UI provides cron expression 2/10 * * ? * *, it wil be internally converted to the equivalent expanded expression 2,12,22,32,42,52 * * ? * *, but the UI (the API consumer) will always get the original provided expression and not the extended one.

Bug fixes

  • Restored the cron expression conversion step

Related issue

Closes #438

[#436] breaking change policies don't make sense if runtime​

Bug fixes

  • You cannot create a policy/metric with timing runtime and strategy breaking change anymore. since it does not make any sense because breaking changes can be detected only at deploy time.

Related issue

Closes #436

[#319] Create a end-to-end test suite (insert - update - evaluate)​

New features and improvements

  • End2end tests for the CGP

Related issue

Closes #319

[#432] Add trigger field to governance entity​

New features and improvements

  • Added trigger field

Related issue

Closes #432

[#324] Coordinator - Integrate similarity check service​

New features and improvements

  • New: Similarity Check Service configuration. Disabled and non-blocking by default

Related issue

Closes #324

[#430] Split create and update API for policies and metrics​

New features and improvements

  • New create metric endpoint POST /v1/governance-platform/metrics
  • New update metric endpoint PUT /v1/governance-platform/metrics/:metricId
  • New update metric status endpoint PUT /v1/governance-platform/metrics/:metricId/status
  • New bump metric version endpoint PATCH /v1/governance-platform/metrics/:metricId

Breaking changes

  • Bump policy version is now reachable at PATCH /v1/governance-platform/policies/:policyId
  • Now when asking for a status change, the response will be the entire updated entity, not just the new status

Related issue

Closes #430

[#406] Headers forwarding​

New features and improvements

  • New: Forwardable headers integration and configuration
    • New configuration provisioning-coordinator.forwardable-headers
    • HTTP scheduler tasks now include an httpHeaders param
    • The provisioning and validation flows now handle an httpHeaders list coming from the API layer
    • HttpClient now accepts headers too
    • A ForwardableHeadersHelper filters the incoming headers that can be forwarded according to the configurations
  • New: Forwardable headers documentation
  • Fix: The integration tests suite is now restored
  • New: Headers forwarding integration test

Impacts

After merging these changes, by default Authorization headers and headers starting with wb-user- will start to be forwarded (if present) to the provisioning and validation services.

They are not mandatory, hence no error is raised if this kind of headers cannot be found.

Related issue

Closes #406, #405

[#431] EvaluationResults for metrics must be filtered when computing status​

New features and improvements

  • Added policy_result_type in evaluation_result table
  • Added a filter in the status method that accepts only the boolean policies
  • Renamed all the fields of evaluation_result in order to include the governance_entity concept

Related issue

Closes #431

[#419] Each Policy must be associate just a one environment​

New features and improvements

  • Modified the way policies can be updated and created (now there is 1 policy for each environment in the Policy table)

Related issue

Closes #419

[#357] Better logging​

New features and improvements -Improved logging in the CGP

Related issue

Closes #357

[#416] Insert field engine and strategy for policy and metric​

New features and improvements

  • Renaming
    • Policy => GovernanceEntity
    • BooleanPolicy => Policy
    • MetricPolicy => Metric
  • use a field strategy to capture the concept of 'resource modification' before being evaluated from engine
    • Default: the input resource has no modifications
    • BreakingChange: the resource consists of the latest version retrieved by the perimeter resolver and the one supplied as input
  • introduce a new field engine
    • cue
    • remote

Breaking changes

Change the table policy in schema cgp:

  • rename execution_strategy => strategy
  • add field engine

Migration

A flyway migration was created to handle these changes

Related issue

Closes #416

[#427] Coordinator calls CGP using HTTP call​

New features and improvements

Http calls to CGP

Related issue

Closes #427

[#427] Coordinator calls CGP using HTTP call​

New features and improvements

Http calls to CGP

Related issue

Closes #427

[#426] Update Mock-Specific-Provisioner for remote policy​

Related issue

Closes #426

[#429-FollowUp] Improvements build and openapi generator​

Bug fixes

Fix ci pipeline to publish just provisioning-coordinator artifact

Related issue

Closes #429

[#429] Resolve "Improvements build and openapi generator"​

New features and improvements

  • Improve build.sbt to publish artifact only for root project (provisioning-coordinator)
  • Remove function validationFunction from Controller (does a check that application serve already does)

Related issue

Closes #429

[#428] Update the documentation for CGP configs​

New features and improvements

  • Added Evaluation configuration docs

Related issue

Closes #428

[#407] UpdatePolicyStatus should run also the evaluate API​

New features and improvements

  • Added the possibility to run an evaluation if the policy changes the status to Enabled
  • Made the above behaviour configurable by CGP

Related issue

Closes #407

[#425] RDisplay name should be filled if available from perimeter resolver​

Bug fixes

  • Resource in the request of a push evaluation result always takes precedence except for the display name resolved by perimeter resolver

Related issue

Closes #425

[#410] move interface specification in the cgp module​

New features and improvements

cgp interface specification in it's own module

Related issue

Closes #410

[#1049] Push evaluation result defects​

New Features and improvements

  • Push evaluation result API has now been splitted into:

POST /metrics/{metricId}/evaluation-results to receive results for local metric policies

POST /policies/{policyId}/evaluation-results to receive results from local policies

Bug fixes

  • Fixed a bug that caused draft and disabled local policies to produce errors in the marketplace

Related issue

Closes #1049

[#404] External policy refactoring​

New features and improvements

Introducing the engine concept for cue and external policies

Related issue

Closes #404

[#374] Documentation - Record some missing configurations​

New features and improvements

Added documentation for:

  • Data catalog provisioning configurations
  • Data deletion configurations
  • Marketplace compliant services configurations

Related issue

Closes #374

[#403] Version consistency between the Docker image and the Helm chart​

Bug fixes

CI configure Docker image correctly in Helm chart values file

Related issue

Closes #403

[#402] CI failure on Package Job​

Bug fixes

restore CI functionality

Related issue

Closes #402

[#401] Consistency between helm and docker versions​

New features and improvements

Updated the ci to set the correct version of the docker image in the values.

Related issue

Closes #401

[#375] CGP - threshold in metrics should not overlap​

New features and improvements

Metric thresholds are checked for overlaps

Related issue

Closes #375

[#400] FIx for migration script​

New features and improvements

Clean migration script

Related issue

Closes #400

[#399] Validate environment when pushing evaluation results​

Bug fixes

The environment param provided to the save method of EvaluationReportService is now validated against the available system environments.

Related issue

Closes #399

[#398] Add field additionalMetadata in openapi specification​

New features and improvements

  • Added additionalMetadata field in PolicyCreationModel and PolicyUpdateModel

Related issue

Closes #398

[#396] "Improvements migrations script"​

Migration

Improvements migration script to populate the table 'provisioning_operation_state'

Related issue

Closes #396

[#395] Create mocked APIs for external policies​

New features and improvements

  • Added APIs for mocking external policies execution with Boolean policy returning object

Related issue

Closes #395

[#394] Remove useless fields in push evaluation results API​

Bug fixes

  • Removed policyStatus, resourceType from push evaluation result API

Related issue

Closes #394

[#392] Split docs kinds for CGP​

New features and improvements

  • Adjusted docs for CGP so that they are visible in public and private docs

Related issue

Closes #392

[#397] Add migration script for policy tags​

New features and improvements

The policy migration script now includes policy tags migration

Related issue

Closes #397

[#393] Fix log name inside CUE validation errors​

Bug fixes

CUE evaluation errors now log the policy id instead of the policy name

Related issue

Closes #393

[#390] handling of http errors in calls to the specific provisioner​

Bug fixes

Errors after invoking specific provisioners should be encapsulated in json objects

Related issue

Closes #390

[#391] Add additional_metadata field to policy table​

New features and improvements

  • Added additional_metadata field in policy and policy_history tables
  • Modified migration script in order to populate the above field
  • Modified queries for inserting and updating policy and policy_history tables

Related issue

Closes #391

Changelog v1.1.1​

[#468] Problem with applying multiple policies at the same time​

Bug fixes

  • Fixed remote engine errors during metric and policy evaluation (now it returns an InternalErrorResult)

Related issue

Closes #468

[#463] Expose endpoints to retrieve metric and policy flags​

New features and improvements

  • Updated CGP docs to latest changes

Related issue

Closes #456 #418

Changelog v1.1.2​

[#481] Migration script uncorrectly migrates 'status' column​

Bug fixes

  • Fixes a bug in the migration where the 'status' column of execution_plan is incorrectly carried out in the table provisioning_operation_state.
  • Fixes a bug where the column 'environment' is not carried out correctly into provisioning_operation_sate

Related issue

Closes #481

[#476] Policy deleted information is lost when migrating​

Bug fixes

Property 'status' of coordinator policy is carried out into 'status' of CGP policy in the migration script

Related issue

Closes #476

Bug fixes

  • Fixed remote engine errors during metric and policy evaluation (now it returns an InternalErrorResult)

Related issue

New features and improvements

  • New CGP endpoint /resources/latest-evaluations
  • Domain and Infra components to retrieve the latest resource evaluations

Related issue

Closes #463

Changelog v1.1.3​

[#487] Scheduler hangs-up during sync validation with a lot of components​

Bug fixes

  • The scheduler now uses a cached thread pool for handling blocking operation instead of a configurable fixed thread pool

Related issue

Closes #487

Marketplace Plugin​

Changelog v1.1.0​

[#70] Inherit schema from dependent output port​

New features and improvements

  • Added dependency resolver configuration and implementation
  • Added docs

Related issue

Closes #70

[#63] Fix graphql queries for upsert​

New features and improvements

  • Modified queries in order to move the version, private_descriptor, and descriptor upsert from DataProduct to DataProductInstance table.
  • Modified how private_descriptor is created (now it includes the publicInfo fields).

Bug fixes

  • Fixed updated_at and published_at fields that didn't update.

Related issue

Closes #63