Changelog 1.1.0+
UI
Changelog v1.1.0
[#1189] fix query graphql
Bug fixes
Fix query graphQL, rename policy in governance_entity
Related issue
Closes #1189
[#1189] Fix query graphQL
Bug fixes
Fix query graphQL, rename policy in governance_entity
Related issue
Closes #1189
[#1172] Control panel is not displaying the correct label
New features and improvements
- Fix bug on threshold control panel component
- Add label field on control panel drawer
- Add tooltip to show value entirely on control panel drawer
- Fix bug on default color for threshold result component
- Fix bug on metric/policy wizard review to display resource preprocessing
Related issue
Closes #1172
[#1183] Hidden field resource preprocessing in metrics wizard
Bug fixes
Set to hidden strategy field and set default values based on form
Related issue
Closes #1183
[#1186] No label metrics are not visualized in the marketplace
Bug fixes
- Fixed "No label" metrics that are not displayed in the Marketplace
- Added the first error in the toast when creating/updating a metric/policy
Related issue
Closes #1186
[#1184] Resolve architectural docs links are broken
Bug fixes
- Fixed broken links in docs pointing to CGP docs
Related issue
Closes #1184
[#1169] In overview of metric and policy insert all field
New features and improvements
Added missing fields to policy and metric overview page
Related issue
Closes #1169
[#1174] Multiple metrics displayed in the marketplace
Bug fixes
- Fixed bug
Related issues
Closes #1174
[#1176] the name in the url does not match the menu item
Bug fixes
Edit wrong url root label
Related issue
Closes #1176
[#1180] Resolve "Change breakingChange to previousVsCurrent"
New features and improvements
Renamed the strategy breaking_change
to previous_vs_current
. This allows the user to understand what this preprocessing does, i.e. place the current version side by side with the latest one in the system
Related issue
Closes #1180
[#1178] Flag and Score Dialog displays incorrect links
Bug fixes
- Fixed incorrect index referred to links
Related issue
Closes #1178
[#1175] Fix UI docs broken
Bug fixes
Removed <br>
from docs.
Related issue
Closes #1175
[#1168] Ordering field in wizard metric and policy
New features and improvements
- Change order of fields inside metric/policy wizard
- Rename strategy field to Resource preprocessing
Related issue
Closes #1168
[#1171] More precise CTA in CGP create button
New features and improvements
- Made call to action for creating policy/metric in CGP clearer and matched the verbs used in documentation ("add" vs "create")
Related issue
Closes #1171
[#1170] Fix top failed policies time range
Bug fixes
Top failed policies time range
Related issue
Closes #1170
[#1167] Fix wizard metric
Bug fixes
Fix bug on metric wizard engine field.
Related issue
Closes #1167
[#1161] Resolve blueprint usage for component adding
Bug fixes
fix entity picker check on fetched entities
Related issue
Closes #1161
[#1148] improve metric wizard
Bug fixes
- Hidden field engine
- Hidden field interaction type, we put always 'User to Platform'
Related issue
Closes #1148
[#1140] Search Results Filters visibility
Bug fixes
Added filters on navigation from policies overview to search page
Related issue
Closes #1140
[#1166] Create a generic component to render the result of metric
New features and improvements
- Create new
ThresholdResult
component - Share default colors and types for thresholds
- Delete previous useless components
- Add new component on each of specified pages
Related issue
Closes #1166
[#1161] Resolve blueprint usage bug when adding a component
Bug fixes
Add the domain value after selecting a data product in the blueprint
Related issue
Closes #1161
[#1158] Switch Policy Overview page endpoints
New features and improvements
- Extended the
GovernanceApi
with stats endpoints - Top Failed and Inspector statistics (Policy Overview page) are now computed by calling the new CGP endpoints
- Added tooltips on the Inspector Table in the Policy Overview page
Related issue
Closes #1158, #982
[#1164] Insert a wrapper when visualize a metric/policy
Bug fixes
- Fix metric/policy overview
- Fix descriptor visualisation
- Fix long names
Related issue
Closes #1164 #1165
[#1163] Rename graphQL query with 'governance_entity'
New features and improvements
Rename graphQL query, table and field with 'governance_entity' concept.
Related issue
Closes #1163
[#1123] Add metric result in marketplace CGP results dialog
New features and improvements
Added metrics result in marketplace flag modal
Bug
Fixed bug on policies overview page when clicking on tags ellipsis on top 10 failed table, after opening the modal there was a wrong navigation on click event.
Related issue
Closes #1123
[#1153] Add metric test page/result
New features and improvements
- Add metric test page
- Add metric result page and related components
Related issue
Closes #1153
[#1162] Add event for new metric
New features and improvements
Add new event when we create new metric in CGP component
Related issue
Closes #1162
[#1116] Resolve bad rendering on Safari
Bug fixes
fix header logo rendering on Safari
Related issue
Closes #1116
[#1126] Threshold definition
New features and improvements
-
Transform environment input on editing from multiselect to select
-
Add new custom component to visualize thresholds and domain
-
Add resize observer to calculate the dimension of the thresholds line dynamically
-
Thresholds and domain inputs have following validation:
- Domain max and min value are required
- Domain max value cannot be higher than min
- Label is required
- Color is required
- Value is required
- Is not possible to insert two equal values
- If expression is (Above/Below)OrEqual
- If expression is Above/Below
- Label cannot have more than 64 chars
- Update app-config.yaml to include custom colors on
metrics:
thresholds:
colors:
- label: << String >>
value: << HexColor or StringColor >>
- Handle different type of threshold expression and update accordingly the chart based on following values:
- Above
- AboveOrEqual
- Below
- BelowOrEqual
- Add number formatter library to allow user to insert decimal and negative numbers
Breaking changes
- Update types to handle new thresholds schema
Related issue #1126
[#1099] Scaffolding errors more visible
New features and improvements
- Fixed the
DismissablePanel
component style to match the one of the errorAlertDisplay
- Fixed the nested Backstage error visualization for the affected actions of the scaffolder (e.g. the Backstage register action)
- Refactor the publish actions.
- Removed unused configuration variable
already_initialized_repository_enabled
. Updated documentation accordingly. - Fixed BitbucketServerRepositoryManager
getProjectSlug
,isRepoEmpty
methods.
Related issue
Closes #1099
[#1157] Restore governance config.d.ts
Bug fixes
- Restored config.d.ts in wb-governance
Related issue
Closes #1157
[#1144] Governance section breadcrumbs
Bug fixes
Fix Wrong name in breadcrumb in specific result page
Related issue
Closes #1144
[#1143] Menu and pages naming
New features and improvements
Change the name of menu item in Governance
Related issue
Closes #1143
[#1156] Add a configuration to disable blueprints
New features and improvements
Added a boolean configuration mesh.builder.scaffolder.blueprints.enabled
to enable/disable the blueprints.
Related issue
Closes #1156
[#1149] Adjust CGP policy and metric testing endpoints
New features and improvements
- Added test endpoint for metrics
- Improved configuration reading for the governance proxy plugin
Breaking changes
mesh.governance
section inapp-config
has been restructured
Migration
We must update the helm charts accordingly with the new app-config
structure for the mesh.governance
section
Bug fixes
- Fixed a bug that made metrics endpoints to be mandatory even if CGP metrics are disabled
Related issue
Closes #1149
[Closes #1147] feature: CI Simplification
New features and improvements
Removed build_vars job
Related issue
Closes #1147
[#1128] Strengthening the test suite for CGP
New features and improvements
- Added a new test to check the policy life circle. Here is a brief description of the steps of the test:
- Create a new cue policy
- Check "Draft" stance. Open the policy menu (three dots) and check available buttons, check that policy is editable
- Check "Grace" stance. Go from "Draft" to "Grace", again check the available buttons and check that policy is not editable
- Check "Disable" stance after "Grace", "New Version" function in that position. Go from "Grace" to "Disable". Check available buttons, create a new version, and check that it is not editable ("Disable"), but the new version has "Draft" status.
- Check "Enable" stance, "Publish" function. Go from "Disable" to "Enable", check available buttons, try to publish the policy, check the published policy button again, check that it is not editable.
- Check "Disable" stance after "Enable", "New Version" function in that position. Check available buttons, check that policy is not editable.
- Check "Deprecated" stance, and available buttons, and that policy is not editable.
- Clean all steps and results.
- Added fixes to existing tests
- Update chromedriver to version 113
Related issue
Closes #1128
[#1109] Refactor page Overview and Results
New features and improvements
Refactor page Overview|Results|Control Panel
Related issue
Closes #1109
[#1078] Auto-logout
New features and improvements
- Added Autologout to automatically logout users on idle after a certain amount of time
Related issue
Closes #1078
[#1145] Change the metric model
Bug fixes
- Fixed metric creation
Related issue
Closes #1145
[#1076] User headers
New features and improvements
- Introduced the
sensitive
flag on user configurations - Now the value of sensitive configurations can be redacted on-demand
- New
wb-user-headers
plugin (api and client to interact with the new user headers functionality) - User headers section in the Advanced Settings page
- User headers configurations under
mesh.userHeaders
- User headers forwarding functionality (toward the Coordinator)
- User headers documentation
- JWT token forwarding documentation
- Refactoring of the user settings section (Advanced Settings page) in order to make it aligned with the user headers one
Impacts
The user headers functionality is disabled by default (config mesh.userHeaders.enabled
to false
). The headers section will not be visible to the user and no headers will be forwarded to the coordinator until that configuration is enabled.
Related issue
Closes #1076, #1077, #1084
[#1135] Problems related to environmental change
New features and improvements
- Added
SelectEnvironmentRadio
component to select just one policy
Bug fixes
- Fixed the fact that updating the policy environment has not a consistent visualization
Related issue
Closes #1135
Default value of the cueScript field must be an empty string
Bug fixes
- Made the cue script to assume empty string as default value
Related issue
Closes #1136
[#590] Improve control panel features discovery
New features and improvements
Introduced some tooltips to better explain the functionalities of the control panel buttons.
Related issue
Closes #590
[Closes #1131] feature: [CGP] Runtime breaking change policies don't make sense
New features and improvements
Both in the new/update policy/metric forms is not possibile anymore to select strategy Breaking Change
when the selected timing is Runtime
Related issue
Closes #1131
[#1139] Resolve the test policy that is not working
Bug fixes
- Added a missing token in the request for the Test policy
Related issue
Closes #1139
[#1137] Selectors are not accepted in the create and update requests
Bug fixes
- Fixed a bug that prevented the creation/update of a policy or metric with selectors
Related issue
Closes #1137
[#1134] Policies Overview top 10 table is not fetching all the failed policies
Bug fixes
- Added
warning
,info
,bot_blocking_error
to the graphql query that fetches the top 10 fail policies. - Changed
Type
intoStrategy
in the top 10 table header
Related issue
Closes #1134
[#1108] Restore html and json fields in informative notification
New features and improvements
- Restore html and json fields in informative notifications and add parameters in notifications/by-event
- Make entity query parameter optional in notifications/by-event
Related issue
Closes #1108
[#979] New CGP policies results' search page
New features and improvements
Add the search policy page with filters and results' table.
Related issues
Closes #979
[#1129] Improve definition of done for issues
New features and improvements
Introduced the Definition of Done for all the Merge Requests.
This is a checklist of things to always check before putting the MR in review.
Since there is an open MR in Gitlab which is 5 years old to add a variable to reference the issue in the MR template, and no one seems to solve it, I decided to make the MR template the default, so the "Closes #{issue}" is automatically added.
Related issue
Closes #1129
[#1132] Resolve a missing validation entity in BlueprintProcessor
Bug fixes
- Added
validateEntityKind
method toBlueprintProcessor
to validate blueprint templates. - Added validation schema for blueprints.
Related issue
Closes #1132
[#1124] View, Test, and Delete buttons visibility in the metrics menu
Bug fixes
- With
cgp.entity.view
permission only set, the Test and Delete buttons are now not visible in each menu of the metrics. - With
cgp.entity.edit
permission set, the View button is now visible in each menu of the metrics.
Related issue
Closes #1124
[#1095] Documentation of change management helper center is very limited
New features and improvements
- Added 2 tables in
Notifications
:- Table to summarize all the events that trigger a notification.
- Table to describe recipients of each event.
- Fixed section in
Witboost configuration
.
Related issue
Closes #1095
[#1110] Introduce field trigger
New features and improvements
- Add new trigger field on policies/metrics
- Update queries to retrieve new field
- Update add/edit wizards to handle new constraints
Breaking changes
- Update types adding new Trigger enum ("Active", "Passive")
- Remove "Passive" from timing
- Update proxy requests to handle new endpoints about metrics
- Update proxy requests to adapt change on new version endpoint (from POST to PATCH)
Related issue
Closes #1110
[#1102] Autotests for CGP module
New features and improvements
- policy life circle up to the "grace"/"disable"/"publish" steps
- test to verify that the edit policy is only in the draft step
- got optimized way of tests (while creating dp or op) to save a LOT of time pipeline running.
- reworked test with negative deploy check (3-5)
- updated tests that check notifications (suit 6)
- fixed existing tests to check CGP module according to the current position
Related issue
Closes #1102
[#1013] Fix EntityPickers that show the internal ids of the entities
New features and improvements
Introduced a lot of classes that define how entities inside witboost are structured (as extensions of the basic Backstage ones). To enforce the definition, we added also the JSON schema for each of them, so methods that need to make validations can leverage the associated validators.
The EntityPicker
, EntitySelectionPicker
, and EntityComponentsPicker
classes have been updated, in order to display the human-friendly names of entities instead of the entities' identifiers.
Breaking changes
The allowArbitraryValues
flag was removed from all the aforementioned pickers since it would increase the possibility for users to make mistakes.
Related issue
Closes #1013
[#1118] Test and Delete buttons not visible in PoliciesRowMenu
New features and improvements
Made buttons visible when the user has the right permissions.
Related issue
Closes #1118
[#1117] hotfix: Errors in policy creation
Bug fixes
- Fixed resultType
- Fixed graphQL query
Related issue
Closes #1117
[#1113] Fix Policies Refactor
Bug fixes
- Fix policies filter ui
- Fix Cron field in create/edit
Related issue
Closes #1113
[#1101] CGP backend must send a request for each environment
New features and improvements
- Modified message toast when creating and updating a policy
- Modified the way the request is sent to the coordinator for policy creation and update
Bug fixes
- Fixed a mismatching between the coordinator response and PolicyValidateResult
Related issue
Closes #1101
[#1114] Update the EvaluationReport DTO
New features and improvements
- Aligned Evaluation Result DTO with the one contained in the coordinator
- Fixed some bugs
Related issue
Closes #1114
[#1083] Update the images in the official doc
New features and improvements
Changed the following images for the documentation:
- blueprint selecting
- selecting new DP while using a blueprint
- DP graph ("getting starting" and "dp details" pages)
- Updated the information about blueprint usage
Related issue
Closes #1083
[#1072] CGP permissions review
Breaking changes
- The governance menu is now visible only if the user has either
cgp.entity.view
orcgp.entity.edit
permissions. - with
cgp.entity.view
the user can only view the policies/metrics from the Registry. - with
cgp.entity.edit
the user can do all on policies/metrics from the Registry (view, create, edit, delete). - in the
Flag and Score
dialog, results are clickable only if either one of the 2 new permissions are set. - Backend blocks actions on policies/metrics (create, edit, delete) if user do not have
cgp.entity.edit
permission. - Frontend shows an
ErrorPanel
if the user does not havecgp.entity.view
orcgp.entity.edit
permissions and tries to navigate to a forbidden route
Migrations
backstage
db ->rbac.permissions
:- changed row id from
builder.policy.template.edit
tocgp.entity.view
- changed row id from
builder.policy.global.edit
tocgp.entity.edit
- changed row id from
Related Issue
Closes #1072
[#989] Restyle the output port marketplace page
New features and improvements
Align Output port page UI with new mock
Related issue
Closes #989 #1059
[#1112] Query params filters get lost when moving between policies and metrics
New features and improvements
- Create custom hook to handle query params using session storage
- Remove useless HOC
- Improve code reusability
Bug fixes
Fix bug on policies/metrics query params
Related issue
Closes #1112
[#1098] Update name field in provisioningPlan response
Bug fixes
Update provisioning plan api response type
Related issue
Closes #1098
[#1096] Refactor metric
New features and improvements
- Adapt metrics to have same features as policies
- Add multi select filters
- Add timing column
- Display badge on filters button
- Get applied filters from url params
- Adapt metrics to new BE changes
- Update metrics add/edit wizard to new constraints
- Update types
Related issue
Closes #1096
[#1103] Resolve "Refactor policies"
Breaking changes
Align policies views with the new BE structure
Related issue
Closes #1103
[#1073] Improving the policy suit
New features and improvements
- Split a bunch of tests that were in the same scenario into different tests
- Added a new test scenario when the user can add valid or invalid cron expression to the runtime policy
- Added new test scenario when the user tries to input invalid cue policy
- Corrected and improved existing autotests according to the current situation
Related issue
Closes #1073
[#967] Resolve "Edit Metric page"
New features and improvements
Add edit metrics page
Related issue
Closes #967
[#934] The Data Product gets added to the Builder even though the creation failed
New features and improvements
- Add a line in the documentation explaining to not create a data product repository/branch with the same name of a repository/branch deleted from Gitlab but not from the builder.
Related issue
Closes #934
[#980] Improve marketplace CGP results dialog
New features and improvements
- Added
i
icon and tooltip to show the description of every warning/error policy in theFlagAndScoreDialog
- Added authorization to view the link of the policy. The permission is set to
builder.policy.global.edit
- Increased width of the Flag Dialog
Related issue
Closes #980
[#1060] Show data product description in the marketplace main table
New features and improvements
Add the description column inside the catalog table.
Related issue
Closes #1060
[#971] Update the Marketplace db schema to handle versions in the right way
Breaking changes
- Modified Hasura queries for the marketplace in order to fetch version, descriptor, and private_descriptor from the DataProductInstance table.
Migration
- Added a migration to move descriptor, version, and private_descriptor field from DataProduct to DataProductInstance and to modify the content of the
private_descriptor
field of DataProductInstance, OutputPort, and Component tables
Related issue
Closes #971
[#1082] Fix helm package job
New features and improvements
Fixed a misconfiguration for the helm package job.
Related issue
Closes #1082
[#1070] Fix version consistency between the Docker image and the Helm chart
Bug fixes
CI configure Docker image correctly in Helm chart values file
Related issue
Closes #1070
[#1065] Policy is not deprecated as instead stated
Bug fixes
- Fix the detail icon tooltip
- Fix policy status inside the detail drawer
- Hide descriptor when empty in detail test
Related issue
Closes #1065 #1066
[#1071] Add the "view" action also when the user has permission to edit policies
New features and improvements
- Add view action for all policy status
- handle policy overview registry breadcrumb to retrieve preselected filters
- change marketplace graph sidebar filters button badge
Related issue
Closes #1071
[#1038] Multiselection on policy filters
New features and improvements
- Add multi-select on policy filters
- Update policies list query to handle arrays
- Build dynamic breadcrumb for Add/Edit policy page with HOC
- Add a filters badge on the button
- Set filters accordion opened if there are selected filters
Related issue
Closes #1038
[#1035] Add a new delimiter for array pickers
Bug fixes
- Added a bottom margin for every element in the ArrayFieldTemplate in order to space them with a blank line
Related Issue
Closes #1035
[#986] Tests for the policies module
New features and improvements
- Update tests 4-1 for general checking CGP module (bookmarks, search and filter functions, environments selection)
- Added tests 4-2 to check policy creation/deleting
- Added tests 4-3 to check policy modification (name, description, strategy, severity, interaction type, timing, and context)
Related issue
Closes #986
[#1069] Fix CI failure on Package Job
Bug fixes
restore ci functionality
Related issue
Closes #1069
### [#1068] Fix consistency between helm and docker versions
New features and improvements
Updated the ci to set the correct version of the docker image in the values.
Related issue
Closes #1068
[#1064] HasuraClaims default breaks deployments
New features and improvements
This MR changes the default role injected by witboost in the requests througout hasura, from user
to admin
.
Related issue
Closes #1064
[#1022] Resolve "Policy registry is not keeping the selection of the environment"
New features and improvements
- Sync registry filters with url query params
- Add validation to url query params (if user changes url manually with a wrong value, a default one is set)
- User can share url with applied filters
- Pagination limit can be set on query params
TBD Breadcrumbs. Currently, we don't have a way to build dynamic breadcrumbs. Now, clicking on it reset filters. A possible solution could be to use session storage but we would lose the shareable url functionality. I don't advice to mix both because it would be hard to debug and maintain.
Related issue
Closes #1022 #1023
[#1047] Policy result page is not adapting to the context
Bug fixes
Hide CUE script in the result page when the policy doesn't require it
Related issue
Closes #1047
[#1045] the flags section does not display the results in warnings and info
Bug fixes
Fix graphQL query to show flags
Related issue
Closes #1045
[#1044] Add field additionalMetadata when create/edit a policy
New features and improvements
- add additionalMetadata to create/edit payload
Bug fixes
- Fx preprocessing input field enable condition
Related issue
Closes #1044
[#1046] Add format 'json' when display Details section
Bug fixes
Fix json formatting
Fix tags spacing and wrap
Related issue
Closes #1046
[#1043] Remove policies docs under platform customizations
Bug fixes
Removed policies docs under platform customizations
Related issue
Closes #1043
[#978] New policies specific results page
New features and improvements
Add the new specific result page
Related issue
Closes #978
[#1042] Adjust blueprint docs title
Bug fixes
- Adjusted title in blueprint docs
Related issue
Closes #1042
[#1040] Blueprint Template dependencies error
Bug fixes
Fix template dependencies inside the blueprint.
Related issue
Closes #1040
[#1036] Resolve the new version feature that sometimes fails silently
New features and improvements
If the branch already exists we are throwing a new Error instead of only returning so that we stop failing silently and have better error message/handling
Related issue
Closes #1036
[#1027] Fix policy error not scrolling anymore to highlight selected error
Bug fixes
- Fixed policy cue editor error highlight
- Fixed auto scrolling on the first error
Related issue
Closes #1027
[#1033] Remove policies docs from ui repo
New features and improvements
- Removed old docs on managing policies
Bug fixes
- Old screenshots for policy tests are now updated
Related issue
Closes #1033
[#1032] Add blueprint documentation
New features and improvements
Add public documentation to explain the following to the user:
- what a blueprint is
- how it works
- how the user can use it to create a new DP
- how the user can use it to add a component to the DP
- what else the user can do with the blueprint
Related issue
Closes #1032
[#1024] Policy launch results not consistent
Bug fixes
Fix policy launch results with the right error status
Related issue
Closes #1024
[#1028] Change the default "rows per page" in pagination
Bug fixes
- Fix bug on builder breadcrumbs
- Change default rows per page in pagination (default 25) with 25/50/100
Related issue
Closes #1028 #1031
[#1037] Make the data sharing agreement field displayed in the marketplace configurable
New features and improvements
Introduced a new configuration mesh.marketplace.ui.outputPortPage.dataSharingAgreementField
that can be used to define which field of the descriptor will be used to display the data sharing agreement details.
Related issue
Closes #1037
[#1025] Fix blueprint components order that is not preserved
Bug fixes
- Fix blueprint template order
- Fix condition to show other components
Related issue
Closes #1025 #1026
[#1034] Fix the list of data products in the blueprint that is not consistent with my data products
New features and improvements
Updated the blueprint documentation to explain better which data products are loaded when selecting one to confront with a blueprint.
Related issue
Closes #1034
[#937] FIx the wrong percentage in the DP Usage card
Bug fixes
Solved an error with the percentage displayed for the marketplace supervision panel.
Related issue
Closes #937
[#1029] Resolve the wrong type displayed in the templates page
Bug fixes
Fixed a bug that displayed the wrong template group name on the "Templates" page.
Related issue
Closes #1029
Changelog v1.1.1
[#1204] Fix regex to handle undefined values
Bug fixes
- Handle cases where input is not set and is undefined so it will fail to match the regex even when regex allows empty values
- Remove the hard limit of length 1 for the input
Related issue
Closes #1204
Changelog v1.2.0
[#1205] Strange label in control panel test
Bug fixes
Fix bug on wrong label in control panel test.
Related issue
Closes #1205
[#1214] Fix breadcrumb in Metric Results
New features and improvements
Update navigation on metrics results
Related issue
Closes #1214
[#1182] Page Search Metrics
New features and improvements
- Add metrics search page
- Refactoring common components
Related issue
Closes #1182
[#1213] Fix flags and scores loading
Bug fixes
Fixed flags and scores loading error
Related issue
Closes #1213
[#1197] Resolve the environment reset across different tabs of CGP
New features and improvements
- Share environment fields across different tabs of cgp
Related issue
Closes #1197
[#1211] Introduce the scroll bar in the error component
New features and improvements
Add scroll bar inside error box component
Related issue
Closes #1211
[#1190] Resolve "Retrieve metric and policy flags through the CGP endpoint"
New features and improvements
- Extended the
GovernanceApi
with latest resource evaluation retrieval endpoint - Flags and Scores in the marketplace data product catalog and in the data product page are now computed by calling the new CGP endpoint
Related issue
Closes #1190
[#981] Add duplicate name entities check when registering location
Bug fixes
- Backstage registering process does not throw an error when trying to import a location that spawns an entity with a duplicate name. This MR patches backstage so that when analyzing a location before registering, checks the location will generate entities with names already existing in the catalog. This works only in register dryRun operations, as normally the register operation does not process the location to discover entities. If the location already exists, then the patch has no effect and the workflow proceeds as normal, prompting the user to refresh the location.
Related issues
Closes #981, #584
[#1151] CGP test suite extension
New features and improvements
- Added a new test that will check the ability to change context, trigger, time, engine, strategy, severity
- Fix AT system
Related issue
Close #1151
[#1209] Output Port datacontract is optional
Bug fixes
Handle optional datacontract/schema/sla
Related issue
Closes #1209
[#1100] Reserved field processor
New features and improvements
- ReservedFieldsProcessor is a processor that checks the ‘reserved fields’ and emits an error in processing phase if they were modified
- Ability to specify the path of the reserved fields in configuration
Related issue
Closes #1100
[#1202] Improvements Test Result section
New features and improvements
- Add fixed dimension to test results panel
- Add scrolling to test results
Related issue
Closes #1202
[#1105] Add validity checks in publish action
New features and improvements
- Add additional checks before performing the publish action on GitLab and BitBucket
- Current additional checks are: YAML parsing, allowed kinds, system/component/domain schema check
Related issue
Closes #1105
[#1204] Fix regex to handle undefined values
Bug fixes
- Handle cases where input is not set and is undefined so it will fail to match the regex even when regex allows empty values
- Remove the hard limit of length 1 for the input
Related issue
Closes #1204
[#1198] Add policy title click action
New features and improvements
- Add cursor pointer on policy/metrics page
- Handle click on title to navigate on view page
- Add underline decoration on hover for uniformity with the rest of the application
Related issue
Closes #1198
[#1191] Schema Editor Discovery Week
New features and improvements
Add schema editor section to OP page.
Related issue
Closes #1191
Provisioning Coordinator
Changelog v1.1.0
[#456 and #418] Resolve "Update CGP documentation"
New features and improvements
- Updated CGP docs to latest changes
Related issue
Closes #456, #418
[#466-FollowUp] Fix CGP Policy migration script
New features and improvements
Modified table names based on new CGP schema
Bug fixes
Fixes the migration script that wouldn't work with the latest versions of CGP
Related issue
Closes #466
[#466] Fix CGP Policy migration script
New features and improvements
Modified table names based on new CGP schema
Bug fixes
Fixes the migration script that wouldn't work with the latest versions of CGP
Related issue
Closes #466
[#465] Change breakingChange to previousVsCurrent
New features and improvements
Renamed the strategy breaking_change
to previous_vs_current
. This allows the user to understand what this preprocessing does, i.e. place the current version side by side with the latest one in the system
Related issue
Closes #465
[#464] Fix conversion for MetricExecutorResultError
Bug fixes
- Handled MetricEvaluationResultError cas in toEvaluationResult method
Related issue
Closes #464
[#461] Metric's values should be validated against a domain
New features and improvements
- Added domain check for metric values
Related issue
Closes #461
[#462] performance improvement during evaluation
New features and improvements
- created specific query for upsert operations
- Added index on table evaluation result
- configured memory sizing on kubernetes
- configured jvm heap
Related issue
Closes #462
[#450] Define policy execution stats endpoints
New features and improvements
- New CGP stats endpoints
- New CGP stats components (repository, adapter, port)
- Modified the provisioning flow to attach a configurable label to the CGP evaluation report it generates
Related issue
Closes #450
[#462] Performance improvement during evaluation
New features and improvements
- created specific query for upsert operations
- Added index on table evaluation result
- configured memory sizing on kubernetes
- configured jvm heap
Related issue
Closes #462
[#462] Performance improvement during evaluation
New features and improvements
- created specific query for upsert operations
- Added index on table evaluation result
- configured memory sizing on kubernetes
- configured jvm heap
Related issue
Closes #462
[#458] Rename table and field with 'governance entity' concept
New features and improvements
This PR contains a renaming of table, fields, column name, logs with governanceEntity
concept
Breaking changes
Some queries in graphQL need to be renamed
Related issue
Closes #458
[#457] Fix status result of report for metric
Bug fixes
This PR contains some improvements on report status for metric evaluation.
Until all the metrics have been evaluated, the status of the report remains pending. When a response has been received for all the metrics (and therefore associated with the relative threshold) the status of the report is calculated only on the policies.This is because metrics are values and do not have the concept of failure as in policies
We added the environment
parameter to the internal evaluate method.
Related issue
Closes #457
[#376] Notification on new policy enabled
New features and improvements
- notification on failed scheduled governance entity
- notification on new governance entity creation
- updated documentation about CGP configuration (now it includes the notification part)
- updated helm chart for inserting two new env variables
Related issue
Closes #376
[#446] Update API to new version of threshold
New features and improvements
- Changed all the APIs that include Threshold as request parameter
- Changed the threshold computation
Related issue
Closes #446
[#455] Split CGP env variables names for config
New features and improvements
- Added
CGP_DATABASE_URL
env variable - Improved logging in policy scheduler
Related issue
Closes #455
[#455] split cgp env variables names for config
New features and improvements
- Added
CGP_DATABASE_URL
env variable - Improved logging in policy scheduler
Related issue
Closes #455
[#453] Timeout on cgp evaluation tasks
New features and improvements
- Added timeout configuration for CGP evaluation
Related issue
Closes #453
[#454] Remove from log the descriptor
New features and improvements
to improve the readability of the logs we have removed, in this MR, the print of the descriptor.
Related issue
Closes #454
[#443] Add metrics testing
New features and improvements
- Added new endpoint
/metrics/:id/test
for testing a metric entity - Changed endpoint
/policies/:id/evaluate
into/policies/:id/test
- Improved error codes on endpoints. 404 when entities are not found. 422 when request is valid but entity cannot accept it due to its internal state. Error handler has been centralized
Bug Fixes
- Renaming of most of the references to policies inside the Evaluator. From
policy
togovernanceEntity
still more to be renamed
Related issue
Closes #443
[#452] Configurable specific provisioner endpoint version
New features and improvements
Added a configuration used to specify the specific provisioner endpoint version
Related issue
Closes #452
[#448] Coordinator async task does not handle sync replies from specific provisioner
Bug fixes
SpecificProvisionerAsyncHttpTask manages specific provisioner sync reply too.
Related issue
Closes #448
Add domain field to governance entity
New features and improvements
- Added a new
domain
field for metrics, representing the range of allowed values accepted by the metric
Related issue
Closes #433
[#447] CI Simplification
New features and improvements
Removed build_vars job
Related issue
Closes #447
[#444] Cron jobs must be executed in isolation
Bug fixes
Unhandled cron job exceptions are not anymore blocking for other scheduler's tasks
Related issue
Closes #444
[#441] Fix policy migration script
New features and improvements
Adds to the SQL migration script from old policies to CGP policies the new fields.
Migration
The script can now be used to migrate from old policies to CGP policies correctly
Bug fixes
Fixes a bug where the script was not aligned with the latest version of CGP so the migration would fail
Related issue
Closes #441
[#440] Fix documentation formatting
New features and improvements
Better formatting on the config documentation for Docusaurus compatibility
Related issue
Closes #440
[#438] Restore the cron expression converter
New features and improvements
- New:
CronExpression
domain object that embeds by-design validation and conversion - New: The original cron expression (that provided by the UI) is now retained along with the equivalent expression in cron4s format. For instance, if the UI provides cron expression
2/10 * * ? * *
, it wil be internally converted to the equivalent expanded expression2,12,22,32,42,52 * * ? * *
, but the UI (the API consumer) will always get the original provided expression and not the extended one.
Bug fixes
- Restored the cron expression conversion step
Related issue
Closes #438
[#436] breaking change policies don't make sense if runtime
Bug fixes
- You cannot create a policy/metric with timing runtime and strategy breaking change anymore. since it does not make any sense because breaking changes can be detected only at deploy time.
Related issue
Closes #436
[#319] Create a end-to-end test suite (insert - update - evaluate)
New features and improvements
- End2end tests for the CGP
Related issue
Closes #319
[#432] Add trigger field to governance entity
New features and improvements
- Added
trigger
field
Related issue
Closes #432
[#324] Coordinator - Integrate similarity check service
New features and improvements
- New: Similarity Check Service configuration. Disabled and non-blocking by default
Related issue
Closes #324
[#430] Split create and update API for policies and metrics
New features and improvements
- New create metric endpoint
POST /v1/governance-platform/metrics
- New update metric endpoint
PUT /v1/governance-platform/metrics/:metricId
- New update metric status endpoint
PUT /v1/governance-platform/metrics/:metricId/status
- New bump metric version endpoint
PATCH /v1/governance-platform/metrics/:metricId
Breaking changes
- Bump policy version is now reachable at
PATCH /v1/governance-platform/policies/:policyId
- Now when asking for a status change, the response will be the entire updated entity, not just the new status
Related issue
Closes #430
[#406] Headers forwarding
New features and improvements
- New: Forwardable headers integration and configuration
- New configuration
provisioning-coordinator.forwardable-headers
- HTTP scheduler tasks now include an
httpHeaders
param - The provisioning and validation flows now handle an
httpHeaders
list coming from the API layer HttpClient
now accepts headers too- A
ForwardableHeadersHelper
filters the incoming headers that can be forwarded according to the configurations
- New configuration
- New: Forwardable headers documentation
- Fix: The integration tests suite is now restored
- New: Headers forwarding integration test
Impacts
After merging these changes, by default Authorization headers and headers starting with wb-user-
will start to be forwarded (if present) to the provisioning and validation services.
They are not mandatory, hence no error is raised if this kind of headers cannot be found.
Related issue
Closes #406, #405
[#431] EvaluationResults for metrics must be filtered when computing status
New features and improvements
- Added policy_result_type in evaluation_result table
- Added a filter in the status method that accepts only the boolean policies
- Renamed all the fields of evaluation_result in order to include the governance_entity concept
Related issue
Closes #431
[#419] Each Policy must be associate just a one environment
New features and improvements
- Modified the way policies can be updated and created (now there is 1 policy for each environment in the Policy table)
Related issue
Closes #419
[#357] Better logging
New features and improvements -Improved logging in the CGP
Related issue
Closes #357
[#416] Insert field engine and strategy for policy and metric
New features and improvements
- Renaming
- Policy => GovernanceEntity
- BooleanPolicy => Policy
- MetricPolicy => Metric
- use a field
strategy
to capture the concept of 'resource modification' before being evaluated from engine- Default: the input resource has no modifications
- BreakingChange: the resource consists of the latest version retrieved by the perimeter resolver and the one supplied as input
- introduce a new field
engine
- cue
- remote
Breaking changes
Change the table policy
in schema cgp
:
- rename
execution_strategy
=>strategy
- add field
engine
Migration
A flyway migration was created to handle these changes
Related issue
Closes #416
[#427] Coordinator calls CGP using HTTP call
New features and improvements
Http calls to CGP
Related issue
Closes #427
[#427] Coordinator calls CGP using HTTP call
New features and improvements
Http calls to CGP
Related issue
Closes #427
[#426] Update Mock-Specific-Provisioner for remote policy
Related issue
Closes #426
[#429-FollowUp] Improvements build and openapi generator
Bug fixes
Fix ci pipeline to publish just provisioning-coordinator artifact
Related issue
Closes #429
[#429] Resolve "Improvements build and openapi generator"
New features and improvements
- Improve
build.sbt
to publish artifact only for root project (provisioning-coordinator) - Remove function
validationFunction
from Controller (does a check that application serve already does)
Related issue
Closes #429
[#428] Update the documentation for CGP configs
New features and improvements
- Added Evaluation configuration docs
Related issue
Closes #428
[#407] UpdatePolicyStatus should run also the evaluate API
New features and improvements
- Added the possibility to run an evaluation if the policy changes the status to Enabled
- Made the above behaviour configurable by CGP
Related issue
Closes #407
[#425] RDisplay name should be filled if available from perimeter resolver
Bug fixes
- Resource in the request of a push evaluation result always takes precedence except for the display name resolved by perimeter resolver
Related issue
Closes #425
[#410] move interface specification in the cgp module
New features and improvements
cgp interface specification in it's own module
Related issue
Closes #410
[#1049] Push evaluation result defects
New Features and improvements
- Push evaluation result API has now been splitted into:
POST /metrics/{metricId}/evaluation-results
to receive results for local metric policies
POST /policies/{policyId}/evaluation-results
to receive results from local policies
Bug fixes
- Fixed a bug that caused draft and disabled local policies to produce errors in the marketplace
Related issue
Closes #1049
[#404] External policy refactoring
New features and improvements
Introducing the engine concept for cue and external policies
Related issue
Closes #404
[#374] Documentation - Record some missing configurations
New features and improvements
Added documentation for:
- Data catalog provisioning configurations
- Data deletion configurations
- Marketplace compliant services configurations
Related issue
Closes #374
[#403] Version consistency between the Docker image and the Helm chart
Bug fixes
CI configure Docker image correctly in Helm chart values file
Related issue
Closes #403
[#402] CI failure on Package Job
Bug fixes
restore CI functionality
Related issue
Closes #402
[#401] Consistency between helm and docker versions
New features and improvements
Updated the ci to set the correct version of the docker image in the values.
Related issue
Closes #401
[#375] CGP - threshold in metrics should not overlap
New features and improvements
Metric thresholds are checked for overlaps
Related issue
Closes #375
[#400] FIx for migration script
New features and improvements
Clean migration script
Related issue
Closes #400
[#399] Validate environment when pushing evaluation results
Bug fixes
The environment param provided to the save
method of EvaluationReportService
is now validated against the available system environments.
Related issue
Closes #399
[#398] Add field additionalMetadata in openapi specification
New features and improvements
- Added
additionalMetadata
field inPolicyCreationModel
andPolicyUpdateModel
Related issue
Closes #398
[#396] "Improvements migrations script"
Migration
Improvements migration script to populate the table 'provisioning_operation_state'
Related issue
Closes #396
[#395] Create mocked APIs for external policies
New features and improvements
- Added APIs for mocking external policies execution with Boolean policy returning object
Related issue
Closes #395
[#394] Remove useless fields in push evaluation results API
Bug fixes
- Removed
policyStatus
,resourceType
from push evaluation result API
Related issue
Closes #394
[#392] Split docs kinds for CGP
New features and improvements
- Adjusted docs for CGP so that they are visible in public and private docs
Related issue
Closes #392
[#397] Add migration script for policy tags
New features and improvements
The policy migration script now includes policy tags migration
Related issue
Closes #397
[#393] Fix log name inside CUE validation errors
Bug fixes
CUE evaluation errors now log the policy id instead of the policy name
Related issue
Closes #393
[#390] handling of http errors in calls to the specific provisioner
Bug fixes
Errors after invoking specific provisioners should be encapsulated in json objects
Related issue
Closes #390
[#391] Add additional_metadata field to policy table
New features and improvements
- Added
additional_metadata
field inpolicy
andpolicy_history
tables - Modified migration script in order to populate the above field
- Modified queries for inserting and updating
policy
andpolicy_history
tables
Related issue
Closes #391
Changelog v1.1.1
[#468] Problem with applying multiple policies at the same time
Bug fixes
- Fixed remote engine errors during metric and policy evaluation (now it returns an InternalErrorResult)
Related issue
Closes #468
[#463] Expose endpoints to retrieve metric and policy flags
New features and improvements
- Updated CGP docs to latest changes
Related issue
Closes #456 #418
Changelog v1.1.2
[#481] Migration script uncorrectly migrates 'status' column
Bug fixes
- Fixes a bug in the migration where the 'status' column of
execution_plan
is incorrectly carried out in the tableprovisioning_operation_state
. - Fixes a bug where the column 'environment' is not carried out correctly into
provisioning_operation_sate
Related issue
Closes #481
[#476] Policy deleted information is lost when migrating
Bug fixes
Property 'status' of coordinator policy is carried out into 'status' of CGP policy in the migration script
Related issue
Closes #476
Bug fixes
- Fixed remote engine errors during metric and policy evaluation (now it returns an InternalErrorResult)
Related issue
New features and improvements
- New CGP endpoint
/resources/latest-evaluations
- Domain and Infra components to retrieve the latest resource evaluations
Related issue
Closes #463
Changelog v1.1.3
[#487] Scheduler hangs-up during sync validation with a lot of components
Bug fixes
- The scheduler now uses a cached thread pool for handling blocking operation instead of a configurable fixed thread pool
Related issue
Closes #487
Marketplace Plugin
Changelog v1.1.0
[#70] Inherit schema from dependent output port
New features and improvements
- Added dependency resolver configuration and implementation
- Added docs
Related issue
Closes #70
[#63] Fix graphql queries for upsert
New features and improvements
- Modified queries in order to move the version, private_descriptor, and descriptor upsert from
DataProduct
toDataProductInstance
table. - Modified how private_descriptor is created (now it includes the publicInfo fields).
Bug fixes
- Fixed updated_at and published_at fields that didn't update.
Related issue
Closes #63