Skip to main content

Changelog 0.4.0

UI​

Changelog v0.7.0​

[#651] Licensing​

Breaking changes

Defined a licensing method that prevents the product from running in case the license is expired.

Migration

The Helm chart and the secrets should be updated. All the instructions are listed in the installation guides.

Related issue

Closes #651

[#718] RBAC subjects should always include namespace​

Bug fixes

  • RBAC now handles both subjects formats (namespaced subjects and implicit namespace subjects)

Related issue

Closes #718

[#536] Template doc not behaving as expected​

New features and improvements

Open the template docs on a page instead of a modal

Related issue

Closes #536

[#562] Notify consumer components owners when a new version is released​

New features and improvements

  • When new_version gets called, a notification is sent to all consumers' owners of the DP.
  • A new relationship is emitted between consumers and consumed components. The relation, according to backstage practices, is bidirectional and of types wb-isReadBy and wb-readsFrom.

Note: a consumer is a component that reads from the output port of another data product.

Related issue

Closes #562

[#689] Move the policies menu into a Governance super-menu​

New features and improvements

  • Move the Policies menu into a governance super-menu
  • Updated menu icons

Related issue

Closes #689

[#692] Improve autotest for General section​

New features and improvements

  • Fixed tests for the new UI changes
  • Added test which works with global search and advanced search functionalities

Related issue

Closes #692

[#710] Poor spacing between menus​

Bug fixes

Increase spacing between menus.

Related issue

Closes #710

[#690] Add the Technical Information card in the marketplace​

New features and improvements

Add the Technical Information card

Related issue

Closes #690

[#709] Clearer exceptions when the user is not found​

Bug fixes

Improve descriptor preview error message handling when a user is not found.

Related issue

Closes #709

[#696] Repository path on scaffolding does not support trailing slash​

Bug fixes

Add regex to handle initial and final slash on user/group text input cloning repository.

Related issue

Closes #696

[#703] Configurable Hasura role in JWT​

New features and improvements

You can now optionally configure the default role and allowed roles injected in the user JWT for Hasura, using two properties

  • mesh.marketplace.hasuraClaims.x-hasura-allowed-roles (default is ['admin'])
  • mesh.marketplace.hasuraClaims.x-hasura-default-role (default is admin)

Related issue

Closes #703

[#697] Change a label to address all YAML types​

Bug fixes

Fix incomplete validation message label on component creation wizard.

Related issue

Closes #697

[#525] Adding permission to Front End​

New features and improvements

Disable actions that the user cannot perform according to his role/permissions

Related issue

Closes #525

[#700] Extension points API overview​

Closes #700

[#672] Refactor roles_subjects RBAC table​

New features and improvements

  • Added new field enabled in roles_subjects RBAC table to enable/disable a grant
  • Removed field subject_kind from roles_subjects RBAC table to remove unnecessary overhead when parsing grants requests

Breaking changes

  • enabled field will be set to true for all existing records found in roles_subjects. No further action is needed.

Migration

  • Migrations are automatically performed by knex. No further action is needed by platform team.

Related issue

Closes #672

[#708] Align Policies query to latest fields names​

Bug fixes

For validation policies:

  • updated_at is now update_time
  • created_at is now creation_time

Related issue

Closes #708

Closes #701

[#670] Remove RBAC permissions from JWT token​

New features and improvements

  • Removed RBAC permissions from JWT token
  • Fixed TechDocs issue caused by huge permissions header in JWT

Related issue

Closes #670

[#675] Fix marketplace breadcrumbs from Output Port to Data Product​

Bug fixes

Fix the output port page breadcrumb error that happens when the user goes back from the output port page to the data product one

Related issue

Closes #675

[#668] Remove diagram tab from DP page​

New features and improvements

Removed diagram tab from DP page

Related issue

Closes #668

[#688] Broken dropdown menu​

Bug fixes

Fix the select environment combo-box on the policy page

Related issue

Closes #688

[#626] Review typography/spacing/layout​

New features and improvements

Implement the new UI reducing the font sizes and re-arranging the pages' elements to improve the user experience.

Related issue

Closes #626

[#384] Fix minor documentation errors for local setup​

Bug fixes

Fix minor documentation errors

Related issue

Closes #384

[#638] Scaffolding initial commit appears as user "witboost"​

New features and improvements

  • set current user as first commit author if not specified otherwise
  • refactoring code to centralize this logic for gitlab and bitbucket
  • add documentation with the priorities of user used for scaffolding operations (configuration first, then templates, then logged in user)

Bug fixes

Fixed a bug that used always a default witboost user for scaffolding initial commit

Related issue

Closes #638

[#684] Insert private descriptor field in Marketplace database​

New features and improvements

  • Added private_descriptor field in the Marketplace DB for DataProduct, OutputPort, and Component tables

Related issue

Closes #684

[#685] Improve and add autotest for the marketplace section​

New features and improvements

  • Added test 03_05 to check whether there will be a popup error if the deployment fails
  • Added test 05_04 to check all tabs in Marketplace
  • Added test 05_05 to check the visualization of graphs and the relationship between the names of circles and the names of the corresponding lines on the right tab
  • Fixed deploy report step

Related issue

Closes #685

[#653] Add RBAC scaffolder action​

New features and improvements

  • Added RBAC Scaffolder Action
  • Added RBAC Role Picker to be used in templates
  • Improved RBAC client and splitted into frontend client and server-to-server client
  • Increased test coverage of wm-rbac-backend
  • On New Version of a Data Product all permissions for the older version will be inherited for the newer version

Breaking changes

  • entityRefField is now a required field for all configured Grant Mechanisms. If it is not given, witboost will complain about that. This is to prevent uniqueness constraint breaking on roles_subjects table.
  • Added new field visibility in roles table. It can assume two values: user, internal. it is a mandatory field.

Migration

  • Make sure all configured Grant Mechanisms in app-config include the entityRefField. If your previous Grant Mechanisms were not including an entity reference then you should change it. Grants created with Grant Mechanisms without entity ref are not supported anymore.
  • Field visibility in roles table will be automatically created by knex on server startup. By default this field will have value: internal

Related issue

Closes #653

[#644] Data Product's page incorrectly retrieves who has access to its components​

Bug fixes

Correctly displays the checkmark if the logged user has access to a component.

Related issue

Closes #644

[#664] Fix the last graph of the Mesh Supervision page​

Bug fixes

  1. Fix missing token in question tab API call
  2. Fix question query in AverageQuestion chart

Related issue

Closes #664

[#633] Component page review​

New features and improvements

  • Removed Link section from component Page
  • Hide Subcomponents table if there aren't subcomponent

Related issue

Closes #633

[#686] Wrong domain URN in roles_subjects​

Bug fixes

Change encoding of value 'Domain' in function encodeKindForURN.

Related issue

Closes #686

[#602] Fix Control Panel user interaction problems​

Bug fixes

Improved the deploy open panel operation

Related issue

Closes #602

[#585] Template policies are applied only to one component​

New features and improvements

Split test table list items in multiple components to improve readability and maintainability.

Bug fixes

Fix bug related to the same policy failure on multiple components.

Related issue

Closes #585

[#683] Test improvements and e2e tests from checklist​

New features and improvements

  • Structuring tests (suits)
  • Add new tests, covering Builder
  • Added test to check the environments with proper values. Change environments both in the dropdown menu and in the label of the table.
  • Added test to check the environments list and pre-existing policies. Both government and template policies in every environment.
  • Added test to check the marketplace section. Displaying and naming text elements: general info, dependencies, review, output port, and question.
  • Improved, fixed, and finish the end-to-end test for the control panel. Fixed step in the marketplace to check DP and all data.
  • Added test which covers the "schema", "linking" and "preview" tabs in the marketplace section.
  • Added a step in an existing test to check values after choosing infrastructure or template list in creating the template policy phase.
  • Changed the pipeline to allow runs on schedule
  • Added a step to deploy test results to the GitLab page (Allure report)

Related issue

Closes #683

[#660] Centralize RBAC docs​

New features and improvements

  • Docs on how to configure RBAC are all under Platform section
  • "How it works" docs of RBAC lies all inside Architecture section

Related issue

Closes #660

[#659] Append user: prefix to users in marketplace migration script​

Bug fixes

  • Added user: in front of identities in marketplace dump
  • Removed INSERT of useless rows in marketplace dump

Related issue

Closes #659

[#591] Migrate Notification table from marketplace to backstage DB​

New features and improvements

  • Moved Notification table from DB marketplace to DB backstage
  • Removed Hasura from notification CRUD layer; now it uses Knex. Hasura is also removed for ACL queries.
  • Created a standalone notification plugin. Routed all CRUD APIs to this plugin.

Breaking changes

The notification plugin uses backstage DB. To ensure continuity of data, the "Notification" table must be moved from the marketplace db into the backstage db.

Migration

There are two ways to migrate the notification table, copying all already stored data to preserve information.

  1. Via dblink A solution is use the Postgres extension dblink. This is the easiest solution to use, if the Postgres client supports it.
  2. Launch the platform, it will apply the necessary migrations.
  3. Open a new SQL script in the backstage DB, paste this snippet and run it:
create extension if not exists dblink;
insert into notifications.notification (select * from dblink('dbname=marketplace', 'select * from marketplace."Notification" n')
as t1(id int, kind text, notification_request jsonb, notification_response jsonb, recipient text, hook_id text, created_at timestamp, updated_at timestamp, sender text, read_at timestamp));
  1. Via shell Another solution is to manually connect to the Postgres instance shell and execute this script. This is for local containers, it will have to be declined with respect to the instance.

pg_dump -U postgres -t notification.\"Notification\" -a -h localhost marketplace | psql -U postgres -d backstage Parameters:

  • -U is a user who has both write and read access to the DBs marketplace and backstage. We use postgres in this example.
  • -h is the address of the Postgres instance. We use localhost in this case.

Related issue

Closes #591

[#657] EntitySelectionPicker disabled feature​

New features and improvements

  • EntitySelectionPicker can be disabled for free user input by using already existing allowArbitraryValues function in the template.yaml like we used for other pickers
  • EntityPicker now has an option to hide like the EntitySelectionPicker #656

Related issue

Closes #657

[#656] Add hidden field option in the EntitySelectionPicker​

New features and improvements

EntitySelectionPicker can now be hidden with including the this option in the template

ui:options:
    ui:widget: 'hidden'

Related issue

Closes #656

[#654] Clarify scopes format in RBAC docs​

New features and improvements

  • Added subjects section in RBAC docs

Bug fixes

  • Clarified scopes formats in RBAC docs

Related issue

Closes #654

[#650] Improve docs for RBAC​

New features and improvements

  • Permissions Scopes explained
  • Divided permissions into Catalog, Builder, Marketplace
  • Added permissions cookbooks
  • Improved grant mechanisms explanation

Related issue

Closes #650

[#652] Data Product components are not visible​

Bug fixes

  • Now users that have visibility over a data product can also view its components

Related issue

Closes #652

[#649] Add docs about creating new dp version​

New features and improvements

  • Added docs describing how to create a new version for a data product

Related issue

Closes #649

[#648] Add docs on how to inspect policy test errors​

New features and improvements

  • Added docs explaining policy errors inspection

Related issue

Closes #648

[#646] Observability tab hidden by default​

New features and improvements

  • Observability tabs are now hidden by default. This is governed by the optional parameter mesh.marketplace.ui.showObservability. Set it to true to enable them.

Related issue

Closes #646

[#647] Remove docs screenshots with Lorem Ipsum samples​

Bug Fixes

  • Changed some screenshots containing "Lorem Ipsum" sample data
  • Some minor fixes in docs

Related issue

Closes #647

[#487] Documentation review​

New features and improvements

  • Documentation restructured in: user, architecture, platform.

Breaking changes

If you have some docs on your branch, please check which is the correct section to apply your modifications since there is a completely new documentation structure.

Related issue

Closes #487

[#637] Create End 2 End report​

New features and improvements

  • Implemented Allure report to the automation tests.
  • Added step in e2e pipeline (Page). It will save and deploy allure report to the GitLab Pages.
  • Variable involved in pipeline schedule settings to run necessary only steps in e2e testing (Check, E2E test, E2E deploy)
  • Improved 02 and 03 tests: added step when a user create policy by uploading cue file.
  • Added test ## 06: a user modifies data, create/add environment in output port by editing repo file.

Related issue

Closes #637

[#643] PayloadTooLargeError error during data product validation​

Features

  • Added possibility to configure payload size limit for the Builder router through mesh.builder.payloadSizeLimit property

Bug fixes

  • Augmented payload size limit in Builder router so that it can accept payloads up to 10mb

Related issue

Closes #643

[#641] Fix wrong error on deployment failure​

Bug fixes

Improved the way witboost backend communicates with the Provisioning Coordinator. This solves an error that appeared in case a deployment failed for validation errors: since the error was not handled, the deploy operation failed with an error in another point of the code that couldn't find the right data. Now errors raised during the calls to the Coordinator should be handled correctly.

Related issue

Closes #641

[#642] Fix Downtime Chart example​

Bug fixes

Display the last 300 values in the Downtime charts

Related issue

Closes #642

[#625] Remove amcharts library​

New features and improvements

Replace amcharts with chartjs

Related issue

Closes #625

[#634] Improve external resources dashboard​

New features and improvements

Updated the dashboard page of the external resources.

Related issue

Closes #634

Provisioning Coordinator​

Changelog v0.6.1​

[#184] Get provisioning plan returns too many tasks​

Related issue

Closes #184

[#183] Bug on provisioning persistence - error serialization​

Related issue

Closes #183

Changelog v0.6.0​

[#181] Provisioning plan query returns no task information​

Related issue

Closes #181

[#179] Update the interface-spec doc​

New features and improvements

  • Updated specific_provisioner_api.md with the deployInfo feature.

Related issue

Closes #179

[#126] Compatibility problems on Windows 10/WSL 2 local environment​

Related issue

Closes #126

[#169] Implement multi-jvm tests​

New features and improvements

Defined a new set of suites to perform multi JVM tests.

Related issue

Closes #169

[#173] provisioning plan save always the same task​

Bug fixes

The update statements was incorrect, a task update apply the update to all record instead of only to the related task id

Related issue

Closes #173

[#172] Infrastructure layer logging​

New features and improvements

Lower logging for repository queries

Related issue

Closes #172

[#171] Template validation version check for useCaseTemplateid / infrastructureTemplateId​

Related issue

Closes #171

[#170] Migration fix​

Bug fixes

fix possible error in provisioning plan query

Related issue

Closes #170

[#170] Fix migration script​

Bug fixes

Fix the database migration script

Related issue

Closes #170

[#170] Migration failure in dev environment​

Bug fixes

Provides default values for newly added columns

Related issue

Closes #170

[#168] Bug during query access​

Bug fixes

Flyway now starts during application startup

Related issue

Closes #168

[#147] Refactor repository implementation​

New features and improvements

New repository implementation

Breaking changes

in some cases there will not be a single transaction for related inserts

Related issue

Closes #147

[#157] Deploy info needs to be sent to marketplace​

New features and improvements

  • Added code for handling deployInfo/buildInfo descriptor fields

Related issue

Closes #157

[#139] Specific provisioner OpenAPI specification is not valid​

Bug fixes

Fix on the OpenAPI specification to make it compliant with standards

Related issue

Closes #139

[#162] Improve docs structure​

New features and improvements

  • Docs structure refactoring

Related issue

Closes #162

[#161] Added deployment model​

New features and improvements

  • Added docs regarding deployment model

Related issue

Closes #161

[#160] The validation must return the validation policies even when they pass​

Bug fixes

  • Added policy validation results even when they passes

Related issue

Closes #160

[#152] unhandled error in validation service​

New features and improvements

resolve error on option.get

Related issue

Closes #152

[#150] Remove useless dependsOn nested field in the getProvisioningPlan response​

New features and improvements

  • Removed useless dependsOnTasks inside the Task object
  • Sorted components task on the basis of dependsOn

Related issue

Closes #150

[#159] Deploy with some validation error should return the correct error message​

Bug fixes

  • Modified DataProductValidationError message

Related issue

Closes #159

Marketplace Plugin​

Changelog v0.6.0​

[#45] Fix question removal​

Bug fixes

  • Removed delete_Question from delete API.

Related issue

Closes #45

[#43] Separate public fields from private in deployInfo/buildInfo​

New features and improvements

  • Introduced private and public descriptor management

Related issue

Closes #43

[#44] Fix for deploymentUpdate​

Related issue

Closes #44

[#41] Fix all the foreign key errors in insertOrUpdate and delete​

New features and improvements

Aligned the db used for integration tests with the one that is in dev

Bug fixes

Fixed all the foreign key constraint during upsert and delete

Closes #41

[#42] Custom changes for the marketplace​

Related issue

Closes #42