Installing Witboost

Witboost is installed in the customer environment and interacts with several external components (database, identity provider, versioning system, etc.). As each customer has a different infrastructure, Witboost tries to be as flexible as possible to accommodate such scenarios.

Before walking through the prerequisites, make sure you read the requirements.

Prerequisites

Database

Once the database has been provided, make sure to create these three databases, you will need them later:

• core
• marketplace
• provisioning_coordinator

They don’t need any special configuration.

If needed, the database names can be changed, but they must be changed accordingly in the configuration file (the values.yaml of the witboost helm chart). Refer to the following table to correctly reconfigure the databases.

Default database	Configuration reference
core	Globals.dbs.coreDB
marketplace	Globals.dbs.marketplaceDB
provisioning_coordinator	Globals.dbs.provisioningCoordinatorDB

Versioning System

Witboost requires a versioning system to store templates, components, and other entities and to manage their lifecycle.

Gitlab is the preferred choice, but Azure DevOps, Bitbucket ad GitHub are also supported.

Regardless of the one you pick, you will need to provide a token that has read permissions on the group/project where all Witboost repositories will be hosted.

note

It is important to know that Witboost will also create repositories in the same group/project. This is performed through the user personal token, which will have also write permissions. This decoupling guarantees that write operations are bound to the user, with the benefit of providing least privileges to the technical user, and allowing for a granular audit. More on this can be found here.

Directory Services

Witboost supports OpenLDAP compatible vendors, as well as Active Directory and FreeIPA. Witboost interacts with a directory service for two main purposes:

• Log in (Authentication)
• Fetch of users/groups (Authorization)

On Prem AD or OpenLDAP/FreeIPA

A bind user is necessary for authentication and to fetch users/groups on a specific search base. For how to configure an LDAP provider, you can refer to this documentation.

Entra ID (Former Azure AD)

An Azure app registration must be configured to enable SSO and to fetch users/groups. One or more groups can be provided to narrow the company tree.

Go to Azure Portal > App registrations and create a new App Registration.

On your app registration's overview page, add a new Web platform configuration, with the configuration:

• Redirect URI: https://your-witboost-ui.com/api/auth/microsoft/handler/frame
• Front-channel logout Url: blank
• Implicit grant and hybrid flows: All unchecked

On the API permissions tab, grant the following Microsoft Graph APIs.

warning

Your company may require you to grant admin consent for these permissions. Even if your company doesn't require admin consent, you may wish to do so as it means users don't need to individually consent the first time they access Witboost. To grant admin consent, a directory admin will need to come to this page and click on the Grant admin consent for COMPANY NAME button.

These APIs are used for SSO purposes, as well as to fetch the EntraID graph required from Witboost.

For how to configure the Azure AD provider you can refer to this documentation.

Platform

Witboost's main components are installed in Kubernetes. As the platform is provided by the user, there can be several variations of the configurations, such as how the customer handles networking, TLS certificates, DNSs, etc. In the following paragraphs we list some considerations.

Docker Registry

Witboost images are publicly accessible through a token that will be provided by AgileLab at installation time. If the network is capped and internet access forbidden, a docker registry must be provided, so that the necessary docker images will be imported.

Networking

Witboost exposes a web interface that must be reachable by the users. To accomplish this, a TLS certificate and a domain name must be available.

TLS certificate

Witboost leverages cert-manager to retrieve a valid certificate from Let’s Encrypt. It is anyway possible to configure Witboost with a customer provided certificate. When choosing let’s encrypt, it is important to decide the validation method, which can be particularly tricky in air-gapped scenarios.

If the customer already has certificates to use, they can be setup directly at ingress controller level, enforcing TLS termination on the ingress.

note

Cert-manager can be configured in the cert-manager block of the values.yaml.

Domain

Witboost leverages external-dns, which allows to control DNS records dynamically via Kubernetes resources in a DNS provider-agnostic way. This is usually used in on-cloud scenarios, where the records are updated in the cloud-specific DNS, accordingly delegated by the customer primary domain.

For example, in an AWS environment, the customer can provide a route53 zone (.witboost.customer.com), delegated from the customer main DNS (.customer.com). Witboost will leverage external-dns to synchronize the records in the route53 zone.

You can refer to DNS delegation for an overview on this topic.

note

External-dns can be configured in the external-dns block of the values.yaml.

note

The Kubernetes cluster must have the permissions to interact (read/write) with the DNS zone.

Secret management

Witboost needs to use secrets (i.e. DB user, LDAP user, GitLab token, etc.).

They must be configured in a secret manager compatible with external-secrets. This is the list of compatible backends. The choice is usually driven by the cloud the cluster lives in, or by the customer company guidelines/policies.

If no backend is available, or the customer prefers to opt for a quicker approach, there are few options:

• Secrets are "manually" created, it might be the case when the customer uses custom pipelines to deploy helm charts into Kubernetes.
• The fake secret store is used: this is usually not recommended, as the secrets are stored in a yaml file that is pushed along with the chart, but it could be judged viable in playgrounds/poc environments

Regardless of the approach used, Witboost expects a kubernetes secret named "witboost-secrets" containing, among others, the following:

• Database credentials
• Azure AD/Entra ID credentials This secret is automatically created from external-secrets, and it will contain all the secrets that have been inserted in the configured secret store.

tip

As a reference of how to fill the secret manager (or directly the secret) an example fake secret store is provided in which a list of needed secrets for a basic installation can be found. This reference will be of help for filling the secret store, but also for the scenario where secrets are manually created.

Manually created secrets

If external-secrets is not used, the secret called “witboost-secrets” must be created with all the items in it.

Installation

Witboost is installed via its Helm chart, provided by Agile Lab.

Third-party installations

Before proceeding with the installation of the witboost helm chart, make sure the following third-party deployments are present if needed.

External secrets

Unless you have opted for the manual creation of Kubernetes secrets, make sure external-secrets is installed. You can double-check it with the following command, which should return external-secrets CRDs:

$ kubectl get crds | grep externalsecrets

If it is not installed, you must

• Install it following the official documentation
• Configure a ClusterSecretStore based on one of the available providers
• Reference the ClusterSecretStore in the values.yaml (globals.externalSecrets.secretStore)

Run the following command to make sure it has been created and is in Ready state.

$ kubectl get clustersecretstore

You should find your secret store, in Ready state. The output should look like the following

$ kubectl get clustersecretstore
NAME                     AGE    STATUS   CAPABILITIES   READY
mysecretstore            296d   Valid    ReadWrite      True

Cert manager

If you are not using custom certificates but you plan to use the default TLS management configuration, make sure cert-manager is installed. You can double-check it with the following command, which should return cert-manager CRDs:

$ kubectl get crds | grep cert-manager

If it is not installed, you must

• install it following the official documentation
• configure a clusterIssuer, for example, the ACME one
• reference the issuer in the values.yaml (globals.certManager.clusterIssuer)

Run the following command to make sure it has been created and is in Ready state.

$ kubectl get clusterIssuer

You should find your secret store, in Ready state. The output should look like the following

$ kubectl get clusterIssuer
NAME                     READY   AGE
letsencrypt-production   True    2y236d

External DNS

Make sure external secrets is installed. You can double-check it with the following command, which should return external-dns pod(s):

$ kubectl get pods --all-namespaces | grep external-dns

If it is not installed, you can either:

• Install it following the official documentation
• Enable it via the external-dns.enabled toggle of the values.yaml

Witboost Helm chart installation

You are now ready to proceed with the Witboost installation.

Let's create a dedicated namespace, we will install Witboost in it:

$ kubectl create ns witboost

Create the regcred to access the Agile Lab docker registry (or whatever registry you host the images in):

$ kubectl create secret docker-registry regcred --docker-server=registry.gitlab.com --docker-username=witboost --docker-password="${REGISTRY_TOKEN}" -n witboost

In order to proceed to the witboost helm chart installation, a configuration file (i.e. values.yaml) must be prepared to properly configure Witboost depending on the aforementioned prerequisites.

tip

A reference values.yaml is provided and must be customized according to the installation environment.

Using the base file provided, look for the “CHANGEME” placeholders and configure them based on your setup.

For the configuration of the main components, you can refer to their documentation:

• UI configuration
• Provisioning Coordinator configuration
• GCP configuration

With the values.yaml prepared, you can proceed in installing the Witboost helm chart.

$ helm repo add --username <username> --password <access_token> witboost https://gitlab.com/api/v4/projects/40979925/packages/helm/stable
$ helm upgrade --install --atomic <release_name> witboost/witboost -n <namespace> -f <my-values.yaml> --version X.Y.Z

After Helm finishes, you should:

• Check that all pods are up and running
• Check Ingresses and wait for DNS & TLS certificates to apply; if using external-dns and cert-manager it can take a couple of minutes
• Go to the web UIs and ensure everything is reachable. The endpoints depend on how you have configured the values.yaml, you can check the ingresses
  • Ensure you can access Witboost UI
  • Ensure you can access Hasura UI
  • Ensure you can access the Documentation UI

Post-installation steps

Add databases in Hasura

Open the Hasura Web UI and log in with the admin token you set in the secrets. First, reload all metadata by going to Settings (gear icon, top right), Metadata, then check the boxes and reload:

Now, go to Data. In the list of databases make sure that marketplace (in the default schema) is present. We now need to add the provisioning_coordinator too. You can go into manage > Connect Database and fill in the info:

• Database Display Name: provisioning_coordinator
• Data Source Driver: PostgreSQL
• Connect Database Via: Database URL
• Database URL: insert here the PostgreSQL URL of the provisioning_coordinator database

You can now press the Connect Database button, and the new database should appear in the database list.

Track tables and relationships in Hasura

Now, track the tables and relations to expose them in the GraphQL API. First, the schema for the Marketplace. Go to Data, select the schema, then use the Track All button:

Next, after the tables are tracked, form the same page track the relations using the Track All button that appears:

Repeat these steps for the Provisioning Coordinator and CGP schemas. If one of these databases is not listed, you can follow the previous chapter to add it.

Configure RBAC

If RBAC is enabled (it is by default), you need to follow the steps in the doc to configure the first roles.

RBAC Configuration

Register EULA

You can add an EULA that the user needs to accept on the first login.

EULA Configuration

Login to Witboost UI

Go to the UI URL and log in with your user; verify that everything works fine by going to the various pages. If you receive a “user not found” error, verify that the users and groups are being pulled from your authentication system via MSGraph or LDAP correctly by looking at the logs. Look for the following:

Reading msgraph users and groups type=plugin target=https://graph.microsoft.com/v1.0 class=MicrosoftGraphOrgEntityProvider taskId=MicrosoftGraphOrgEntityProvider:https://graph.microsoft.com/v1.0:refresh

Read users from group membership type=plugin target=https://graph.microsoft.com/v1.0 class=MicrosoftGraphOrgEntityProvider taskId=MicrosoftGraphOrgEntityProvider:https://graph.microsoft.com/v1.0:refresh groupCount=1 userCount=22

Read 22 msgraph users and 2 msgraph groups in 1.2 seconds. Committing... type=plugin target=https://graph.microsoft.com/v1.0 class=MicrosoftGraphOrgEntityProvider taskId=MicrosoftGraphOrgEntityProvider:https://graph.microsoft.com/v1.0:refresh 

Committed 22 msgraph users and 2 msgraph groups in 0.0 seconds. type=plugin target=https://graph.microsoft.com/v1.0 class=MicrosoftGraphOrgEntityProvider taskId=MicrosoftGraphOrgEntityProvider:https://graph.microsoft.com/v1.0:refresh 

In case no users/groups are being added, check what the errors are. If you get permission errors, double-check that the required permissions were granted to the app registration/LDAP bind user. If no permission errors are being reported, double-check the user and group filters in the configuration.

If you get SSO errors, double-check that the redirect URI in the app registration is correct and that all permissions are granted; also check the credentials in the secret.

Configure the Practice Shaper

You can now configure the practice shaper, refer to the following link to understand what it is and how to configure it. As a starting point, you can refer to the data landscape preset.

Practice Shaper Configuration

Register Access Request Template

Follow the steps in the documentation to create and register your Access/Revoke Request Templates.

Access Request Template

Register Domains

Define and register your domains.

Domain registration

Register Templates

Register your templates; as a starting point you can use the ones included in the Starter Kit.

Template registration

Test a simple project

You can now create, release and deploy a simple project to verify everything works.