Installation
This page will help you install Port's GitLab integration (powered by the Ocean framework).
This page outlines the following steps:
- How to create a GitLab group access token to give the integration permissions to query your GitLab account.
- How to configure and customize the integration before deploying it.
- How to deploy the integration in the configuration that fits your use case.
Prerequisitesโ
- A GitLab account with admin privileges.
- A GitLab group account with the
api
scope. - If you choose the real-time & always-on installation method, a Kubernetes cluster to install the integration on.
- Your Port user role is set to
Admin
.
Setupโ
Create a GitLab group access tokenโ
A group access token can be used for the group it was generated at, as well as for all sub-groups underneath it.
The GitLab integration is able to query multiple GitLab root groups. To do so, it will require multiple group access tokens, each at the correct root group.
GitLab group access tokens example
For example, let's assume the following GitLab account structure:
GitLab account
.
โโโ microservices-group
โย ย โโโmicroservice1-group
โย ย โโโmicroservice2-group
โโโ apis-group
โย ย โโโ rest-api-group
โย ย โโโ graphql-api-group
In this example:
- To map only the
microservices-group
, we require one group access token - one for themicroservices-group
. - To map the
microservices-group
and all of its subgroups, we require only one group access token - one for themicroservices-group
. - To map the
microservices-group
, theapis-group
and all of their subgroups, we require only two group access tokens - one for themicroservices-group
and one for theapis-group
. - To map the
microservice1-group
, we have 2 options:- Create a group access token for the
microservices-group
and use the token mapping to select just themicroservice1-group
. - Create a group access token for the
microservice1-group
directly.
- Create a group access token for the
See the Token Mapping section for more information.
The following steps will guide you how to create a GitLab group access token.
-
Sign in to GitLab and go to your desired group's settings page:
-
In the "Access Tokens" section, you need to provide the token details, including the name and an optional expiration date. Additionally, select the api scope, and then proceed to click on the "Create access token" button.
-
Click "Create group access token".
-
Copy the generated token and use it when deploying the integration in the following steps.
Token Mappingโ
The GitLab integration supports fetching data related to specific paths in your GitLab groups. The integration is also able to fetch data from different GitLab parent groups by providing additional group tokens. In order to do so, you need to map the desired paths to the relevant access tokens.
The tokenMapping
parameter supports specifying the paths that the integration will search for files and information in, using globPatterns.
Mapping format:
{"MY_FIRST_GITLAB_PROJECT_GROUP_TOKEN": ["**/MyFirstGitLabProject/**","**/MySecondGitLabProject/*"]}
Example:
{"glpat-QXbeg-Ev9xtu5_5FsaAQ": ["**/DevopsTeam/*Service", "**/RnDTeam/*Service"]}
When using the tokenMapping
parameter in the integration's Helm installation, make sure to escape the necessary characters, for example:
--set integration.secrets.tokenMapping="\{\"glpat-oh1YXc54pR4eofx6hYy5\": [\"**\"]\}"
Multiple GitLab group access tokens example:
{"glpat-QXbeg-Ev9xtu5_5FsaAQ": ["**/DevopsTeam/*Service", "**/RnDTeam/*Service"],"glpat-xF7Ae-vXu5ts5_QbEgAQ9": ["**/MarketingTeam/*Service"]}
Configure Realtime webhook eventsโ
The appHost
parameter is used specifically to enable the real-time functionality of the integration.
If it is not provided, the integration will continue to function correctly. In such a configuration, to retrieve the latest information from the target system, the scheduledResyncInterval
parameter has to be set, or a manual resync will need to be triggered through Port's UI.
In order for the GitLab integration to update the data in Port on every change in the GitLab repository, you need to specify the appHost
parameter.
The appHost
parameter should be set to the url
of your GitLab integration instance. In addition, your GitLab instance (whether it is GitLab SaaS or a self-hosted version of GitLab) needs to have the option to send webhook requests to the GitLab integration instance, so please configure your network accordingly.
The default webhook events behaviorโ
The GitLab integration supports listening to GitLab webhooks and updating the relevant entities in Port accordingly.
Supported webhooks are Group webhooks and System hooks.
As part of the installation process, the integration will create a webhook in your GitLab instance, and will use it to listen to the relevant events.
There are a few points to consider before deciding on which webhook to choose :
-
If you choose system hooks, the integration will create a single webhook for the entire GitLab instance. If you choose group webhooks, the integration will create a webhook for each root group in your GitLab instance, unless you provide the
tokenGroupHooksOverrideMapping
parameter- and then it will create a webhook for each specified group in this parameter. -
The system hooks has much less event types than the group webhooks.
-
Group Webhooks supported event types:
push_events
issues_events
jobs_events
merge_requests_events
pipeline_events
releases_events
tag_push_events
subgroup_events
confidential_issues_events
user_remove_from_group
user_update_for_group
user_add_to_group
-
System Hooks supported event types:
push_events
merge_requests_events
repository_update_events
This means that if you choose system hooks, the integration will not be able to update the relevant entities in Port on events such as
issues_events
,pipeline_events
and etc.
-
-
Creating a system hook requires admin privileges in GitLab. Due to this, the integration supports that the system hook will be created manually, and the integration will use it to listen to the relevant events.
Specific Group Webhooksโ
By default, if appHost
is provided, the integration will create group webhooks for each root group in your GitLab instance. If you need to create webhooks only for specific groups, you should configure the tokenGroupHooksOverrideMapping
parameter.
System Webhooksโ
To create a system hook in GitLab, you must set useSystemHook: true
and choose one of the following setup methods:
- Automatic Setup - Provide a token with admin privileges using the
tokenMapping
parameter. - Manual Setup - Create the system hook yourself in GitLab:
- Follow GitLab's system hook setup guide.
- Set the URL to
{appHost}/integration/system/hook
(e.g.,https://my-gitlab-integration.com/integration/system/hook
). - Enable the supported triggers:
push
andmerge_request
.
Due to GitLab webhook limitations, new repositories require special handling:
- A repository will only appear in Port after its first commit (GitLab webhooks don't support
project_create
events). - Empty repositories (no commits) will only appear after the next scheduled resync.
For more details, see GitLab's webhook events documentation.
Listen to specified groupsโ
the integration can support listening to webhooks on specified groups, by configuring the tokenGroupHooksOverrideMapping
parameter. this parameter is not required, and when you don't use it, the integration will listen to all of the root groups (if not using useSystemHooks=true
)
Mapping format:
{"MY_FIRST_GROUPS_TOKEN": {"groups:"{"MY_FIRST_GROUP_FULL_PATH": {"events": [CHOSEN_EVENT_TYPES]}, "MY_OTHER_GROUP_FULL_PATH": {"events": [CHOSEN_EVENT_TYPES]}}}}
Example:
{"glpat-QXbeg-Ev9xtu5_5FsaAQ": {"groups": {"path/to/my-first-group": {"events": ["push_events", "merge_requests_events]}, "path/to/my-other-group": {"events": ["pipelines_events"]}}}}
You can configure multiple tokens, and multiple groups per token (the token should have admin access to those groups), but there are some rules:
- All of the tokens mentioned here must be contained in
tokenMapping
. - A "groups" key is required for each token.
- All of the groups in all of the tokens must be non-hierarchical to each other, and not identical (duplicated).
- The group path is the full path in gitlab. If a group path is incorrect, the webhook will not be created.
- The events for each group must match the supported event types mentioned below. if you would like to have all the events provided in the webhook, you can use:
{"events" = []}
, but not eliminate this key completely, because it is required.
Deploy the GitLab integrationโ
Choose one of the following installation methods:
- Hosted by Port
- Real-time (self-hosted)
- Scheduled (CI)
Using this installation option means that the integration will be hosted by Port, with a customizable resync interval to ingest data into Port.
The Hosted by Port
option is currently in beta, and is still undergoing final testing before its official release.
Should you encounter any bugs or functionality issues, please let us know so we can rectify them as soon as possible.
Your help is greatly appreciated! โญ
Live event support
Currently, live events are not supported for integrations hosted by Port.
Resyncs will be performed periodically every 1 hour by default (can be configured differently after installation), or manually triggered by you via Port's UI.
Therefore, real-time events (including GitOps) will not be ingested into Port immediately.
Support for live events is WIP and will be supported in the near future.
Installation
To install, follow the following steps:
-
Go to the Data sources page of your portal.
-
Click on the
+ Data source
button in the top-right corner. -
Click on the relevant integration in the list.
-
Under
Select your installation method
, chooseHosted by Port
. -
Configure the
integration settings
andapplication settings
as you wish (see below for details).
Application settings
Every integration hosted by Port has the following customizable application settings, which are configurable after installation:
-
Resync interval
: The frequency at which Port will ingest data from the integration. There are various options available, ranging from every 1 hour to once a day. -
Send raw data examples
: A boolean toggle (enabled
by default). If enabled, raw data examples will be sent from the integration to Port. These examples are used when testing your mapping configuration, they allow you to run yourjq
expressions against real data and see the results.
Integration settings
Every integration has its own tool-specific settings, under the Integration settings
section.
Each of these settings has an โ icon next to it, which you can hover over to see a description of the setting.
Port secrets
Some integration settings require sensitive pieces of data, such as tokens.
For these settings, Port secrets will be used, ensuring that your sensitive data is encrypted and secure.
When changing such a setting, you will be prompted to choose an existing secret or create a new one:
Port source IP addresses
When using this installation method, Port will make outbound calls to your 3rd-party applications from static IP addresses.
You may need to add these addresses to your allowlist, in order to allow Port to interact with the integrated service:
- Europe (EU)
- United States (US)
54.73.167.226
63.33.143.237
54.76.185.219
3.234.37.33
54.225.172.136
3.225.234.99
Using this installation option means that the integration will be able to update Port in real time using webhooks.
Prerequisites
To install the integration, you need a Kubernetes cluster that the integration's container chart will be deployed to.
Please make sure that you have kubectl
and helm
installed on your machine, and that your kubectl
CLI is connected to the Kubernetes cluster where you plan to install the integration.
If you are having trouble installing this integration, please refer to these troubleshooting steps.
For details about the available parameters for the installation, see the table below.
- Helm
- ArgoCD
To install the integration using Helm:
-
Go to the Gitlab data source page in your portal.
-
Select the
Real-time and always on
method: -
A
helm
command will be displayed, with default values already filled out (e.g. your Port cliend ID, client secret, etc).
Copy the command, replace the placeholders with your values, then run it in your terminal to install the integration.
The baseUrl
, port_region
, port.baseUrl
, portBaseUrl
, port_base_url
and OCEAN__PORT__BASE_URL
parameters are used to select which instance or Port API will be used.
Port exposes two API instances, one for the EU region of Port, and one for the US region of Port.
- If you use the EU region of Port (https://app.getport.io), your API URL is
https://api.getport.io
. - If you use the US region of Port (https://app.us.getport.io), your API URL is
https://api.us.getport.io
.
To install the integration using ArgoCD:
- Create a
values.yaml
file inargocd/my-ocean-gitlab-integration
in your git repository with the content:
Remember to replace the placeholders for GITLAB_TOKEN_MAPPING
.
initializePortResources: true
scheduledResyncInterval: 120
integration:
identifier: my-ocean-gitlab-integration
type: gitlab
eventListener:
type: POLLING
secrets:
tokenMapping: GITLAB_TOKEN_MAPPING
- Install the
my-ocean-gitlab-integration
ArgoCD Application by creating the followingmy-ocean-gitlab-integration.yaml
manifest:
Remember to replace the placeholders for YOUR_PORT_CLIENT_ID
YOUR_PORT_CLIENT_SECRET
and YOUR_GIT_REPO_URL
.
Multiple sources ArgoCD documentation can be found here.
ArgoCD Application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-ocean-gitlab-integration
namespace: argocd
spec:
destination:
namespace: my-ocean-gitlab-integration
server: https://kubernetes.default.svc
project: default
sources:
- repoURL: 'https://port-labs.github.io/helm-charts/'
chart: port-ocean
targetRevision: 0.1.14
helm:
valueFiles:
- $values/argocd/my-ocean-gitlab-integration/values.yaml
parameters:
- name: port.clientId
value: YOUR_PORT_CLIENT_ID
- name: port.clientSecret
value: YOUR_PORT_CLIENT_SECRET
- name: port.baseUrl
value: https://api.getport.io
- repoURL: YOUR_GIT_REPO_URL
targetRevision: main
ref: values
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
The baseUrl
, port_region
, port.baseUrl
, portBaseUrl
, port_base_url
and OCEAN__PORT__BASE_URL
parameters are used to select which instance or Port API will be used.
Port exposes two API instances, one for the EU region of Port, and one for the US region of Port.
- If you use the EU region of Port (https://app.getport.io), your API URL is
https://api.getport.io
. - If you use the US region of Port (https://app.us.getport.io), your API URL is
https://api.us.getport.io
.
- Apply your application manifest with
kubectl
:
kubectl apply -f my-ocean-gitlab-integration.yaml
This table summarizes the available parameters for the installation.
Note the parameters specific to this integration, they are last in the table.
Parameter | Description | Example | Required |
---|---|---|---|
port.clientId | Your Port client id | โ | |
port.clientSecret | Your Port client secret | โ | |
port.baseUrl | Your Port API URL - https://api.getport.io for EU, https://api.us.getport.io for US | โ | |
integration.secrets.tokenMapping | The token mapping configuration used to query GitLab | โ | |
integration.config.appHost | The host of the Port Ocean app. Used to set up the integration endpoint as the target for webhooks created in GitLab | https://my-ocean-integration.com | โ |
integration.config.gitlabHost | (for self-hosted GitLab) the URL of your GitLab instance | https://my-gitlab.com | โ |
integration.secrets.tokenGroupHooksOverrideMapping | The token group hooks override mapping configuration used to create custom webhooks on groups | โ | |
integration.eventListener.type | The event listener type. Read more about event listeners | โ | |
integration.type | The integration to be installed | โ | |
scheduledResyncInterval | The number of minutes between each resync. When not set the integration will resync for each event listener resync event. Read more about scheduledResyncInterval | โ | |
initializePortResources | Default true, When set to true the integration will create default blueprints and the port App config Mapping. Read more about initializePortResources | โ | |
sendRawDataExamples | Enable sending raw data examples from the third party API to port for testing and managing the integration mapping. Default is true | โ |
For advanced configuration such as proxies or self-signed certificates, click here.
This pipeline will run the GitLab integration once and then exit, this is useful for scheduled ingestion of data.
If you want the integration to update Port in real time using webhooks you should use the Real-time (self-hosted) installation option.
- GitLab
- Jenkins
Make sure to configure the following GitLab Variables:
Parameter | Description | Example | Required |
---|---|---|---|
OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING | The token mapping configuration used to query GitLab | โ | |
OCEAN__PORT__CLIENT_ID | Your Port client (How to get the credentials) id | โ | |
OCEAN__PORT__CLIENT_SECRET | Your Port client (How to get the credentials) secret | โ | |
OCEAN__PORT__BASE_URL | Your Port API URL - https://api.getport.io for EU, https://api.us.getport.io for US | โ | |
OCEAN__INITIALIZE_PORT_RESOURCES | Default true, When set to true the integration will create default blueprints and the port App config Mapping. Read more about initializePortResources | โ | |
OCEAN__INTEGRATION__CONFIG__GITLAB_HOST | (for self-hosted GitLab) the URL of your GitLab instance | โ | |
OCEAN__INTEGRATION__IDENTIFIER | The identifier of the integration that will be installed | โ |
Here is an example for .gitlab-ci.yml
workflow file:
stages:
- deploy_gitlab
variables:
# Define non-secret variables
INTEGRATION_TYPE: "gitlab"
VERSION: "latest"
# These variables should be set in GitLab's CI/CD variables for security
# OCEAN__PORT__CLIENT_ID: $OCEAN__PORT__CLIENT_ID
# OCEAN__PORT__CLIENT_SECRET: $OCEAN__PORT__CLIENT_SECRET
# OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING: $OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING
deploy_gitlab:
image: docker:24.0.7
stage: deploy_gitlab
services:
- docker:24.0.7-dind
script:
- image_name="ghcr.io/port-labs/port-ocean-$INTEGRATION_TYPE:$VERSION"
- |
docker run -i --rm --platform=linux/amd64 \
-e OCEAN__EVENT_LISTENER='{"type":"ONCE"}' \
-e OCEAN__INITIALIZE_PORT_RESOURCES=true \
-e OCEAN__SEND_RAW_DATA_EXAMPLES=true \
-e OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING="$OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING" \
-e OCEAN__PORT__CLIENT_ID=$OCEAN__PORT__CLIENT_ID \
-e OCEAN__PORT__CLIENT_SECRET=$OCEAN__PORT__CLIENT_SECRET \
-e OCEAN__PORT__BASE_URL='https://api.getport.io' \
$image_name
only:
- main
The baseUrl
, port_region
, port.baseUrl
, portBaseUrl
, port_base_url
and OCEAN__PORT__BASE_URL
parameters are used to select which instance or Port API will be used.
Port exposes two API instances, one for the EU region of Port, and one for the US region of Port.
- If you use the EU region of Port (https://app.getport.io), your API URL is
https://api.getport.io
. - If you use the US region of Port (https://app.us.getport.io), your API URL is
https://api.us.getport.io
.
When saving the OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING
variable, be sure to save it as-is, for example given the following token mapping:
{"glpat-QXbeg-Ev9xtu5_5FsaAQ": ["**/DevopsTeam/*Service", "**/RnDTeam/*Service"]}
(Note that this is a one-liner)
Save it as a GitLab variable without any changes (there is no need to wrap it in single-quotes ('
) or double-quotes ("
).
Also make sure to keep the double-quotes ("
) when passing the OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING
parameter to the Docker CLI (see the pipeline example above).
Your Jenkins agent should be able to run docker commands.
Make sure to configure the following Jenkins Credentials of Secret Text
type:
Parameter | Description | Example | Required |
---|---|---|---|
OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING | The token mapping configuration used to query GitLab | โ | |
OCEAN__PORT__CLIENT_ID | Your Port client (How to get the credentials) id | โ | |
OCEAN__PORT__CLIENT_SECRET | Your Port client (How to get the credentials) secret | โ | |
OCEAN__PORT__BASE_URL | Your Port API URL - https://api.getport.io for EU, https://api.us.getport.io for US | โ | |
OCEAN__INITIALIZE_PORT_RESOURCES | Default true, When set to true the integration will create default blueprints and the port App config Mapping. Read more about initializePortResources | โ | |
OCEAN__INTEGRATION__CONFIG__GITLAB_HOST | (for self-hosted GitLab) the URL of your GitLab instance | โ | |
OCEAN__INTEGRATION__IDENTIFIER | The identifier of the integration that will be installed | โ |
Here is an example for Jenkinsfile
groovy pipeline file:
pipeline {
agent any
stages {
stage('Run GitLab Integration') {
steps {
script {
withCredentials([
string(credentialsId: 'OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING', variable: 'OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING'),
string(credentialsId: 'OCEAN__PORT__CLIENT_ID', variable: 'OCEAN__PORT__CLIENT_ID'),
string(credentialsId: 'OCEAN__PORT__CLIENT_SECRET', variable: 'OCEAN__PORT__CLIENT_SECRET'),
]) {
sh('''
#Set Docker image and run the container
integration_type="gitlab"
version="latest"
image_name="ghcr.io/port-labs/port-ocean-${integration_type}:${version}"
docker run -i --rm --platform=linux/amd64 \
-e OCEAN__EVENT_LISTENER='{"type":"ONCE"}' \
-e OCEAN__INITIALIZE_PORT_RESOURCES=true \
-e OCEAN__SEND_RAW_DATA_EXAMPLES=true \
-e OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING="$OCEAN__INTEGRATION__CONFIG__TOKEN_MAPPING" \
-e OCEAN__PORT__CLIENT_ID=$OCEAN__PORT__CLIENT_ID \
-e OCEAN__PORT__CLIENT_SECRET=$OCEAN__PORT__CLIENT_SECRET \
-e OCEAN__PORT__BASE_URL='https://api.getport.io' \
$image_name
exit $?
''')
}
}
}
}
}
}
The baseUrl
, port_region
, port.baseUrl
, portBaseUrl
, port_base_url
and OCEAN__PORT__BASE_URL
parameters are used to select which instance or Port API will be used.
Port exposes two API instances, one for the EU region of Port, and one for the US region of Port.
- If you use the EU region of Port (https://app.getport.io), your API URL is
https://api.getport.io
. - If you use the US region of Port (https://app.us.getport.io), your API URL is
https://api.us.getport.io
.
For advanced configuration such as proxies or self-signed certificates, click here.