Yodamad's blog
Backstage on Docker with GitLab authentificaiton

Backstage on Docker with GitLab authentificaiton

Matthieu Vincent's photo
Matthieu Vincent
·

8 min read

Few days ago, I decided to finally get some time to discover #Backstage and understand the main concepts and how to use it.

In this article, I’ll explain the steps (and ambushes 😉) I followed to install it locally running on #Docker and with authentication based on #GitLab

Follow the official doc

So to begin, I’ve tried to follow the official documentation: https://backstage.io/docs/getting-started/

First, we need to create our app

$ npx @backstage/create-app@latest
? Enter a name for the app [required] hashnode-article

Creating the app...

 Checking if the directory is available:
  checking      hashnode-article ✔

 Creating a temporary app directory:

 Preparing files:
  copying       .dockerignore ✔
  templating    .gitignore.hbs ✔
  templating    .eslintrc.js.hbs ✔
  copying       README.md ✔
...
  executing     yarn install ✔
  executing     yarn tsc ✔

🥇  Successfully created hashnode-article


 All set! Now you might want to:
  Run the app: cd hashnode-article && yarn dev
  Set up the software catalog: https://backstage.io/docs/features/software-catalog/configuration
  Add authentication: https://backstage.io/docs/auth/

Good, we are ready to go ! We can test that locally it works

$ yarn dev
yarn run v1.22.22
$ concurrently "yarn start" "yarn start-backend"
$ yarn workspace app start
$ yarn workspace backend start
$ backstage-cli package start
$ backstage-cli package start
[0] Loaded config from app-config.yaml
[0] <i> [webpack-dev-server] Project is running at:
[0] <i> [webpack-dev-server] Loopback: http://localhost:3000/, http://[::1]:3000/
[0] <i> [webpack-dev-server] Content not from webpack is served from '/private/tmp/hashnode-article/packages/app/public' directory
[0] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[0] <i> [webpack-dev-middleware] wait until bundle finished: /
...
[0] <i> [webpack-dev-middleware] wait until bundle finished: /
[0] webpack compiled successfully

It works !

Ok, but this is not very convenient to share this with your team. Let’s ship it with Docker, it would be easier to deploy.

Package with Docker

First surprise : there is no official Docker image available (you’ll understand why later in this article 😉). So let’s follow follow the doc again to build our own image : https://backstage.io/docs/deployment/docker

First, we need to build the application

yarn build:backend --config ../../app-config.yaml

Now, we can build our image

docker image build . -f packages/backend/Dockerfile --tag backstage:demo

Let’s start it

$ docker run -it -p 7007:7007 backstage:demo
Loading config from MergedConfigSource{FileConfigSource{path="/app/app-config.yaml"}, FileConfigSource{path="/app/app-config.production.yaml"}, EnvConfigSource{count=0}}
{"level":"info","message":"Found 0 new secrets in config that will be redacted","service":"backstage"}
{"level":"info","message":"Listening on :7007","service":"rootHttpRouter"}
{"level":"info","message":"Plugin initialization started: 'app', 'proxy', 'scaffolder', 'techdocs', 'auth', 'catalog', 'permission', 'search', 'kubernetes'","service":"backstage","type":"initialization"}
/app/node_modules/@backstage/backend-defaults/dist/database.cjs.js:457
        throw new Error(
              ^

Error: Failed to connect to the database to make sure that 'backstage_plugin_app' exists, Error: connect ECONNREFUSED 127.0.0.1:5432
    at PgConnector.getClient (/app/node_modules/@backstage/backend-defaults/dist/database.cjs.js:457:15)
    at runNextTicks (node:internal/process/task_queues:60:5)
    at listOnTimeout (node:internal/timers:538:9)
    at process.processTimers (node:internal/timers:512:7)

Node.js v18.20.4

Oops, even with the default config, we need a postgres database. No big deal, we can start one with Docker

$ docker network create backstage
$ docker run -d --name postgres -p 5432:5432 -e POSTGRES_HOST_AUTH_METHOD=trust --restart always --network backstage postgres

Note that we link it to a backstage network in Docker

Let’s retry to start backstage

$ docker run --rm -it -p 7007:7007 -e POSTGRES_HOST=postgres -e POSTGRES_USER=postgres --network backstage backstage:demo
...
ForwardedError: Plugin 'auth' startup failed; caused by Error: Failed to initialize guest auth provider, The guest provider cannot be used outside of a development environment
    at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:677:19
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:11
    ... 3 lines matching cause stack trace ...
    at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5) {
  cause: Error: Failed to initialize guest auth provider, The guest provider cannot be used outside of a development environment
      at bindProviderRouters (/app/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:1544:17)
      at createRouter (/app/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:2267:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Object.init [as func] (/app/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:2335:24)
      at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:11
      at async Promise.all (index 4)
      at async #doStart (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:630:5)
      at async BackendInitializer.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:556:5)
      at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5)
}

Hum, it seems that we have to setup a authentication provider, guest mode is not authorized by default. As I have the chance to be a GitLab Hero, let’s try to connect with GitLab as there is a provider for it.

Authentication with GitLab

So, let’s follow the official documentation : https://backstage.io/docs/integrations/gitlab/locations

We have to update the app-config.yaml file to add GitLab integration

...
integrations:
  gitlab:
    - host: gitlab.com
      token: ${GITLAB_TOKEN}

Let’s rebuild our image to include this new configuration and try to re-run it (adding the GitLab token as environment variable)

$ docker image build . -f packages/backend/Dockerfile --tag backstage:demo
...
$ docker run --rm -it -p 7007:7007 -e POSTGRES_HOST=postgres -e POSTGRES_USER=postgres -e GITLAB_TOKEN=<your_token> --network backstage backstage:demo
...
MigrationLocked: Plugin 'catalog' startup failed; caused by MigrationLocked: Migration table is already locked
    at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:677:19
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:11
    ... 3 lines matching cause stack trace ...
    at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5) {
  cause: MigrationLocked: Migration table is already locked
      at /app/node_modules/knex/lib/migrations/migrate/Migrator.js:343:13
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Migrator._runBatch (/app/node_modules/knex/lib/migrations/migrate/Migrator.js:359:7)
      at async applyDatabaseMigrations (/app/node_modules/@backstage/plugin-catalog-backend/dist/cjs/CatalogBuilder-C7ANIkk3.cjs.js:1481:3)
      at async CatalogBuilder.build (/app/node_modules/@backstage/plugin-catalog-backend/dist/cjs/CatalogBuilder-C7ANIkk3.cjs.js:6982:7)
      at async Object.init [as func] (/app/node_modules/@backstage/plugin-catalog-backend/dist/alpha.cjs.js:242:46)
      at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:11
      at async Promise.all (index 5)
      at async #doStart (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:630:5)
      at async BackendInitializer.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:556:5)
      at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5)
}

Oops, looks like a bug 🙃 : https://github.com/backstage/backstage/issues/24284

So let’s do one of the workarounds : drop the database and re-run backstage

$ docker rm -f postgres
$ docker run -d --name postgres -p 5432:5432 -e POSTGRES_HOST_AUTH_METHOD=trust --restart always --network backstage postgres
$ docker run --rm -it -p 7007:7007 -e POSTGRES_HOST=postgres -e POSTGRES_USER=postgres -e GITLAB_TOKEN=<your_token> --network backstage backstage:demo
...
ForwardedError: Plugin 'auth' startup failed; caused by Error: Failed to initialize guest auth provider, The guest provider cannot be used outside of a development environment
    at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:677:19
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:11
    ... 3 lines matching cause stack trace ...
    at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5) {
  cause: Error: Failed to initialize guest auth provider, The guest provider cannot be used outside of a development environment
      at bindProviderRouters (/app/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:1544:17)
      at createRouter (/app/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:2267:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Object.init [as func] (/app/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:2335:24)
      at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:11
      at async Promise.all (index 4)
      at async #doStart (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:630:5)
      at async BackendInitializer.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:556:5)
      at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5)
}

Then we need to specify the provider configuration : https://backstage.io/docs/auth/gitlab/provider/#configuration

😱 OMG: Need to modify the sources !! I understand know, why there is no official Docker image : when you want to enable a plugin, you’ll need to modify the source code of your application and re-build the Docker image …

Ok, let’s follow the doc and modify the source code as explain. Then modify again the app-config.yaml to include provider information :

...
auth:
  environment: local
  providers:
    gitlab:
      local:
        clientId: <your_gitlab_app_client_id>
        clientSecret: <your_gitlab_app_client_secret>
        audience: https://gitlab.com
        signIn:
          resolvers:
            - resolver: emailMatchingUserEntityProfileEmail
            - resolver: usernameMatchingUserEntityName
...
catalog:
  providers:
    gitlab:
      yourProviderId:
        host: gitlab.com
        orgEnabled: true
        group: fun_with # One of my groups on gitlab.com
        allowInherited: true
        groupPattern: '[\s\S]*'
        schedule:
          frequency: { minutes: 3 }
          timeout: { minutes: 3 }

We re-build our image and run it (and probably the postgres container also…)

$ yarn build:backend --config ../../app-config.yaml
...
$ docker image build . -f packages/backend/Dockerfile --tag backstage:demo
...
$ docker run --rm -it -p 7007:7007 -e POSTGRES_HOST=postgres -e POSTGRES_USER=postgres -e GITLAB_TOKEN=<your_token> --network backstage backstage:demo
...
ForwardedError: Plugin 'kubernetes' startup failed; caused by Error: Kubernetes configuration is missing
    at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:677:19
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:11
    at async Promise.all (index 8)
    ... 2 lines matching cause stack trace ...
    at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5) {
  cause: Error: Kubernetes configuration is missing
      at KubernetesBuilder.build (/app/node_modules/@backstage/plugin-kubernetes-backend/dist/index.cjs.js:1576:15)
      at Object.init [as func] (/app/node_modules/@backstage/plugin-kubernetes-backend/dist/alpha.cjs.js:143:42)
      at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:676:33
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Promise.all (index 8)
      at async #doStart (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:630:5)
      at async BackendInitializer.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:556:5)
      at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:832:5)
}

Oops, looks like another bug 🙃 : even if you don’t want to enable Kubernetes management in your instance, you have to provide configuration.

Found a workaround : https://github.com/backstage/backstage/pull/26223

Add this configuration to the app-config.yaml:

...
kubernetes:
  serviceLocatorMethod:
    type: 'multiTenant'
  clusterLocatorMethods:
    - type: 'config'
      clusters: []

Let’s go for another round of image building and run

$ docker image build . -f packages/backend/Dockerfile --tag backstage:demo
...
$ docker run --rm -it -p 7007:7007 -e POSTGRES_HOST=postgres -e POSTGRES_USER=postgres -e GITLAB_TOKEN=<your_token> --network backstage backstage:demo
...
{"level":"info","message":"Listening on :7007","service":"rootHttpRouter"}
...

🎉 it starts ! Let’s check it in a browser :

Sounds good but ...

Impossible to connect 🤨

Backstage based on authentication provider should create the users from the group I’ve defined in the configuration (the fun_with one). But looking at the logs of the container, it seems that Backstage doesn’t find any user

Need to import users from a gitlab

https://backstage.io/docs/integrations/gitlab/org

Don't understand why, let's debug. Looking at the logs of the container, we can see that Backstage don’t find any users

{"class":"GitlabOrgDiscoveryEntityProvider","level":"info","message":"Scanned 0 users and processed 0 users","plugin":"catalog","service":"backstage","target":"GitlabOrgDiscoveryEntityProvider:yourProviderId","taskId":"GitlabOrgDiscoveryEntityProvider:yourProviderId:refresh","taskInstanceId":"da68ba04-34f3-4796-9050-526e598121b7"}
{"class":"GitlabOrgDiscoveryEntityProvider","level":"info","message":"Scanned 14 groups and processed 0 groups","plugin":"catalog","service":"backstage","target":"GitlabOrgDiscoveryEntityProvider:yourProviderId","taskId":"GitlabOrgDiscoveryEntityProvider:yourProviderId:refresh","taskInstanceId":"da68ba04-34f3-4796-9050-526e598121b7"}

Let’s deep dive into Backstage code and find that is not working for public groups on gitlab.com (NB : because seats concept is only for paid organization)

https://github.com/backstage/backstage/blob/3c587ef7898bf7b584dc218419d2468b472e4e12/plugins/catalog-backend-module-gitlab/src/lib/client.ts#L143

return this.listGroupMembers(groupPath, {
  ...options,
  show_seat_info: true,
}

So I created an Issue on Github to ask for a fix. (After some discussions, first it was defined as the normal behavior, but after some exchanges, a fix will be applied to have the possibility to disable this flag)

So let’s try to switch to my own GitLab instance, I need to adapt the configuration

...
integrations:
  gitlab:
    - host: ${GITLAB_HOST}
      token: ${GITLAB_TOKEN}
      apiBaseUrl: https://${GITLAB_HOST}/api/v4
      baseUrl: https://${GITLAB_HOST}
...
catalog:
  providers:
    gitlab:
      yourProviderId:
        host: ${GITLAB_HOST}
        orgEnabled: true
        groupPattern: '[\s\S]*'
        schedule:
          frequency: { minutes: 3 }
          timeout: { minutes: 3 }

And let’s build and run a last (?) time our Docker image

$ yarn build:backend --config ../../app-config.yaml
...
$ docker image build . -f packages/backend/Dockerfile --tag backstage
...
$ docker run --rm -it -p 7007:7007 -e POSTGRES_HOST=postgres -e POSTGRES_USER=postgres -e GITLAB_TOKEN=<your_token> --network backstage backstage:demo
...
{"class":"GitlabOrgDiscoveryEntityProvider","level":"info","message":"Scanned 5 users and processed 5 users","plugin":"catalog","service":"backstage","target":"GitlabOrgDiscoveryEntityProvider:yourProviderId","taskId":"GitlabOrgDiscoveryEntityProvider:yourProviderId:refresh","taskInstanceId":"3792184e-6eac-412c-968c-9cf02041b6f1"}
...

The logs looks better, let’s try to connect

Finally !

In summary…

After some little troubles, I finally succeed to install Backstage on a Docker image with a GitLab provider for authentication. Now my instance is fully operational.

What have I learn during my first experience with Backstage is that :

  • it seems very customizable

  • but

    • you have to modify the sources to enable plugins and some features

    • there are some bugs and little issues

  • the team is super reactive, I had a feedback to my new issue less than 2 days after

So I’ll probably continue to explore Backstage because I want to understand how helpful it can be in companies to enable a developer portal but the fact that it’s not easily configurable with some configuration for plugins and to have to modify the source code each time is quite disturbing for me 😉

Did you find this article valuable?

Support Yodamad's blog by becoming a sponsor. Any amount is appreciated!

backstage.ioGitLabDocker