Skip to content
BranchesTags

Repository files navigation

Cheetah development infrastructure

This repository is used to setup infrastructure when developing locally using Kafka/OpenSearch.

The repository consists of a set of docker-compose files which are all referenced in the .env file. This allows invoking docker compose up <service-name> on a service in any of the docker-compose files, from the root of the repository.

See also: https://docs.cheetah.trifork.dev/reference/development-infrastructure

Start infrastructure

docker compose up --quiet-pull

First-time boot note: OpenSearch performs a one-time security cert and index bootstrap on cold start that can take a few minutes, especially on slower machines. Its healthcheck's start_period is set to 120s to accommodate this. If OpenSearch still isn't healthy after ~2 minutes, dependent services (opensearch-dashboards, opensearch-configurer) won't start - just re-run docker compose up once OpenSearch is healthy and everything else will come up.

Prerequisites

  1. Follow: https://docs.cheetah.trifork.dev/getting-started/guided-tour/prerequisites#run-standard-jobs
  2. Add 127.0.0.1 keycloak to your hosts file (/etc/hosts on Linux/macOS, C:\Windows\System32\drivers\etc\hosts on Windows). Keycloak is served under the keycloak hostname so browser and host-side tooling resolve to the same name the in-cluster services use.

Resource requirements

The infrastructure requires a lot of resources, especially memory when running all services at once.

Here is some basic profiling done while running through WSL2 with 16GB RAM:

# See if your docker supports memory limits
docker info --format '{{json .MemoryLimit}}'
# Get total memory for docker
docker info --format '{{json .MemTotal}}' | numfmt --from=auto --to=iec
# Get total CPUs for docker
docker info --format '{{json .NCPU}}'
Profile MEM USAGE / LIMIT
core 2.4GB / 4.4GB
kafka 1.3GB / 2.2GB
opensearch 1.9GB / 2.9GB
full 2.9GB / 5.2GB

Estimated requirements:

Profile CPUs Docker available memory (RAM) Disk space (Images)
Minimum 2 4GB >6.6GB
Recommended 8 8GB >20GB
Best 16 16GB >40GB

Security model

The development infrastructure follows the Reference Security Model.
For local development we are using Keycloak inside docker-compose/keycloak.yaml as a local IDP.

See sections below for details on security model configuration.

Kafka

The kafka setup consists of different services:

  • kafka - Strimzi Kafka with the Cheetah KRaft Kafka Authorizer
  • redpanda - A Console provides a user interface to manage multiple Kafka connect clusters. https://docs.redpanda.com/docs/manage/console/
  • kafka-setup - A bash script which sets up a Kafka User for redpanda to use when connecting to Kafka, as well as some predefined topics. The topics to be created are determined by the environment variable INITIAL_KAFKA_TOPICS, which can be set in the .env file or overritten in your local environment. It also pre-creates the internal topics that Apicurio Registry 3.x depends on (kafkasql-journal-v3, kafkasql-snapshots, registry-events).
  • schema-registry - Apicurio Registry 3.1.x, running the kafkasql storage backend against the local Kafka broker via OAuth.
  • kafka-minion - Kafka Prometheus exporter

Running Kafka and its associated services

Run:

docker compose --profile=kafka up -d

This brings up kafka, kafka-setup, redpanda, schema-registry, and their Keycloak dependency. kafka-minion lives under the observability (or full) profile — add --profile=observability if you want Prometheus metrics too.

When all of the services are running, you can go to:

Listeners

5 different listeners is setup for Kafka on different internal and external ports (see kraft.properties for the configuration):

  • localhost:9092 - Used for connecting to kafka with OAuth2 authentication from outside the docker environment.
  • localhost:9093 - Used for connecting to kafka without authentication from outside the docker environment.
  • kafka:19092 - Used for connecting to kafka with OAuth2 authentication from a docker container in the cheetah-infrastructure docker network.
  • kafka:19093 - Used for connecting to kafka without authentication from a docker container in the cheetah-infrastructure docker network.
  • kafka:19094 - Only used by Redpanda, since it does not support Oauth2.

Authentication

To require Oauth2 authentication when connecting to kafka, you can remove ;User:ANONYMOUS from the super.users property in kraft.properties.
This will cause all connections from unauthenticated sources to be rejected by CheetahKRaftAuthorizer.

OpenSearch

The OpenSearch setup consists of different services:

  • OpenSearch - OpenSearch data storage solution
  • OpenSearch-Dashboard - Dashboard solution for interacting with OpenSearch API
  • OpenSearch Configurer - Uses OpenSearch Template Configuration Script to setup Index Templates and more.

Files placed in any subdirectory of config/opensearch-configurer/ are automatically applied to the OpenSearch instance.

Running OpenSearch and its associated services

Run:

docker compose --profile=opensearch up -d

When all of the services are running, you can go to:

Authentication

Services should connect using the OAuth2 protocol.
You can choose to set DISABLE_SECURITY_DASHBOARDS_PLUGIN=true and DISABLE_SECURITY_PLUGIN=true to disable security completely.

There are two ways to authenticate against OpenSearch, backed by two separate user directories:

Path Credentials Where the user lives When to use it
HTTP Basic Auth admin / admin OpenSearch's internal user DB (internal_users.yml) Break-glass, API/curl scripting, when Keycloak is down
OIDC (OAuth2) developer / developer Keycloak realm (local-development.json) Mirrors production auth — how a real user would log in

The admin user does not exist in Keycloak, and the developer user does not exist in OpenSearch. They're two entirely different accounts in two different databases with two different auth mechanisms. Consequently, admin:admin has full cluster admin, while developer:developer gets only the roles mapped to it via Keycloak scopes (read-only data access, no cluster-admin actions).

Basic auth access (admin:admin)

Note: OpenSearch has anonymous access enabled by default, but the anonymous user has no permissions. Browsers won't prompt for credentials automatically.

For browser access, use a browser extension or tool that supports basic authentication, or use the OpenSearch Dashboard at http://localhost:5602 instead.

For API/command line access, use curl with the admin:admin credentials:

curl -k -s -u "admin:admin" http://localhost:9200/
curl -k -s -u "admin:admin" http://localhost:9200/_cat/indices

Or set the OPENSEARCH_URL variable:

curl -k -s -u "admin:admin" $OPENSEARCH_URL/_cat/indices

OAuth2 token

Tokens are minted by Keycloak and must carry aud: opensearch - enforced by required_audience in config/opensearch/security/config.yml. The opensearch scope on a Keycloak client stamps this audience automatically, so as long as you request scope=opensearch you're fine.

If you do not want to use basicauth locally, you can get a token using this curl command:

ACCESS_TOKEN=$(curl -s -X POST $OPENSEARCH_TOKEN_URL \
     -H "Content-Type: application/x-www-form-urlencoded" \
     -d "grant_type=client_credentials&client_id=$OPENSEARCH_CLIENT_ID&client_secret=$OPENSEARCH_CLIENT_SECRET&scope=$OPENSEARCH_SCOPE" \
     | jq -r '.access_token')
     #| grep -o '"access_token":"[^"]*' | grep -o '[^"]*$')

And query OpenSearch like this:

curl -k -s -H "Authorization: Bearer $ACCESS_TOKEN" $OPENSEARCH_URL/_cat/indices

OIDC login via Dashboards (developer:developer)

On http://localhost:5602 click "Log in with single sign-on" → you'll be redirected to Keycloak → sign in as developer / developer. First-time login prompts for profile fields (email, first/last name - any placeholder like developer@localhost works; values persist for the container's lifetime).

The developer user only has read access and a couple of dev roles, so some cluster-admin UI features (e.g. "Manage data sources") will return 403. That's expected - for full admin access, use admin:admin via the internal-user form on the same login page.

PostgreSQL

Services:

  • postgres-build-oidc-validator — init container; provisions pg_oidc_validator.so (upstream Percona) into a shared volume on first boot.
  • postgres — OAuth-protected PostgreSQL 18.
  • pgadmin — browser GUI.

Running

docker compose --profile=postgres up -d

pgAdmin: http://localhost:5050 — login admin@admin.com / admin. On first Connect to cheetah-postgres, enter database password admin (cached for the life of pgadmin-data).

Authentication

  • OAuth-only for services. Issuer http://keycloak:1852/realms/local-development, scope postgres.
  • One scram-sha-256 exception: the pgadmin role (password admin, defined in config/postgres/init/01-roles.sql), scoped via pg_hba.conf.
  • The validator maps the JWT azp claim (authorized party = the OAuth client_id) → PostgreSQL role. Roles: default-access, default-read, default-write.

Host-side psql

Requires PostgreSQL 18 client + libpq-oauth and 127.0.0.1 keycloak in /etc/hosts. libpq enforces HTTPS issuer URLs by default; for local-dev HTTP, prepend PGOAUTHDEBUG=UNSAFE. libpq runs the OAuth device flow — it prints a URL + code; visit it, log in as developer/developer, authorize the client.

Use the following command to connect with psql and filter out everything except the device flow auth url and the device code during authorization.

PGOAUTHDEBUG=UNSAFE psql 'host=localhost port=5432 dbname=cheetah-postgres user=default-access oauth_issuer=http://keycloak:1852/realms/local-development oauth_client_id=default-access oauth_client_secret=default-access-secret oauth_scope=postgres' 2>&1 | grep --line-buffered -vE '^\[libcurl\]'

List of all profiles in docker compose

List of profiles:

  • full
  • core
  • kafka
  • opensearch
  • observability
  • postgres

Here is further explanation on what each profile starts.

Images / profiles kafka-core opensearch-core schema-registry-core core kafka opensearch observability postgres full
Keycloak x x x x x x x x x
Kafka x x x x x x
Redpanda console x x
Opensearch x x x x
Opensearch dashboard x x
Opensearch configurer x x x x
Schema registry x x x x
Prometheus x x
Grafana x x
PostgreSQL x

Keycloak

Keycloak is used as a local identity provider, to be able to mimic a production security model with service to service authentication.

Useful urls:

Default clients:

A set of default clients have been defined which covers most common usecases.

All roles are mapped to the roles claim in the JWT. This configuration is defined in local-development.json and is applied to keycloak using the keycloak-setup service. To modify the configuration either go to the admin console (Username: admin Password: admin) or edit local-development.json following this guide.

  • Default access
    • Description: Read and write access to all data Kafka, OpenSearch, Schema registry and PostgreSQL
    • client_id: default-access
    • client_secret: default-access-secret
    • default_scopes: [ ]
    • optional_scopes:
      • kafka
        • Roles:
          • Kafka_*_all
      • opensearch
        • Roles:
          • opensearch_default_access
      • schema-registry
        • Roles:
          • sr-producer
      • postgres
        • Roles:
          • postgres_access
  • Default write
    • Description: Write access to all data in Kafka, OpenSearch, Schema registry and PostgreSQL
    • client_id: default-write
    • client_secret: default-write-secret
    • default_scopes: [ ]
    • optional_scopes:
      • kafka
        • Roles:
          • Kafka_*_write
      • opensearch
        • Roles:
          • opensearch_default_write
          • opensearch_default_delete
      • schema-registry
        • Roles:
          • sr-producer
      • postgres
        • Roles:
          • postgres_access
  • Default read
    • Description: Read access to all data in Kafka, OpenSearch and PostgreSQL (plus schema-registry producer role where configured)
    • client_id: default-read
    • client_secret: default-read-secret
    • default_scopes: [ ]
    • optional_scopes:
      • kafka
        • Roles:
          • Kafka_*_read
      • opensearch
        • Roles:
          • opensearch_default_read
      • postgres
        • Roles:
          • postgres_access
  • Users
    • Description: User login via browser such as OpenSearch Dashboard (See Users for user details)
    • client_id: users
    • client_secret: users-secret
    • default_scopes: [ ]
    • optional_scopes:
      • kafka
      • opensearch
      • schema-registry
      • postgres
  • Custom client
    • Description: A custom client which can be configured using Environment variables. Useful for pipelines where services require custom roles.
    • client_id: $DEMO_CLIENT_NAME
    • client_secret: $DEMO_CLIENT_SECRET
    • default_scopes:
      • custom-client
        • Roles: $DEMO_CLIENT_ROLES - Should be a comma separated list e.g. (my_view_role,my_edit_role,my_admin_role)
    • optional_scopes: [ ]

Users:

  • developer
    • Username: developer
    • Password: developer
    • Roles:
      • opensearch_developer
      • opensearch_default_read
      • Kafka_*_all
      • sr-producer
      • postgres_access

About

No description or website provided.

Topics

cheetah

Resources

Readme
Activity
Custom properties

Stars

2 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages