How to add a production worker?

[northcode7]

I started Dataplane with the default docker-compose.yml. And I added an extra workergroup like so:

dataplane-worker-prod: image: dataplane/dataplane-worker-python:latest mem_limit: 512MB cpus: 3 deploy: replicas: 6 volumes: - code-files:/dataplane/code-files/ environment: DP_CODE_FOLDER: "/dataplane/code-files/" secret_db_host: postgres secret_db_user: postgres secret_db_pwd: "Hello123!" secret_db_ssl: "disable" secret_db_port: "5432" secret_db_database: "dataplane" secret_jwt_secret: "117943e8-9d43-49ac-aa4d-9c64004c8992" DP_DATABASE: "timescaledb" DP_NATS: "nats://nats:4222, nats://nats-r_1:4222, nats://nats-r_2:4222" DP_MODE: "production" DP_DEBUG: "true" DP_DB_DEBUG: "false" DP_MQ_DEBUG: "false" DP_METRIC_DEBUG: "false" DP_SCHEDULER_DEBUG: "false" DP_WORKER_HEARTBEAT_SECONDS: "1" DP_WORKER_GROUP: "python_production" DP_WORKER_CMD: "/bin/sh" DP_WORKER_TYPE: "container" DP_WORKER_LB: "roundrobin" DP_WORKER_ENV: "Production" DP_WORKER_PORT: "9005" DP_WORKER_LANGUAGES: "Python" DP_WORKER_LOAD_PACKAGES: "Python"

The workergroup is started. I can see the workers coming up as docker containers, but in the main-app there is no new workergroup visible. In the main app in the docker-compose.yml I also added 'production' like so:
DP_MODE: "development, production"
But it doesn't work. Can you guys help me out?

• Is the preferred way of hosting on Kubernetes?

• When do you think there will be a v1.0 release?

Thanks alot for Dataplane and the effort! Looks very promising.

[saul-data]

Hi there, thanks for the feedback. I’d be happy to go through implementing both docker compose and Kubernetes with you. Can you please send me your docker-compose file? I don’t know if it’s possible to attach it on here or send a link to a repo? I can try it on my side.

Leave DP_MODE as development. That’s good feedback, it doesn’t do much, could be confusing and maybe should be removed.

Still working through some bugs for v1 - mostly around permissions. I’d like to add an api trigger and open to other suggestions.

[saul-data]

Only a pleasure, I’m glad it’s working. I’ll update our notes in the documentation on how to add workers under different environments. Do let me know how you get along and any further feedback and if you still need any assistance with deploying to Kubernetes :)

[northcode7]

Very kind. Thank you! I think I could use some help with a deployment on Kubernetes. If I'm at that point I will reach out. Do you guys also have a Discord channel or something?

[saul-data]

Happy to help with the Kubernetes implementation when you are ready. I opened a Discord channel and added the shield to the channel on the Github repo. Here is the link to the channel: https://discord.com/channels/1000429753205014528/

[northcode7]

Hi Saul, I'm not sure if the channel is public. I can't join with the link.

[saul-data]

Can you please try this link: https://discord.gg/Ztu4ASNky8