PHEMS (short for "Pediatric Hospitals as European drivers for multi-party computation and synthetic data generation capabilities across clinical specialities and data types") is a Europe-wide consortium of paediatric hospitals that:
...aims to revolutionize the way health data is managed and utilized across Europe. This project is particularly focused on addressing the challenges posed by privacy concerns and the complexity of data sharing due to varying interpretations of the EU General Data Protection Regulation (GDPR). By developing a decentralized and open health data ecosystem, PHEMS strives to facilitate easier access to health data, thereby advancing federated health data analysis and creating services for generating shareable synthetic datasets.
As a technical partner of the project Aridhia has developed the Federated Node an open source component for running federated tasks.
The Federated Node is based on three existing open source projects:
The Common API provides the structure of the API calls, Keycloak is used for token and user management, and Traefik is used as a reverse proxy. The FN needs to be deployed to a Kubernetes cluster, and requires a Postgres database for storing user credentials.
| Description | |
|---|---|
| 1a | Before creating the task pod, the FN checks if the docker image needed can be found in any of the docker container registries associated with the FN |
| 1b | The task pod is created and the results are saved in the storage account |
| 2 | On /results calls, if the task pod is on completed status, a job is created. |
| 3 | The job's pod will have the 2 storage environments mounted. It fetches the tasks result folder and zips it |
| 4 | The webserver reads the zip contents from the live job pod and saves it in its own storage account environment. |
| 5 | The resulting archive is returned to the end user |
Licences for the component projects can be found here.
Python dependencies are declared in pyproject.toml files within each component directory (e.g. webserver/, build/db-connector/, build/alpine/, build/kc-init/). Locked requirements.txt files are generated from these using pip-tools via the pip_compile Makefile target.
Install pip-tools in your local environment:
python -m pip install pip-toolsRun make pip_compile with the target component directory as an argument:
# Lock dependencies for the webserver
make pip_compile webserver
# Lock dependencies for a build component
make pip_compile build/db-connectorBy default this writes requirements.txt in the given directory. To write to a different output file, pass it as a second positional argument:
make pip_compile webserver requirements-dev.txtAny additional flags supported by pip-compile can be appended after the directory (and optional output file) arguments.
To view the default flags see scripts/pip-compile.sh.
- Edit the
dependencieslist in the relevantpyproject.toml. Use[project.optional-dependencies]for dev/test-only extras. - Re-run
make pip_compile <dir>to regenerate the lockedrequirements.txt. - Commit both
pyproject.tomlandrequirements.txt.
Use ~= (compatible-release) specifiers in pyproject.toml to constrain the minor version while allowing patch updates, e.g. "flask~=3.1.3". The locked requirements.txt pins exact versions with hashes for reproducible installs.
hadolint lints all Dockerfiles in the project. It runs inside Docker, so no local installation is required beyond Docker itself.
make hadolintThis prints any issues directly to the terminal in a readable format. It also writes a JUnit XML report to artifacts/hadolint.xml, which is consumed by CI.
See the Run Locally Wiki Page
See the How to deploy Wiki Page.