integrity-checker¶

This service uses a config file to analyse a dataset and provide a report about the dataset integrity.

The API documentation is automatically generated by FastAPI using the OpenAPI standard. A user friendly interface provided by Swagger is available under the /docs route, where the endpoints of the service are described.

This simple service only has one route /compute that takes two input: a config file and the dataset to nalayse, and provide a report in a .csv file.

Environment variables¶

All environment variables are described in the .env file.

The environment variables can be overwritten during the CI/CD pipeline described in the [GitHub workflow files] (https://github.com/swiss-ai-center/integrity-checker/tree/main/kubernetes).

Start the service locally with Python¶

In the integrity-checker directory, start the service with the following commands.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Generate the virtual environment
python3 -m venv .venv

# Activate the virtual environment
source .venv/bin/activate

# Install the requirements
pip install \
    --requirement requirements.txt \
    --requirement requirements-all.txt

Start the application.

1
2
3
4
5

# Switch to the `src` directory
cd src

# Start the application
uvicorn --reload --port 9595 main:app

Access the service documentation on http://localhost:9595/docs.

Run the tests with Python¶

For each module a test file is available to check the correct behavior of the code. The tests are run using the pytest library with code coverage check. To run the tests, use the following command inside the integrity-checker folder:

1

pytest --cov-report term:skip-covered --cov-report term-missing --cov=. -s --cov-config=.coveragerc

Start the service locally with minikube and the Docker image hosted on GitHub¶

Start the service with the following commands. This will start the service with the official Docker images that are hosted on GitHub.

In the integrity-checker directory, start the service with the following commands.

1
2
3
4
5

# Start the integrity-checker backend
kubectl apply \
    -f kubernetes/integrity-checker.config-map.yml \
    -f kubernetes/integrity-checker.stateful.yml \
    -f kubernetesintegrity-checker.service.yml

Create a tunnel to access the Kubernetes cluster from the local machine. The terminal in which the tunnel is created must stay open.

1
2

# Open a tunnel to the Kubernetes cluster
minikube tunnel --bind-address 127.0.0.1

Access the integrity-checker documentation on http://localhost:9595/docs.

Access the Core Engine documentation on http://localhost:8080/docs to validate the backend has been successfully registered to the Core Engine.

Start the service locally with minikube and a local Docker image¶

Note: The service StatefulSet (integrity-checker.stateful.yml file) must be deleted and recreated every time a new Docker image is created.

Start the service with the following commands. This will start the service with the a local Docker image for the service.

In the integrity-checker directory, build the Docker image with the following commands.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

# Access the Minikube's Docker environment
eval $(minikube docker-env)

# Build the Docker image
docker build -t ghcr.io/swiss-ai-center/integrity-checker:latest .

# Exit the Minikube's Docker environment
eval $(minikube docker-env -u)

# Edit the `kubernetes/integrity-checker.stateful.yml` file to use the local image by uncommented the line `imagePullPolicy`
#
# From
#
#        # imagePullPolicy: Never
#
# To
#
#        imagePullPolicy: Never

In the integrity-checker directory, start the service with the following commands.

1
2
3
4
5

# Start the integrity-checker backend
kubectl apply \
    -f kubernetes/integrity-checker.config-map.yml \
    -f kubernetes/integrity-checker.stateful.yml \
    -f kubernetes/integrity-checker.service.yml

Create a tunnel to access the Kubernetes cluster from the local machine. The terminal in which the tunnel is created must stay open.

1
2

# Open a tunnel to the Kubernetes cluster
minikube tunnel --bind-address 127.0.0.1

Access the integrity-checker documentation on http://localhost:9595/docs.

Access the Core Engine documentation on http://localhost:8080/docs to validate the backend has been successfully registered to the Core Engine.