ae-ano-detection¶

This service uses an autoencoder model to detect an anomaly in a time series.

The service is built in two steps:

1. Model creation - The creation of the model from the data
2. Model serving - The serving of the built model

Model creation¶

The goal of this step is to prepare the data and train a new model. All further commands are ran in the model-creation directory.

Set up the environment¶

Set up the environment with the following commands.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Generate the virtual environment
python3.10 -m venv .venv

# Activate the virtual environment
source .venv/bin/activate

# Install the requirements
pip install \
    --requirement requirements.txt \
    --requirement requirements-all.txt

Run the experiment¶

Run a new training using the following commands.

1
2
3
4
5
6
7
8
9

# Export the MinIO S3 credentials (ask them to other members of the team)
export AWS_ACCESS_KEY_ID=***
export AWS_SECRET_ACCESS_KEY=***

# Pull the required data for the experiment from MinIO
dvc pull

# Reproduce the ML experiment with DVC
dvc repro

The DVC pipeline is described in the dvc.yaml file.

Each stage describes the dependencies and the outputs of the stage. Every time a dependency of the experiment is updated, running dvc repro will run the stages of the pipeline that are affected and keep the results in cache to speed up future runs.

More information on their website: Get Started: Data Pipelines - dvc.org.

Push new data/results to MinIO¶

In order to push new results to MinIO, use the following commands (similar to Git). Note: DVC automatically adds files that are specified in the pipelines. In other words, there are no needs to explicitly add those files with dvc add. Don't forget to then add the DVC metadata files to Git as well.

1
2
3
4
5
6
7
8

# Get the data status
dvc status

# Add the required files to DVC
dvc add <the files you would add to DVC>

# Push the data to DVC
dvc push

Model serving¶

The goal of this step is to serve the model made in the previous step. All further commands are ran in the model-serving directory.

The API documentation is automatically generated by FastAPI using the OpenAPI standard. A user friendly interface provided by Swagger is available under the /docs route, where the endpoints of the service are described.

This simple service only has one route /compute that takes an image as input, which will be used to guess the number.

Set up the environment¶

Set up the environment with the following commands.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Generate the virtual environment
python3.10 -m venv .venv

# Activate the virtual environment
source .venv/bin/activate

# Install the requirements
pip install \
    --requirement requirements.txt \
    --requirement requirements-all.txt

Retrieve the model¶

Run the following command to get the model created from the previous step.

1
2

# Copy the model from the creation directory
cp ../model-creation/model/ae_model.h5 ./model/ae_model.h5

How to run¶

Environment variables¶

All environment variables are described in the .env file.

The environment variables can be overwritten during the CI/CD pipeline described in the ae-ano-detection.yml GitHub workflow file.

Start the service locally with Python¶

Start the application.

1
2
3
4
5

# Switch to the `src` directory
cd src

# Start the application
uvicorn --reload --port 8282 main:app

Access the service documentation on http://localhost:8282/docs.

Run the tests with Python¶

For each module a test file is available to check the correct behavior of the code. The tests are run using the pytest library with code coverage check. To run the tests, use the following command inside the src folder:

1

pytest --cov-report term:skip-covered --cov-report term-missing --cov=. -s --cov-config=.coveragerc

Start the service locally with minikube and the Docker image hosted on GitHub¶

Start the service with the following commands. This will start the service with the official Docker images that are hosted on GitHub.

In the ae-ano-detection/model-serving directory, start the service with the following commands.

1
2
3
4
5

# Start the ae-ano-detection backend
kubectl apply \
    -f kubernetes/ae-ano-detection.config-map.yml \
    -f kubernetes/ae-ano-detection.stateful.yml \
    -f kubernetes/ae-ano-detection.service.yml

Create a tunnel to access the Kubernetes cluster from the local machine. The terminal in which the tunnel is created must stay open.

1
2

# Open a tunnel to the Kubernetes cluster
minikube tunnel --bind-address 127.0.0.1

Access the ae-ano-detection documentation on http://localhost:8282/docs.

Access the Core Engine documentation on http://localhost:8080/docs to validate the backend has been successfully registered to the Core Engine.

Start the service locally with minikube and a local Docker image¶

Note: The service StatefulSet (ae-ano-detection.stateful.yml file) must be deleted and recreated every time a new Docker image is created.

Start the service with the following commands. This will start the service with the a local Docker image for the service.

In the ae-ano-detection/model-serving directory, build the Docker image with the following commands.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

# Access the Minikube's Docker environment
eval $(minikube docker-env)

# Build the Docker image
docker build -t ghcr.io/swiss-ai-center/ae-ano-detection:latest .

# Exit the Minikube's Docker environment
eval $(minikube docker-env -u)

# Edit the `kubernetes/ae-ano-detection.stateful.yml` file to use the local image by uncommented the line `imagePullPolicy`
#
# From
#
#        # imagePullPolicy: Never
#
# To
#
#        imagePullPolicy: Never

In the ae-ano-detection/model-serving directory, start the service with the following commands.

1
2
3
4
5

# Start the ae-ano-detection backend
kubectl apply \
    -f kubernetes/ae-ano-detection.config-map.yml \
    -f kubernetes/ae-ano-detection.stateful.yml \
    -f kubernetes/ae-ano-detection.service.yml

Create a tunnel to access the Kubernetes cluster from the local machine. The terminal in which the tunnel is created must stay open.

1
2

# Open a tunnel to the Kubernetes cluster
minikube tunnel --bind-address 127.0.0.1

Access the ae-ano-detection documentation on http://localhost:8282/docs.

Access the Core Engine documentation on http://localhost:8080/docs to validate the backend has been successfully registered to the Core Engine.