innovativeproject ml models management

Maisie - ML Models Management System

Maisie
Maisie Sphinx Theme
Documentation
PyPI
DockerHub

• 1   Features
• 2   Installing
• 3   Deploying
  • 3.1   Using docker-compose
  • 3.2   Fetching individual images
• 4   Contributing
  • 4.1   Getting started
  • 4.2   Fixing existing issues
  • 4.3   Implementing new features
  • 4.4   Commiting the changes
• 5   Documentation

1   Features

Could archiving, storing, managing and organizing machine learning models be done efficiently and with great focus on user experience? Sure, Maisie does just that.

Maisie is a friendly, easy to use assistant that consists of:

• Web Application written in React
• Backend API written in Python, Flask
• Client Application/Package written in Python and hosted on PyPI

It integrates seamlessly with your favorite tools and provides you with all the important data, such as:

• Git revisions for all trained models, as well as information about source branches
• Searchable, filterable hyperparameters, parameters and metrics
• A single identifying dataset name, as well as an optional description
• Permanent URLs for easy sharing and downloading of stored models

2   Installing

You can get the most current package from PyPI

$ pip install Maisie

Using it in your training environment is fairly straightforward:

import maisie
from sklearn.externals import joblib

# Define your model here

model.fit(X, y)
model_filename = "example_model.pkl"
joblib.dump(model, model_filename)

# Define your metrics, fetch parameters and hyperparameters

models = maisie.Models()
models.upload(
    name="My first uploaded model",
    filename=model_filename,
    dataset_name="Singly Identifying Dataset Name",
    metrics={"accuracy": accuracy},
    hyperparameters=hyperparameters,
    parameters=parameters,
)

3   Deploying

3.1   Using docker-compose

This repository provides a pre-configured docker-compose.yml file that contains sensible default options.

Before starting the containers, you should create a local .env file using the included .env.sample.

To start up all services, run:

$ docker-compose up

To stop your services, you can press Ctrl+C/Ctrl+D. If you started the services in the background using docker-compose up -d, the correct way to do this would be:

$ docker-compose stop

You can learn more about Docker Compose by clicking here.

3.2   Fetching individual images

Both frontend and backend images are automatically published to Docker Hub as soon as new stable release is made available.

Links

• Frontend image on Docker Hub
• Backend image on Docker Hub
• ...other services

For reference, you can look at the sample Ansible playbook that deploys all containers to a specified host using the locally configured .env file.

4   Contributing

4.1   Getting started

1. Clone the repository from the develop branch

$ git clone -b develop [email protected]:nokia-wroclaw/innovativeproject-ml-models-management.git

2. Install pre-commit
3. Inside the project's root directory install all required githooks:

$ pre-commit install

4. To start all required services for local development, run:

$ docker-compose up

5. You're all set up!

4.2   Fixing existing issues

1. Pick one of the open issues or create a new one
2. Create a new branch named issue-[number]-[short description] derived from the develop branch, for example

$ git checkout -b issue-42-project-removal-permissions develop

3. Make sure your implementation fixes the actual problem and is well tested.

4.3   Implementing new features

When implementing new features, you should start by creating a new branch named feature-[short description] derived from the develop branch, for example

$ git checkout -b feature-new-user-profile develop

4.4   Commiting the changes

To run all tests and check whether all required pre-commit githooks are satisfied, run

$ pre-commit run --all-files

Your commit message should briefly summarize the changes (if possible) in plain English. To learn how to write a proper commit message, check out this article.

When ready, create a new pull request compared with the develop branch set as a base branch.

5   Documentation

For the lastest stable release, the documentation can be seen at docs.maisie.dev.