Skip to content

Kedro starter templates using Scikit-learn and optionally MLflow

Notifications You must be signed in to change notification settings

Minyus/kedro-starters-sklearn

Repository files navigation

kedro-starters-sklearn

This repository provides the following starter templates for Kedro 0.18.14.

  • sklearn-iris trains a Logistic Regression model using Scikit-learn.
  • sklearn-mlflow-iris adds experiment tracking feature using MLflow.

Pipeline visualized by Kedro-viz

sklearn-iris template

Iris dataset

Iris dataset is included and used in default.

  • Modification: for each species, setosa is encoded to 0, versicolor is encoded to 1, and virginica samples were removed.
  • Split: for each species, first 25 samples were included in train.csv, and last 25 samples were included in test.csv.

How to use

  1. Install dependencies.

    pip install 'kedro==0.18.14' pandas scikit-learn 
  2. Generate your Kedro starter project from sklearn-iris directory.

    kedro new --starter https://github.com/Minyus/kedro-starters-sklearn.git --directory sklearn-iris

    As explained by Kedro's documentaion, enter project_name, repo_name, and python_package.

    Note: As your Python package name, choose a unique name and avoid a generic name such as "test" or "sklearn" used by another package. You can see the list of importable packages by running python -c "help('modules')".

  3. Change the current directory to the generated project directory.

    cd /path/to/project/directory
  4. Run the project.

    kedro run

Option to use Kaggle Titanic dataset

  1. Download Kaggle Titanic dataset
  2. Replace train.csv and test.csv in /path/to/project/directory/data/01_raw directory
  3. Modify /path/to/project/directory/base/parameters.yml to set parameters appropriate for the dataset (commented out in default)

sklearn-mlflow-iris template

This template integrates MLflow to Kedro using PipelineX. Even without writing MLflow code. You can:

  • configure MLflow Tracking
  • log inputs and outputs of Python functions set up as Kedro nodes as parameters (e.g. features used to train the model) and metrics (e.g. F1 score).
  • log execution time for each Kedro node and DataSet loading/saving as metrics.
  • log artifacts (e.g. models, execution time Gantt Chart visualized by Plotly, parameters.yml file)

In this template, MLflow logging is configured in Python code at src/<python_package>/mlflow/mlflow_config.py

See here for details.

How to use

  1. Install dependencies.

    pip install 'kedro==0.18.14' pandas scikit-learn mlflow 'pipelinex>=0.7.7' plotly
  2. Generate your Kedro starter project from sklearn-mlflow-iris directory.

    kedro new --starter https://github.com/Minyus/kedro-starters-sklearn.git --directory sklearn-mlflow-iris
  3. Follow the same steps as sklearn-iris template.

Access MLflow web UI

To access the MLflow web UI, launch the MLflow server.

mlflow server --host 127.0.0.1 --port 8080 --backend-store-uri sqlite:///mlruns/sqlite.db --default-artifact-root ./mlruns

Logged metrics shown in MLflow's UI

Gantt chart for execution time, generated using Plotly, shown in MLflow's UI

About

Kedro starter templates using Scikit-learn and optionally MLflow

Resources

Stars

Watchers

Forks

Packages

No packages published