We use cookies to enhance your experience on the site

MLflow - tracking your experiments

Three lessons ago you trained a random forest in scikit-learn, then a caravan of trees in XGBoost, and at the end a neural network in PyTorch. Every time you changed something small: the depth of a tree here, the number of estimators there, the learning rate somewhere else. Every time you looked at the accuracy printed on the screen and told yourself "ah, better now". So answer me this without looking anything up, @name: which exact set of settings gave you that best result? And where is the file with that model?

I know that silence. I have heard it in more than one research camp: twenty trackers come back from the savanna, each carrying a handful of measurements on a scrap of paper or in their head, and by the evening fire it turns out that nobody can reproduce the observation that looked the most interesting. The note got soaked, the card slipped out of a pocket, and its author only remembers that "the animal was big". In machine learning the part of the soaked note is played by the notebook cell you overwrote, and by the variable you assigned for the second time.

This lesson is about keeping an expedition logbook that cannot be lost. You are going to meet MLflow - a tool that on every training run writes three things into one place: the parameters the model was built with, the metrics it achieved, and the model itself, as a file ready to be loaded back. You will see how to log your first experiment, how to inspect the results in a browser, how to hand the logging over to a machine, and how to pull the winner out of twenty attempts with a single command.

The logbook nobody keeps

Before we reach for a tool, let us look at experiment tracking done by hand. Almost everybody goes through this stage: you start an ordinary list, after every training run you append a dictionary with the settings and the score, and at the end you pick the best entry with the built-in

max

function. Its

key

parameter tells it what to compare the elements by - here, by the value stored under the

accuracy

key. This works, and it beats nothing at all, so let us look at it honestly before we start finding fault with it.

1# manual experiment tracking: a list of dicts
2results = []
3
4results.append({"n_estimators": 100, "max_depth": 10, "accuracy": 0.973})
5results.append({"n_estimators": 300, "max_depth": 6, "accuracy": 0.981})
6
7print(max(results, key=lambda r: r["accuracy"]))
8# {'n_estimators': 300, 'max_depth': 6, 'accuracy': 0.981}

The

max

function pointed at the second entry, because 0.981 beats 0.973. Now pay attention to what this list does not contain, and never did. There is no model in it - there are numbers describing a model, but not the model, so to use the winner you have to train it all over again and hope you hit the same conditions. There is no date and no data version in it, so a week from now you will not tell a score from before a fix in the field notebook apart from a score after it. There is not a single entry in it that you did not add by hand, and hands forget, especially on the twentieth training run at midnight. And the most important one: this list lives in the memory of a process. Close the notebook or let the kernel restart, and the whole expedition is gone.

What MLflow is, and what it is not

MLflow is a platform for tracking machine learning experiments: it records parameters, metrics and models. That one sentence answers the question of why anybody installs this tool at all, and it is worth remembering in exactly that shape - parameters, metrics, models. Everything else MLflow can do grows out of those three.

Let us push three misunderstandings out of the way straight away, because they cling to this name stubbornly. First, MLflow is not there only to train models - it does not train at all. There is not a single learning algorithm inside it. Training is still done by scikit-learn, XGBoost or PyTorch, exactly as in the previous lessons, while MLflow stands beside them with a notebook and writes down what happened. If you removed it from the project, the models would come out exactly the same, only nobody would have recorded how they came to be.

Second, MLflow is not there to build user interfaces. This misunderstanding is a sneaky one, because in a moment we will start the MLflow web interface and you will see a table of results in your browser. That interface, however, is finished and fixed: it is a viewer for your experiments, not a tool for building a screen for the users of your own application. Building your own interfaces is the job of completely different libraries, Streamlit or Gradio on the Python side, for example.

Third, MLflow is not there to write unit tests. Unit tests check whether a function returns what it should, and you write them in

pytest

unittest

. MLflow does not rule on whether your code is correct - it notes the results of a training run even when that run was a complete miss. A model with fifteen percent accuracy gets recorded just as diligently as the best one, because an expedition logbook records the failed expeditions too.

Installation and four words to start with

You install MLflow with a single package manager command, the same way you installed Pandas or XGBoost earlier. The package is a big one, because a web server and a database sit inside it, so the installation will take longer than usual. That is normal.

1pip install mlflow

After the installation you have two things at once: a library you will import in your code, and a program you run from the terminal with the

mlflow

command. It is the same package in two disguises, and we are about to use both. Nothing else in your code changed - no scikit-learn or NumPy import needs any correction after this, and no file of yours was touched.

Before we write the first line, four words that will keep coming back until the end of the lesson. A run is one training execution: one set of parameters, one set of metrics, one model. An experiment is a named collection of runs concerning the same problem, for example every attempt at recognizing species. A parameter is a number or a piece of text that you set before the training and that describes the configuration. A metric is a number that came out of the training and that measures quality. The difference between a parameter and a metric is not cosmetic: MLflow treats the two differently, as you will find out in the section on comparing runs.

The field notebook for this lesson

We need something to train on, but modestly - the hero of this lesson is the logbook, not the data. We will collect three hundred specimens of three species, each described by two measurements: mass in kilograms and top speed in kilometres per hour. You know the

np.random.default_rng

generator from the NumPy lesson and

train_test_split

from the supervised learning lesson, so let me only remind you that

stratify=y

keeps the species proportions equal in both parts, and

random_state=42

makes the split identical every single time. This time we hand the

normal

method two lists instead of two numbers: the first holds the means of both measurements, the second their spreads, and

size=(100, 2)

gives a hundred rows of two columns.

1import numpy as np
2from sklearn.model_selection import train_test_split
3
4rng = np.random.default_rng(7)
5
6# each species: 100 specimens, columns are mass [kg] and speed [km/h]
7lion = rng.normal([190, 58], [30, 8], size=(100, 2))
8cheetah = rng.normal([55, 110], [10, 12], size=(100, 2))
9hyena = rng.normal([60, 65], [12, 9], size=(100, 2))
10
11X = np.vstack([lion, cheetah, hyena])
12y = np.repeat([0, 1, 2], 100)          # 0 = lion, 1 = cheetah, 2 = hyena
13
14X_train, X_test, y_train, y_test = train_test_split(
15    X, y, test_size=0.2, random_state=42, stratify=y
16)
17
18print(X_train.shape, X_test.shape)     # (240, 2) (60, 2)

Two hundred and forty animals go to training, sixty wait for the exam. The

np.vstack

function glued the three herds one below another, so the order of the rows matches the order of the labels from

np.repeat

. Notice that fixing the generator seed has nothing to do with MLflow - it is plain reproducibility hygiene, and here it pays off twice over, because in a moment we will be comparing runs against each other and we want the differences to come from the model settings, not from a different draw of animals.

Your first run: an experiment and a parameter

We start keeping the logbook. Three calls are enough to begin with. The

mlflow.set_experiment

function takes the name of an experiment and makes it the current one; if no such experiment exists yet, MLflow creates it. As a warm-up we set up a separate experiment called "safari-trials", so that these first, incomplete runs do not clutter the real logbook. The

mlflow.start_run

function opens a single run, and we use it inside a

with

block, which I will say more about in a moment. The optional

run_name

argument gives the run a readable name - without it you get a randomly generated nickname along the lines of "silent-hound-42", amusing but useless when you are comparing results. Finally,

mlflow.log_param

records one parameter: its name first, then its value.

1import mlflow
2
3mlflow.set_experiment("safari-trials")
4
5with mlflow.start_run(run_name="random_forest_v1"):
6    mlflow.log_param("n_estimators", 100)
7    mlflow.log_param("max_depth", 10)
8
9    print(mlflow.active_run().info.run_id)

Once you leave the

with

block, the run is closed and saved. The program printed a long string of characters - that is the run identifier, the unique fingerprint of this one execution, which we will use later to load a model back. I am not quoting it here, because yours will be different; MLflow generates it at random for every run. Notice what did not change: no file that you would have named yourself appeared in the project directory, and your data is sitting exactly where it was. MLflow created an

mlruns

directory next to the script and keeps the whole logbook in there. Remember that name, because in a moment it will explain why the web interface has to be started from that same folder.

Let us also look closely at the logging call itself, because its order trips people up. Read element by element it goes:

mlflow

log_param

"n_estimators", 100

. The package first, then the dot, then the function name, the opening parenthesis, inside it the name of the parameter as text, after the comma its value, and the closing parenthesis at the end. The order of the arguments cannot be reversed: had you written

mlflow.log_param(100, "n_estimators")

, the logbook would have gained a parameter named "100" with the value "n_estimators". Single and double quotes are equivalent in Python, so that middle piece can just as well be written

'n_estimators', 100

- the meaning does not change one bit.

While we are on

with

: you can open a run the old way, calling

mlflow.start_run()

without

with

and closing it afterwards with

mlflow.end_run()

. My recommendation is unambiguous: always use
with
. The reason is practical, not aesthetic. If your training throws an exception halfway through, and it will sooner or later, the

with

version still closes the run and marks it as interrupted. The version without

with

leaves it open, because the

end_run

call never happens. Everything you log afterwards quietly attaches itself to that crippled run, and when you try to open the next one, MLflow stops the program with a message that a run is already active. Half an hour of hunting for why "nothing is being saved in the new run" is the price of three saved characters.

A metric is a result, a parameter is a decision

The parameters are recorded, but a logbook without results is only a wish list. Recording a result is the job of

mlflow.log_metric

, which takes the same two arguments as

log_param

: a name and a value. The difference sits elsewhere. A parameter may be set only once within a run and may be anything at all - a number, a piece of text, the name of an algorithm. A metric has to be a number, because MLflow draws charts from it and sorts tables by it; trying to log text as a metric ends in an error. In exchange, a metric may be recorded many times during a single run, which comes in handy with neural networks, where you want to note the current loss after every epoch.

So let us train a random forest, whose syntax you know from the scikit-learn lesson, and record its accuracy. You met

accuracy_score

in the supervised learning lesson - it is the share of correct predictions.

1from sklearn.ensemble import RandomForestClassifier
2from sklearn.metrics import accuracy_score
3
4with mlflow.start_run(run_name="random_forest_v2"):
5    mlflow.log_param("n_estimators", 100)
6    mlflow.log_param("max_depth", 10)
7
8    model = RandomForestClassifier(
9        n_estimators=100, max_depth=10, random_state=42
10    )
11    model.fit(X_train, y_train)
12
13    accuracy = accuracy_score(y_test, model.predict(X_test))
14    mlflow.log_metric("accuracy", accuracy)

The accuracy will land somewhere around 98 percent. I am not putting the exact number in a comment, because it depends on the scikit-learn version, so run the code and check it for yourself. Something else matters more here: notice that the training looks exactly the way it looked in the scikit-learn lesson. The same class, the same hyperparameters, the same

fit

method. MLflow did not get in the model's way by a millimetre, did not change its score and did not slow the computation down in any way you could notice. All we added were three calls describing what was going to happen anyway. Notice, too, the repeated numbers: the hundred and the ten each appear twice, once in

log_param

and once in the model constructor. That is the weakest point of manual logging, and two sections from now I will show you how to get rid of it.

The model is a result worth saving too

That leaves the third element of the promised trio. A model is a Python object, so it fits neither in a parameter nor in a metric - MLflow has separate functions for saving it, one per library. This split is called a flavor:

mlflow.sklearn

handles scikit-learn models,

mlflow.xgboost

handles XGBoost models,

mlflow.pytorch

handles PyTorch networks. You import the flavor module separately, next to

mlflow

itself. The

log_model

function takes the model and the name of the subdirectory it should be written into inside the run; the convention is to simply put "model" there.

1import mlflow.sklearn
2
3with mlflow.start_run(run_name="random_forest_v3"):
4    model = RandomForestClassifier(n_estimators=100, random_state=42)
5    model.fit(X_train, y_train)
6
7    mlflow.sklearn.log_model(model, "model")

After that call, a complete set of files landed in the

mlruns

directory next to the parameters and metrics: the serialized model, a list of the library versions needed to load it, and a description of the input it expects. Thanks to that, last month's winner can be loaded without retraining, and on a different machine at that. The second argument goes by different names: older documentation calls it

artifact_path

, newer documentation calls it

name

. Passing it positionally, as above, spares you the choice, and if you ever see a warning about a deprecated argument name, this is exactly the place it is talking about. The

model

object itself did not change in the slightest: you can still call

predict

on it, because

log_model

only read it.

A model is not the end of the list of things worth keeping. You can attach any file to a run - a feature importance chart, a CSV of misclassifications, a text note - and

mlflow.log_artifact

is what does it, given the path to a file that already exists. I am not showing it in a block of its own, because it would mean generating a chart first, but remember that the option is there: in MLflow's vocabulary an artifact is any file pinned to a run, and a model is simply a special case of one.

A complete run from start to finish

Let us now gather everything into one recipe, because this is precisely the shape you will be using in practice. The snippet is worth memorizing as a skeleton: you open a run, log the decisions, train, measure, log the results and save the model. There is one new thing in it - the

mlflow.log_params

function in the plural, which takes a dictionary and records all of its pairs at once. We then hand that same dictionary to the model constructor using the double star

**

, which unpacks it into keyword arguments, so no value is written down in two places any more.

1import mlflow
2import mlflow.sklearn
3from sklearn.ensemble import RandomForestClassifier
4from sklearn.metrics import accuracy_score
5
6params = {"n_estimators": 200, "max_depth": 8, "random_state": 42}
7
8mlflow.set_experiment("safari-classification")
9
10with mlflow.start_run(run_name="random_forest_final"):
11    mlflow.log_params(params)
12    mlflow.log_param("training_specimens", len(X_train))
13
14    model = RandomForestClassifier(**params)
15    model.fit(X_train, y_train)
16
17    accuracy = accuracy_score(y_test, model.predict(X_test))
18    mlflow.log_metric("accuracy", accuracy)
19
20    mlflow.sklearn.log_model(model, "model")

That is the full set: parameters, a metric and a model in one run, which is exactly the trio we started the lesson with. Notice that the

set_experiment

call moved us from the warm-up "safari-trials" to the real "safari-classification" - the earlier, incomplete runs stayed where they were and will not muddy the comparisons. I logged the number of training specimens as a parameter rather than a metric, and that is no accident: it is a fact about the input data, not the result of a quality measurement. Had it slipped in among the metrics, MLflow would offer you a chart of specimen count over time, which means nothing at all. Notice finally that the

params

dictionary was used twice but defined once, so it is no longer possible to log settings other than the ones actually used. The

with

block itself returned no value for later use and printed nothing - the entire trace of this execution went to disk.

mlflow ui - the browser for your experiments

The logbook exists, but so far you have only been looking at it through a keyhole, that is, through prints in the terminal. Time to unroll the camp map. You start the MLflow web interface with the

mlflow ui

command typed in the terminal, not in Python, in the same directory that holds the

mlruns

folder. That last part matters: the command reads the logbook from a relative path, so started one level up it will show you an empty table and you will be convinced nothing was ever saved.

1mlflow ui
2mlflow ui --port 5000

Both lines do the same thing, because 5000 is the default port - I am showing the second version so that you know what to change when the port is taken by another program. Once it starts, open the address printed in the terminal in your browser and you will see a table of runs: name, time, parameters and metrics in columns. You can sort by a metric, tick several runs and compare them on a chart. The server keeps going until you stop it, and interrupting it with a keyboard shortcut deletes nothing - the data sits in

mlruns

, the interface merely displays it.

Three other commands that people try to type at this point will not work, and it is worth knowing why.

python mlflow.py

has no chance of succeeding, because no such file exists in your project - and if you create one yourself so that there is "something to run", you will do yourself real harm: a file named

mlflow.py

in the working directory shadows the actual library and

import mlflow

stops working across the whole project.

mlflow start

ends with a message about an unknown command, because the

mlflow

program has no

start

subcommand; it has

ui

server

run

and

models

, among others.

run mlflow

is back to front - a command always begins with the name of the program, and besides,

mlflow run

exists to run an MLflow project from a directory or a repository, not to show you the interface.

Autolog - when you cannot be bothered to write log_param

Let us come back to that awkwardness with numbers repeated in two places. A scikit-learn model knows its own settings after all - they sit inside it as attributes - so copying them into the logbook by hand is work a machine ought to be doing. And it does. Calling

mlflow.sklearn.autolog()

makes MLflow log the parameters and metrics of scikit-learn models by itself, without a single

log_param

from you. You only have to call it once, before the training.

1import mlflow
2import mlflow.sklearn
3from sklearn.ensemble import RandomForestClassifier
4
5mlflow.sklearn.autolog()
6
7with mlflow.start_run(run_name="random_forest_auto"):
8    model = RandomForestClassifier(n_estimators=150, max_depth=12)
9    model.fit(X_train, y_train)

There is not one explicit logging call in that run, and yet the logbook will hold the forest's full set of hyperparameters, metrics computed on the training set, and the saved model. The mechanism is simple: autolog swaps the

fit

method for its own version, which first notes the settings, then calls the original, and finally records the results. That is exactly why it only works for libraries MLflow knows about. Notice that the training did not change one jot - it is still your

fit

call doing all the work, with autolog merely eavesdropping on it.

Since we are on the limits of this mechanism, let us spell out what it does not do. Autolog does not clean data: missing measurements and bad entries will stay in the table exactly where they were, and tidying up is the job of the Pandas methods you met with the field notebook. Autolog does not train models - if you deleted the line with

fit

, nothing but an empty run would appear in the logbook, because there would be nothing to eavesdrop on. Autolog does not create PDF reports, nor any other documents; you look at results in the web interface or pull them into your code, which is what comes next. It does one thing: it automatically logs the parameters and metrics of models from a given library.

The equivalents for other libraries are named predictably:

mlflow.xgboost.autolog()

for XGBoost and

mlflow.pytorch.autolog()

for PyTorch. There is also

mlflow.autolog()

with no flavor name, which turns automatic logging on for every recognized library at once. In practice a combination of both approaches works best: autolog takes the model hyperparameters off your hands, and you add by hand the things the library knows nothing about - the data version, the tracker's name, the number of the expedition.

Comparing runs without clicking

The web interface is comfortable for looking around, but when there are a hundred and fifty runs you want to ask your question in code rather than with your eyes. That is what

mlflow.search_runs

is for, and it returns a DataFrame, the one you know from the Pandas lesson - with everything you are able to do to one. First, though, you have to say which experiment you are asking about, and

mlflow.get_experiment_by_name

does that: it takes a name and returns an experiment object, from which we pull the

experiment_id

field. The search function expects a list of identifiers, because it lets you search several experiments at once, so we wrap our single identifier in square brackets.

1import mlflow
2
3experiment = mlflow.get_experiment_by_name("safari-classification")
4runs = mlflow.search_runs(experiment_ids=[experiment.experiment_id])
5
6print(runs.shape)
7print(runs[["params.n_estimators", "params.max_depth", "metrics.accuracy"]].head())

What you get is an ordinary table: one row per run, and in the columns everything you logged. The column names carry prefixes -

params.

for parameters and

metrics.

for metrics - because without them a parameter and a metric of the same name would collide in a single column. There is one trap here that everybody falls into: the values in the
params.
columns are strings, not numbers, because MLflow stores parameters as text. Sorting by

params.n_estimators

will therefore put "100" before "50", the way a dictionary would rather than the way mathematics would. The

metrics.

columns, on the other hand, are genuine floating point numbers and sort normally.

Since the metrics are numbers, finding the winner is a single

sort_values

call from the Pandas lesson, with

ascending=False

for descending order. We take the first row of the sorted table with

iloc[0]

, that is, by position, and out of it the

run_id

column - the same identifier we printed at the very first run. We load the model with the

load_model

function, passing an address in the format

runs:/identifier/subdirectory_name

. We assemble that address with an f-string, because the identifier is a variable.

1import mlflow.sklearn
2
3best = runs.sort_values("metrics.accuracy", ascending=False)
4winner_id = best.iloc[0]["run_id"]
5
6winner = mlflow.sklearn.load_model(f"runs:/{winner_id}/model")
7print(winner.predict(X_test)[:5])

The loaded model is an ordinary scikit-learn object and answers

predict

as if it had just walked out of training - and it could have been built a month ago on a different machine. That is the whole point of saving the model together with the metrics. Notice that

sort_values

did not change the

runs

table, it returned a sorted copy, exactly as in the Pandas lesson, and that the

model

segment of the address is the same subdirectory name we passed earlier as the second argument to

log_model

. Had you written "forest" there, the address would have to read

runs:/identifier/forest

And out of this comes a practical rule worth building your whole working discipline around: log the model in every run that stands any chance of winning. If a run in which you saved the metric alone - like our warm-up "random_forest_v2" - landed at the top of the sorted table, loading would end in an error about a missing artifact, because there is simply nothing at that address. That is exactly why we keep the trials in a separate experiment and let only complete runs into the real one.

Model Registry - one place for the production model

An address with a run identifier is precise but unwieldy: nobody memorizes thirty-two hexadecimal characters, and in application code a string like that looks like an accident. That is why MLflow has a model registry - a catalogue of named models with numbered versions. A model enters the registry at the moment of logging, if you add the

registered_model_name

argument. The first time round, version one is created, and every subsequent log under the same name adds version two, three and so on. Your previous models do not disappear anywhere in the process.

1with mlflow.start_run(run_name="registry_candidate"):
2    model = RandomForestClassifier(n_estimators=200, random_state=42)
3    model.fit(X_train, y_train)
4
5    mlflow.sklearn.log_model(
6        model, "model", registered_model_name="SafariClassifier"
7    )

From now on the model has a name a human can read and a version, and a separate registry tab shows up in the web interface. Nothing beyond that changed - the run looks like any other, you log parameters and metrics the same way, and the

model

object itself is still perfectly usable in the same script.

Loading from the registry looks familiar; only the prefix of the address changes:

models:

instead of

runs:

, then the model name, and after the slash the version number.

1registered = mlflow.sklearn.load_model("models:/SafariClassifier/1")
2print(registered.predict(X_test)[:5])

The one at the end is a version number, not a count of models - to reach for a newer one you write a two. Code that uses the model no longer has to know anything about runs or identifiers. In older material you will meet a form with a stage name in this place, for example

models:/SafariClassifier/Production

; that is a leftover from the stages mechanism, which MLflow declared obsolete and replaced with aliases - labels of your own along the lines of

@production

, which you pin to a chosen version and repin when the winner changes. The principle is the same: the application asks for a name, and you decide which version hides behind it.

A tuning loop with the logbook running in the background

To finish, let us join the logbook to the hyperparameter tuning from the previous lesson. A manual sweep of combinations is where MLflow shows its full worth: every combination gets a run of its own, so once the loop is done you have a ready-made table to compare instead of staring at a scrolling terminal. We will use

cross_val_score

, which you know from the supervised learning lesson - it splits the training data into five parts and returns five scores, one per split. Out of those five we log two numbers: the mean, which is the quality you can expect, and the standard deviation, which is how stable the result is across splits.

1import mlflow
2import mlflow.xgboost
3import xgboost as xgb
4from sklearn.model_selection import cross_val_score
5
6mlflow.set_experiment("safari-xgboost-tuning")
7
8combinations = [
9    {"max_depth": 3, "learning_rate": 0.1},
10    {"max_depth": 5, "learning_rate": 0.1},
11    {"max_depth": 3, "learning_rate": 0.05},
12    {"max_depth": 5, "learning_rate": 0.05},
13]
14
15for params in combinations:
16    with mlflow.start_run():
17        mlflow.log_params(params)
18
19        model = xgb.XGBClassifier(**params, n_estimators=100, random_state=42)
20        scores = cross_val_score(model, X_train, y_train, cv=5)
21
22        mlflow.log_metric("cv_mean", scores.mean())
23        mlflow.log_metric("cv_std", scores.std())
24
25        model.fit(X_train, y_train)
26        mlflow.log_metric("test_accuracy", model.score(X_test, y_test))
27        mlflow.xgboost.log_model(model, "model")

Four combinations, four runs, twelve metrics - and not one number written on a scrap of paper. I am not quoting the results in comments, because they depend on the XGBoost version and on the data, and besides, they are not the point here: the point is that once the loop is over,

mlflow.search_runs

from the previous section is all you need to see the four attempts side by side. Notice two things. First, the

with

sits inside the loop, not outside it. Were it the other way round, every combination would fall into a single run, and on the second turn of the loop

log_params

would stop the program - remember that a parameter may only be recorded once per run, and the second combination is trying to swap

max_depth

from a three to a five. Second, we save the XGBoost model with the

mlflow.xgboost

flavor rather than

mlflow.sklearn

, even though

XGBClassifier

does a fine impression of a scikit-learn classifier; every library has its own way of serializing and its own set of versions to remember.

Summary

MLflow is for tracking ML experiments: it records parameters, metrics and models. It does not train models, it is not for creating user interfaces and it does not replace unit tests.
A run is one training execution, an experiment is a named collection of runs. You open a run with
```
with mlflow.start_run():
```
- the version without
```
with
```
leaves the run open after every exception.
You log a parameter with six elements:
```
mlflow
```
,
```
.
```
,
```
log_param
```
,
```
(
```
,
```
"n_estimators", 100
```
,
```
)
```
- the name first, then the value.
```
log_params
```
takes many at once, as a dictionary.
A metric has to be a number and may be recorded many times within one run; a parameter may be anything, but only once.
You save a model with the flavor matching the library:
```
mlflow.sklearn.log_model
```
,
```
mlflow.xgboost.log_model
```
,
```
mlflow.pytorch.log_model
```
. Any other file is attached with
```
log_artifact
```
.
You start the web interface in the terminal with
```
mlflow ui
```
, in the directory holding the
```
mlruns
```
folder. Not
```
python mlflow.py
```
, not
```
mlflow start
```
, not
```
run mlflow
```
.
```
mlflow.sklearn.autolog()
```
automatically logs the parameters and metrics of scikit-learn models. It does not clean data, does not train models and does not generate PDF reports.
```
mlflow.search_runs
```
returns a DataFrame - the values in the
```
params.
```
columns are strings, those in
```
metrics.
```
are numbers.
The model registry gives a model a name and numbered versions:
```
registered_model_name
```
when saving, the address
```
models:/Name/1
```
when loading.

Three projects close this module: a classifier with a Pipeline and cross-validation, XGBoost with hyperparameter tuning, and a neural network in PyTorch. Do every one of them inside

with mlflow.start_run():

- it is ten extra minutes of work, and instead of three loose scripts you will be left with an expedition logbook you can come back to in six months. Because a model with no parameters written down is like a specimen with no label: beautiful in the display case, useless to science.

Go to CodeWorlds→