Scikit-learn - the species tracker

Imagine, @name, that you come back from an expedition with a notebook full of measurements: fang length, wingspan, body weight. You want the computer to work out on its own which animal each row describes. That is exactly the job of classic machine learning - and its most popular tool in Python is scikit-learn.

1pip install scikit-learn

One install gives you dozens of ready-made models plus everything you need to prepare data for them. Nothing on this savannah has to be built from scratch.

One ritual, many models

Before you meet any particular model, remember the single most important thing in this lesson. Every model in scikit-learn - whether it is a tree, a forest, a border on a map or a circle of neighbors - speaks the same three verbs:

• model.fit(X_train, y_train)

  - learn: show the model the measurements

  X_train

  and the correct answers

  y_train

  .

• model.predict(X_test)

  - recognize: for new measurements, return the predicted species.

• model.score(X_test, y_test)

  - grade: check what share of the guesses were right.

In code the whole ritual is three lines, and those three lines never change:

1model.fit(X_train, y_train)
2predictions = model.predict(X_test)
3accuracy = model.score(X_test, y_test)

Think of it as the common language of trackers: learn it once and every new model is just another tracker speaking the same tongue. The models below differ in how they think, but you operate them identically - the only line that ever changes is the one that creates the model.

Decision tree - the tracker who asks questions

The simplest tracker is the one who fires off a series of questions: "Does it weigh over 100 kg? Does it have a mane?". That is how a decision tree works - it splits the animals with questions until a single species is left standing.

1from sklearn.tree import DecisionTreeClassifier
2from sklearn.datasets import load_iris
3
4iris = load_iris()
5X, y = iris.data, iris.target
6
7tree = DecisionTreeClassifier(max_depth=3, random_state=42)
8tree.fit(X_train, y_train)
9
10accuracy = tree.score(X_test, y_test)

Notice that two of our three verbs are already here:

fit

teaches,

score

grades. The

max_depth=3

parameter caps how long the chain of questions can get - the deeper the tree, the more tightly it hugs the training data, and the more easily it overfits and gets lost on animals it has never seen. That is the first real machine learning trade-off you will feel in practice.

The great advantage of a tree is that you can literally see it - you can draw the questions it asks:

1from sklearn.tree import plot_tree
2import matplotlib.pyplot as plt
3
4plt.figure(figsize=(20, 10))
5plot_tree(tree, feature_names=iris.feature_names, class_names=iris.target_names, filled=True)
6plt.show()

That picture is your best friend when you have to explain a model's decision to somebody else - every branch is one question the model asked, and every leaf is a verdict. Very few models let you look inside this easily, so enjoy it while you can.

Random Forest - a council of many trackers

A single tree can be temperamental. Swap a handful of animals in the training set and it starts asking completely different questions. So what if you asked a hundred trackers and took the majority answer? That is exactly what Random Forest is: an ensemble of many decision trees, each grown on a slightly different slice of the data, all voting on the final species.

1from sklearn.ensemble import RandomForestClassifier
2
3forest = RandomForestClassifier(
4    n_estimators=100,    # number of trees in the council
5    max_depth=10,        # maximum depth of each tree
6    min_samples_split=5, # min samples needed to split a node
7    random_state=42
8)
9forest.fit(X_train, y_train)
10
11importance = forest.feature_importances_
12for name, imp in zip(feature_names, importance):
13    print(f"{name}: {imp:.4f}")

n_estimators=100

is the number of trackers sitting on the council - more trees usually means a steadier answer, at the cost of time. Keep the definition straight: a forest is not a single decision tree, because averaging away the whims of one tree is the entire point. It is not a neural network either, and it is not a clustering algorithm - clustering groups animals that carry no labels at all, while a forest needs

y_train

with the correct species to learn from. The forest also hands you a gift a lone tree gives far less reliably:

feature_importances_

tells you which measurements helped most in recognizing the species. Remember that property - in practice it is what tells you which measurements are worth taking at all.

SVM - the tracker who draws the border

A different way of thinking: instead of asking questions, draw a border across the map of measurements that separates the species. SVM (Support Vector Machine) hunts for the optimal hyperplane - the border that leaves the widest possible margin between the groups.

1from sklearn.svm import SVC
2
3svm = SVC(
4    kernel='rbf',        # shape of the border: linear, poly, rbf, sigmoid
5    C=1.0,               # regularization strength
6    gamma='scale',
7    probability=True     # lets you ask for probabilities
8)
9svm.fit(X_train, y_train)
10
11proba = svm.predict_proba(X_test)

The

kernel

parameter decides whether the border is straight (

linear

) or free to bend around the herds (

rbf

). Turning on

probability=True

buys you something valuable:

predict_proba

returns not just a species but the model's confidence - "80 percent lion, 20 percent cheetah", say. That is handy when you would rather reject an uncertain sighting than guess. And do not file SVM away as a regression tool only: an

SVR

variant exists for numbers, but the classic job of an SVM is classification by finding the optimal hyperplane. It has nothing to do with generating text or compressing images - all it ever does is draw borders in the space of your measurements.

KNN - the tracker who looks at the neighbors

The most intuitive tracker barely learns at all. Given a new animal, KNN (K-Nearest Neighbors) simply looks at which known specimens it resembles most and adopts their species. It classifies based on the k nearest neighbors - no gradient descent, no trees being grown, no neural network anywhere in sight.

1from sklearn.neighbors import KNeighborsClassifier
2
3knn = KNeighborsClassifier(
4    n_neighbors=5,       # how many neighbors we poll
5    weights='uniform',   # uniform or distance
6    metric='euclidean'
7)
8knn.fit(X_train, y_train)
9predictions = knn.predict(X_test)

n_neighbors=5

means "look at the 5 closest known animals and go with the majority". Training here is little more than memorizing the notebook, which is why

fit

is almost instant and

predict

is the slow part. Watch out for one trap: KNN measures distances, so if one column is in tons and another in millimeters, the first one will bulldoze the result on its own. That is why scaling comes next in this lesson - for KNN it is not optional.

Regression - when the answer is a number

So far we have been guessing the species, a category. But what if you want to predict a number - the weight of an animal from its dimensions? That is regression, and it leans on exactly the same three verbs.

1from sklearn.linear_model import LinearRegression, Ridge, Lasso
2
3linear = LinearRegression()
4linear.fit(X_train, y_train)
5
6ridge = Ridge(alpha=1.0)    # penalty on oversized coefficients (L2)
7ridge.fit(X_train, y_train)
8
9lasso = Lasso(alpha=0.1)    # can zero out useless features (L1)
10lasso.fit(X_train, y_train)
11
12print(f"Coefficients: {linear.coef_}")
13print(f"Intercept: {linear.intercept_}")

LinearRegression

looks for a straight relationship between the measurements and the result.

Ridge

and

Lasso

are its more cautious cousins - both add a penalty for going overboard, so the model cannot cling too desperately to the training data. The difference:

Lasso

can drive a useless feature's coefficient all the way to zero, which makes it a selection tool as much as a model - it tells you which measurements to keep in the notebook.

When the relationship is anything but straight, the same forest you met in classification has a version for numbers:

1from sklearn.ensemble import RandomForestRegressor
2
3rf_reg = RandomForestRegressor(n_estimators=100, random_state=42)
4rf_reg.fit(X_train, y_train)
5predictions = rf_reg.predict(X_test)

Notice how the code is a twin of the classifier - the only thing that changed is the end of the name,

Regressor

instead of

Classifier

. That is the common language again: master one model and you recognize the next one on sight, without reading a single page of documentation.

Scaling - level the units before training

Back to the trap we hit with KNN. Any model that measures distances gets lost when the features live on wildly different scales. Before you train anything, raw field notes go through a fixed order of operations: first you load the raw data, then you handle missing values (NaN), then you encode categorical variables, and only at the very end do you standardize numerical features. The order is not negotiable - a scaler cannot average a column that still has holes in it, and it cannot touch a column that is still a species name rather than a number.

1from sklearn.preprocessing import StandardScaler, MinMaxScaler, LabelEncoder, OneHotEncoder
2
3scaler = StandardScaler()          # mean 0, standard deviation 1
4X_scaled = scaler.fit_transform(X_train)
5X_test_scaled = scaler.transform(X_test)
6
7minmax = MinMaxScaler()            # squeeze into the 0-1 range
8X_normalized = minmax.fit_transform(X_train)
9
10le = LabelEncoder()                # lion / cheetah -> 0 / 1 / 2
11y_encoded = le.fit_transform(['lion', 'elephant', 'cheetah'])
12
13ohe = OneHotEncoder(sparse_output=False)  # categories -> 0/1 columns
14species_onehot = ohe.fit_transform(species_array.reshape(-1, 1))

There is one rule here that is easy to break and that quietly ruins your results: you fit the scaler on the training data only, with

scaler.fit_transform(X_train)

, and you merely transform the test data with

transform

. Otherwise the model gets a peek at the test set and your score comes out flattered. The two encoders at the bottom turn species names into numbers, because models understand nothing else -

LabelEncoder

for the answers,

OneHotEncoder

for categorical features.

Pipeline - pack the whole ritual into one object

Since every model needs its data scaled first, mistakes are cheap to make: scale the training set one way and the test set another, or forget the step entirely on a tired evening. A Pipeline glues the preparation and the model into a single object that keeps the order straight for you.

1from sklearn.pipeline import Pipeline
2from sklearn.preprocessing import StandardScaler
3from sklearn.ensemble import RandomForestClassifier
4
5pipeline = Pipeline([
6    ('scaler', StandardScaler()),
7    ('classifier', RandomForestClassifier(n_estimators=100))
8])
9
10pipeline.fit(X_train, y_train)
11predictions = pipeline.predict(X_test)

Read that in order and you have the recipe: import Pipeline, define the steps as

(scaler, classifier)

pairs, create the Pipeline from that list, then call fit() and finally call predict(). And here is the best part: the

pipeline

speaks our three verbs too - one

fit

scales and trains the whole chain, while

predict

automatically scales new data with the very same scaler. The scaling trap disappears on its own, because doing it in the wrong order is no longer possible.

Take one thing away from this lesson: in scikit-learn you are not learning ten different tools, you are learning one shared language -

fit

,

predict

,

score

- that every model speaks. Everything else is just picking the tracker who suits the hunt.