NumPy - the array that computes a whole column at once

You get back from a tour of five reserves, @name, and your notebook holds five numbers: 120, 450, 85, 200 and 330 head. The questions the expedition leader is about to ask are the same every season. What is the average per reserve? How much does each population grow if the increase comes to ten percent? Which herds have dropped below a hundred head? An ordinary Python list will store those five numbers without the slightest trouble, and it will answer none of those questions directly.

Before we reach for a new tool, it is worth seeing exactly where the old one gives out. The built-in function

sum

adds up the elements of a list and

len

counts how many there are, so one divided by the other gives you the average and nothing goes wrong there. Rescaling is another matter. Multiplying a list by a whole number does not multiply the values, it repeats the whole list end to end, and multiplying by a fraction does not go through at all - it stops on a

TypeError

.

1populations = [120, 450, 85, 200, 330]
2
3print(sum(populations) / len(populations))  # 237.0
4
5print(populations * 2)
6# [120, 450, 85, 200, 330, 120, 450, 85, 200, 330]
7
8# populations * 1.1
9# TypeError: can't multiply sequence by non-int of type 'float'

Three lines and three completely different behaviours. The average came out right because we computed it by hand, step by step. The second printout shows that for a list the multiplication sign means "repeat the contents", so instead of doubled head counts we got ten elements where there had been five. The third line is commented out for a good reason: had you run it, the program would have stopped on an error, because Python cannot repeat a list a fraction of a time. Notice what is not here - not a single number rescaled by ten percent. To get one you would have to write a loop, and with five thousand readings off GPS collars a Python loop turns painfully slow.

NumPy, short for Numerical Python, is the library that settles both problems at once. It brings a data type of its own - the array - in which every element is of the same kind and all of them sit side by side in memory. That is what lets an arithmetic operation run across the whole array in one go, with no loop, and the arithmetic itself is carried out by compiled C code rather than by the Python interpreter. NumPy is also the foundation the rest of a data scientist's kit stands on: Pandas, Matplotlib and the machine learning libraries all count in NumPy arrays underneath.

What you will learn

• How to install NumPy and bring it into a program under the alias the whole world uses.
• How to build arrays out of your own measurements, and how to order ready-made arrays filled with zeros, ones and evenly spaced numbers.
• How to read an array's ID card: its shape, its number of dimensions, its element count and its data type.
• How to compute whole-herd statistics in a single call: mean, median, standard deviation, minimum, maximum and sum.
• How to rescale every measurement at once, without a loop, and how to keep only the ones that meet a condition.
• How to cut pieces out of arrays, rearrange their shape and do matrix arithmetic.

Installing NumPy and the

np

alias

NumPy is not part of the Python standard library, so you have to install it first. That job belongs to

pip

, the Python package manager - the program that fetches a library off the network and puts it away in your environment. You type the command into a terminal, not into a file of code, and you run it once per environment rather than every time the script starts. If you work in Google Colab or in the Anaconda distribution, NumPy is already in place and you can skip this step entirely.

1pip install numpy

After that command the package is sitting on disk, but not one of your code files knows about it yet. Bringing the library into a program takes a separate instruction, this time in Python itself. The keyword

import

pulls the library into memory, and the keyword

as

gives it an alias, a second and shorter name for exactly the same thing. NumPy is shortened to the two letters

np

by convention, and that convention is so universal that every example in the documentation, every answer online and every task in this module assumes it.

1import numpy as np

That single line has four parts and their order is fixed: first the word

import

, then the full package name

numpy

, then the word

as

, and the alias

np

last. Commit that quartet to memory, because you will type it at the top of every analysis script you ever write. Nothing else happened here - no array was created and nothing was computed. The plain

import numpy

is valid too, but then every call has to spell out

numpy.array

, which gets tiring by the thirtieth call in a script. What will not work is

import np

: the package is called

numpy

, so Python stops with

ModuleNotFoundError: No module named 'np'

. You hand out the alias yourself, on this very line, and only from here on does it exist.

Your first array:

np.array

Now that the library is within reach, let us trade the notebook for an array. The function that does it is called

array

, and it takes a single argument: a list of values. What comes back is an object of class

ndarray

, short for n-dimensional array, an array with any number of dimensions. We will check that straight away with the built-in function

type

, which answers the question "what kind of object is this". Fix the name

np.array

in your head in exactly that form, because it is the one and only road from a list to an array.

1import numpy as np
2
3arr = np.array([1, 2, 3, 4, 5])
4
5print(arr)        # [1 2 3 4 5]
6print(type(arr))  # <class 'numpy.ndarray'>

The first thing that jumps out of the printout is the missing commas. A Python list prints as

[1, 2, 3, 4, 5]

, a NumPy array as

[1 2 3 4 5]

- the elements are separated by nothing but spaces. That is the quickest way to tell one from the other at a glance in the console. The second printout confirms that what you got is a new kind of object rather than an improved list. And something just as important: the list you passed inside the brackets did not change one bit. NumPy copied the values out of it into a structure of its own and left the original alone.

Several names float around the business of creating arrays that sound perfectly sensible and do not exist. There is no

np.list

, because a list is a Python type and not a NumPy one. There is no

np.create

and no

np.new

, even though other libraries do use names like that. There is no

numpy.new

either, and for two independent reasons at once: no such function is in the package, and after

import numpy as np

the name

numpy

is not defined in your program at all. You can check this instead of taking my word for it, with the function

hasattr

, which answers the question "does this object hold a name that sounds like this".

1print(hasattr(np, "array"))   # True
2print(hasattr(np, "list"))    # False
3print(hasattr(np, "create"))  # False
4print(hasattr(np, "new"))     # False

One truth and three falsehoods, in black and white. Calling

np.list([1, 2, 3])

ends with the message

AttributeError: module 'numpy' has no attribute 'list'

, and the other two variants end the same way. So remember one road only: an array is built from a list by

np.array

and by nothing else. The

hasattr

calls themselves built nothing and changed nothing along the way - they only reported what is in the library and what is not. Keep the square brackets on the inside in mind too:

np.array

takes one list, so

np.array(1, 2, 3)

fails with an error about too many arguments.

One array, one type

A NumPy array differs from a list in more than how it prints. A Python list can hold a number, a piece of text and a logical value side by side, whereas an array must have one shared type for all of its elements - and that requirement is exactly what buys it speed, because the computer knows in advance how many bytes each element takes. You read an array's type off the attribute

dtype

, short for data type. Let us see what happens when we drop a single fraction in among whole numbers.

1whole = np.array([1, 2, 3])
2mixed = np.array([1, 2, 3.5])
3
4print(whole.dtype)  # int64
5print(mixed.dtype)  # float64
6print(mixed)        # [1.  2.  3.5]

The first array got the type

int64

, whole numbers stored in sixty-four bits. The second got

float64

, floating point numbers, even though you typed two whole numbers into it and only one fraction. NumPy picked the type that holds every value without loss, and quietly turned the one and the two into

1.0

and

2.0

- you can see it in the last printout, in the dots with no digits after them. Your data was not damaged, but it did change form, so it is worth glancing at. One practical note: on some systems, mainly Windows, the default whole-number type is

int32

instead of

int64

. That is the same number in a smaller box and it changes nothing in your code.

This one-type rule looks innocent as long as you are counting measurements and nothing else. In the next lesson you will find that for a field notebook, where a head count sits next to a species name and a note on whether the animal is endangered, it is a hard limit - and that is precisely why the Pandas library exists.

The array's ID card:

shape

,

ndim

,

size

An array does not have to be a single row of numbers. Hand it a list of lists and you get a two-dimensional array, a matrix: the first inner list becomes the first row, the second becomes the second, and so on. Three attributes describe a structure like that. The attribute

shape

returns a tuple, an immutable sequence of values written in round brackets, in the order rows, columns. The attribute

ndim

says how many dimensions the array has, and

size

says how many elements there are altogether. All three are attributes rather than methods, so you write them without brackets.

1arr = np.array([[1, 2, 3], [4, 5, 6]])
2
3print(arr.shape)  # (2, 3)
4print(arr.ndim)   # 2
5print(arr.size)   # 6
6print(arr.dtype)  # int64

Four numbers that are easy to mix up, so let us read them slowly. The shape is

(2, 3)

, and it always reads the same way: two rows by three columns, never the other way round. You passed two inner lists of three numbers each, so there are two rows and three columns - the answer

(3, 2)

would describe an entirely different array, one with three rows and two columns. The number

6

is

size

, the total count of elements, and not the shape. Mind the brackets as well: the shape comes back in round brackets because it is a tuple, not in square ones, because it is not a list. The array itself is exactly the same after those four printouts as it was before them - reading an attribute changes nothing.

Since the shape is a tuple, you can address it by position, just as you would a list. The expression

arr.shape[0]

gives you the number of rows and

arr.shape[1]

the number of columns, and you will use the first of those endlessly to check how many observations your data set holds.

1print(type(arr.shape))  # <class 'tuple'>
2print(arr.shape[0])     # 2
3print(arr.shape[1])     # 3
4
5# arr.shape()
6# TypeError: 'tuple' object is not callable

The last two lines are commented out on purpose and are worth remembering as a warning. Adding brackets to

shape

ends in an error, because you are trying to call something that is already a finished tuple as if it were a function. That is the most common slip on first contact with NumPy: you call methods with brackets and read attributes without them. The first three printouts, meanwhile, confirm that

shape

really does hold an ordinary Python tuple, with everything that follows from it - you can unpack it into two variables or compare it with another tuple.

Arrays made to order:

zeros

and

ones

Not every array comes from measurements. Very often what you need is an empty sheet of a given size that you will fill with results later - a grid of three species by four quarters, say. The function

np.zeros

creates such an array filled with zeros, and its only argument is a tuple of dimensions, once again in the order rows, columns. That detail matters: the whole tuple

(3, 4)

is one argument and not two, which is why you see two pairs of brackets in the call.

1grid = np.zeros((3, 4))
2
3print(grid)
4# [[0. 0. 0. 0.]
5#  [0. 0. 0. 0.]
6#  [0. 0. 0. 0.]]
7
8print(grid.shape)  # (3, 4)
9print(grid.dtype)  # float64
10print(type(grid))  # <class 'numpy.ndarray'>

The result is a matrix of three rows and four columns, twelve zeros set out in a rectangle. Read the printout again and count: three lines of four zeros each. The reverse reading, a four by three matrix, would describe a different array, one with four lines of three zeros. Nor is this a list of three Python lists, even though the square brackets look similar - the last printout shows the class

numpy.ndarray

in black and white. And there is no error here either: passing a tuple as a single argument is exactly what

np.zeros

expects. Note the dots beside the zeros and the type

float64

as well: the zeros are floating point, because that is the default type for these functions.

The twin function

np.ones

works identically but fills the array with ones, and it comes in handy when you are building multipliers or weights. Standing apart from both is

np.eye

, which creates an identity matrix: a square array with ones down the diagonal and zeros everywhere else. That last one takes a single number, because an identity matrix has as many rows as columns by definition.

1print(np.ones((2, 3)))
2# [[1. 1. 1.]
3#  [1. 1. 1.]]
4
5print(np.eye(3))
6# [[1. 0. 0.]
7#  [0. 1. 0.]
8#  [0. 0. 1.]]

Two rows of three ones, and a three by three square with a diagonal of ones - exactly what the prose promised. The identity matrix plays the same role in algebra that the number one plays in ordinary multiplication: multiplying by it changes nothing, which is why it serves as the starting point when you invert a matrix. Notice that

np.eye

takes a bare number and no tuple, because the second dimension follows from the first. None of these functions needs your data at all - they all build an array from nothing but a size.

Runs of numbers:

arange

and

linspace

To describe a time axis or the numbers of successive observations you need a rising run of numbers rather than zeros. NumPy offers two functions for that and the difference between them is the source of a great many mistakes. The function

np.arange

behaves like the built-in

range

: you give it a start, a stop and a step, and it fills the array in increments of whatever you asked for, leaving the stop out. The function

np.linspace

asks for something else: a start, a stop and a number of points, and it works out the spacing itself so the interval divides evenly, with the stop included in the result.

1print(np.arange(0, 10, 2))     # [0 2 4 6 8]
2print(np.linspace(0, 1, 5))    # [0.   0.25 0.5  0.75 1.  ]

The first array has five elements and finishes at eight, because ten as the stop does not make it into the result. The second also has five elements, but the last one is exactly one, because

linspace

treats the stop as a target value rather than a boundary. The third argument means something completely different in the two functions: in

arange

it is the gap between values, in

linspace

it is the count of values. My advice, @name: when you know the step and want to number your measurements, take

arange

, and when you need exactly a hundred points spread evenly from zero to one, take

linspace

- otherwise you end up dividing the interval by hand and collecting a rounding error at the far end.

Made-up data:

np.random.rand

Before the real collar readings come in, a made-up data set you can test the whole script against is worth having. Random numbers come from the module

np.random

, and the simplest of its functions is

rand

, which returns values between zero and one. Its dimensions are passed as separate arguments rather than as a tuple - an exception to the rule you just met with

np.zeros

, and an easy one to trip over. So that the results can be reproduced, we set the generator's seed beforehand with

np.random.seed

, and to shorten the printout we use the method

round

, which rounds every value in the array at once.

1np.random.seed(42)
2sample = np.random.rand(2, 3)
3
4print(sample.round(2))
5# [[0.37 0.95 0.73]
6#  [0.6  0.16 0.16]]

Six random numbers laid out in two rows of three columns, every one of them smaller than one. Thanks to the seed set to forty-two you will get precisely these values on your own machine - that is not magic, just a repeatable generator, and repeatability is the entire point when you are testing. Without the

seed

call the numbers would differ on every run and you would have no way to reproduce last week's result. Notice that

round

broke nothing inside

sample

- it returned a new, rounded array while the original still holds the full expansions. For new projects I recommend the newer form

rng = np.random.default_rng(42)

, because it keeps the generator state in a separate object instead of globally, but you have to know the

seed

version, because you will meet it in thousands of existing scripts.

Statistics in a single call

Back to the notebook from the first page of this lesson. Five head counts, one array and a set of questions that need answering. NumPy has a separate function for each of them.

np.mean

computes the arithmetic mean,

np.median

the median, which is the middle value once the numbers are put in order, and

np.std

the standard deviation, which says how widely the measurements scatter around the mean. Three obvious ones join them:

np.min

,

np.max

and

np.sum

. Each takes an array and returns a single number.

1populations = np.array([120, 450, 85, 200, 330])
2
3print(np.mean(populations))    # 237.0
4print(np.median(populations))  # 200.0
5print(np.std(populations))     # 135.77923257994942
6print(np.min(populations))     # 85
7print(np.max(populations))     # 450
8print(np.sum(populations))     # 1185

Six numbers that describe the whole herd better than the raw list does. The mean is two hundred and thirty-seven while the median is two hundred - the gap comes from the elephants' four hundred and fifty pulling the mean upward while leaving the median untouched, because the median only looks at what stands in the middle of the row. A standard deviation of roughly a hundred and thirty-six against a mean of two hundred and thirty-seven is a signal that the reserves differ enormously from one another. One note for the record:

np.std

computes the population deviation by default, dividing by the number of measurements, while Pandas in the same place computes the sample deviation, dividing by that number minus one. For these same five numbers that gives 151.81 instead of 135.78, so do not be surprised when you see two different values. The array

populations

still holds the same five numbers after those six printouts.

The mean call is worth taking apart piece by piece, because you will write it more often than any other. It consists of six elements in a fixed order:

np

,

.

,

mean

,

(

,

arr

,

)

. First the library alias, then the dot, then the function name, the opening bracket, the array name and the closing bracket. There is a second form as well, in which the mean is a method on the array itself.

1arr = np.array([1, 2, 3, 4, 5])
2
3print(np.mean(arr))  # 3.0
4print(arr.mean())    # 3.0

The same result by two roads, and that is no coincidence - it is literally the same operation written two ways. The form

np.mean(arr)

has one advantage I recommend you exploit: it works on a plain Python list too, so

np.mean([1, 2, 3, 4, 5])

also returns three, whereas

[1, 2, 3, 4, 5].mean()

ends in an error because a list has no such method. When you are writing a script that receives data from several places, that robustness can be priceless. Neither call changes the array - computing a statistic is a pure read.

Vectorization: one operation across the whole array

Now back to the question the list stumbled on: how far the populations grow at a ten percent increase. In NumPy you write it exactly the way it sounds in English - you multiply the array by 1.1. That style of writing is called vectorization: a single number is stretched across every element of the array and the operation runs separately for each one, with no loop anywhere. NumPy calls the underlying mechanism broadcasting.

1growth = populations * 1.1
2
3print(growth)      # [132.  495.   93.5 220.  363. ]
4print(populations) # [120 450  85 200 330]
5print(growth.dtype)  # float64

Five new head counts out of one line of code, with no loop and no indices. The second printout matters most here:

populations

still holds the original measurements, because an arithmetic operation returns a new array rather than correcting the old one. The third printout shows the side effect of multiplying by a fraction - the result moved to the type

float64

, because ninety-three and a half does not fit in whole numbers. Biologically half a cheetah is nonsense, but mathematically that is simply how multiplication behaves, and

round

is there for the rounding. The other operations behave the same way:

populations + 10

adds ten to every value and

populations / 2

halves each of them.

Vectorization will handle an expression made of several steps at once as well. The classic example is normalization, bringing measurements onto a common scale: you subtract the mean from every value and divide by the standard deviation. The result says how many deviations a given population lies above or below the average, which makes it comparable across completely different quantities - a herd size and a rainfall figure in millimetres.

1normalized = (populations - np.mean(populations)) / np.std(populations)
2
3print(normalized.round(2))  # [-0.86  1.57 -1.12 -0.27  0.68]
4print(np.mean(normalized).round(10))  # 0.0
5print(np.std(normalized).round(10))   # 1.0

Five numbers again, this time on both sides of zero. The negative values are the reserves below average and the positive ones above it, and the elephants at 1.57 sit one and a half deviations over the mean. The last two printouts confirm that the transformation did what its definition promises: a normalized set has a mean of zero and a deviation of one. The whole operation is three calls and one division applied to the entire array at once - in plain Python it would have taken a loop and two passes over the data. The original

populations

came through untouched once more, because each of these operations built a new array alongside it.

Indexing and slices

You reach single values exactly as you would in a Python list: by giving a position in square brackets, counting from zero. A negative index counts from the end, so minus one is the last element. A slice is written as two numbers separated by a colon: the first is the starting position, the second is the position the slice stops before - the end of the range does not make it into the result. Leaving out the number before the colon means "from the beginning" and leaving out the one after it means "to the end". A third number, after a second colon, is the step.

1arr = np.array([10, 20, 30, 40, 50])
2
3print(arr[0])     # 10
4print(arr[-1])    # 50
5print(arr[1:4])   # [20 30 40]
6print(arr[:3])    # [10 20 30]
7print(arr[::2])   # [10 30 50]

Five expressions and five different answers out of the same array. The slice

arr[1:4]

gives three elements rather than four, because position four is the boundary and does not come along - a rule that confuses everybody at first. The expression

arr[:3]

takes positions zero, one and two, the first three values. The last one,

arr[::2]

, walks the whole array taking every second element, which is why it returns ten, thirty and fifty. It is worth noting that all of these printouts are NumPy arrays and not lists, so you can compute a mean on them right away.

There is one difference from Python lists, though, that can cost you an hour of bug hunting. A list slice creates a copy, while a NumPy slice creates a view, a window onto the very same data in memory. Changing a value in the view changes the original. Let us check that on a separate array, so the effect leaves no room for doubt.

1readings = np.array([10, 20, 30, 40, 50])
2window = readings[1:4]
3window[0] = 999
4
5print(readings)  # [ 10 999  30  40  50]
6
7safe = readings[1:4].copy()
8safe[0] = 111
9
10print(readings)  # [ 10 999  30  40  50]
11print(safe)      # [111  30  40]

The first part of the code spoils the data silently: an assignment into

window

replaced a value inside

readings

, even though nobody touched that array directly. The second part shows the cure - the method

copy

makes an independent copy of the slice, and from that moment the two arrays live separate lives, which you can see from

readings

not budging after

111

was written. My advice here is unambiguous, @name: if you intend to write anything into a slice, add

copy

straight away, without stopping to work out whether a view would do harm in this particular spot. For reading only, the view is better, because it copies no memory and works instantly.

Boolean masks: picking by condition

The question about herds below a hundred head is solved in NumPy in two steps, and both are worth seeing separately. Comparing an array with a number does not return a single

True

and be done with it. It performs the comparison for every element on its own and hands back an array of logical values of the same length. An array like that is called a boolean mask. Put back inside square brackets, it selects exactly those elements whose mask entry is

True

.

1print(populations < 100)
2# [False False  True False False]
3
4print(populations[populations < 100])  # [85]
5print(populations)  # [120 450  85 200 330]

The first printout is the decision and nothing else: five yes-or-no answers in the same order as the measurements, and not a single head count - a mask stores no data, only a verdict on each element. The second printout applies it and leaves the one reserve below the threshold, the cheetahs' eighty-five. The third printout says the most important thing:

populations

still holds all five values, because filtering returned a new array. You will meet this same mechanism, literally the same one, in the next lesson when we filter Pandas tables - so it is worth understanding now, on five numbers, rather than later on three thousand rows.

Two-dimensional arrays up close

In a matrix a single position is not enough, because you have to name a row and a column. You give both inside one pair of square brackets, separated by a comma, always in the order row, column. A bare colon in place of either one means "all of them", which is why

[:, 0]

reads as "every row, column zero", that is, the whole first column.

1matrix = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
2
3print(matrix[0, 1])  # 2
4print(matrix[:, 0])  # [1 4 7]
5print(matrix[1, :])  # [4 5 6]

Three queries and three differently shaped answers. The first returns a single number, because you named exactly one cell: row zero, column one, which holds a two. The second returns an array of three numbers gathered from top to bottom - it is a column, so what you see in the result are values from three different rows. The third returns the whole middle row. The form

matrix[0][1]

works too, since it pulls out the row first and then an element from it, but I recommend the comma version: it is shorter, it builds no intermediate array on the way and it lets you slice both dimensions at once, as in

matrix[0:2, 1:3]

.

Changing shape:

reshape

,

flatten

and

T

The same numbers can be laid out in different ways, and NumPy lets you do that without rewriting the data. The method

reshape

rearranges an array into a given shape, on one condition: the product of the new dimensions has to match the number of elements. The method

flatten

does the opposite - it flattens any array into a single row. The attribute

T

is the transpose, the swap of rows for columns, and like

shape

we write it without brackets.

1arr = np.arange(12)
2matrix = arr.reshape(3, 4)
3
4print(matrix)
5# [[ 0  1  2  3]
6#  [ 4  5  6  7]
7#  [ 8  9 10 11]]
8
9print(matrix.flatten())  # [ 0  1  2  3  4  5  6  7  8  9 10 11]
10print(matrix.T.shape)    # (4, 3)

Twelve numbers from zero to eleven arranged themselves into three rows of four, filled row by row from left to right. Flattening returned them to a single run in the same order, and the transpose flipped the shape from

(3, 4)

to

(4, 3)

- another chance to remind yourself that the first number in a shape is always the rows. A call to

arr.reshape(5, 4)

would end with

ValueError: cannot reshape array of size 12 into shape (5,4)

, because five times four is twenty and there are twelve elements. There is a convenient shortcut for this: put minus one in place of one of the dimensions and NumPy will work it out for you, so

arr.reshape(3, -1)

gives the same shape

(3, 4)

. The array

arr

itself is still a flat run of twelve numbers after all of this.

Matrix operations

With two matrices comes a distinction that is fundamental in mathematics and comes down to a single character in code. Addition and multiplication written with the ordinary signs work element by element: NumPy pairs up the cells sitting at the same positions. Matrix multiplication, the one from linear algebra, is written with the operator

@

or the function

np.dot

, and it computes something else entirely: every element of the result comes from multiplying a row by a column and adding the products up.

1A = np.array([[1, 2], [3, 4]])
2B = np.array([[5, 6], [7, 8]])
3
4print(A + B)
5# [[ 6  8]
6#  [10 12]]
7
8print(A * B)
9# [[ 5 12]
10#  [21 32]]
11
12print(A @ B)
13# [[19 22]
14#  [43 50]]

Compare the second result with the third, because this is one of the most common traps in machine learning code. Multiplication with the star gave five in the top left corner, one times five - a plain pairing of cells. Multiplication with

@

gave nineteen in that same corner, because it computed one times five plus two times seven. Both are correct operations, they simply answer different questions, and the program will raise no error if you pick the one you did not mean. Remember it this way: the star acts on pairs of cells,

@

multiplies matrices. The function

np.dot(A, B)

returns exactly what

A @ B

returns, but the operator reads better and it is the one I recommend in new code.

Linear algebra: the

linalg

module

The heavier algebra tools live in the submodule

np.linalg

, short for linear algebra. The function

det

computes the determinant of a matrix, a single number that tells you, among other things, whether the matrix can be inverted. The function

inv

returns the inverse matrix, the one that gives the identity matrix - the one with ones down the diagonal - when multiplied by the original. A determinant equal to zero means there is no inverse, so in practice you check it before inverting.

1A = np.array([[1, 2], [3, 4]])
2
3print(np.linalg.det(A))         # -2.0000000000000004
4print(round(np.linalg.det(A), 2))  # -2.0
5print(np.linalg.inv(A))
6# [[-2.   1. ]
7#  [ 1.5 -0.5]]

The first printout is a fine lesson in humility towards the computer: the determinant of this matrix is exactly minus two, and NumPy shows minus two and a shade more. That is not a bug in the library but a consequence of writing fractions in binary, where many numbers have no exact counterpart. This is why floating point results are never compared with an equals sign but rounded, or checked against a tolerance - the second printout shows the simplest way. The inverse came out exactly as the textbook says, and

A

after both calls is still the same matrix we typed in. The same module also houses

np.linalg.eig

, which returns a pair of results at once: the eigenvalues and the eigenvectors of a matrix. You will meet it when you reach dimensionality reduction in machine learning.

Why this is fast

It is worth understanding at the end where NumPy's advantage comes from, because that explains all of its quirks. A Python list stores pointers to objects scattered around memory, and each of those objects carries its own type information. A NumPy array keeps raw numbers side by side in one continuous block of memory, and describes the type once for the whole array - that is the

dtype

we were looking at earlier. Thanks to that, the loop over elements runs in compiled C code, and the processor can chew through several numbers in a single tick. Hence the one-type requirement, hence the rigid shape, hence views instead of copies on slices: every one of those rules buys speed. At five numbers you will not notice the difference; at five million it is the difference between a second and a quarter of an hour.

Recap

• You bring the library in with four words in a fixed order:

  import

  ,

  numpy

  ,

  as

  ,

  np

  .
• An array is built from a list by

  np.array

  and nothing else - the functions

  np.list

  ,

  np.create

  and

  np.new

  do not exist in NumPy.

• np.zeros((3, 4))

  returns an

  ndarray

  of three rows and four columns filled with zeros. The tuple of dimensions is a single argument, so there is no error here.
• You read the shape off the attribute

  shape

  , without brackets, and the result is a tuple in the order rows, columns: for

  np.array([[1,2,3], [4,5,6]])

  it is

  (2, 3)

  . The number

  6

  is

  size

  , not the shape.
• You compute the mean with six pieces:

  np

  ,

  .

  ,

  mean

  ,

  (

  ,

  arr

  ,

  )

  . Standing beside it are

  median

  ,

  std

  ,

  min

  ,

  max

  and

  sum

  .
• An arithmetic operation on an array runs across all elements at once and returns a new array, leaving the original untouched.
• A boolean mask,

  populations < 100

  for instance, returns an array of

  True

  and

  False

  values, and put inside square brackets it filters the data.
• A slice of an array is a view onto the same data rather than a copy - when you intend to write into it, add

  copy

  .
• The star multiplies element by element and

  @

  performs matrix multiplication. Those are two different operations, not two spellings of the same one.

In the next lesson we will add column names and mixed types in one table to these numbers, because a field notebook is more than a column of measurements. For now remember one thing, @name: a NumPy array is the tracker's carabiner - one type, one shape, no ornaments, and the whole herd recomputed in a single pull.