Tutorial

Types, Shapes & Tensors

Alrighty, let's get started.

Can't wait!

What is 5?

Is 5 an int?

Excellent! Do you know other ints?

Hmm... 0 and -42?

They are ints too.

What about -42.42?

-42.42 is a float, different from int.

How?

Machines use different formats to store int values and float values. They may behave differently under the same mathematical operations.*
So they are different types. A type describes a set of values that share the same behavior.

Interesting. Are there values of other types?

Here's one: [1, 2, 3].

It groups three ints together! What do we call it?

We call [1, 2, 3] a List[int].

So List[t] is a type as long as t is a type.
Then there must be a type called List[List[int]], right?

Yes, Lists can be nested, such as [[1, 2, 3], [4, 5, 6]].

What about [[1, 2, 3], [4, 5]]?

That's also a List[List[int]], since its first element is a List[int]...

... because all elements inside that element are ints.
Then its second element is also a List[int], since all elements there are ints.
Phew... that's a lot of reasoning.

Fortunately, machines can do that reasoning for us, rigorously and efficiently.
Now there is a stricter way to group things.

Go for it!

Tensor([[1, 2, 3], [4, 5, 6]])

It just wraps this Tensor thing around the list.** How is it different?

It has the type Tensor[int][[2, 3]]. A Tensor knows two things about its elements: their type and their shape.

[2, 3] is a List[int] that describes the shape of Tensor([[1, 2, 3], [4, 5, 6]]), since the outer layer contains two List[int]s and each inner layer contains three ints?

Not quite, but close.
Tensor([[1, 2, 3], [4, 5, 6]]) is the shorter way to write Tensor([Tensor([1, 2, 3]), Tensor([4, 5, 6])]).
It contains two elements; both have type Tensor[int][[3]].
Each element also contains three elements; all have type Tensor[int][[]].

I see.
So int and Tensor[int][[]] are the same type.
Tensor[int][[2, 3]] and Tensor[Tensor[int][[3]]][[2]] are also the same type.

By convention, values of int or float are called scalars.

Can we make [[1, 2, 3], [4, 5]] a Tensor?

Good question. What's its shape?

Hmm... I don't know how to describe its shape.

Neither do I. We cannot make [[1, 2, 3], [4, 5]] a Tensor, since it does not have a shape!

So Tensors are special because of shapes!

Yes. Tensor[t][s] is a type as long as t is a type and s is a List[int].
With shapes, Tensors describe their values more accurately than Lists do.
This precision enables many cool things.

Such as?

Running programs efficiently in parallel.
Since all elements in a tensor share the same shape, it is easier to run commands on all of them together.

Sounds cool! Let's see an example!

We will, in the next chapter. Now it's time to take a break.

See you there!

* In this tutorial, we use int for 64-bit integers and float for 64-bit floating-point numbers for simplicity. In practice, people prefer smaller and more efficient variants, such as float32 or bfloat16, which are available in PyPie.
** Tensor is available through from pypie import Tensor.