Skip to content

PyTorch Fundamentals

Exercise 1

A big part of deep learning (and learning to code in general) is getting familiar with the documentation of a certain framework you're using. We'll be using the PyTorch documentation a lot throughout the rest of this course. So I'd recommend spending 10-minutes reading the following (it's okay if you don't get some things for now, the focus is not yet full understanding, it's awareness):

  • The tensor datatype has a lot of methods and attributes available, will go back when needing more information
  • Utilizing CUDA threads is a very good way to speed up the software, instead of running all operations on the CPU. On my current machine (MacBook Air M2) I do not have CUDA cores available, but will dive into this when I have my desktop PC running again from my office.

Exercise 2

Create a random tensor with shape (7, 7).

Create this random tensor with torch.rand():

1
2
3
def e2():
    random_tensor = torch.rand(7, 7)
    print("Shape: ", random_tensor.shape)

Exercise 3

Perform matrix multiplication on the tensor from Exercise 2 with another random tensor with shape (1, 7)

To perform matrix multiplication we do this with torch.matmul. But we have to transpose the second tensor since the number of rows do not align with the number or columns in the first tensor. This is done with the transpose() before making the matrix multiplication:

1
2
3
4
5
6
7
8
def e3():
    rt1 = torch.rand(7, 7)
    rt2 = torch.rand(1, 7)
    rt2_t = rt2.transpose(0, 1)
    mat_multi = torch.matmul(rt1, rt2_t)

    print(f"Matrix multiplication {mat_multi=}",
          f"With shape {mat_multi.shape=}")

Exercise 4

Set the random seed to 0 and do exercises 2 and 3 again

We are setting the random seed on the CPU with torch.random.manual_seed(). Then we perform the same as in exercises 2 and 3: 

1
2
3
4
5
6
7
8
def e4():
    torch.random.manual_seed(0)
    rt1 = torch.rand(size=(7, 7))
    rt2 = torch.rand(size=(1, 7))
    rt2_t = rt2.transpose(0, 1)
    mat_multi = torch.matmul(rt1, rt2_t)
    print(f"Matrix multiplication {mat_multi=}",
          f"With shape {mat_multi.shape=}")

Exercise 5

Speaking of random seeds, we saw how to set it with torch.manual_seed() but is there a GPU equivalent? (hint: you'll need to look into the documentation for torch.cuda for this one)

Since I do not have CUDA cores available we skip this exercise for now. But you set the random seed on one CUDA core with torch.cuda.manual_seed()

1
2
def e5():
    print(f"CUDA: {torch.cuda.is_available()=}")

Exercise 6

Create two random tensors of shape (2, 3) and send them both to the GPU (you'll need access to a GPU for this). Set torch.manual_seed(1234) when creating the tensors (this doesn't have to be the GPU random seed).

As in exercise 5, we do not have CUDA cores available. But we can check this with torch.cuda.is_available(). So I do this and sets the seed on the CUDA cores if they are available, if not I seed the CPU and do what is asked and returning the tensors, since they are used in the next exercise:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
def e6():
    if torch.cuda.is_available():
        torch.cuda.manual_seed(1234)
    else:
        torch.random.manual_seed(1234)

    rt1 = torch.rand(2, 3)
    rt2 = torch.rand(2, 3)
    print(f"{rt1=}")
    print(f"{rt2=}")

    return rt1, rt2

Exercise 7

Perform a matrix multiplication on the tensors you created in exercise 6 (again, you may have to adjust the shapes of one of the tensors).

Since the number of rows in the second tensor not is matching the number of columns in the first tensor I have to transpose again before doing the matrix multiplication with torch.matmul. I also return the matrix product since I have to use this in exercises 8 and 9:

1
2
3
4
5
def e7(rt1: torch.Tensor, rt2: torch.Tensor):
    rt2_t = rt2.transpose(0, 1)
    mat_multi = torch.matmul(rt1, rt2_t)
    return mat_multi
    print(f"{mat_multi=}")

Exercise 8

Find the maximum and minimum values of the output of exercise 7

This is done with the method max() and min() which is attributes to the tensor class:

1
2
3
4
def e8(mat_multi: torch.Tensor):
    mx = mat_multi.max()
    mn = mat_multi.min()
    print(f"{mat_multi=}\n max: {mx=} \n min: {mn=}")

Exercise 9

Find the maximum and minimum index values of the output of exercise 7.

This is done with the argmin() and argmax() methods:

1
2
3
4
def e9(mat_multi: torch.Tensor):
    argmx = mat_multi.argmax()
    argmn = mat_multi.argmin()
    print(f"{mat_multi=}\n argmax: {argmx=} \n argmin: {argmn=}")

Exercise 10

Make a random tensor with shape (1, 1, 1, 10) and then create a new tensor with all the 1 dimensions removed to be left with a tensor of shape (10). Set the seed to 7 when you create it and print out the first tensor and it's shape as well as the second tensor and it's shape.

Once again I set the seed with torch.random.manual_seed() and then I am defining the first multi dimensional tensor with the size argument. I then create a new tensor by selecting the first indices in the first tensor

1
2
3
4
5
6
7
def e10():
    torch.random.manual_seed(7)
    rt1 = torch.rand(size=(1, 1, 1, 10))
    rt2 = rt1[0, 0, 0]

    print(rt1, rt1.shape)
    print(rt2, rt2.shape)