Creating Tensors

A tensor is PyTorch's only data structure: a typed, shaped, device-aware array. Every model input, weight, and gradient is one. Getting shape, dtype, and device right at creation prevents most of the shape errors you will hit later.

The idea

Three properties define a tensor:

• shape — the size along each axis. (32, 3, 224, 224) is a batch of 32 RGB images.
• dtype — torch.float32 for weights and activations, torch.int64 for indices and labels, torch.bool for masks.
• device — cpu, cuda, or mps. Every tensor in one operation must live on the same device.

Build a tensor from Python data with torch.tensor, or allocate directly by shape with torch.zeros, torch.ones, torch.empty, and torch.full. torch.tensor copies its input; torch.as_tensor reuses the same memory when dtype and device already match.

Dtype inference is a common trap. torch.tensor([1, 2]) gives int64, while torch.tensor([1.0, 2.0]) gives float32. Feed integer tensors into a float model and PyTorch raises a dtype error, so add a decimal point or pass dtype= explicitly.

One more knob: requires_grad=True marks a leaf tensor for autograd tracking, the subject of a later lesson. torch.empty returns uninitialized memory — fast, but every entry must be overwritten before use.

Worked example

Build a tiny linear layer by hand:

• x = torch.tensor([[1.0, 2.0], [3.0, 4.0]]) → shape (2, 2).
• w = torch.ones(2, 3) → shape (2, 3).
• b = torch.zeros(3) → shape (3,), broadcast across the batch.
• y = x @ w + b → shape (2, 3).

Check the metadata: x.dtype is torch.float32, b.shape is torch.Size([3]), and x.device is cpu. torch.zeros(2, 3).dtype is also float32, but torch.zeros(2, 3, dtype=torch.int64) is not.

In code

import torch

x = torch.tensor([[1.0, 2.0], [3.0, 4.0]])   # (2, 2) float32
w = torch.ones(2, 3)                          # (2, 3)
b = torch.zeros(3)                            # (3,)
y = x @ w + b
print(y.shape, y.dtype)                       # torch.Size([2, 3]) torch.float32

empty = torch.empty(4, 4)                        # uninitialized memory
full = torch.full((2, 2), 7.0)                   # every entry 7.0
mask = torch.tensor([True, False])               # bool
print(empty.shape, full[0, 0].item(), mask.dtype)
# torch.Size([4, 4]) 7.0 torch.bool

Check yourself

1. What dtype does torch.tensor([1, 2, 3]) have, and why does it matter for a loss function?
2. What is the difference between torch.tensor(data) and torch.as_tensor(data)?
3. If a is on cuda and b is on cpu, what happens when you compute a + b?

Key takeaways

• A tensor is shape plus dtype plus device; get all three right before debugging math.
• torch.tensor copies, as_tensor can alias, empty is uninitialized.
• Integer literals produce int64; add a decimal point or pass dtype for floats.