PyTorch 张量完全指南：深度学习的核心数据结构
核心认知：张量（Tensor）是深度学习世界的”通用语言”——无论是图像、文本、语音还是视频，都会被转换为张量进行处理。理解张量的创建、变换和运算，是掌握 PyTorch 的第一步，也是构建神经网络的基础。
张量是 PyTorch 中最基本的数据结构，类似于 NumPy 的 ndarray，但增加了 GPU 加速和自动求导等深度学习特性。本文将系统讲解张量的核心概念、创建方式、维度变换和常用运算。
目录
- 什么是张量？
- 创建张量
- 张量属性
- 索引与切片
- 维度变换（核心重点）
- 张量运算
- 设备管理（CPU/GPU）
- 广播机制
- 实战案例
- 总结速查表
什么是张量？
从标量到高阶张量
张量是标量、向量、矩阵的泛化，是 AI 中统一的数据格式。
| 阶 (Rank) |
名称 |
数学表示 |
示例 |
形状 (Shape) |
| 0 |
标量 (Scalar) |
单个数字 |
5 |
() |
| 1 |
向量 (Vector) |
一维数组 |
[1, 2, 3] |
(3,) |
| 2 |
矩阵 (Matrix) |
二维表格 |
[[1, 2], [3, 4]] |
(2, 2) |
| 3 |
3阶张量 |
三维数组 |
彩色图片 (高, 宽, 通道) |
(224, 224, 3) |
| 4 |
4阶张量 |
四维数组 |
一批图片 (批次, 高, 宽, 通道) |
(32, 224, 224, 3) |
| 5 |
5阶张量 |
五维数组 |
视频数据 (批次, 帧, 通道, 高, 宽) |
(16, 30, 3, 224, 224) |
张量的三个核心属性
每个张量都有三个核心属性：
1
2
3
4
5
6
7
| import torch
tensor = torch.rand(3, 4)
print(f"形状 (shape): {tensor.shape}")
print(f"数据类型 (dtype): {tensor.dtype}")
print(f"设备 (device): {tensor.device}")
|
| 属性 |
说明 |
常见取值 |
| shape |
每个维度的大小 |
(3, 224, 224) |
| dtype |
元素类型 |
torch.float32, torch.int64, torch.bool |
| device |
存储位置 |
cpu, cuda:0 |
创建张量
从已有数据创建
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| import torch
import numpy as np
tensor_list = torch.tensor([[1, 2], [3, 4]])
print(tensor_list)
np_array = np.array([1, 2, 3])
tensor_numpy = torch.from_numpy(np_array)
print(tensor_numpy)
tensor_float = torch.tensor([1, 2, 3], dtype=torch.float32)
tensor_int = torch.tensor([1.5, 2.7], dtype=torch.int64)
|
创建特殊值的张量
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
zeros = torch.zeros(2, 3)
print(zeros)
ones = torch.ones(2, 3)
eye = torch.eye(3)
print(eye)
full = torch.full((2, 3), 7)
empty = torch.empty(2, 3)
|
创建随机张量
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
rand_uniform = torch.rand(2, 3)
rand_normal = torch.randn(2, 3)
rand_uniform_range = torch.empty(2, 3).uniform_(0, 10)
rand_int = torch.randint(0, 10, (2, 3))
normal = torch.normal(mean=0, std=1, size=(2, 3))
|
创建序列张量
1
2
3
4
5
6
7
8
9
|
arange_1 = torch.arange(5)
arange_2 = torch.arange(1, 10, 2)
linspace = torch.linspace(0, 1, 5)
logspace = torch.logspace(0, 2, 5)
|
与 NumPy 互转
1
2
3
4
5
6
7
8
9
10
11
12
13
|
np_array = np.array([1, 2, 3])
tensor = torch.from_numpy(np_array)
tensor = torch.tensor([1, 2, 3])
np_array = tensor.numpy()
tensor = torch.tensor([1, 2, 3])
np_array = tensor.numpy()
tensor[0] = 100
print(np_array)
|
张量属性
查看和修改属性
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| tensor = torch.rand(3, 4)
print(tensor.shape)
print(tensor.size())
print(tensor.numel())
print(tensor.dtype)
tensor_int = tensor.to(torch.int64)
tensor_float16 = tensor.half()
tensor_float32 = tensor.float()
print(tensor.device)
print(tensor.requires_grad)
tensor.requires_grad_(True)
|
形状的表示
1
2
3
4
5
6
7
8
9
|
shape = torch.rand(2, 3, 4).shape
print(len(shape))
print(shape[0])
print(shape[1])
print(shape[2])
dim = tensor.dim()
|
索引与切片
基础索引
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
| tensor = torch.arange(24).reshape(4, 6)
print(tensor)
element = tensor[1, 2]
first_row = tensor[0]
first_col = tensor[:, 0]
sub = tensor[1:3, 2:4]
every_other = tensor[::2, ::2]
|
高级索引
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| tensor = torch.arange(12).reshape(3, 4)
indices = tensor[[0, 2], [1, 3]]
mask = tensor > 5
selected = tensor[mask]
result = torch.where(tensor > 5, tensor, torch.zeros_like(tensor))
|
维度变换（核心重点）
reshape 与 view
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| tensor = torch.arange(12)
reshaped = tensor.reshape(3, 4)
reshaped = tensor.reshape(2, 2, 3)
reshaped = tensor.reshape(-1, 4)
viewed = tensor.view(3, 4)
tensor = torch.rand(3, 4).t()
tensor.reshape(2, 6)
|
squeeze 与 unsqueeze
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
tensor = torch.rand(1, 3, 1, 4)
squeezed = tensor.squeeze()
squeezed_dim = tensor.squeeze(0)
squeezed_dim = tensor.squeeze(2)
tensor = torch.rand(3, 4)
unsqueezed = tensor.unsqueeze(0)
unsqueezed = tensor.unsqueeze(1)
unsqueezed = tensor.unsqueeze(-1)
single_image = torch.rand(224, 224, 3)
batch_image = single_image.unsqueeze(0)
|
transpose 与 permute
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
matrix = torch.tensor([[1, 2], [3, 4]])
transposed = matrix.t()
transposed = torch.transpose(matrix, 0, 1)
tensor = torch.rand(2, 3, 4)
permuted = tensor.permute(0, 2, 1)
image = torch.rand(32, 224, 224, 3)
image_nchw = image.permute(0, 3, 1, 2)
|
维度变换速查表
| 操作 |
代码 |
输入形状 |
输出形状 |
说明 |
| 展平 |
tensor.flatten() |
(2, 3, 4) |
(24,) |
一维化 |
| 展平（保留batch） |
tensor.flatten(1) |
(32, 3, 224, 224) |
(32, 3×224×224) |
保留第一维 |
| reshape |
tensor.reshape(4, 6) |
(24,) |
(4, 6) |
任意改变形状 |
| 增维 |
tensor.unsqueeze(0) |
(3, 4) |
(1, 3, 4) |
在第0维添加 |
| 降维 |
tensor.squeeze() |
(1, 3, 1, 4) |
(3, 4) |
移除size=1的维度 |
| 转置 |
tensor.t() |
(3, 4) |
(4, 3) |
2D转置 |
| 交换维度 |
tensor.transpose(0, 1) |
(2, 3, 4) |
(3, 2, 4) |
交换指定两维 |
| 重排 |
tensor.permute(1, 2, 0) |
(2, 3, 4) |
(3, 4, 2) |
任意重排顺序 |
张量运算
基本算术运算（逐元素）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])
print(a + b)
print(a - b)
print(a * b)
print(a / b)
print(a ** 2)
print(torch.pow(a, 2))
print(torch.sqrt(a))
print(torch.exp(a))
print(torch.log(a))
|
矩阵乘法
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
A = torch.rand(2, 3)
B = torch.rand(3, 4)
C = A @ B
C = torch.matmul(A, B)
C = torch.mm(A, B)
batch_A = torch.rand(16, 2, 3)
batch_B = torch.rand(16, 3, 4)
batch_C = torch.bmm(batch_A, batch_B)
|
聚合操作
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
| tensor = torch.tensor([[1, 2, 3], [4, 5, 6]], dtype=torch.float32)
print(tensor.sum())
print(tensor.sum(dim=0))
print(tensor.sum(dim=1))
print(tensor.mean())
print(tensor.mean(dim=0))
print(tensor.max())
print(tensor.max(dim=1))
print(tensor.prod())
print(tensor.std())
print(tensor.var())
sum_keepdim = tensor.sum(dim=1, keepdim=True)
|
设备管理（CPU/GPU）
基本设备操作
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
| import torch
print(torch.cuda.is_available())
print(torch.cuda.device_count())
print(torch.cuda.get_device_name(0))
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tensor_on_cpu = torch.rand(3, 4)
tensor_on_gpu = torch.rand(3, 4, device="cuda")
tensor_on_device = torch.rand(3, 4, device=device)
tensor = torch.rand(3, 4)
tensor = tensor.to(device)
tensor = tensor.cuda()
tensor = tensor.cpu()
print(tensor.device)
|
模型与数据设备对齐
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| import torch.nn as nn
model = nn.Linear(10, 2)
data = torch.rand(32, 10)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
data = data.to(device)
output = model(data)
|
广播机制
当两个张量形状不同时，PyTorch 会自动尝试广播（Broadcasting），使它们形状兼容。
广播规则
- 从尾部维度开始对齐
- 如果两个维度大小相等，或其中一个为 1，或其中一个不存在，则可以广播
- 否则无法广播
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
a = torch.rand(3, 4)
b = 2
c = a * b
a = torch.rand(3, 4)
b = torch.rand(4)
c = a + b
a = torch.rand(3, 4)
b = torch.rand(3, 1)
c = a + b
a = torch.rand(3, 4)
b = torch.rand(2, 4)
a = torch.rand(3, 1)
b = torch.rand(1, 3)
c = a + b
|
手动广播
1
2
3
4
5
6
|
a = torch.rand(3, 1)
expanded = a.expand(3, 4)
repeated = a.repeat(1, 4)
|
实战案例
案例1：图像数据预处理（NHWC ↔ NCHW）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| import torch
images_nhwc = torch.rand(32, 224, 224, 3)
print(f"原始形状: {images_nhwc.shape}")
images_nchw = images_nhwc.permute(0, 3, 1, 2)
print(f"转换后形状: {images_nchw.shape}")
images_normalized = images_nchw / 255.0
mean = torch.tensor([0.5, 0.5, 0.5]).view(1, 3, 1, 1)
std = torch.tensor([0.5, 0.5, 0.5]).view(1, 3, 1, 1)
images_standardized = (images_normalized - mean) / std
|
案例2：批量矩阵计算
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
batch_size = 100
vector_dim = 256
A = torch.rand(batch_size, vector_dim)
B = torch.rand(batch_size, vector_dim)
dot_products = (A * B).sum(dim=1)
dot_products = torch.einsum('ij,ij->i', A, B)
Q = torch.rand(32, 8, 64)
K = torch.rand(32, 8, 64)
scores = torch.einsum('bhd,bhd->bh', Q, K)
|
案例3：掩码操作
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
seq_lengths = [3, 5, 2, 4]
max_len = max(seq_lengths)
batch_size = len(seq_lengths)
mask = torch.zeros(batch_size, max_len, dtype=torch.bool)
for i, length in enumerate(seq_lengths):
mask[i, :length] = 1
print(mask)
data = torch.rand(batch_size, max_len, 10)
masked_data = data[mask]
|
案例4：创建神经网络层
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
| import torch.nn as nn
class SimpleCNN(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(3, 16, kernel_size=3, padding=1)
self.conv2 = nn.Conv2d(16, 32, kernel_size=3, padding=1)
self.fc = nn.Linear(32 * 56 * 56, 10)
def forward(self, x):
x = self.conv1(x)
x = nn.ReLU()(x)
x = nn.MaxPool2d(2)(x)
x = self.conv2(x)
x = nn.ReLU()(x)
x = nn.MaxPool2d(2)(x)
x = x.view(x.size(0), -1)
x = self.fc(x)
return x
model = SimpleCNN()
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
x = torch.rand(16, 3, 224, 224).to(device)
output = model(x)
print(output.shape)
|
总结：张量操作速查表
| 操作类型 |
常用函数 |
说明 |
| 创建 |
torch.tensor(), torch.zeros(), torch.ones(), torch.rand() |
创建张量 |
| 转换 |
.numpy(), torch.from_numpy() |
与 NumPy 互转 |
| 形状 |
.shape, .size(), .numel() |
查看形状和大小 |
| 重塑 |
.reshape(), .view(), .flatten() |
改变形状 |
| 增/删维度 |
.unsqueeze(), .squeeze() |
操作 size=1 的维度 |
| 重排 |
.permute(), .transpose() |
交换/重排维度 |
| 索引 |
tensor[1, 2], tensor[mask] |
获取子集 |
| 运算 |
+, -, *, /, @ |
基本运算和矩阵乘法 |
| 聚合 |
.sum(), .mean(), .max() |
统计聚合 |
| 设备 |
.to(device), .cuda(), .cpu() |
移动张量 |