使用预签名 URL 上传

本文为希望允许用户上传视频和其他资源的网络应用提供指导。我们建议在服务器端生成一个预签名 URL，使用户能够直接将文件上传到你的云存储，而无需通过你的服务器传输文件。

🌐 This article provides guidance for webapps wanting to allow users to upload videos and other assets. We recommend to generate a presigned URL server-side that allows a user to directly upload a file into your cloud storage without having to pass the file through your server.

你可以设置诸如最大文件大小和文件类型的限制，应用速率限制，要求身份验证，并预先定义存储位置。

🌐 You can set constraints such as maximal file size and file type, apply rate limiting, require authentication, and predefine the storage location.

为什么使用预签名 URL？

🌐 Why use presigned URL?

实现文件上传的传统方法是让客户端将文件上传到服务器，然后服务器将文件存储在磁盘上或转发上传到云存储。虽然这种方法可行，但由于多种原因，它并不理想。

🌐 The traditional way of implementing a file upload would be to let the client upload the file onto a server, which then stores the file on disk or forwards the upload to cloud storage. While this approach works, it's not ideal due to several reasons.

减少负载：如果许多客户端恰好在同一服务器上上传大文件，该服务器可能会变慢甚至在负载下崩溃。使用预签名工作流，服务器只需要创建预签名 URL，这比处理文件传输能减少服务器负载。
减少垃圾邮件：为了防止用户将你的上传功能当作免费存储空间使用，如果他们超出你的配额，你可以拒绝向他们提供预签名 URL。
数据安全：由于如今很多托管解决方案是临时或无服务器的，因此文件不应存储在这些解决方案中。无法保证服务器重启后文件仍然存在，而且你可能会用完磁盘空间。

AWS 示例

🌐 AWS Example

这是一个将用户上传内容存储在 AWS S3 的示例。

🌐 Here is an example for storing user uploads are stored in AWS S3.

权限

🌐 Permissions

在 AWS 控制台中的你的存储桶里，转到权限并通过 CORS 允许 PUT 请求：

🌐 In your bucket on the AWS console, go to Permissions and allow PUT requests via CORS:

Cross-origin resource sharing (CORS) policy
[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["PUT"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": [],
    "MaxAgeSeconds": 3000
  }
]

note

允许通过 CORS 使用 GET 方法可能也会很有用，这样你可以在上传后获取资源。

你的 AWS 用户策略至少必须具有放置对象并将其设为公开的能力：

🌐 Your AWS user policy must at least have the ability to put an object and make it public:

User role policy
{
  "Sid": "Presign",
  "Effect": "Allow",
  "Action": ["s3:PutObject", "s3:PutObjectAcl"],
  "Resource": ["arn:aws:s3:::{YOUR_BUCKET_NAME}/*"]
}

预签名 URL

🌐 Presigning URLs

首先，在你的前端接收一个文件，例如使用 <input type="file">。你应该会得到一个 File，从中你可以确定内容类型和内容长度：

🌐 First, accept a file in your frontend, for example using <input type="file">. You should get a File, from which you can determine the content type and content length:

App.tsx
const contentType = file.type || 'application/octet-stream';
const arrayBuffer = await file.arrayBuffer();
const contentLength = arrayBuffer.byteLength;

此示例使用 @aws-sdk/s3-request-presigner 和从 @remotion/lambda 导入的 AWS SDK。通过调用下面的函数，将生成两个 URL：

🌐 This example uses @aws-sdk/s3-request-presigner and the AWS SDK imported from @remotion/lambda. By calling the function below, two URLs are generated:

presignedUrl 是可以上传文件的 URL
readUrl 是可以读取该文件的 URL。

generate-presigned-url.ts
import {getSignedUrl} from '@aws-sdk/s3-request-presigner';
import {AwsRegion, getAwsClient} from '@remotion/lambda/client';

export const generatePresignedUrl = async (contentType: string, contentLength: number, expiresIn: number, bucketName: string, region: AwsRegion): Promise<{presignedUrl: string; readUrl: string}> => {
  if (contentLength > 1024 * 1024 * 200) {
    throw new Error(`File may not be over 200MB. Yours is ${contentLength} bytes.`);
  }

  const {client, sdk} = getAwsClient({
    region: process.env.REMOTION_AWS_REGION as AwsRegion,
    service: 's3',
  });

  const key = crypto.randomUUID();

  const command = new sdk.PutObjectCommand({
    Bucket: bucketName,
    Key: key,
    ACL: 'public-read',
    ContentLength: contentLength,
    ContentType: contentType,
  });

  const presignedUrl = await getSignedUrl(client, command, {
    expiresIn,
  });

  // The location of the asset after the upload
  const readUrl = `https://${bucketName}.s3.${region}.amazonaws.com/${key}`;

  return {presignedUrl, readUrl};
};

解释：

🌐 Explanation:

首先，会检查上传请求是否符合限制条件。在此示例中，我们拒绝超过200MB的上传。你可以添加更多限制或添加速率限制。
AWS SDK 使用 getAwsClient() 导入。如果你不使用 Remotion Lambda，请直接安装 @aws-sdk/client-s3 包。
一个 UUID 被用作文件名以避免名称冲突。
最后，预签名 URL 和输出 URL 会被计算并返回。

Next.js 示例代码

🌐 Next.js example code

这是一个 Next.js 应用路由的示例片段。
该端点可通过 api/upload/route.ts 获取。

🌐 Here is a sample snippet for the Next.js App Router.
The endpoint is available under api/upload/route.ts.

app/api/upload/route.ts
import {NextResponse} from 'next/server';
import {getSignedUrl} from '@aws-sdk/s3-request-presigner';
import {AwsRegion, getAwsClient} from '@remotion/lambda/client';

const generatePresignedUrl = async ({contentType, contentLength, expiresIn, bucketName, region}: {contentType: string; contentLength: number; expiresIn: number; bucketName: string; region: AwsRegion}): Promise<{presignedUrl: string; readUrl: string}> => {
  if (contentLength > 1024 * 1024 * 200) {
    throw new Error(`File may not be over 200MB. Yours is ${contentLength} bytes.`);
  }

  const {client, sdk} = getAwsClient({
    region: process.env.REMOTION_AWS_REGION as AwsRegion,
    service: 's3',
  });

  const key = crypto.randomUUID();

  const command = new sdk.PutObjectCommand({
    Bucket: bucketName,
    Key: key,
    ACL: 'public-read',
    ContentLength: contentLength,
    ContentType: contentType,
  });

  const presignedUrl = await getSignedUrl(client, command, {
    expiresIn,
  });

  // The location of the asset after the upload
  const readUrl = `https://${bucketName}.s3.${region}.amazonaws.com/${key}`;

  return {presignedUrl, readUrl};
};

export const POST = async (request: Request) => {
  if (!process.env.REMOTION_AWS_BUCKET_NAME) {
    throw new Error('REMOTION_AWS_BUCKET_NAME is not set');
  }

  if (!process.env.REMOTION_AWS_REGION) {
    throw new Error('REMOTION_AWS_REGION is not set');
  }

  const json = await request.json();
  if (!Number.isFinite(json.size)) {
    throw new Error('size is not a number');
  }
  if (typeof json.contentType !== 'string') {
    throw new Error('contentType is not a string');
  }

  const {presignedUrl, readUrl} = await generatePresignedUrl({
    contentType: json.contentType,
    contentLength: json.size,
    expiresIn: 60 * 60 * 24 * 7,
    bucketName: process.env.REMOTION_AWS_BUCKET_NAME as string,
    region: process.env.REMOTION_AWS_REGION as AwsRegion,
  });

  return NextResponse.json({presignedUrl, readUrl});
};

这是你在前端调用它的方式：

🌐 This is how you can call it in the frontend:

Uploader.tsx
const presignedResponse = await fetch('/api/upload', {
  method: 'POST',
  body: JSON.stringify({
    size: file.size,
    contentType: file.type,                  
const file: File
  }),
});

const json = (await presignedResponse.json()) as {
  presignedUrl: string;
  readUrl: string;
};

note

此示例未实现任何速率限制或身份验证。

正在执行上传

🌐 Performing the Uploading

使用 fetch()

🌐 Using fetch()

将预签名的 URL 返回给客户端。之后，你现在可以使用内置的 fetch() 函数进行上传：

🌐 Send the presigned URL back to the client. Afterwards, you can now perform an upload using the built-in fetch() function:

upload-with-fetch.ts

await fetch(presignedUrl, {
  method: 'PUT',
  body: arrayBuffer,
  headers: {
    'content-type': contentType,
  },
});

跟踪上传进度

🌐 Tracking the upload progress

截至2024年10月，如果你需要跟踪上传进度，你需要使用 XMLHTTPRequest。

🌐 As of October 2024, if you need to track the progress of the upload, you need to use XMLHTTPRequest.

upload-with-progress.ts
export type UploadProgress = {
  progress: number;
  loadedBytes: number;
  totalBytes: number;
};

export type OnUploadProgress = (options: UploadProgress) => void;

export const uploadWithProgress = ({file, url, onProgress}: {file: File; url: string; onProgress: OnUploadProgress}): Promise<void> => {
  return new Promise((resolve, reject) => {
    const xhr = new XMLHttpRequest();

    xhr.open('PUT', url);

    xhr.upload.onprogress = function (event) {
      if (event.lengthComputable) {
        onProgress({
          progress: event.loaded / event.total,
          loadedBytes: event.loaded,
          totalBytes: event.total,
        });
      }
    };

    xhr.onload = function () {
      if (xhr.status === 200) {
        resolve();
      } else {
        reject(new Error(`Upload failed with status: ${xhr.status}`));
      }
    };

    xhr.onerror = function () {
      reject(new Error('Network error occurred during upload'));
    };

    xhr.setRequestHeader('content-type', file.type);
    xhr.send(file);
  });
};

另请参阅

🌐 See also

处理用户视频上传

为什么使用预签名 URL？​

AWS 示例​

权限​

预签名 URL​

Next.js 示例代码​

正在执行上传​

使用 fetch()​

跟踪上传进度​

另请参阅​