How to Upload Files to Cloudflare R2 Using Go A Complete Guide

99 min read

Cloudflare R2 provides a cost-effective alternative to AWS S3 for object storage, with no egress fees. In this tutorial, we'll explore how to use Go to upload files to Cloudflare R2 using the AWS SDK for Go v2, taking advantage of R2's S3-compatible API.

Prerequisites

Before we begin, make sure you have:

  • Go installed on your system
  • A Cloudflare account with R2 enabled
  • Your R2 credentials (Account ID, Access Key ID, and Secret Access Key)
  • Basic familiarity with Go programming

Setting Up Your Project

First, create a new Go project and install the required AWS SDK packages:

mkdir r2-upload
cd r2-upload
go mod init r2-upload
go get github.com/aws/aws-sdk-go-v2
go get github.com/aws/aws-sdk-go-v2/config
go get github.com/aws/aws-sdk-go-v2/credentials
go get github.com/aws/aws-sdk-go-v2/service/s3

Implementation

Here's a complete example showing how to upload a file to Cloudflare R2:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/credentials"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
    // R2 credentials
    accountID := "your_account_id"
    accessKeyID := "your_access_key_id"
    accessKeySecret := "your_access_key_secret"

    // Custom endpoint resolver for R2
    r2Resolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
        return aws.Endpoint{
            URL: fmt.Sprintf("https://%s.r2.cloudflarestorage.com", accountID),
        }, nil
    })

    // Configure AWS SDK
    cfg, err := config.LoadDefaultConfig(context.TODO(),
        config.WithEndpointResolverWithOptions(r2Resolver),
        config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKeyID, accessKeySecret, "")),
        config.WithRegion("auto"),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Create S3 client
    client := s3.NewFromConfig(cfg)

    // Open the file to upload
    file, err := os.Open("example.txt")
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    // Upload the file
    _, err = client.PutObject(context.TODO(), &s3.PutObjectInput{
        Bucket: aws.String("your-bucket-name"),
        Key:    aws.String("example.txt"),
        Body:   file,
    })
    if err != nil {
        log.Fatal(err)
    }

    log.Println("File uploaded successfully")
}

Code Breakdown

Let's break down the key components of the code:

1. Configuration Setup

r2Resolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
    return aws.Endpoint{
        URL: fmt.Sprintf("https://%s.r2.cloudflarestorage.com", accountID),
    }, nil
})

This creates a custom endpoint resolver that points to your R2 bucket's endpoint. The URL format is specific to Cloudflare R2.

2. AWS SDK Configuration

cfg, err := config.LoadDefaultConfig(context.TODO(),
    config.WithEndpointResolverWithOptions(r2Resolver),
    config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKeyID, accessKeySecret, "")),
    config.WithRegion("auto"),
)

This configures the AWS SDK with your R2 credentials and custom endpoint resolver. Note that we use "auto" as the region since R2 doesn't use traditional AWS regions.

3. File Upload

_, err = client.PutObject(context.TODO(), &s3.PutObjectInput{
    Bucket: aws.String("your-bucket-name"),
    Key:    aws.String("example.txt"),
    Body:   file,
})

This uploads the file to your R2 bucket. The Key parameter determines the file's name in the bucket.

Best Practices and Tips

  1. Error Handling: Always check for errors at each step, especially during file operations and uploads.

  2. File Management: Use defer file.Close() to ensure files are properly closed after use.

  3. Security: Never hardcode credentials in your code. Use environment variables or a configuration file instead:

accountID := os.Getenv("R2_ACCOUNT_ID")
accessKeyID := os.Getenv("R2_ACCESS_KEY_ID")
accessKeySecret := os.Getenv("R2_ACCESS_KEY_SECRET")
  1. Content Type: You can specify the content type of your uploaded file:
_, err = client.PutObject(context.TODO(), &s3.PutObjectInput{
    Bucket:      aws.String("your-bucket-name"),
    Key:         aws.String("example.txt"),
    Body:        file,
    ContentType: aws.String("text/plain"),
})

Common Issues and Solutions

  1. Connection Issues
  • Double-check your Account ID and endpoint URL
  • Verify your R2 bucket exists and is properly configured
  • Ensure your firewall isn't blocking the connection
  1. Authentication Errors
  • Verify your Access Key ID and Secret are correct
  • Check if your R2 token has the necessary permissions
  • Ensure your credentials haven't expired

Conclusion

Using Go with Cloudflare R2 provides a powerful and cost-effective way to handle object storage in your applications. The AWS SDK's S3 compatibility makes it straightforward to implement, while R2's pricing model (especially the lack of egress fees) makes it an attractive option for many use cases.

Remember to always follow security best practices and properly handle errors in your production code. For more advanced usage, you might want to explore additional features like multipart uploads for large files or implementing retry logic for better reliability.

Resources