Introduction

Say you uploaded 200 GB to S3 last night. This morning you want to know one thing: did it arrive intact?

The obvious way to find out is to download it and compare. That means 200 GB of egress, an hour of waiting, and a bill. All to learn a single yes or no.

There is a better way. S3 will checksum a piece of an object for you, on its own hardware, and hand you back the answer in a response header. The bytes never move.

mito is built on that one trick. It asks S3 for thousands of those little checksums at once, computes the matching ones on your side using your CPU's built-in CRC32 instructions, and compares the two lists.

    The obvious way                     What mito does

   ┌──────────────┐                   ┌──────────────┐
   │   S3 object  │                   │   S3 object  │
   │    200 GB    │                   │    200 GB    │
   └──────┬───────┘                   └──────┬───────┘
          │                                  │
          │  200 GB over the wire            │  ~25,600 tiny checksums
          │  ~1 hour                         │  bytes stay in S3
          │  egress charges                  │  seconds to minutes
          ▼                                  ▼
   ┌──────────────┐                   ┌──────────────┐
   │  your disk   │                   │  32-bit CRCs │
   └──────┬───────┘                   └──────┬───────┘
          │                                  │
          └────────► same? ◄─────────────────┘

Same answer. Almost none of the cost. Architecture explains how the trick works.

What follows from that

Once you can compare a remote file without downloading it, everything else falls out.

mito compares by content, in any direction: local to local, local to S3, S3 to S3. And sync can use the same machinery to transfer only the parts of a big file that actually changed, instead of the whole thing.

What people use it for

• Checking that an upload really landed, byte for byte
• Comparing two copies of a dataset that live in different places
• Keeping a bucket in step with a directory, moving only what changed
• Finding out what your S3 API traffic is costing you
• Hunting down failed uploads that are quietly still on your bill

Before you trust it

This is experimental software (0.x). It is a personal project, not operational tooling. It does real work on real buckets, so treat it like anything else that can delete your data: read what it prints before you type --force.

Three things are worth knowing up front.

sync does not checksum to decide what to copy. It looks at file size and modification time, which is fast and cheap. It skips a file only when the sizes match and the destination is newer. That catches most edits, but it trusts two clocks. A restore that sets an old timestamp (tar -x, cp -p, rsync -t) looks older than the copy already there, so it gets skipped. When you need to be sure, use diff, which does checksum.

The no-download trick needs a real S3 feature. It depends on S3 additional checksums. Not every S3-compatible service implements them. See S3-compatible storage.

Interfaces are settled, not frozen. Commands, flags and exit codes are documented and stable enough to script against. But this is still 0.x, and a version bump may change them.

Where to start

New here? Read Install, then Quickstart.

Already have it running and just need an answer? The command pages have the flags. The reference pages explain why things work the way they do. Security and privacy covers what mito reads, what it writes, and what it can destroy.

Disclaimer

MitoSync is a personal project, written to learn hardware checksums, C++ concurrency and the S3 API. It is provided for educational and experimental purposes only, with no warranty of any kind. It is not intended or recommended for use in production or business-critical systems. Evaluate it carefully and run it at your own risk on any system or bucket you rely on.

Licensed under Apache-2.0.

Install

Download the package for your platform from a GitHub Release.

The quick way

This works on Linux and macOS. It figures out your platform, installs under $HOME/.local, and never asks for sudo:

sh -c 'set -eu; os=$(uname -s); arch=$(uname -m); case "$os" in Linux) os=linux;; Darwin) os=macos;; *) echo "unsupported OS: $os" >&2; exit 1;; esac; case "$arch" in x86_64|amd64) arch=x86_64;; aarch64) arch=aarch64;; arm64) [ "$os" = macos ] && arch=arm64 || arch=aarch64;; *) echo "unsupported arch: $arch" >&2; exit 1;; esac; curl -fsSL "https://github.com/echemythia/mitosync/releases/latest/download/mito-$os-$arch.install.sh" | sh'

Or pick your platform yourself

# Linux x86_64
curl -fsSL https://github.com/echemythia/mitosync/releases/latest/download/mito-linux-x86_64.install.sh | sh

# Linux ARM64
curl -fsSL https://github.com/echemythia/mitosync/releases/latest/download/mito-linux-aarch64.install.sh | sh

# macOS Apple Silicon
curl -fsSL https://github.com/echemythia/mitosync/releases/latest/download/mito-macos-arm64.install.sh | sh

# macOS Intel
curl -fsSL https://github.com/echemythia/mitosync/releases/latest/download/mito-macos-x86_64.install.sh | sh

If you would rather download first and run it afterwards, that works the same way:

sh ./mito-<version>-linux-x86_64.install.sh

# or choose a different prefix you own
sh ./mito-<version>-linux-x86_64.install.sh --prefix "$HOME/.local"

The installer puts the binary at $HOME/.local/bin/mito and the docs under $HOME/.local/share/doc/mito. Make sure $HOME/.local/bin is on your PATH.

System packages on Linux

sudo apt install ./mito-<version>-linux-x86_64.deb
# or
sudo dnf install ./mito-<version>-linux-x86_64.rpm

These install mito as /usr/bin/mito. Tarballs are also published if you would rather place things by hand.

Building from source

You need these first:

You do not install the libraries yourself. vcpkg handles those.

git clone --recurse-submodules git@github.com:echemythia/mitosync.git
cd mitosync

./bootstrap.sh                 # one-time: set up vcpkg and fetch dependencies

cmake --preset linux-release   # or macos-release
cmake --build build

./build/mito --version         # prints: mito 0.7.0
./build/mito --help
sudo cmake --install build --prefix /usr/local

The version comes from project(mito VERSION …) in CMakeLists.txt. That is the single source of truth: the number the binary reports, the number in vcpkg.json, and the number in a release artifact's name are all the same number by construction.

Cloned without --recurse-submodules? Fix it with:

git submodule update --init --recursive

Presets

What you get

Release builds turn on -O3, link-time optimization, -fstack-protector-strong, and full RELRO on Linux. Everything is compiled with -Wall -Wextra -Wpedantic.

No architecture flags are set, on purpose. zlib-ng brings its own accelerated CRC32 code and picks between the versions at run time, based on what the CPU actually reports. So a binary built on one machine runs, and stays fast, on another.

In build/ you will find:

• mito, the tool
• roughly twenty mito_tests_* binaries, one per area

Dependencies

Resolved by vcpkg from vcpkg.json:

Platform support

zlib-ng chooses among these by asking the CPU what it supports. Anything else falls back to portable C, which is correct but slower. There is no Windows build.

AWS credentials

mito reads credentials from the standard AWS chain and stores none of its own:

1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN)
2. ~/.aws/credentials and ~/.aws/config, including named profiles
3. An ECS task role

You can pick a different profile for each S3 side, which is what makes cross-account work possible. See Cross-account S3.

EC2 instance roles do not work. mito disables instance metadata lookup at startup and overwrites any value you set. See Files on disk.

Running the tests

ctest --test-dir build --output-on-failure      # the whole suite

./build/mito_tests                              # one binary of it
./build/mito_tests --gtest_filter=CRC32Test.*   # one group inside that binary

The suite is split across roughly twenty binaries, so ctest is the only thing that runs all of it. ls build/mito_tests_* lists them.

Run the suite serially. SyncTaskTest and LocalToLocalSyncTest share fixed temporary directories and clobber each other under ctest -j, producing failures that do not reproduce when the tests run one at a time.

Quickstart

Five minutes. You need a working mito and AWS credentials in the usual place.

First, how to name things

Every command takes paths in the same four shapes. Learn these and the rest is easy.

  /home/me/report.pdf        one local file
  /home/me/data/             a local folder  (note the slash)
  s3://my-bucket/report.pdf  one S3 object
  s3://my-bucket/data/       an S3 folder    (note the slash)

The trailing slash matters. It is the only thing that tells mito whether you mean one file or a whole tree.

You can also pin a region by adding @region to any S3 path:

s3://my-bucket/data/@eu-west-1

Without it, mito asks S3 where the bucket lives. That costs one extra API call, and it fails on some non-AWS services. Pinning is faster and more portable, so it is a good habit.

Compare two local folders

mito ./data/ /mnt/backup/data/

diff is the default command, so you can leave the word out. Exit code 0 means everything matched.

Compare local against S3

mito diff ./data/ s3://my-bucket/data/

Nothing big gets downloaded. That is the whole point. See Architecture.

Get a report a script can read

mito diff ./data/ s3://my-bucket/data/ -o results.json

The file extension picks the format: .json, .csv, or plain text for anything else.

When a file differs, the report does not just say "this file is wrong". It narrows the problem down for you:

  file.bin  ──►  8 MiB chunks  ──►  the one that differs  ──►  64 KiB blocks
                 [ ][ ][X][ ]              chunk 2            [ ][X][ ][ ]

  result: "bytes 16777216-16842751 differ"

So you learn where two copies diverged, not just that they did.

Sync, carefully

Always dry-run first. --delete really does delete.

mito sync ./data/ s3://my-bucket/backup/ --dry-run   # look
mito sync ./data/ s3://my-bucket/backup/             # do it

Direction comes from the order of the arguments. Source first, destination second. Swap them to download instead of upload.

Remember: sync decides by size and modification time, not by checksum. It skips a file only when the sizes match and the destination is newer. That catches an ordinary edit. It does not catch a restore that put an old timestamp back. When you need certainty, run diff.

Delete objects

rm will not delete anything until you add --force. Without it, you get a preview.

mito rm s3://my-bucket/old-data/ --recursive          # preview
mito rm s3://my-bucket/old-data/ --recursive --force  # do it

See what it cost

mito stats

The counts add up across every run and are kept on disk. See stats.

Clean up what failed

mito leftovers my-bucket --older-than 7d           # list
mito leftovers my-bucket --older-than 7d --abort   # clean

When a big upload fails halfway, S3 keeps the parts it already received. It also keeps charging you for them. They show up in no object listing, so nobody ever notices. This is how you find them.

Where to go next

• diff and sync for the full flag lists
• Exit codes if you are scripting this
• S3-compatible storage if you are not using AWS

diff: compare by checksum

mito [diff] <source1> <source2> [OPTIONS]

diff answers one question: are these two things the same?

It answers it by checksum, not by size or timestamp. If it says two files match, it read every byte of both, one way or another. This is the command you run when you need to be sure.

It is also the default command, so mito a/ b/ and mito diff a/ b/ mean the same thing. Either side can be local or S3, in any combination.

Options

Examples

# Two local folders
mito /path/to/dir1/ /path/to/dir2/

# One local file against one S3 object
mito ./file.bin s3://my-bucket/file.bin

# Local tree against an S3 prefix, with a report a script can read
mito diff ./data/ s3://my-bucket/data/ -o results.json

# Two buckets in different accounts and regions
mito diff s3://acctA-bucket/data/@us-east-1 s3://acctB-bucket/data/@eu-west-1 \
  --source-profile acctA --dest-profile acctB

What actually happens

Every file is cut into 8 MiB chunks. Each chunk gets its own checksum. Then the two lists of checksums are compared.

The two sides get there differently:

   LOCAL SIDE                          S3 SIDE

   mmap the chunk                      ask S3 to checksum
   (no copy, pages in                  that byte range
    only what is read)                 (bytes never move)
          │                                   │
          ▼                                   ▼
   zlib-ng CRC32                       read the answer from
   using your CPU's                    the x-amz-checksum-crc32
   CRC instructions                    response header
          │                                   │
          ▼                                   ▼
   [c0][c1][c2][c3]  ◄── compare ──►   [c0][c1][c2][c3]

Both sides run at the same time. So the whole thing takes as long as the slower of the two, not both added together. That matters, because they are limited by completely different things: your disk and CPU on one side, network round trips on the other.

Finding where two files differ

Knowing "chunk 47 is wrong" is not very useful. An 8 MiB chunk is a big haystack.

So when a chunk disagrees, diff goes back and re-checks just that chunk at 64 KiB granularity, 128 blocks of it:

  Pass 1, coarse:  [ ][ ][ ][X][ ][ ][ ][ ]   ← 8 MiB chunks, cheap
                            │
                            ▼
  Pass 2, fine:    [ ][ ][X][ ] ... (128 blocks of 64 KiB)
                          │
                          ▼
              "bytes 25165824-25231359 differ"

Why two passes instead of just checking everything at 64 KiB? Because that would multiply the number of S3 calls by 128, on a comparison that usually finds nothing at all. Doing the fine pass only where the coarse pass found trouble keeps the normal case cheap.

A partial listing stops the run

If mito cannot read one side completely, diff fails. It does not report on the part it managed to read.

This sounds unhelpful until you think about what the alternative would print. A folder that was only half-listed does not look half-listed. It looks small. Every file it failed to mention would be reported as ONLY <other source>, which turns a typo or a permissions problem into a confident, wrong answer.

That matters more here than almost anywhere else, because diff is the command people run to decide whether it is safe to delete the other copy.

The error names which side was incomplete. The specific paths it could not read are logged just above it. sync refuses for the same reason.

Report formats

The extension you give to -o decides the format. Anything that is not .json or .csv comes out as text.

All three list only what is wrong: files that differ, files present on one side only, and files that could not be read. Matching files are left out. That is deliberate. It is what keeps a report over a million matching objects small enough to read.

JSON

A summary, then an array of problems.

{
  "source_a": "./data/",
  "source_b": "s3://my-bucket/data/",
  "summary": {
    "total_files": 1204,
    "matching": 1201,
    "different": 2,
    "only_in_./data/": 1,
    "only_in_s3://my-bucket/data/": 0,
    "errors": 0,
    "elapsed_seconds": 14.30
  },
  "files": [
    { "path": "logs/app.bin", "status": "MISMATCH", "size_a": 8388608, "size_b": 8388608 }
  ]
}

different and errors are separate counters. That is how a script tells "these two copies genuinely differ" from "I could not read one of them". The exit code cannot tell you this, so if you care, read the report. See Exit codes.

Two things to know before you parse this:

• The only_in_* key names change every run. They are built from the source strings you typed. Read them by position or by prefix, never by literal name.
• Strings are not escaped. A path or error message containing a " or a \ will produce JSON that does not parse. Ordinary paths are fine. Unusual ones are worth knowing about before a nightly job trips over one.

CSV

A header row, then one row per problem file.

status,path,size_a,size_b,error
MISMATCH,"logs/app.bin",8388608,8388608,""
ONLY ./data/,"logs/new.bin",4096,-1,""

A size of -1 means the file is not there on that side. There is no summary block. The counts only appear in the JSON and text formats. The same escaping caveat applies.

Text

The default for any other extension. Written for a person to read.

Directory Comparison Results
============================

Source A: ./data/
Source B: s3://my-bucket/data/

Summary:
  Total files:      1204
  Matching:         1201
  Different:        2
  Only in ./data/: 1
  Only in s3://my-bucket/data/: 0
  Errors:           0
  Time:             14.30s

Differences:
  [MISMATCH] logs/app.bin
  [ONLY ./data/] logs/new.bin

The Differences: section disappears entirely when there is nothing to report.

Exit status

Code 1 covers both "they differ" and "it broke". If your script needs to tell those apart, write a report and read the counters. See Exit codes.

Tuning

The defaults are meant to be fine. If you are tuning anyway:

• Lots of small files are limited by folder scanning. Give them a wide comparison window.
• A few huge files are limited by chunk work. Give them more threads per file instead.
• --ramp-up helps when a cold DNS cache is what is really slowing down the start.

Most of this is worked out automatically by measuring throughput as it goes. See Parallelism.

sync: make one side match

mito sync <source> <destination> [OPTIONS]

sync makes the destination look like the source. It works between a local folder and S3, in either direction, or between two S3 prefixes.

Direction comes from the order you type things. Source first, destination second.

Options

Examples

mito sync ./data/ s3://my-bucket/backup/            # upload
mito sync s3://my-bucket/backup/ ./data/            # download
mito sync s3://my-bucket/a/ s3://my-bucket/b/       # S3 to S3, copied inside AWS
mito sync ./data/ s3://my-bucket/backup/ --delete   # upload, prune extras
mito sync ./data/ s3://my-bucket/backup/ --dry-run  # preview only

How it decides what to copy

This is the most important thing to understand about sync, so here it is on its own line:

A file is skipped only when the sizes match and the destination is newer than the source. Everything else gets transferred.

   for each file:

   size different?  ──── yes ──►  TRANSFER
          │
          no
          ▼
   destination newer
   than source?     ──── no  ──►  TRANSFER
          │
          yes
          ▼
        SKIP

Notice what is missing from that diagram: any checksum. Both signals, size and timestamp, come out of the folder listing each side already produced. So sync can plan a run over a huge tree without reading a single byte of file content. That is what makes it fast.

Size alone would not be enough. Plenty of edits do not change a file's length: a fixed-width record overwritten in place, a binary patched byte for byte, content swapped for different content of the same size. In all of those the size is identical and only the timestamp moves.

Why "newer" and not "not older"

The check is strictly newer. Here is why that matters.

Both timestamps are whole seconds. If you edit a file in the same second that the destination was written, the two stamps come out equal. Neither one will ever change again on its own. So if "equal" counted as a skip, that file would be skipped forever.

Requiring the destination to be strictly newer costs you at most one redundant transfer. After that the destination's timestamp moves ahead and the file settles down into being skipped.

An unknown timestamp means transfer

If either side reports no timestamp, sync transfers. A zero proves nothing. Copying something you did not need to costs bandwidth; skipping something you needed leaves stale data. The first mistake is cheaper.

What this still gets wrong

Two clocks are being compared, with no tolerance for skew. So:

• A restore that puts an old timestamp back gets skipped. tar -x, cp -p and rsync -t all set the extracted file's mtime to the original's. That is older than the destination written after it, so the destination looks current and your restore is silently not propagated.
• A wrong local clock either hides edits or re-copies everything, depending on which way it is wrong.

When being correct matters more than being fast, run diff. It checksums.

A partial listing stops the run

If either side cannot be listed completely, sync refuses to do anything at all.

That is stricter than it sounds, and it is deliberate. A half-listed folder does not look half-listed. It looks small. Without --delete, mito would copy part of your source and report success. With --delete, every file the failed listing forgot to mention looks exactly like a file that should be removed from the destination.

So the run stops with an error. The paths it could not read are logged just above it.

Differential transfer

Once a file is picked for transfer, its size decides how it moves.

   file < 8 MiB          send the whole thing

   file >= 8 MiB         compare chunk checksums first,
                         send only the chunks that differ

                         [c0][c1][c2][c3][c4]   source
                          ✓   ✓   X   ✓   X     compare
                                  │       │
                                  ▼       ▼
                              send only these two

For a large file with a small change, this is the difference between moving gigabytes and moving megabytes.

Differential transfer uses the same remote-checksum trick as diff. On a service that does not implement S3 additional checksums, it fails for files above the chunk size. See S3-compatible storage.

--delete

--delete removes anything at the destination that has no counterpart at the source.

Combined with a mistyped source path, it will empty a prefix. Dry-run first. Every time.

mito sync ./data/ s3://my-bucket/backup/ --delete --dry-run

--delete is also the reason a partial source listing is refused instead of tolerated. See above.

S3 to S3

An S3-to-S3 sync is a server-side copy. The bytes move inside AWS and never travel through your machine. That is faster and avoids egress charges.

There is a catch that surprises people, and it is worth understanding before you try it across accounts:

   Your machine
        │
        │ 1. list source      (uses --source-profile)
        │ 2. list destination (uses --dest-profile)
        │ 3. "copy A to B"    (uses --dest-profile)
        │        │
        ▼        │
   ┌─────────────┼──────────────┐
   │  AWS        ▼              │
   │   source ──────► dest      │   bytes move in here,
   │                            │   under the DESTINATION's
   └────────────────────────────┘   credentials

The copy is one API call, sent to the destination, naming the source in a header. There is only one set of credentials on that request, and they belong to the destination profile. So the destination principal has to be allowed to read the source bucket.

--source-profile authenticates the source listing. It cannot authenticate the copy.

A real cross-account copy therefore also needs a policy on the source bucket. See Cross-account S3.

Exit status

An interrupted sync leaves the destination partly updated. There is no rollback. Running the same command again converges. See Exit codes.

rm: delete S3 objects

mito rm <s3-url> [OPTIONS]

rm deletes one S3 object, or everything under a prefix.

It is the only command here that destroys data on purpose, so it is built to be hard to trigger by accident.

Options

Two locks on the door

You have to get past both of these before anything is deleted.

   mito rm s3://bucket/logs/
        │
        ▼
   is it a prefix, with no -r ?  ──► yes ──►  ERROR, exit 2
        │                                     (not a no-op,
        no                                     not a mass delete)
        ▼
   was -f given ?                ──► no  ──►  PREVIEW only,
        │                                     nothing deleted
        yes
        ▼
   delete

The first lock means a prefix always needs --recursive. Pointing rm at s3://bucket/logs/ without it is an error. It does not quietly do nothing, and it certainly does not delete everything.

The second means nothing is deleted without --force. Leave it off and you get a preview of exactly what would go.

mito rm s3://my-bucket/old-data/            # error: prefix needs -r
mito rm s3://my-bucket/old-data/ -r         # preview of the recursive delete
mito rm s3://my-bucket/old-data/ -rf        # do it
mito rm s3://my-bucket/file.txt --force     # a single object

The habit worth building: run it once without -f, read the list, then add -f to the same command.

--batch

By default each object is deleted with its own request, -t of them at a time.

--batch switches to the DeleteObjects API, which removes up to 1000 keys in a single request. Over a large prefix this is much faster. It is also much more likely to draw throttling responses from S3.

Use it when you are deleting a lot and you are not competing with production traffic.

On AWS S3, DELETE requests are free either way. So this choice is about speed and throttling, not cost.

Exit status

See Exit codes.

stats: API usage and cost

mito stats [OPTIONS]

stats tells you what mito has been doing to your AWS bill.

Every command records the S3 calls it makes. stats prints the running totals and estimates what they cost.

Options

The counts add up over time

The counters are cumulative and live on disk. So mito stats answers "what has this tool cost me so far", not "what did the last command cost".

If you want to measure one specific job, reset first:

mito stats --reset
mito diff ./data/ s3://my-bucket/data/
mito stats

For each kind of operation it tracks calls, successes, failures, retries, bytes up, bytes down, bytes moved server-side, and latency (average and worst).

Why the operation types look oddly specific

The types are split more finely than the S3 API names are, because the billing is split more finely:

The same-bucket versus cross-bucket split is decided by comparing the source bucket named in the copy header against the destination bucket. It matters because a same-bucket copy is priced as a request, while a cross-region one also moves data.

Where the prices come from

Built-in per-region defaults, compiled into the binary. Nothing is fetched.

   built-in per-region defaults  ──► cached on disk ──► estimate

Fetching live rates from the AWS Pricing API is not implemented (#106). There is a cache file with a 24-hour lifetime, but what it caches is the same built-in table, so it changes nothing today.

The practical consequence: the rates are a snapshot someone typed in, not what AWS is charging you this month. They will drift.

mito stats --json

{
  "total": {
    "calls": 0,
    "success": 0,
    "failures": 0,
    "retries": 0,
    "bytes_uploaded": 0,
    "bytes_downloaded": 0,
    "avg_latency_ms": 0.00
  },
  "operations": {},
  "cost_breakdown": {
    "requests_usd": 0.000000,
    "data_transfer_out_usd": 0.000000,
    "total_usd": 0.000000
  },
  "region": "us-east-1"
}

Treat the number as an estimate, not a bill

It only covers requests mito itself made. It knows nothing about:

• storage costs
• your negotiated pricing
• free-tier allowances
• traffic from anything else on the account

Its real value is comparative. Run a workload two ways and see which one made an order of magnitude fewer PUT-class calls. That comparison is reliable even when the dollar figure is not.

If you point mito at a non-AWS service, the call counts stay accurate and the dollars become meaningless, because everything is priced at AWS rates. See S3-compatible storage.

Where the state lives

See Files on disk.

--reset does not delete the metrics file. It clears the counters and writes an empty file over the top. If that write fails, the command says so and exits 1, rather than claiming a reset that did not happen. It leaves the pricing cache alone.

leftovers: incomplete uploads

mito leftovers <bucket> [OPTIONS]

leftovers finds multipart uploads that were started and never finished, and can clean them up.

The problem it solves

Here is a story that happens all the time.

You start uploading a 200 GB file. S3 splits it into parts. At 180 GB, the network drops.

Those 180 GB of parts are still sitting in your bucket. S3 is still charging you for them. And here is the part that catches people out:

   ListObjectsV2 on the bucket   ──►  nothing
   the AWS console               ──►  nothing
   your monthly bill             ──►  180 GB

Incomplete uploads appear in no object listing. They are invisible to every normal way of looking at a bucket. The only place they show up is the bill.

So every failed upload, every cancelled CI job that was pushing an artifact, every laptop that went to sleep mid-transfer, leaves a silent recurring charge behind. This is one of the most common reasons an S3 bill drifts away from what the object listing says it should be.

mito's own comparison path creates scratch multipart uploads and always aborts them. But any client can leave leftovers, and one badly timed network failure is enough.

Options

The bucket can be given bare or as a URL. Both my-bucket and s3://my-bucket work.

Examples

# What is in there?
mito leftovers my-bucket

# Only the stale ones, under one prefix
mito leftovers my-bucket --prefix uploads/ --older-than 7d

# Clean them up
mito leftovers my-bucket --older-than 7d --abort

The safe habit is the same as with rm: list first with --older-than, read the output, then add --abort to the identical command.

Use --older-than even when cleaning. It protects uploads that are legitimately in flight right now. A blanket --abort would kill somebody's running transfer.

Duration format

A number and a unit: 30m, 1h, 1d, 7d. Anything it cannot parse is a usage error and exits 2.

A permanent fix instead

This command is for finding out what is already there, and for buckets whose configuration you do not control.

If you do control the bucket, set a lifecycle rule with AbortIncompleteMultipartUpload on it. Then S3 cleans up after itself and you never have to think about this again.

Exit status

See Exit codes.

Architecture

The idea

To compare two copies of a file, you normally have to read both copies.

That is an expensive sentence when one copy lives in S3. You pay for the egress. You wait for the download. And you learn nothing you could not have learned from a checksum.

So mito never downloads the big ones. It gets S3 to do the checksumming.

The trick

S3 supports a feature called additional checksums. Here is the useful consequence.

If you ask S3 to copy a byte range of an existing object (an UploadPartCopy), S3 reads those bytes on its own hardware, computes a CRC32 over them, and returns the value to you in a response header called x-amz-checksum-crc32.

That is a checksum of the remote bytes. And the bytes never crossed the network.

So the S3 side of a comparison works like this:

  1.  CreateMultipartUpload  on a scratch key
      └─► gives you an upload ID to hang the requests off

  2.  UploadPartCopy  × one per 8 MiB chunk, all in parallel
      each one naming a byte range of the object you care about
      └─► each response carries x-amz-checksum-crc32

  3.  decode that header from Base64 into a uint32_t

  4.  AbortMultipartUpload  on the scratch upload
      always. including every error path.

A 200 GB object becomes roughly 25,000 small API calls instead of 200 GB of egress.

Step 4 is not optional. A scratch upload left behind keeps billing for its parts, which is exactly the problem leftovers exists to find. mito aborts on every path out, including the ones where something went wrong.

The local side

Local chunks are checksummed in parallel:

1. Open the file once. One file descriptor no matter how many chunks, so a wide comparison does not run the process out of descriptors.
2. mmap the chunk. Zero-copy, and it pages in only what actually gets read.
3. Prefetch, then run the CRC32.
4. munmap, report progress.

Both sides at once

                          ┌──────────────┐
                          │ Orchestrator │
                          └──────┬───────┘
                    ┌────────────┴────────────┐
                    ▼                         ▼
          ┌───────────────────┐    ┌─────────────────────┐
          │ S3 task           │    │ Local task          │
          │ UploadPartCopy    │    │ mmap + crc32_hw     │
          │ per 8 MiB chunk   │    │ per 8 MiB chunk     │
          └─────────┬─────────┘    └──────────┬──────────┘
                    │                         │
                    ▼                         ▼
             vector<uint32_t>          vector<uint32_t>
                    └────────────┬────────────┘
                                 ▼
                          compare, report

The two halves run at the same time, so the total is max(local, S3) rather than the sum. That is worth doing because the two are limited by entirely different things. The local side waits on disk and CPU. The S3 side waits on network round trips.

The checksum

IEEE CRC32, the same polynomial S3 uses, computed by zlib-ng using whatever the CPU offers:

zlib-ng checks the CPU once, on the first CRC32 call, and picks one.

mito writes no CRC arithmetic of its own. It used to. A hand-written PCLMUL path returned wrong values for 16-byte-aligned buffers for its entire life, undetected, until it was deleted (issue #18). Using the library is the fix.

Because the choice is made at run time inside the library, the build sets no architecture flags. It used to pass -msse4.2 or -march=armv8-a+crc to the whole program, which tied the binary to the machine that built it. On ARM that was worse than untidy: CRC32 is optional before ARMv8.1, so the binary would crash on a chip without it instead of falling back. Today a binary built on one machine runs, and stays fast, on another.

CRC32 is a checksum, not a cryptographic hash

It is used here for exactly one reason: S3 computes it for free on its own hardware, and that is the entire reason a remote comparison can skip the download.

It catches accidental corruption, truncation and drift. Those are the failure modes that "did this upload land intact?" is really asking about.

It is no defence against someone who wants two different files to look identical. Building a 32-bit CRC collision is trivial. If your threat model includes a motivated adversary, use a hash designed for that instead.

The polynomial is load-bearing

IEEE CRC32 and CRC32-C (Castagnoli) use the same instruction family on some architectures and produce completely different numbers.

Picking the wrong one gives you a checksum that is internally consistent, passes every local test, and disagrees with S3 on every single object.

The test suite therefore has a case whose only job is to assert the result is not the CRC32-C value, next to the canonical "123456789" vector.

Narrowing a mismatch

A chunk is 8 MiB, which is a vague answer to "where do these differ?"

So when two chunks disagree, that one chunk is re-analyzed at 64 KiB granularity: 128 blocks. The report can then name byte ranges instead of just a filename.

The two-level scheme is deliberate. Checksumming everything at 64 KiB would multiply the S3 request count by 128, on a comparison that usually finds nothing. Running the fine pass only where the coarse pass found something keeps the common case cheap.

Constants

One path does not use BLOCK_SIZE. Two local files small enough to read whole (at most five chunks) are compared block by block directly, at a granularity scaled to the file size: 1 KiB below 64 KiB, up to 128 KiB above 16 MiB. The scaling aims for roughly 64 to 256 blocks whatever the size, so the report stays readable. That is the block_size you will see in a report for a small local file. Everywhere else it is 64 KiB.

8 MiB does three jobs. It is the checksum chunk, the threshold where sync switches to differential transfer, and the size below which an S3 object is simply downloaded and checksummed locally instead of compared remotely.

Error handling

Fail fast, clean up always.

• A chunk-level failure aborts the whole operation rather than reporting a partial result. A comparison that quietly skipped a chunk would claim a match it never verified.
• Parallel errors go into an atomic counter, checked once the fan-out finishes.
• AbortMultipartUpload runs on every path out, including the error paths.
• Errors are logged with context through spdlog, not printed bare.

Failure modes worth knowing

A service without additional checksums accepts the UploadPartCopy and just leaves the header out. That fails the comparison for objects above 8 MiB. See S3-compatible storage.

A service that answers a ranged GET without Content-Range gets that read rejected rather than trusted, because nothing then proves which bytes came back. This affects reads at an offset (block narrowing, chunked downloads), not whole-object reads. See S3-compatible storage.

A chunk count mismatch between the two sides means the objects are different sizes. That is reported as a difference, not an error.

GetBucketLocation denied by a cross-account policy blocks region detection. Pin the region with @region instead. See Cross-account S3.

Parallelism

A directory comparison runs in parallel in three different places, for three different reasons. This page walks through each one and the settings that reach it.

   1. DISCOVERY          walking the folder trees
          │
          ▼
   2. FILE COMPARISON    many files at once, window width adapts
          │
          ▼
   3. CHUNK WORK         many chunks of one file at once

1. Discovery: walking the trees

Both sides are enumerated as a breadth-first walk, one level at a time. The directories at each level are spread across a thread pool.

Level 0:  [root]                    1 directory, single-threaded
Level 1:  [dir1, dir2, dir3, ...]   N directories, in parallel
Level 2:  [sub1, sub2, sub3, ...]   M directories, in parallel
...

Each level is a barrier. Every directory at the current level finishes before the next level starts.

Why one level at a time, instead of just letting recursion run free?

• Memory stays bounded. Only the current and next level are held, not the whole tree.
• Progress is reportable. You can say "level 3 of 6". A depth-first walk cannot.
• Symlink loop detection stays cheap. Canonical targets already visited go in a set behind a mutex. Only symlinks need tracking, because an ordinary directory cannot be its own ancestor.

S3 listing is capped at 80 workers whatever you ask for, to stay under request rate limits. A higher value gets clamped down, with a log line saying so. Local scanning gets the full budget.

2. File comparison: the adaptive window

This is the interesting one.

Files are compared through a sliding window. The controller measures how fast work is completing and adjusts how wide that window is.

┌──────────────────────────────────────────────────────┐
│  Thread pool, size = min(max_concurrency, num_files) │
└──────────────────────────────────────────────────────┘
                          ▲
                          │ the window bounds in-flight tasks
                          │
┌──────────────────────────────────────────────────────┐
│  Adaptive controller                                 │
│   start at initial_concurrency                       │
│   measure throughput every N completions             │
│   improved by >10%  → double the window              │
│   flat twice in a row → lock it                      │
└──────────────────────────────────────────────────────┘

Why bother adapting? Because the right concurrency for 100,000 tiny objects and the right concurrency for four 50 GB objects differ by orders of magnitude. No single default serves both.

Too narrow, and an S3 comparison sits idle waiting on round trips. Too wide, and you run out of threads, memory and request quota all at once.

So the controller walks toward the knee of the curve and stops when walking stops paying.

Where it starts depends on the shape of the work:

--threads is a ceiling, not a target. Real concurrency is num_threads / threads_per_file, adapted downward from there by measurement.

The file-descriptor ceiling above all of it

Before any of that, --threads gets clamped to what the process can actually open.

Each concurrent operation is budgeted 3 descriptors, with 50 held back for everything else. So the real ceiling is:

   (RLIMIT_NOFILE soft limit − 50) / 3

mito first tries to raise its own soft limit toward the hard limit. If that is still not enough, it reduces the thread count and tells you, naming the ulimit -n value that would have allowed what you asked for:

Reducing threads from 1024 to 324 due to file descriptor limit.
To use 1024 threads, run: ulimit -n 3172

That example is a soft limit of 1024: (1024 − 50) / 3 = 324. And 1024 × 3 + 100 = 3172 to lift it.

This is the usual reason a -t 1024 run behaves like a much smaller one. A soft limit above 1,000,000 is treated as unlimited and nothing gets capped. The clamp applies to diff, sync and rm alike.

3. Chunk work inside one file

Within a single large file, chunks are checksummed concurrently.

100 MB file, 8 MiB chunks:
┌─────────┬─────────┬─────────┬─────────┬─────────┐
│ Chunk 0 │ Chunk 1 │ Chunk 2 │ Chunk 3 │ Chunk 4 │
└────┬────┴────┬────┴────┬────┴────┬────┴────┬────┘
  Thread1   Thread2   Thread3   Thread4   Thread1
     ▼         ▼         ▼         ▼         ▼
   CRC32     CRC32     CRC32     CRC32     CRC32

This trades directly against level 2. More threads per file means one file finishes sooner and fewer files are in flight. The threads_per_file table above is where that trade gets made.

Choosing settings

The defaults are meant to work. If you are tuning anyway, these are reasonable starting points:

Costs worth keeping in mind

Thread pools are not free. They are created per enumeration:

Teardown costs more than creation, because stopping a pool means waking every idle thread so it can notice the stop flag and exit.

Stacks are reserved per thread, typically 1 to 8 MB depending on the platform. So 128 workers reserve somewhere between 128 MB and 1 GB of address space. Not necessarily committed memory, but it is the practical reason the worker count is capped at 128.

Cancellation

Every pool watches a shared atomic cancellation flag, checked both in the submission loop and inside each worker.

Cancellation is cooperative. A task finishes the unit of work it is in before it notices. So Ctrl-C during an 8 MiB chunk checksum returns after that chunk, not instantly.

That is on purpose: a half-written destination file is worse than a second of waiting. The exit code is 130.

Synchronization

Condition variables for the sliding window and the level barriers. Mutexes around the shared result collections and the visited-symlink set. Atomics for progress counters and the cancellation flag.

Exit codes

   0  ─── it worked
   1  ─── it ran, and the answer was bad OR it broke partway
   2  ─── it never started. your command line was wrong.
  130  ─── you stopped it

0

The command did what you asked.

For diff this also means every single file matched, so this reads correctly:

mito diff a/ b/ && echo "identical"

1

Two quite different situations share this code:

• diff found a real difference
• something failed

That is a genuine limitation, and it is worth stating plainly rather than hiding.

If your script needs to tell those apart, do not read the exit code. Write a report and read the counters, where the two are separate:

mito diff ./data/ s3://my-bucket/data/ -o results.json
if [ $? -ne 0 ]; then
  jq '.summary | {different, errors}' results.json
fi

summary.different counts genuine mismatches. summary.errors counts files that could not be compared. See the diff report format.

leftovers also returns 1, when one or more aborts failed.

2

Something was wrong with the command line or the configuration. Every command can return this. Concretely:

• an unknown option
• missing arguments: no S3 URL or bucket, or fewer than two sources for diff
• a prefix passed to rm without --recursive
• a local path given to sync that is not a directory
• an --older-than duration that could not be parsed
• a bucket region that could not be detected

The defining feature of a 2 is that nothing was attempted. Running the same command again will fail exactly the same way, so a retry loop should treat it as terminal and give up.

130

Ctrl-C, following the usual 128 + signal convention.

Cancellation is cooperative, so the process finishes the unit of work it is in (one chunk checksum, one S3 request) before exiting. It is not instant. That is deliberate: a half-written destination file is worse than a second of delay.

An interrupted sync leaves the destination partly updated. There is no transaction and no rollback.

Running the same sync again converges, though. Anything already transferred now matches by size and carries a destination timestamp newer than its source, so it gets skipped on the second pass. Anything caught mid-transfer does not match, and gets redone.

Scripting notes

Gate a deploy on the upload having landed intact:

if ! mito diff ./build/ s3://artifacts/build/ -q; then
  echo "artifact mismatch, refusing to promote" >&2
  exit 1
fi

Nightly cleanup that does not retry a configuration mistake:

mito leftovers my-bucket --older-than 7d --abort || rc=$?
[ "${rc:-0}" -eq 2 ] && echo "config problem, not retrying" >&2

One gotcha under set -e: mito diff exiting 1 on a legitimate mismatch will abort your script. Usually that is exactly what you want. When it is not, use || true or wrap it in an explicit if.

S3-compatible storage

mito talks to Storj, MinIO, Ceph RGW and other S3-compatible services through --endpoint-url. All four commands accept it: diff, sync, rm and leftovers.

mito sync ./data/ s3://my-bucket/backup/ --endpoint-url https://gateway.storjshare.io
mito rm s3://my-bucket/old/ --recursive --force --endpoint-url https://gateway.storjshare.io

Setting an endpoint does two things beyond changing the hostname:

1. Switches to path-style addressing (https://host/bucket/key instead of https://bucket.host/key), which these services generally require
2. Turns off region auto-detection, explained below

Region detection is off with an endpoint

Auto-detection uses the AWS GetBucketLocation API. Other services may not implement it. And for storage that has no regions, the question has no answer anyway.

So when --endpoint-url is set, the default region is used instead.

If your service cares about the region string, pin it:

mito sync ./data/ s3://my-bucket/backup/@us-1 --endpoint-url https://gateway.example.com
mito leftovers my-bucket --region us-1 --endpoint-url https://gateway.example.com

The 8 MiB limit

This is the big one, so read this section before committing to a non-AWS service.

Objects larger than 8 MiB cannot be compared on a service that lacks additional checksums.

The no-download comparison works by reading x-amz-checksum-crc32 from an UploadPartCopy response. That header is part of an AWS feature called additional checksums. See Architecture.

Here is what makes it awkward. A service that does not implement it does not reject the request. It accepts it and simply leaves the header out.

   mito:     UploadPartCopy, please checksum bytes 0-8388607
   gateway:  200 OK    (and no x-amz-checksum-crc32 header)
   mito:     ...I have no checksum. I will not guess.  ──► error

mito fails the comparison with an error naming the cause, rather than reporting a match it never actually verified.

Storj behaves this way. So do some MinIO and Ceph builds.

What that means per command:

So the boundary is not the service, it is the object size. A bucket full of small objects works completely. A bucket of large ones does not.

One softer edge: sync treats a missing timestamp as "transfer it". A service whose listing omits LastModified therefore re-transfers everything, every run. That is correct behaviour, but it is not cheap, and it looks like sync ignoring its own rules. See how sync decides.

Checking your service

Test with one large object:

mito diff ./big-file.bin s3://my-bucket/big-file.bin \
  --endpoint-url https://gateway.example.com

Anything over 8 MiB will either compare cleanly or report the missing checksum header.

But that one test does not clear the whole service. It only exercises one of the two headers mito depends on. If the files match, the comparison never reaches the code that reads a byte range at an offset. So run a second test against a file you know differs:

mito diff ./changed-file.bin s3://my-bucket/changed-file.bin \
  --endpoint-url https://gateway.example.com

Ranged reads must carry Content-Range

Reading part of an object sends Range: bytes=<start>-<end>. This happens when narrowing a mismatched chunk to 64 KiB blocks, and when downloading a large file in parallel chunks.

mito requires the response to say which bytes it is carrying. RFC 9110 makes Content-Range mandatory on a 206, and AWS, MinIO and Ceph RGW were all measured sending it.

A response that omits the header, or whose header names a different range, is rejected rather than trusted. Here is the failure that rule prevents:

   mito:     give me 64 KiB starting at offset 65536
   gateway:  (ignores Range, returns the whole object)
   mito:     reads the first 64 KiB
             → right length, wrong bytes
             → reported as a difference in the wrong place

That is worse than an error, because it looks like a real answer.

A read starting at offset 0 is exempt. A whole-object answer to that is either the same bytes or a different length, and a different length is already caught. So a service that omits Content-Range still handles small files and whole-file downloads, and fails on large-file sync and on block-level narrowing. The error names the object and the range: answered the range 65536-131071 without a Content-Range.

diff and sync accept --allow-unverified-ranges for that case. It excuses a missing header only. A Content-Range naming the wrong range is still refused, because that is positive evidence the answer is not what was asked for.

Turning it on means trusting bytes you cannot place. Use it against a service you have already established omits the header, not as a general workaround. The run logs a warning saying so.

mito diff ./data/ s3://my-bucket/data/ \
  --endpoint-url https://gateway.example.com --allow-unverified-ranges

Cost accounting on non-AWS endpoints

mito stats prices everything at AWS rates.

Against a third-party service the call counts stay accurate and the dollar figure does not mean anything. Read the operation counts and ignore the total. Or reset between runs and use it to compare one workload against another.

Cross-account S3

sync and diff let each S3 side authenticate as a different AWS profile, using --source-profile and --dest-profile. Leave one off to use the default credential chain.

These are ordinary named profiles from ~/.aws/credentials and ~/.aws/config, including ones that assume a role.

# Compare two buckets in two accounts
mito diff s3://acctA-bucket/data/ s3://acctB-bucket/data/ \
  --source-profile acctA --dest-profile acctB

# Copy between them
mito sync s3://acctA-bucket/data/ s3://acctB-bucket/data/ \
  --source-profile acctA --dest-profile acctB

Two things will bite you, in this order

1. Region detection needs GetBucketLocation

Auto-detection calls GetBucketLocation on each bucket. Cross-account, that call is very often denied even when object access is granted, because bucket-level and object-level permissions are handed out separately.

The symptom: a failure before any comparison even starts.

The fix: stop asking.

mito sync s3://acctA-bucket/data/@us-east-1 s3://acctB-bucket/data/@eu-west-1 \
  --source-profile acctA --dest-profile acctB

Pinning @region is worth doing anyway. It saves a round trip per bucket.

2. An S3-to-S3 copy needs a policy on the source bucket

This is the one that catches everybody. It is a property of S3, not of mito.

A server-side copy is one API call, sent to the destination, naming the source in the x-amz-copy-source header. There is only one set of credentials on that request.

   ┌──────────────────────────────────────────────┐
   │  request: "copy acctA-bucket/x to here"      │
   │  sent to: acctB (the destination)            │
   │  signed with: --dest-profile                 │
   └──────────────────────────────────────────────┘
                       │
                       │  so the DESTINATION principal
                       │  must be able to read the SOURCE
                       ▼
   source bucket policy must grant acctB:
       s3:GetObject   on the objects
       s3:ListBucket  on the bucket

--source-profile authenticates the source listing only. It does not, and cannot, authenticate the copy itself.

So a real cross-account copy also needs a source bucket policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "AWS": "arn:aws:iam::DEST-ACCOUNT-ID:role/YourRole" },
    "Action": ["s3:GetObject", "s3:ListBucket"],
    "Resource": [
      "arn:aws:s3:::acctA-bucket",
      "arn:aws:s3:::acctA-bucket/*"
    ]
  }]
}

Without it you get a confusing partial failure: the listing succeeds, so mito knows exactly which objects to move, and then cannot move a single one of them.

What needs no extra grant

Only the server-side copy has this requirement. All of these are fine with per-side profiles alone:

• Cross-account diff. Each side is read with its own profile, independently.
• Upload. Local to S3, one profile.
• Download. S3 to local, one profile.

The pattern is simple once you see it:

   bytes pass through your machine   ──►  each side authenticated separately,
                                          no cross-account request to authorize

   bytes move inside AWS             ──►  one request, one credential,
                                          needs the extra grant

A server-side copy is faster precisely because the bytes skip your machine. That is also exactly why it needs the extra permission.

Object ownership after a copy

An object copied into a bucket is owned by the account that wrote it, which is the destination principal.

Depending on the destination bucket's Object Ownership setting, that can leave objects the bucket owner cannot read. If you control the destination, setting Bucket owner enforced avoids the whole class of problem.

mito sets no ACLs and does not try to manage ownership.

Files on disk

mito keeps two small pieces of state outside your project. Neither one holds credentials, and neither names a bucket or an object.

Security and privacy covers the wider picture.

   ~/.local/share/mitosync/cloud_metrics.json   API call counters   (Linux)
   ~/Library/Application Support/MitoSync/      same               (macOS)

   ~/.mitosync/pricing_cache.json               S3 price table     (both)

   ./results.json                               only when you ask with -o

Metrics

The cumulative API counters behind mito stats, stored as cloud_metrics.json:

The directory is created on first use, recursively, because ~/.local/share does not exist on a minimal install.

Deleting the file resets the counters.

mito stats --reset does not delete it. It clears the counters in memory and writes an empty metrics file over the top. If that write fails, the command reports the failure and exits 1 rather than claiming a reset that never happened.

Pricing cache

The S3 price table used for cost estimates, cached as pricing_cache.json in ~/.mitosync/, keyed by region with a 24-hour lifetime.

The prices are built-in per-region defaults compiled into the binary, not fetched from AWS. Live lookup against the AWS Pricing API is not implemented (#106), so the cache currently stores the same table the binary already contains and the file earns nothing. It exists for the day the lookup lands.

Deleting the file is harmless.

Note that the two files live in different directories. The pricing cache uses ~/.mitosync/ on every platform, while metrics follow the platform convention. That is a wrinkle, not a design statement.

AWS credentials

mito reads the standard AWS chain and writes nothing to it:

• AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN
• ~/.aws/credentials and ~/.aws/config, including named profiles
• An ECS task role

Named profiles are selected per S3 side with --source-profile and --dest-profile. See Cross-account S3.

EC2 instance roles do not work.

mito sets AWS_EC2_METADATA_DISABLED=true at startup, unconditionally, overwriting any value you set.

The reason: off an EC2 instance, an instance-metadata lookup can only ever time out, and it adds that timeout to every cold start. Disabling it removes the delay.

The cost: on an actual EC2 instance, it also removes instance-role credentials from the chain. A run that relies on them will fail to authenticate. Use environment variables, a profile, or an ECS task role there.

What gets written where you run it

Report files, and only when you ask for one:

mito diff ./data/ s3://my-bucket/data/ -o results.json

Nothing else is written to the working directory.

There is one exception, as a fallback. If the application data directory cannot be determined and cannot be created, metrics land in the current directory instead.

Scratch state in S3

Comparing a remote object creates a scratch multipart upload, and aborts it on every path out including the error paths. See Architecture.

If a process is killed hard enough to skip that abort, the parts are left behind and S3 keeps billing for them. mito leftovers is how you find and clear those.

Security and privacy

mito holds AWS credentials and can delete objects. Both of those deserve a plain explanation.

Credentials

mito reads the standard AWS chain and writes nothing back to it: environment variables, ~/.aws/credentials and ~/.aws/config including named profiles, or an ECS task role.

It stores no credential of its own. None reaches a report, a log line, or the metrics file.

Per-side profiles (--source-profile, --dest-profile) pick among those same named profiles. See Cross-account S3.

Instance metadata lookup is disabled at startup. mito sets AWS_EC2_METADATA_DISABLED=true unconditionally, overwriting any value you set.

Off an EC2 instance, that removes a lookup that can only time out. On one, it removes instance-role credentials from the chain entirely, so a run relying on them fails to authenticate. Use a profile, environment variables, or an ECS task role there.

What it writes

Two small state files, neither of which names a bucket or an object:

• cloud_metrics.json — per-operation call counts, byte totals, latencies, and the regions seen
• pricing_cache.json — the built-in S3 price table used for cost estimates, keyed by region

Paths are in Files on disk.

Reports are the sensitive artefact

A report written with -o names local paths, bucket names and object keys. That is the file worth thinking about.

It is created with an ordinary open, which means:

• your umask decides its permissions. Run umask 077 first if that matters.
• an existing file at that path is truncated.
• a symlink at that path is followed.

So treat -o the way you would treat any shell redirection. Do not point it at a directory another user can write to.

Destructive operations

Three independent gates stand between a typo and data loss.

   1. rm without --force        ──►  preview only, nothing deleted

   2. rm on a prefix without -r ──►  error, not a mass delete

   3. a listing that failed     ──►  the whole run stops
                                     nothing planned, nothing deleted

The third one deserves the most explanation, because it is the least obvious.

If either side could not be enumerated completely, sync plans nothing and deletes nothing, and diff reports nothing. A partial listing is indistinguishable from a smaller tree. Under --delete, every file the failed listing forgot to mention looks exactly like a destination orphan to be removed. In a comparison, those same files look exactly like files present on only one side.

See sync and diff.

--dry-run on sync, and the no---force preview on rm, are the intended way to check a command before it does anything.

Scratch state in S3

Comparing a remote object creates a scratch multipart upload and aborts it on every path out, including the error paths.

A process killed hard enough to skip that abort leaves parts behind. S3 keeps billing for them, and they appear in no object listing. mito leftovers finds and clears them.

Object ownership

An object written by an S3-to-S3 copy is owned by the account that wrote it, which is the destination principal.

Depending on the destination bucket's Object Ownership setting, that can leave objects the bucket owner cannot read. mito sets no ACLs and does not manage ownership. See Cross-account S3.

Transport

All S3 traffic goes through the AWS SDK for C++ over TLS, using OpenSSL as linked by vcpkg.

mito does not disable certificate verification and offers no flag to do so.

--endpoint-url changes the host and switches to path-style addressing. It does not relax any of this.