find -mtime +7 -size +1M -delete: clearing old, oversized logs

find -mtime +7 -size +1M -delete: clearing old, oversized logs

Scenario: /srv/kl-logs on a Docker host is filling the disk. The job is to delete every .log file under it, in any subdirectory, that is older than 7 days and larger than 1 MiB. Everything else stays: recent logs, small old logs, and the compressed .gz archives, however old or large they are. One find command does it. But every test in that command has a quirk, and you want to know them before you point -delete at a disk in production.

This is the exercise "Clear out the old, oversized logs" (linux-060) from awesome_labs, my free, open-source collection of hands-on labs that run on local VMs. You get a real shell, you do the task, and the lab checks the result.

Build the expression one test at a time

Never write the -delete version first. Start wide and add one test at a time, and look at what is left after each step:

kpatronas@docker-host:/srv/kl-logs$ find . -type f -name '*.log'
./web/web-x.log
./web/web-1.log
./web/web-2.log
./web/web-3.log
./worker/worker-1.log
./worker/worker-2.log
./worker/worker-3.log
./billing/billing-1.log
./billing/billing-3.log
./billing/billing-2.log
./api/api-3.log
./api/api-1.log
./api/api-2.log
./api/api-y.log

The .gz archives under each archive/ directory don't show up. -name '*.log' has to match the whole file name, so api-old.log.gz doesn't match: it ends in .gz. Next, only the old ones:

kpatronas@docker-host:/srv/kl-logs$ find . -type f -name '*.log' -mtime +7
./web/web-x.log
./web/web-2.log
./worker/worker-1.log
./worker/worker-2.log
./worker/worker-3.log
./billing/billing-1.log
./billing/billing-2.log
./api/api-3.log
./api/api-1.log

api-y.log has dropped out. It's big, but it was written yesterday, so it stays. Now only the old and large ones:

kpatronas@docker-host:/srv/kl-logs$ find . -type f -name '*.log' -mtime +7 -size +1M
./web/web-x.log
./web/web-2.log
./worker/worker-1.log
./worker/worker-2.log
./api/api-3.log
./api/api-1.log

worker-3.log, billing-1.log and billing-2.log are old but small, so they stay too. These six files are exactly what has to go.

How it works

  • .: start here and go down into every subdirectory. find is recursive by default.
  • -type f: regular files only, no directories or symlinks.
  • -name '*.log': the base name has to match the glob. Quote it. Without quotes, the shell expands *.log first if any .log files exist in the current directory. With two or more, GNU find fails with paths must precede expression. With exactly one, it quietly looks for that single name.
  • -mtime +7: last modified more than 7 whole days ago (see the rounding trap below).
  • -size +1M: larger than 1 MiB. M is mebibytes (1,048,576 bytes), not megabytes.
  • Tests written next to each other are joined by an implied -a (AND), so a file must pass all of them.

Permission denied, even on a file you could write

Swap the last test's output for -delete and run it:

kpatronas@docker-host:/srv/kl-logs$ find . -type f -name '*.log' -mtime +7 -size +1M -delete
find: cannot delete ‘./web/web-x.log’: Permission denied
find: cannot delete ‘./web/web-2.log’: Permission denied
find: cannot delete ‘./worker/worker-1.log’: Permission denied
find: cannot delete ‘./worker/worker-2.log’: Permission denied
find: cannot delete ‘./api/api-3.log’: Permission denied
find: cannot delete ‘./api/api-1.log’: Permission denied

The files belong to root. But the file's own permissions are not what decides this. Deleting a file removes its entry from the directory, so you need write (and execute) permission on the directory. I checked it with a log file anyone can write to (666) inside a root-owned 755 directory. An unprivileged user still can't delete it:

$ ls -ld a a/big.log
drwxr-xr-x 2 root root    4096 Nov 18 13:59 a
-rw-rw-rw- 1 root root 3072000 Nov  8 13:59 a/big.log

$ find . -type f -name '*.log' -mtime +7 -size +1M -delete    # as user nobody
find: cannot delete ‘./a/big.log’: Permission denied

That user could empty the file with : > a/big.log, but can't remove it. (The sticky bit on directories like /tmp adds one more rule: there, only the file's owner can delete it.) So the fix here is sudo:

kpatronas@docker-host:/srv/kl-logs$ sudo find . -type f -name '*.log' -mtime +7 -size +1M -delete
kpatronas@docker-host:/srv/kl-logs$

No output at all. -delete is an action, and once an expression has an action, find no longer adds its default -print. If you want a record of what was removed, ask for it. Put -print before -delete, so each name is printed and then deleted:

sudo find /srv/kl-logs -type f -name '*.log' -mtime +7 -size +1M -print -delete

The rounding traps

Both -mtime and -size round before they compare, and they round in different directions. I made a set of test files with head -c and touch -d and ran them through GNU findutils 4.9.0 (Ubuntu 24.04):

$ ls a/
1M1K.log  7d20h.log  8d1h.log  exact1M.log  old.log.gz  small.log  tiny.log
#  1M1K   = 1025 KiB, 10 days old     7d20h = 3000 KiB, 7 days 20 hours old
#  exact1M = 1024 KiB, 10 days old    8d1h  = 3000 KiB, 8 days 1 hour old
#  small  = 500 KiB, 10 days old      tiny  = 1 KiB, 1 day old

$ find . -name '*.log' -mtime +7 | sort
./a/1M1K.log
./a/8d1h.log
./a/exact1M.log
./a/small.log

-mtime rounds the age down to whole days. The file that is 7 days and 20 hours old counts as "7 days", and 7 is not more than 7, so -mtime +7 skips it. In practice -mtime +7 means "at least 8 full days old". If you want exactly 7 × 24 hours, count in minutes:

$ find . -name '*.log' -mmin +$((7*24*60)) | sort
./a/1M1K.log
./a/7d20h.log
./a/8d1h.log
./a/exact1M.log
./a/small.log

-size rounds the size up to whole units. With M, a 1-byte file and a 1,048,576-byte file both count as "1". So +1M behaves as you would expect, strictly more than 1 MiB:

$ find . -name '*.log' -size +1M | sort
./a/1M1K.log
./a/7d20h.log
./a/8d1h.log

exact1M.log is not included, and 1025 KiB is. But turn it around and it breaks. -size -1M means "rounds up to less than 1 MiB", and only an empty file does that:

$ find . -name '*.log' -size -1M
$ find . -name '*.log' -size -1024k
./a/small.log
./a/tiny.log

The first one finds nothing, not even the 1 KiB file. To find small files, use a smaller unit (k) so the rounding hardly matters.

Never put -delete first

find evaluates the expression from left to right, for every file. -delete is not a setting you can put anywhere. It runs when evaluation reaches it, and the tests after it don't protect anything:

$ ls b
a.log  c.log  new.txt
$ find b -delete -name '*.log' -mtime +7
find: ‘b/a.log’: No such file or directory
find: ‘b/c.log’: No such file or directory
$ ls b
ls: cannot access 'b': No such file or directory

Every file went, including new.txt, which isn't a log and isn't old, and so did the directory b itself. The two errors are -mtime trying to check files that -delete had just removed. -delete also turns on -depth (it handles a directory's contents before the directory), which is how the empty directory could be removed too. So the safe habit is the one the lab's hints teach: run the exact command with -print, read the list, then press up-arrow and replace -print with -delete. Don't change anything else.

The disk is still full?

If a process still has one of the deleted logs open, its name is gone but its blocks are not freed until the process closes the file. df won't move. List those files with:

sudo lsof +L1

For a log that is still being written, empty it instead of deleting it (sudo truncate -s 0 file.log), or restart the process that holds it. There's more on this in how to delete a file only when it's not used by any process. If this cleanup keeps coming back, the long-term fix is a logrotate rule with compress and maxage. The find line is for the day the disk is already full.

Checklist

  • Build the expression one test at a time, and read the output each time.
  • Quote the -name pattern.
  • -mtime +7 means 8 or more whole days. Use -mmin when the hours matter.
  • -size +1M is fine. -size -1M only finds empty files.
  • Permission denied on delete is about the directory, not the file.
  • -delete goes last. Use -print -delete if you want a list of what went.

Try it yourself

Reading about find is easy. Typing it against a disk that's filling up is better practice. You can find this exercise and many other real-world ones in awesome_labs, for free. There are over 1,500 exercises across Linux, Docker, Kubernetes, networking, Git, Terraform, Ansible, cloud, databases, security and more. Each has a scenario, hints you reveal one at a time, and a full solution if you get stuck. It runs on macOS and Linux:

git clone https://github.com/kpatr1981/awesome_labs.git
cd awesome_labs
make build
./awesome_labs install   # sets up Lima and anything else that's missing
awesome_labs             # browse the tracks and pick an exercise

If you'd rather not build it, there are ready-made binaries for Linux and macOS on the releases page. This one is linux-060 in Linux — Files, services & recovery. The file sizes and ages are random on every reset, so you can't just memorise the list.

$ ./buy-me-a-coffee ☕ // found this useful? I write these in my spare time