non-Euclidean filesystem

qaz · edit-2 1 month ago

non-Euclidean filesystem

@digdilem · 2 months ago

This is a common thing one needs to do. Not all linux gui tools are perfect, and some calculate number differently (1000 vs 1024 soon mounts up to big differences). Also, if you’re running as a user, you’re not going to be seeing all the files.

Here’s how I do it as a sysadmin:

As root, run:

du /* -shc |sort -h

“disk usage for all files in root, displaying a summary instead of listing all sub-files, and human-readable numbers, with a total. Then sort the results so that the largest are at the bottom”

Takes a while (many minutes, up to hours or days if you’ve slow disks, many files or remote filesystems) to run on most systems and there’s no output until it finishes because it’s piping to sort. You can speed it up by omitting the “|sort -h” bit, and you’ll get summaries when each top level dir is checked, but you won’t have a nice sorted output.

You’ll probably get some permission errors when it goes through /proc or /dev

You can be more targetted by picking some of the common places, like /var - here’s mine from a debian system, takes a couple of seconds. I’ll often start with /var as it’s a common place for systems to start filling up along with /home.

root@scrofula:~# du /var/* -shc |sort -h
0       /var/lock
0       /var/run
4.0K    /var/local
4.0K    /var/mail
4.0K    /var/opt
168K    /var/tmp
4.1M    /var/spool
5.5M    /var/backups
781M    /var/log
787M    /var/cache
8.3G    /var/www
36G     /var/lib
46G     total

Here we can see /var/lib has a lot of stuff in it, so we can look into that with du /var/lib/* -shc|sort -h - it turns out mine has some big databases in /var/lib/mysql and a bunch of docker stuff in /var/lib/docker, not surprising.

Sometimes you just won’t be able to tally what you’re seeing with what you’re using. Often that might be due to a locked file having been deleted or truncated, but the lock’s still preventing the OS from seeing the recovered space. That generally sorts itself out with various timeouts, but you can try and find it with lsof, or if the machine isn’t doing much, a quick reboot.

@deadbeef79000@lemmy.nz · 2 months ago

I tend to use du -hxd1 / rather than -hs so that it stays on one filesystem (usually I’m looking for usage of only one file system) and descends one directory.

@digdilem · 1 month ago

Good thinking. That would speed things up on some systems for sure.

t0m5k1 · 2 months ago

This is the way.