Wednesday, November 30, 2011

Disk usage not accounted for in Linux

I had a filesystem nearly at full capacity (99%, in this case.) This server had a single root filesystem and du -sh /*listed only about 20% of the used capacity.

I suspected an open file handle that hadn't been released, even though the actual file had been deleted. Normally, if a process dies, it will release the handles, though a child process might still be around. Anyway, I turned to lsof and looked for deleted:

lsof | grep deleted

syslogd   30139      root    7w      REG              253,0 6794843241    1307979 /var/log/ldap.log (deleted)

Syslog was the culprit. After a restart of syslogd, the handle was released and all was well.