[Tfug] disk usage mystery..

[JD Rogers]

On Tue, Mar 6, 2012 at 8:32 AM, JD Rogers <rogersjd at gmail.com> wrote:
>> If you have the time, I would suggest rebooting (or using telinit to
>> switch) into a single user run level.  Then you should just have the
>> kernel, init, and your shell running.  If it is *still* leaking, then
>> it seems that perhaps you have a kernel bug?
>
> Not a bad idea.. I may try that when I can.. probably wednesday.

Haven't had a chance to wait a day in single user mode yet, but I did
try "telinit 1" with interesting results. The disk was full but after
entering runlevel 1, the disk had 900MB free.

The disk was freed up even though switching to singleuser mode did not
unmount the ecryptfs homedir.  I think the errors I was seeing from
ecryptfs are unrelated to the disk usage issue. The fact that if I
don't start firefox, those errors never arise points to that, but for
a conclusive test I'll have to try letting the disk fill up while not
ever starting FF.

In any case, there certainly seems to be a process somewhere that is
tying up diskspace, but the fact that it does not show up using lsof
or du is very perplexing. I guess the next step will be waiting for a
full disk and then killing daemons and processes recklessly until I
see a change. Then maybe I'll try the no FF thing or maybe waiting in
singleuser mode to see if the disk fills.

Still a very strange issue.

> [snip]
>> Here's another shot in the dark: if it *is* in fact a kernel file
>> system bug, then perhaps it is related the the ext4 journal?  I should
>> think it has a relatively small maximum size, but maybe it is growing
>> out of control and this data is released by an unmount?  Seems like
>> you would encounter other effects from this, though.
>>
>> Another unlikely but possible culprit could be ecryptfs.  You didn't
>> say if /home (or wherever the encrypted data is stored) is on its own
>> partition.  If it is sharing space with root, maybe it's using up the
>> extra space somehow?
>
> I'm leaning towards ecryptfs. Sheepishly, I must admit I didn't notice
> this earlier.. I am apparently also getting bazillions of these in my
> logs:
> [ 8923.440844] Valid eCryptfs headers not found in file header region
> or xattr region
> [ 8923.440849] Either the lower file is not in a valid eCryptfs
> format, or the key could not be retrieved. Plaintext passthrough mode
> is not enabled; returning -EIO
>
> I found a few bug reports that mostly seem to be duplicates of
> https://bugs.launchpad.net/ecryptfs/+bug/509180
> but my symptoms are not really the same, and this might be totally
> unrelated, because these errors stop when I close firefox. Yet the
> disk usage does not resolve itself.
>
> It appears I have some more digging to do. At least I feel like I have
> some things to try. Thanks for the suggestions!