Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   EXT3 Users (http://www.linux-archive.org/ext3-users/)
-   -   Numbers behind "df" and "tune2fs" (http://www.linux-archive.org/ext3-users/713479-numbers-behind-df-tune2fs.html)

Nicolas Michel 09-16-2013 10:16 AM

Numbers behind "df" and "tune2fs"
 
Hello guys,

I have some difficulties to understand what really are the numbers
behing "df" and tune2fs. You'll find the output of tune2fs and df
below, on which my maths are based.

Here are my maths:

A tune2fs on an ext3 FS tell me the FS size is 3284992 block large. It
also tell me that the size of one block is 4096 (bytes if I'm not
wrong?). So my maths tell me that the disk is 3284992 * 4096 =
13455327232 bytes or 13455327232 / 1024 /1024 /1024 = 12.53 GB.

A df --block-size=1 on the same FS tell me the disk is 13243846656
which is 211480576 bytes smaller than what tune2fs tell me.

In gigabytes, it means:
* for df, the disk is 12.33 GB
* for tune2fs, the disk is 12.53 GB

I thought that maybe df is only taking into account the real blocks
available for users. So I tried to remove the reserved blocks and the
GDT blocks:
(3284992 - 164249 - 801) * 4096 = 12779282432
or in GB : 12779282432 / 1024 / 1024 / 1024 = 11.90 Gb ...

My last thought was that "Reserved block" in tune2fs was not only the
reserved blocks for root (which is 5% per default on my system) but
take into account all other reserved blocks fo the fs internal usage.
So:
(3284992 - 164249) * 4096 = 12782563328
In GB : 11.90 Gb (the difference is not significative with a precision of 2.

So I'm lost ...

Is someone have an explanation? I would really really be grateful.
Nicolas

------------------------------
---------

Here is the output of df and tune2fs :

$ tune2fs -l /dev/mapper/datavg-datalogslv
tune2fs 1.41.9 (22-Aug-2009)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 4e5bea3e-3e61-4fc8-9676-e5177522911c
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index
filetype needs_recovery sparse_super large_file
Filesystem flags: unsigned_directory_hash
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 822544
Block count: 3284992
Reserved block count: 164249
Free blocks: 3109325
Free inodes: 822348
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 801
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8144
Inode blocks per group: 509
Filesystem created: Wed Aug 28 08:30:10 2013
Last mount time: Wed Sep 11 17:16:56 2013
Last write time: Thu Sep 12 09:38:02 2013
Mount count: 18
Maximum mount count: 27
Last checked: Wed Aug 28 08:30:10 2013
Check interval: 15552000 (6 months)
Next check after: Mon Feb 24 07:30:10 2014
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 28
Desired extra isize: 28
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: ad2251a9-ac33-4e5e-b933-af49cb4f2bb3
Journal backup: inode blocks

$ df --block-size=1 /dev/mapper/datavg-datalogslv
Filesystem 1B-blocks Used Available Use% Mounted on
/dev/mapper/datavg-datalogslv 13243846656 563843072 12007239680 5% /logs


--
Nicolas MICHEL

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

Eric Sandeen 09-16-2013 02:39 PM

Numbers behind "df" and "tune2fs"
 
On 9/16/13 5:16 AM, Nicolas Michel wrote:
> Hello guys,
>
> I have some difficulties to understand what really are the numbers
> behing "df" and tune2fs. You'll find the output of tune2fs and df
> below, on which my maths are based.
>
> Here are my maths:
>
> A tune2fs on an ext3 FS tell me the FS size is 3284992 block large. It
> also tell me that the size of one block is 4096 (bytes if I'm not
> wrong?). So my maths tell me that the disk is 3284992 * 4096 =
> 13455327232 bytes or 13455327232 / 1024 /1024 /1024 = 12.53 GB.
>
> A df --block-size=1 on the same FS tell me the disk is 13243846656
> which is 211480576 bytes smaller than what tune2fs tell me.

By default, df on extN assumes that metadata used by the filesystem
was never available for your use and is not part of the filesystem
space.

Documentation/filesystems/ext3.txt says:

bsddf (*) Make 'df' act like BSD.
minixdf Make 'df' act like Minix.

which is pretty unhelpful I suppose. ;)

The mount man page is a little more helpful:

bsddf|minixdf
Set the behaviour for the statfs system call. The minixdf
behaviour is to return in the f_blocks field the total number
of blocks of the filesystem, while the bsddf behaviour (which
is the default) is to subtract the overhead blocks used by the
ext2 filesystem and not available for file storage.

You're seeing the latter behavior. if you mount with -o minixdf you should
see what you expect. (Too bad there's no "linuxdf?") :)

> In gigabytes, it means:
> * for df, the disk is 12.33 GB
> * for tune2fs, the disk is 12.53 GB
>
> I thought that maybe df is only taking into account the real blocks
> available for users. So I tried to remove the reserved blocks and the
> GDT blocks:
> (3284992 - 164249 - 801) * 4096 = 12779282432
> or in GB : 12779282432 / 1024 / 1024 / 1024 = 11.90 Gb ...

you're on the right track, but you forgot the journal space, all the
preallocated inode table blocks, etc.

-Eric

> My last thought was that "Reserved block" in tune2fs was not only the
> reserved blocks for root (which is 5% per default on my system) but
> take into account all other reserved blocks fo the fs internal usage.
> So:
> (3284992 - 164249) * 4096 = 12782563328
> In GB : 11.90 Gb (the difference is not significative with a precision of 2.
>
> So I'm lost ...
>
> Is someone have an explanation? I would really really be grateful.
> Nicolas
>
> ------------------------------
> ---------
>
> Here is the output of df and tune2fs :
>
> $ tune2fs -l /dev/mapper/datavg-datalogslv
> tune2fs 1.41.9 (22-Aug-2009)
> Filesystem volume name: <none>
> Last mounted on: <not available>
> Filesystem UUID: 4e5bea3e-3e61-4fc8-9676-e5177522911c
> Filesystem magic number: 0xEF53
> Filesystem revision #: 1 (dynamic)
> Filesystem features: has_journal ext_attr resize_inode dir_index
> filetype needs_recovery sparse_super large_file
> Filesystem flags: unsigned_directory_hash
> Default mount options: (none)
> Filesystem state: clean
> Errors behavior: Continue
> Filesystem OS type: Linux
> Inode count: 822544
> Block count: 3284992
> Reserved block count: 164249
> Free blocks: 3109325
> Free inodes: 822348
> First block: 0
> Block size: 4096
> Fragment size: 4096
> Reserved GDT blocks: 801
> Blocks per group: 32768
> Fragments per group: 32768
> Inodes per group: 8144
> Inode blocks per group: 509
> Filesystem created: Wed Aug 28 08:30:10 2013
> Last mount time: Wed Sep 11 17:16:56 2013
> Last write time: Thu Sep 12 09:38:02 2013
> Mount count: 18
> Maximum mount count: 27
> Last checked: Wed Aug 28 08:30:10 2013
> Check interval: 15552000 (6 months)
> Next check after: Mon Feb 24 07:30:10 2014
> Reserved blocks uid: 0 (user root)
> Reserved blocks gid: 0 (group root)
> First inode: 11
> Inode size: 256
> Required extra isize: 28
> Desired extra isize: 28
> Journal inode: 8
> Default directory hash: half_md4
> Directory Hash Seed: ad2251a9-ac33-4e5e-b933-af49cb4f2bb3
> Journal backup: inode blocks
>
> $ df --block-size=1 /dev/mapper/datavg-datalogslv
> Filesystem 1B-blocks Used Available Use% Mounted on
> /dev/mapper/datavg-datalogslv 13243846656 563843072 12007239680 5% /logs
>
>

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

Nicolas Michel 09-16-2013 02:44 PM

Numbers behind "df" and "tune2fs"
 
Thanks for you help. I also tried adding some other informations as you suggest:
I can also take into account:
- "Reserved block count: XXXXXXX" from tune2fs that gives me the
number of blocks reserved for root
- Reserved GDT blocks: XXX

But I didn't thought about the FS journal. How can I gather
information about it? (it's size and any other information?)

2013/9/16 Eric Sandeen <sandeen@redhat.com>:
> On 9/16/13 5:16 AM, Nicolas Michel wrote:
>> Hello guys,
>>
>> I have some difficulties to understand what really are the numbers
>> behing "df" and tune2fs. You'll find the output of tune2fs and df
>> below, on which my maths are based.
>>
>> Here are my maths:
>>
>> A tune2fs on an ext3 FS tell me the FS size is 3284992 block large. It
>> also tell me that the size of one block is 4096 (bytes if I'm not
>> wrong?). So my maths tell me that the disk is 3284992 * 4096 =
>> 13455327232 bytes or 13455327232 / 1024 /1024 /1024 = 12.53 GB.
>>
>> A df --block-size=1 on the same FS tell me the disk is 13243846656
>> which is 211480576 bytes smaller than what tune2fs tell me.
>
> By default, df on extN assumes that metadata used by the filesystem
> was never available for your use and is not part of the filesystem
> space.
>
> Documentation/filesystems/ext3.txt says:
>
> bsddf (*) Make 'df' act like BSD.
> minixdf Make 'df' act like Minix.
>
> which is pretty unhelpful I suppose. ;)
>
> The mount man page is a little more helpful:
>
> bsddf|minixdf
> Set the behaviour for the statfs system call. The minixdf
> behaviour is to return in the f_blocks field the total number
> of blocks of the filesystem, while the bsddf behaviour (which
> is the default) is to subtract the overhead blocks used by the
> ext2 filesystem and not available for file storage.
>
> You're seeing the latter behavior. if you mount with -o minixdf you should
> see what you expect. (Too bad there's no "linuxdf?") :)
>
>> In gigabytes, it means:
>> * for df, the disk is 12.33 GB
>> * for tune2fs, the disk is 12.53 GB
>>
>> I thought that maybe df is only taking into account the real blocks
>> available for users. So I tried to remove the reserved blocks and the
>> GDT blocks:
>> (3284992 - 164249 - 801) * 4096 = 12779282432
>> or in GB : 12779282432 / 1024 / 1024 / 1024 = 11.90 Gb ...
>
> you're on the right track, but you forgot the journal space, all the
> preallocated inode table blocks, etc.
>
> -Eric
>
>> My last thought was that "Reserved block" in tune2fs was not only the
>> reserved blocks for root (which is 5% per default on my system) but
>> take into account all other reserved blocks fo the fs internal usage.
>> So:
>> (3284992 - 164249) * 4096 = 12782563328
>> In GB : 11.90 Gb (the difference is not significative with a precision of 2.
>>
>> So I'm lost ...
>>
>> Is someone have an explanation? I would really really be grateful.
>> Nicolas
>>
>> ------------------------------
>> ---------
>>
>> Here is the output of df and tune2fs :
>>
>> $ tune2fs -l /dev/mapper/datavg-datalogslv
>> tune2fs 1.41.9 (22-Aug-2009)
>> Filesystem volume name: <none>
>> Last mounted on: <not available>
>> Filesystem UUID: 4e5bea3e-3e61-4fc8-9676-e5177522911c
>> Filesystem magic number: 0xEF53
>> Filesystem revision #: 1 (dynamic)
>> Filesystem features: has_journal ext_attr resize_inode dir_index
>> filetype needs_recovery sparse_super large_file
>> Filesystem flags: unsigned_directory_hash
>> Default mount options: (none)
>> Filesystem state: clean
>> Errors behavior: Continue
>> Filesystem OS type: Linux
>> Inode count: 822544
>> Block count: 3284992
>> Reserved block count: 164249
>> Free blocks: 3109325
>> Free inodes: 822348
>> First block: 0
>> Block size: 4096
>> Fragment size: 4096
>> Reserved GDT blocks: 801
>> Blocks per group: 32768
>> Fragments per group: 32768
>> Inodes per group: 8144
>> Inode blocks per group: 509
>> Filesystem created: Wed Aug 28 08:30:10 2013
>> Last mount time: Wed Sep 11 17:16:56 2013
>> Last write time: Thu Sep 12 09:38:02 2013
>> Mount count: 18
>> Maximum mount count: 27
>> Last checked: Wed Aug 28 08:30:10 2013
>> Check interval: 15552000 (6 months)
>> Next check after: Mon Feb 24 07:30:10 2014
>> Reserved blocks uid: 0 (user root)
>> Reserved blocks gid: 0 (group root)
>> First inode: 11
>> Inode size: 256
>> Required extra isize: 28
>> Desired extra isize: 28
>> Journal inode: 8
>> Default directory hash: half_md4
>> Directory Hash Seed: ad2251a9-ac33-4e5e-b933-af49cb4f2bb3
>> Journal backup: inode blocks
>>
>> $ df --block-size=1 /dev/mapper/datavg-datalogslv
>> Filesystem 1B-blocks Used Available Use% Mounted on
>> /dev/mapper/datavg-datalogslv 13243846656 563843072 12007239680 5% /logs
>>
>>
>



--
Nicolas MICHEL

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

Eric Sandeen 09-16-2013 04:25 PM

Numbers behind "df" and "tune2fs"
 
On 9/16/13 9:44 AM, Nicolas Michel wrote:
> Thanks for you help. I also tried adding some other informations as you suggest:
> I can also take into account:
> - "Reserved block count: XXXXXXX" from tune2fs that gives me the
> number of blocks reserved for root
> - Reserved GDT blocks: XXX
>
> But I didn't thought about the FS journal. How can I gather
> information about it? (it's size and any other information?)

# dumpe2fs /dev/$YOUR_DEVICE | grep Journal
dumpe2fs 1.41.12 (17-May-2010)
Journal inode: 8
Journal backup: inode blocks
Journal features: journal_incompat_revoke
Journal size: 128M
Journal length: 32768

But you also need to take into account inode tables, inode
allocation bitmaps, block allocation bitmaps ...

-Eric

> 2013/9/16 Eric Sandeen <sandeen@redhat.com>:
>> On 9/16/13 5:16 AM, Nicolas Michel wrote:
>>> Hello guys,
>>>
>>> I have some difficulties to understand what really are the numbers
>>> behing "df" and tune2fs. You'll find the output of tune2fs and df
>>> below, on which my maths are based.
>>>
>>> Here are my maths:
>>>
>>> A tune2fs on an ext3 FS tell me the FS size is 3284992 block large. It
>>> also tell me that the size of one block is 4096 (bytes if I'm not
>>> wrong?). So my maths tell me that the disk is 3284992 * 4096 =
>>> 13455327232 bytes or 13455327232 / 1024 /1024 /1024 = 12.53 GB.
>>>
>>> A df --block-size=1 on the same FS tell me the disk is 13243846656
>>> which is 211480576 bytes smaller than what tune2fs tell me.
>>
>> By default, df on extN assumes that metadata used by the filesystem
>> was never available for your use and is not part of the filesystem
>> space.
>>
>> Documentation/filesystems/ext3.txt says:
>>
>> bsddf (*) Make 'df' act like BSD.
>> minixdf Make 'df' act like Minix.
>>
>> which is pretty unhelpful I suppose. ;)
>>
>> The mount man page is a little more helpful:
>>
>> bsddf|minixdf
>> Set the behaviour for the statfs system call. The minixdf
>> behaviour is to return in the f_blocks field the total number
>> of blocks of the filesystem, while the bsddf behaviour (which
>> is the default) is to subtract the overhead blocks used by the
>> ext2 filesystem and not available for file storage.
>>
>> You're seeing the latter behavior. if you mount with -o minixdf you should
>> see what you expect. (Too bad there's no "linuxdf?") :)
>>
>>> In gigabytes, it means:
>>> * for df, the disk is 12.33 GB
>>> * for tune2fs, the disk is 12.53 GB
>>>
>>> I thought that maybe df is only taking into account the real blocks
>>> available for users. So I tried to remove the reserved blocks and the
>>> GDT blocks:
>>> (3284992 - 164249 - 801) * 4096 = 12779282432
>>> or in GB : 12779282432 / 1024 / 1024 / 1024 = 11.90 Gb ...
>>
>> you're on the right track, but you forgot the journal space, all the
>> preallocated inode table blocks, etc.
>>
>> -Eric
>>
>>> My last thought was that "Reserved block" in tune2fs was not only the
>>> reserved blocks for root (which is 5% per default on my system) but
>>> take into account all other reserved blocks fo the fs internal usage.
>>> So:
>>> (3284992 - 164249) * 4096 = 12782563328
>>> In GB : 11.90 Gb (the difference is not significative with a precision of 2.
>>>
>>> So I'm lost ...
>>>
>>> Is someone have an explanation? I would really really be grateful.
>>> Nicolas
>>>
>>> ------------------------------
>>> ---------
>>>
>>> Here is the output of df and tune2fs :
>>>
>>> $ tune2fs -l /dev/mapper/datavg-datalogslv
>>> tune2fs 1.41.9 (22-Aug-2009)
>>> Filesystem volume name: <none>
>>> Last mounted on: <not available>
>>> Filesystem UUID: 4e5bea3e-3e61-4fc8-9676-e5177522911c
>>> Filesystem magic number: 0xEF53
>>> Filesystem revision #: 1 (dynamic)
>>> Filesystem features: has_journal ext_attr resize_inode dir_index
>>> filetype needs_recovery sparse_super large_file
>>> Filesystem flags: unsigned_directory_hash
>>> Default mount options: (none)
>>> Filesystem state: clean
>>> Errors behavior: Continue
>>> Filesystem OS type: Linux
>>> Inode count: 822544
>>> Block count: 3284992
>>> Reserved block count: 164249
>>> Free blocks: 3109325
>>> Free inodes: 822348
>>> First block: 0
>>> Block size: 4096
>>> Fragment size: 4096
>>> Reserved GDT blocks: 801
>>> Blocks per group: 32768
>>> Fragments per group: 32768
>>> Inodes per group: 8144
>>> Inode blocks per group: 509
>>> Filesystem created: Wed Aug 28 08:30:10 2013
>>> Last mount time: Wed Sep 11 17:16:56 2013
>>> Last write time: Thu Sep 12 09:38:02 2013
>>> Mount count: 18
>>> Maximum mount count: 27
>>> Last checked: Wed Aug 28 08:30:10 2013
>>> Check interval: 15552000 (6 months)
>>> Next check after: Mon Feb 24 07:30:10 2014
>>> Reserved blocks uid: 0 (user root)
>>> Reserved blocks gid: 0 (group root)
>>> First inode: 11
>>> Inode size: 256
>>> Required extra isize: 28
>>> Desired extra isize: 28
>>> Journal inode: 8
>>> Default directory hash: half_md4
>>> Directory Hash Seed: ad2251a9-ac33-4e5e-b933-af49cb4f2bb3
>>> Journal backup: inode blocks
>>>
>>> $ df --block-size=1 /dev/mapper/datavg-datalogslv
>>> Filesystem 1B-blocks Used Available Use% Mounted on
>>> /dev/mapper/datavg-datalogslv 13243846656 563843072 12007239680 5% /logs
>>>
>>>
>>
>
>
>

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

Nicolas Michel 09-17-2013 06:14 AM

Numbers behind "df" and "tune2fs"
 
OK. Thanks for the journal information. I thought tune2fs -l and
dumpe2fs were the same. In reality it's almost the same but not
entirely ^^

I hear you about all the internal mecanisms that make the FS working
or give it some features, and I do understand that it takes some place
on the disk. However what I don't understand is why the number given
in the "available column" is called "available" if it's not really the
case and we have to remove some other thousand/million of bytes for
some internal mecanisms.

In other words I don't understand why the "used" percentage given by
df does not reflects the values given by itself in the other columns.

I can live with it but I really would like to understand why things
are what they are. Is there an historic reason? Or maybe a technical
reason that makes thoses numbers some added values?

The least would be to have the df algorithms documented somewhere? A
document that explains intentions and how the values are obtained.
The same for tune2fs and dumpe2fs (what really means the given numbers?)

2013/9/16 Eric Sandeen <sandeen@redhat.com>:
> On 9/16/13 9:44 AM, Nicolas Michel wrote:
>> Thanks for you help. I also tried adding some other informations as you suggest:
>> I can also take into account:
>> - "Reserved block count: XXXXXXX" from tune2fs that gives me the
>> number of blocks reserved for root
>> - Reserved GDT blocks: XXX
>>
>> But I didn't thought about the FS journal. How can I gather
>> information about it? (it's size and any other information?)
>
> # dumpe2fs /dev/$YOUR_DEVICE | grep Journal
> dumpe2fs 1.41.12 (17-May-2010)
> Journal inode: 8
> Journal backup: inode blocks
> Journal features: journal_incompat_revoke
> Journal size: 128M
> Journal length: 32768
>
> But you also need to take into account inode tables, inode
> allocation bitmaps, block allocation bitmaps ...
>
> -Eric
>
>> 2013/9/16 Eric Sandeen <sandeen@redhat.com>:
>>> On 9/16/13 5:16 AM, Nicolas Michel wrote:
>>>> Hello guys,
>>>>
>>>> I have some difficulties to understand what really are the numbers
>>>> behing "df" and tune2fs. You'll find the output of tune2fs and df
>>>> below, on which my maths are based.
>>>>
>>>> Here are my maths:
>>>>
>>>> A tune2fs on an ext3 FS tell me the FS size is 3284992 block large. It
>>>> also tell me that the size of one block is 4096 (bytes if I'm not
>>>> wrong?). So my maths tell me that the disk is 3284992 * 4096 =
>>>> 13455327232 bytes or 13455327232 / 1024 /1024 /1024 = 12.53 GB.
>>>>
>>>> A df --block-size=1 on the same FS tell me the disk is 13243846656
>>>> which is 211480576 bytes smaller than what tune2fs tell me.
>>>
>>> By default, df on extN assumes that metadata used by the filesystem
>>> was never available for your use and is not part of the filesystem
>>> space.
>>>
>>> Documentation/filesystems/ext3.txt says:
>>>
>>> bsddf (*) Make 'df' act like BSD.
>>> minixdf Make 'df' act like Minix.
>>>
>>> which is pretty unhelpful I suppose. ;)
>>>
>>> The mount man page is a little more helpful:
>>>
>>> bsddf|minixdf
>>> Set the behaviour for the statfs system call. The minixdf
>>> behaviour is to return in the f_blocks field the total number
>>> of blocks of the filesystem, while the bsddf behaviour (which
>>> is the default) is to subtract the overhead blocks used by the
>>> ext2 filesystem and not available for file storage.
>>>
>>> You're seeing the latter behavior. if you mount with -o minixdf you should
>>> see what you expect. (Too bad there's no "linuxdf?") :)
>>>
>>>> In gigabytes, it means:
>>>> * for df, the disk is 12.33 GB
>>>> * for tune2fs, the disk is 12.53 GB
>>>>
>>>> I thought that maybe df is only taking into account the real blocks
>>>> available for users. So I tried to remove the reserved blocks and the
>>>> GDT blocks:
>>>> (3284992 - 164249 - 801) * 4096 = 12779282432
>>>> or in GB : 12779282432 / 1024 / 1024 / 1024 = 11.90 Gb ...
>>>
>>> you're on the right track, but you forgot the journal space, all the
>>> preallocated inode table blocks, etc.
>>>
>>> -Eric
>>>
>>>> My last thought was that "Reserved block" in tune2fs was not only the
>>>> reserved blocks for root (which is 5% per default on my system) but
>>>> take into account all other reserved blocks fo the fs internal usage.
>>>> So:
>>>> (3284992 - 164249) * 4096 = 12782563328
>>>> In GB : 11.90 Gb (the difference is not significative with a precision of 2.
>>>>
>>>> So I'm lost ...
>>>>
>>>> Is someone have an explanation? I would really really be grateful.
>>>> Nicolas
>>>>
>>>> ------------------------------
>>>> ---------
>>>>
>>>> Here is the output of df and tune2fs :
>>>>
>>>> $ tune2fs -l /dev/mapper/datavg-datalogslv
>>>> tune2fs 1.41.9 (22-Aug-2009)
>>>> Filesystem volume name: <none>
>>>> Last mounted on: <not available>
>>>> Filesystem UUID: 4e5bea3e-3e61-4fc8-9676-e5177522911c
>>>> Filesystem magic number: 0xEF53
>>>> Filesystem revision #: 1 (dynamic)
>>>> Filesystem features: has_journal ext_attr resize_inode dir_index
>>>> filetype needs_recovery sparse_super large_file
>>>> Filesystem flags: unsigned_directory_hash
>>>> Default mount options: (none)
>>>> Filesystem state: clean
>>>> Errors behavior: Continue
>>>> Filesystem OS type: Linux
>>>> Inode count: 822544
>>>> Block count: 3284992
>>>> Reserved block count: 164249
>>>> Free blocks: 3109325
>>>> Free inodes: 822348
>>>> First block: 0
>>>> Block size: 4096
>>>> Fragment size: 4096
>>>> Reserved GDT blocks: 801
>>>> Blocks per group: 32768
>>>> Fragments per group: 32768
>>>> Inodes per group: 8144
>>>> Inode blocks per group: 509
>>>> Filesystem created: Wed Aug 28 08:30:10 2013
>>>> Last mount time: Wed Sep 11 17:16:56 2013
>>>> Last write time: Thu Sep 12 09:38:02 2013
>>>> Mount count: 18
>>>> Maximum mount count: 27
>>>> Last checked: Wed Aug 28 08:30:10 2013
>>>> Check interval: 15552000 (6 months)
>>>> Next check after: Mon Feb 24 07:30:10 2014
>>>> Reserved blocks uid: 0 (user root)
>>>> Reserved blocks gid: 0 (group root)
>>>> First inode: 11
>>>> Inode size: 256
>>>> Required extra isize: 28
>>>> Desired extra isize: 28
>>>> Journal inode: 8
>>>> Default directory hash: half_md4
>>>> Directory Hash Seed: ad2251a9-ac33-4e5e-b933-af49cb4f2bb3
>>>> Journal backup: inode blocks
>>>>
>>>> $ df --block-size=1 /dev/mapper/datavg-datalogslv
>>>> Filesystem 1B-blocks Used Available Use% Mounted on
>>>> /dev/mapper/datavg-datalogslv 13243846656 563843072 12007239680 5% /logs
>>>>
>>>>
>>>
>>
>>
>>
>



--
Nicolas MICHEL

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

Nicolas Michel 09-17-2013 06:34 AM

Numbers behind "df" and "tune2fs"
 
In fact the thing I really want to achieve is to be able to find the
values and the algorithm that enable me to reproduce the percentage
given by df (and to understand deeply what it means).

Why do I need it? Because I'm trying to write some script to do
capacity planning and space problem forecast. Currently I don't really
know which values I should use to do it. (I could use the percentage
given by df but it lacks some precisions to make usefull forecasts)

2013/9/17 Nicolas Michel <be.nicolas.michel@gmail.com>:
> OK. Thanks for the journal information. I thought tune2fs -l and
> dumpe2fs were the same. In reality it's almost the same but not
> entirely ^^
>
> I hear you about all the internal mecanisms that make the FS working
> or give it some features, and I do understand that it takes some place
> on the disk. However what I don't understand is why the number given
> in the "available column" is called "available" if it's not really the
> case and we have to remove some other thousand/million of bytes for
> some internal mecanisms.
>
> In other words I don't understand why the "used" percentage given by
> df does not reflects the values given by itself in the other columns.
>
> I can live with it but I really would like to understand why things
> are what they are. Is there an historic reason? Or maybe a technical
> reason that makes thoses numbers some added values?
>
> The least would be to have the df algorithms documented somewhere? A
> document that explains intentions and how the values are obtained.
> The same for tune2fs and dumpe2fs (what really means the given numbers?)
>
> 2013/9/16 Eric Sandeen <sandeen@redhat.com>:
>> On 9/16/13 9:44 AM, Nicolas Michel wrote:
>>> Thanks for you help. I also tried adding some other informations as you suggest:
>>> I can also take into account:
>>> - "Reserved block count: XXXXXXX" from tune2fs that gives me the
>>> number of blocks reserved for root
>>> - Reserved GDT blocks: XXX
>>>
>>> But I didn't thought about the FS journal. How can I gather
>>> information about it? (it's size and any other information?)
>>
>> # dumpe2fs /dev/$YOUR_DEVICE | grep Journal
>> dumpe2fs 1.41.12 (17-May-2010)
>> Journal inode: 8
>> Journal backup: inode blocks
>> Journal features: journal_incompat_revoke
>> Journal size: 128M
>> Journal length: 32768
>>
>> But you also need to take into account inode tables, inode
>> allocation bitmaps, block allocation bitmaps ...
>>
>> -Eric
>>
>>> 2013/9/16 Eric Sandeen <sandeen@redhat.com>:
>>>> On 9/16/13 5:16 AM, Nicolas Michel wrote:
>>>>> Hello guys,
>>>>>
>>>>> I have some difficulties to understand what really are the numbers
>>>>> behing "df" and tune2fs. You'll find the output of tune2fs and df
>>>>> below, on which my maths are based.
>>>>>
>>>>> Here are my maths:
>>>>>
>>>>> A tune2fs on an ext3 FS tell me the FS size is 3284992 block large. It
>>>>> also tell me that the size of one block is 4096 (bytes if I'm not
>>>>> wrong?). So my maths tell me that the disk is 3284992 * 4096 =
>>>>> 13455327232 bytes or 13455327232 / 1024 /1024 /1024 = 12.53 GB.
>>>>>
>>>>> A df --block-size=1 on the same FS tell me the disk is 13243846656
>>>>> which is 211480576 bytes smaller than what tune2fs tell me.
>>>>
>>>> By default, df on extN assumes that metadata used by the filesystem
>>>> was never available for your use and is not part of the filesystem
>>>> space.
>>>>
>>>> Documentation/filesystems/ext3.txt says:
>>>>
>>>> bsddf (*) Make 'df' act like BSD.
>>>> minixdf Make 'df' act like Minix.
>>>>
>>>> which is pretty unhelpful I suppose. ;)
>>>>
>>>> The mount man page is a little more helpful:
>>>>
>>>> bsddf|minixdf
>>>> Set the behaviour for the statfs system call. The minixdf
>>>> behaviour is to return in the f_blocks field the total number
>>>> of blocks of the filesystem, while the bsddf behaviour (which
>>>> is the default) is to subtract the overhead blocks used by the
>>>> ext2 filesystem and not available for file storage.
>>>>
>>>> You're seeing the latter behavior. if you mount with -o minixdf you should
>>>> see what you expect. (Too bad there's no "linuxdf?") :)
>>>>
>>>>> In gigabytes, it means:
>>>>> * for df, the disk is 12.33 GB
>>>>> * for tune2fs, the disk is 12.53 GB
>>>>>
>>>>> I thought that maybe df is only taking into account the real blocks
>>>>> available for users. So I tried to remove the reserved blocks and the
>>>>> GDT blocks:
>>>>> (3284992 - 164249 - 801) * 4096 = 12779282432
>>>>> or in GB : 12779282432 / 1024 / 1024 / 1024 = 11.90 Gb ...
>>>>
>>>> you're on the right track, but you forgot the journal space, all the
>>>> preallocated inode table blocks, etc.
>>>>
>>>> -Eric
>>>>
>>>>> My last thought was that "Reserved block" in tune2fs was not only the
>>>>> reserved blocks for root (which is 5% per default on my system) but
>>>>> take into account all other reserved blocks fo the fs internal usage.
>>>>> So:
>>>>> (3284992 - 164249) * 4096 = 12782563328
>>>>> In GB : 11.90 Gb (the difference is not significative with a precision of 2.
>>>>>
>>>>> So I'm lost ...
>>>>>
>>>>> Is someone have an explanation? I would really really be grateful.
>>>>> Nicolas
>>>>>
>>>>> ------------------------------
>>>>> ---------
>>>>>
>>>>> Here is the output of df and tune2fs :
>>>>>
>>>>> $ tune2fs -l /dev/mapper/datavg-datalogslv
>>>>> tune2fs 1.41.9 (22-Aug-2009)
>>>>> Filesystem volume name: <none>
>>>>> Last mounted on: <not available>
>>>>> Filesystem UUID: 4e5bea3e-3e61-4fc8-9676-e5177522911c
>>>>> Filesystem magic number: 0xEF53
>>>>> Filesystem revision #: 1 (dynamic)
>>>>> Filesystem features: has_journal ext_attr resize_inode dir_index
>>>>> filetype needs_recovery sparse_super large_file
>>>>> Filesystem flags: unsigned_directory_hash
>>>>> Default mount options: (none)
>>>>> Filesystem state: clean
>>>>> Errors behavior: Continue
>>>>> Filesystem OS type: Linux
>>>>> Inode count: 822544
>>>>> Block count: 3284992
>>>>> Reserved block count: 164249
>>>>> Free blocks: 3109325
>>>>> Free inodes: 822348
>>>>> First block: 0
>>>>> Block size: 4096
>>>>> Fragment size: 4096
>>>>> Reserved GDT blocks: 801
>>>>> Blocks per group: 32768
>>>>> Fragments per group: 32768
>>>>> Inodes per group: 8144
>>>>> Inode blocks per group: 509
>>>>> Filesystem created: Wed Aug 28 08:30:10 2013
>>>>> Last mount time: Wed Sep 11 17:16:56 2013
>>>>> Last write time: Thu Sep 12 09:38:02 2013
>>>>> Mount count: 18
>>>>> Maximum mount count: 27
>>>>> Last checked: Wed Aug 28 08:30:10 2013
>>>>> Check interval: 15552000 (6 months)
>>>>> Next check after: Mon Feb 24 07:30:10 2014
>>>>> Reserved blocks uid: 0 (user root)
>>>>> Reserved blocks gid: 0 (group root)
>>>>> First inode: 11
>>>>> Inode size: 256
>>>>> Required extra isize: 28
>>>>> Desired extra isize: 28
>>>>> Journal inode: 8
>>>>> Default directory hash: half_md4
>>>>> Directory Hash Seed: ad2251a9-ac33-4e5e-b933-af49cb4f2bb3
>>>>> Journal backup: inode blocks
>>>>>
>>>>> $ df --block-size=1 /dev/mapper/datavg-datalogslv
>>>>> Filesystem 1B-blocks Used Available Use% Mounted on
>>>>> /dev/mapper/datavg-datalogslv 13243846656 563843072 12007239680 5% /logs
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>
>
>
> --
> Nicolas MICHEL



--
Nicolas MICHEL

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

Eric Sandeen 09-18-2013 03:25 PM

Numbers behind "df" and "tune2fs"
 
On 9/17/13 1:34 AM, Nicolas Michel wrote:
> In fact the thing I really want to achieve is to be able to find the
> values and the algorithm that enable me to reproduce the percentage
> given by df (and to understand deeply what it means).
>
> Why do I need it? Because I'm trying to write some script to do
> capacity planning and space problem forecast. Currently I don't really
> know which values I should use to do it. (I could use the percentage
> given by df but it lacks some precisions to make usefull forecasts)
>

If you want "the truth" just mount -o minixdf, tune2fs to 0 blocks reserved,
and you'll get the actual number of blocks contained in the filesystem,
the actual number of blocks used, and the actual blocks free. Why extN
made it so complicated, I don't really know.

If you want to see how the sausage is made, look at ext3_statfs()
for all the hairy calculations. (ext4_statfs() is even more complex).

Until recently, it was all complicated enough that even the kernel code got
it wrong. ;)

0875a2b448fcaba67010850cf9649293a5ef653d ext4: include journal blocks in df overhead calcs
b72f78cb63fb595af63fc781dced0a6fd354e572 ext4: fix overhead calculations in ext4_stats, again
952fc18ef9ec707ebdc16c0786ec360295e5ff15 ext4: fix overhead calculation used by ext4_statfs()
...

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users


All times are GMT. The time now is 03:13 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.