Vserver en:playing around with quota

I plan to help the linux-vserver people by helping them testing quota on the 2.6 kernel's

First off all the log with Bertl:

<Bertl> google should be able to find relevant info from both, the IRC logs 
and the ML
<crypt> ok
<Bertl> use the site:irc.13thfloor.at or site:linux-vserver.org
<crypt> i'll search
<Bertl> IIRC, at least two folks started some quota test scripts
<Bertl> so maybe you can pick up where they stopped 
<Bertl> regardless of that, we definitely need to have a test tool which knows 
how to test/handle the following cases:
<Bertl> - inode allocation/deallocation
<Bertl> - block allocation/reserve/deallocation
<Bertl> - user and group id checks
<Bertl> and it should be able to do that on all major filesystems
<Bertl> i.e. ext2/3 for a start and jfs, reiser, xfs later
<Bertl> if you need something, just ask, I'll be around (just busy)

<hmeyer> ok...this works
<daniel_hozac> crypt: IIRC brc was the one who got furthest with the testing.
<Bertl> crypt: http://irc.13thfloor.at/LOG/2006-08/LOG_2006-08-09.txt
<Bertl> time index 1155155073
<crypt> ok thanks
<Bertl> http://irc.13thfloor.at/LOG/2005-07/LOG_2005-07-14.txt
<Bertl> time index 1121344767
<Bertl> crypt: as said on irc before (see logs) only the last step is missing 
(in devel) i.e. extending the quota hashes to several per context hashes
<Bertl> crypt: so, what is required, is a script or tool verifying all the 
possible quota changes and special cases (checks) for 'normal' quota

<Bertl> http://vserver.13thfloor.at/Experimental/OLD-DELTA/delta-cq0.01.diff was the last one for 2.6
<crypt> Bertl, are these thoughts correct? http://paste.linux-vserver.org/680
<Bertl> hmm, well that's all nice, but:
<Bertl> - we will not need vroot at the beginning, it is a proxy for security and we do not care about security atm
<crypt> ok
<Bertl> - quota consists of user and group quota, with soft (grace) and hard limits
<Bertl> - the following cases need some 'reproduceable' testing
<Bertl>   + single inode/block for user/group
<Bertl>   + several inodes/blocks for user/group add/remove
<Bertl>   + over soft (grace) limit for user/group
<Bertl>   + hitting hard limit for user/group
<Bertl> - as quota is also inode/block based, setting and overriding quota as root has to be tested too
<Bertl> - as quota is also inode/block based, setting and overriding quota as root has to be tested too
<crypt> do you now, whether there is a document out in the world, where such functions are described of quota?
<Bertl> http://souptonuts.sourceforge.net/quota_tutorial.html
<crypt> thanks a lot
<crypt> i copied all the things you mentioned
<Bertl> there is also an API document, looking for that right now

Because of these I will make the following steps:

- 100M partition, use the existing testfs.sh framework to create reproduce able filesystems and setups
mount the partition with the following options: loop,rw,usrquota,grpquota
setup usrquota and test on that partition
setup grpquota and test on that partition
-> these tests should work without any problems
create a skript like testfs.sh to create reproduceable filesystems for the next step of testing
Later I will post my test scripts so maybe other person's could help me testing

First step: http://vserver.13thfloor.at/Experimental/TOOLS/setugid.c How setugid works:

<Bertl> hey crypt!
<crypt> Bertl, one question what does the setugid util do?
<daniel_hozac> sets the uid/gid and executes a program? :)
<Bertl> it sets the user and/or group id for a process
<crypt> ok you said to me that this programm would be useful for the testing with quota
<daniel_hozac> yes, as you'll need to use a specific uid/gid (the one you set the quota for)
<crypt> ok so i have to set the vserver's root dir to a specific uid and gid with that tool?
<daniel_hozac> for the quota tests, you shouldn't think of vservers at all.
<daniel_hozac> all you have is the uids/gids (initially, later xid will be added).
<crypt> ok I think i have a general understanding problem off what to do exactly for the tests ;)
<crypt> do I have to create afew dirs in the shared partiotn set some uid gids on that dir's and have to test wether the quota i added for these dirs is handeld correct?
<Bertl> you need not only test for uid=gid=0 (root) but more important for uid!=0 and gid!=0 cases as well
<Bertl> as testing with different users and groups becomes complicated, as it usually depends on a passwd file :) this tool helps you to simplify testsw
<Bertl> -w
<crypt> so i create quotas on dir's where the users are not in passwd
<crypt> file
<daniel_hozac> the lack of passwd entries helps with the reproducibility of the tests.
<Bertl> quotas are not related to dirs, that seems to be amisconception on your side
<crypt> ahhh ok
<Bertl> on the contrary, a directory will also add to quota (in this case inode quota not block quota)
<crypt> ok so let's say: on the shared partition i create a quota for a user that doesn't exist like uid and gid 7000
<crypt> inode quota
<Bertl> as I said last time, play around alittle with quota first
<crypt> yeah i tried to do that ;) and that works as i thought so ;)
<Bertl> get a feeling for how quota works, when it changes and who is affected from quota limits in what ways
<crypt> ok ;) i'll read a lot more about quota
<daniel_hozac> crypt: btw, did i understand correctly before that you are adding quota tests to testfs?
<crypt> daniel_hozac, yes
<crypt> i'm using the ext3 filesystem
<crypt> cerated by testfs.sh
<daniel_hozac> testfs should create lots of different filesystems.
<crypt> yes but i have limit it to ext3 after seeing, that the rest works as well ;)
<crypt> or is that not correct?
<daniel_hozac> sure, creating the tests on a known-good filesystem seems like a good idea.
<crypt> ok thanks ;)
<crypt> so next steps for me (informing more about how quota is working), using the setugid tool to set limits for user's who doesn't exist and after that testing the quota stuff again in this nvironement
<crypt> environment
<Bertl> hmm, no .. you can set limits for all kind of uid/gid which do not exists easily
<crypt> hmm but for what using xour tool?
<Bertl> but you will have a hard time to test them without a tool like the setugid :)
<crypt> -x+y
<Bertl> i.e. you cannot easily extend a file as uid=7 without it
<Bertl> and you cannot query quota for uid=7 (as user) without it
<crypt> so i create/expand files through your util for the user who does not exist
<Bertl> for example, to verify that he is accounted properly for, and quota follows/limits
<Bertl> look, let me give an example here:
<Bertl> you want to test quota as non root, okay?
<crypt> yes
<Bertl> so you ahve to become a user, right?
<crypt> yes
<Bertl> what user will you become?
<crypt> the user who is limited through quota
<Bertl> okay, so you need to create a user (which does not own files)
<crypt> right
<Bertl> that is something which will be kind of hard to do in a test script, right?
<crypt> yes this is right
<Bertl> instead of doing that, and possibly destroying a setup, you can simply 'switch' to some uid
<Bertl> that's what the tool is for
<crypt> a nice :)
<crypt> thanks a lot for that full description!
<Bertl> you're welcome!
<crypt> Bertl, what filesystem user id do i have to use for the quota tests? same as user id?
<Bertl> make yourself more comfortable with the term fsuid, effective uid and uid first, same goes for the guid, and test which one actually affects the filesystem
<crypt> thanks
<Bertl> also note the difference between 'echo test >a' and 'touch b'
<crypt> k

My first tests:

./testfs.sh -t -D /dev/loop0 -M /vservers/
Linux-VServer FS Test [V0.14] Copyright (C) 2005-2006 H.Poetzl
Linux 2.6.18-vs2.1.1-gentoo-r1 i686/0.30.211
VCI:  0002:0102 273 031183f6 (ugid24)
---
testing ext3 filesystem ...
mke2fs 1.39 (29-May-2006)
[000]. (ext3 format)
xid related tests ...
[001]. [002]. [011]. [012]. [014]. [015]. [019].
[020]. [021]. [022]. [023]. [024]. [025]. [026]. [027]. [028].
[033]. [034]. [035]. [037].
xattr related tests ...
[101]. [102]. [103]. [104]. [106]. [108]. [109].
[112]. [113]. [114]. [115]. [116]* [117]. [118]. [119].
[121]. [122]. [123]. [124]. [199].
mount /dev/loop0 /vservers/ -o tagxid,usrquota,grpquota
dd if=/dev/zero of=test.img bs=2M count=5000
for i in $(seq 15); do touch "test_$i"; done

For the moment all works fine :) More will follow