Jeremy Katz wrote:

On Tue, 2008-05-13 at 23:55 +1200, Nigel Jones wrote:

Anyway, what this means, is that when nagios has been complaining about
cvs-int recently, in particular the fact that /git has reached WARNING.
After a bit of hunting around, I found /repo/pkgs using 168GiB of the
192GiB available, which is understandable, Fedora has got huge.


Problem here, is that there are a LOT of old tarballs in that folder,
which leaves me wondering if we should do a spring clean ~1 mo after
release.



We can't do this. We need to keep the tarballs for packages which were
released (in any form, be it rawhide, an update, or the actual release
-- and probably even updates-testing) for 3 years to comply with the
terms of the GPL.

Not to mention that removing them would break any ability to go back and
try to bisect or find the source of a problem in an automated fashion.
And yes, I've actually had to go back to ancient history to do so in the
past with some packages :-/

We were just discussing this on #fedora-admin, the other alternative
would be (as I mentioned) move them to some sort of publically available
archive storage setup, with the Makefile pointing towards that instead
of where we normally point.


Anyway, I'm just trying to kick the ball about, I'm sure there is a
miracle solution but imo at the very least it'd be somewhat nice to
unladen cvs-int of some of the burden.


- Nigel

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

Nigel Jones (dev@nigelj.com) said:
> Problem here, is that there are a LOT of old tarballs in that folder, which
> leaves me wondering if we should do a spring clean ~1 mo after release.

Until we get something like correspondingsource up and running, we don't
have a good mechanism for weeding out only the tarballs that correspond
to builds that were not/are not shipped.

Bill

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

On Tuesday 13 May 2008, Nigel Jones wrote:
> Hey guys,
>
> I seem to have deleted the nagios notification that I want to mention in
> particular, but as I noted in SOP/Nagios the %ages noted in the e-mails
> are what is left, so inode=99% doesn't mean that 99% of the inodes are
> used, it means 99% are still free.
>
> Anyway, what this means, is that when nagios has been complaining about
> cvs-int recently, in particular the fact that /git has reached WARNING.
> After a bit of hunting around, I found /repo/pkgs using 168GiB of the
> 192GiB available, which is understandable, Fedora has got huge.
>
> Problem here, is that there are a LOT of old tarballs in that folder,
> which leaves me wondering if we should do a spring clean ~1 mo after
> release.
<snip>

We already have plans to move the lookaside cache to the netapp. at least
that was the last plan i was aware of.

> Diskspace isn't cheap, so I like delete old tarballs, I also like this
> option because it's not like they disappear completely, they should be
> in the src.rpm's already on archive.fp.o and if we accidentally delete 1
> or 2 that are still needed, well grab it from src.rpm...
>
> This leads on to my second item...
>
> xenbuilder2 has run out of diskspace in /, it's down to 32M, thankfully
> koji has disabled it so it's safe for now, but wouldn't it be nice to
> throw say a 50GB partition dedicated solely for /var/lib/mock &
> /mnt/build? Yes, yes, I know money, but once again, builds are getting
> bigger so 'it'd be nice'.

xenbuilder1 and hammer2 are the oldest boxes we have in the buildsystem.
closely followed by ppc1 xenbuilder2 is also quite an old box now. im
cleaning up /var/lib/mock on xenbuilder2 now. failed builds dont get cleaned
up automatically. we probably should reap them more often.

Dennis
_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

On Tue, 2008-05-13 at 08:58 -0500, Dennis Gilmore wrote:
> On Tuesday 13 May 2008, Nigel Jones wrote:
> > Hey guys,
> >
> > I seem to have deleted the nagios notification that I want to mention in
> > particular, but as I noted in SOP/Nagios the %ages noted in the e-mails
> > are what is left, so inode=99% doesn't mean that 99% of the inodes are
> > used, it means 99% are still free.
> >
> > Anyway, what this means, is that when nagios has been complaining about
> > cvs-int recently, in particular the fact that /git has reached WARNING.
> > After a bit of hunting around, I found /repo/pkgs using 168GiB of the
> > 192GiB available, which is understandable, Fedora has got huge.
> >
> > Problem here, is that there are a LOT of old tarballs in that folder,
> > which leaves me wondering if we should do a spring clean ~1 mo after
> > release.
> <snip>
>
> We already have plans to move the lookaside cache to the netapp. at least
> that was the last plan i was aware of.
>
> > Diskspace isn't cheap, so I like delete old tarballs, I also like this
> > option because it's not like they disappear completely, they should be
> > in the src.rpm's already on archive.fp.o and if we accidentally delete 1
> > or 2 that are still needed, well grab it from src.rpm...
> >
> > This leads on to my second item...
> >
> > xenbuilder2 has run out of diskspace in /, it's down to 32M, thankfully
> > koji has disabled it so it's safe for now, but wouldn't it be nice to
> > throw say a 50GB partition dedicated solely for /var/lib/mock &
> > /mnt/build? Yes, yes, I know money, but once again, builds are getting
> > bigger so 'it'd be nice'.
>
> xenbuilder1 and hammer2 are the oldest boxes we have in the buildsystem.
> closely followed by ppc1 xenbuilder2 is also quite an old box now. im
> cleaning up /var/lib/mock on xenbuilder2 now. failed builds dont get cleaned
> up automatically. we probably should reap them more often.

Buildroots for failed Koji builds do get cleaned up after 4 hours (to
give someone time to diagnose the problem if desired). However,
xenbuilder2 is also running plague, which never cleans up buildroots as
far as I know.


_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list