FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Build System

 
 
LinkBack Thread Tools
 
Old 01-20-2011, 10:31 PM
Mike McLean
 
Default Cleaning scratch & work directories

On 01/20/2011 05:05 PM, Anthony Messina wrote:
> I'm continuing work a on a private Koji instance (Kerberos) and am now
> working on automating cleanup, etc.
>
> It is mentioned here
> http://lists.fedoraproject.org/pipermail/buildsys/2010-December/003415.html
> that "Koji leaves it to the administrator to clean out the scratch build
> directory."
>
> Is there anything special about cleaning up scratch builds or the
> work/{cli-build,cli-import,tasks} directories for that matter?
>
> Do I need to interact with the Koji database in any way to let koji know
> these files are gone? What I mean is, can I simply remove the files and
> directories without any Koji integration?

The contents of work and scratch are not tracked in the database. The
mechanism I use is a simple cron job that simply looks at modification
times.

The main things you want to watch out for is deleting things too early.
Scratch builds aren't much use to anyone if you delete them before the
builder gets a chance to poke at them. The work directory is used as a
temporary holding zone by some tasks, so you don't want to delete stuff
out from under a running task. A reasonable time-based check is
sufficient to avoid this.

In my koji instance we keep content under work/ for two weeks and
content under scratch/ for one week.

Using tmpwatch is a another possibility. I believe it looks at access
times by default.
--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
 
Old 01-22-2011, 12:37 AM
Anthony Messina
 
Default Cleaning scratch & work directories

On 01/20/2011 05:31 PM, Mike McLean wrote:
> On 01/20/2011 05:05 PM, Anthony Messina wrote:
>> > I'm continuing work a on a private Koji instance (Kerberos) and am now
>> > working on automating cleanup, etc.
>> >
>> > It is mentioned here
>> > http://lists.fedoraproject.org/pipermail/buildsys/2010-December/003415.html
>> > that "Koji leaves it to the administrator to clean out the scratch build
>> > directory."
>> >
>> > Is there anything special about cleaning up scratch builds or the
>> > work/{cli-build,cli-import,tasks} directories for that matter?
>> >
>> > Do I need to interact with the Koji database in any way to let koji know
>> > these files are gone? What I mean is, can I simply remove the files and
>> > directories without any Koji integration?
> The contents of work and scratch are not tracked in the database. The
> mechanism I use is a simple cron job that simply looks at modification
> times.
>
> The main things you want to watch out for is deleting things too early.
> Scratch builds aren't much use to anyone if you delete them before the
> builder gets a chance to poke at them. The work directory is used as a
> temporary holding zone by some tasks, so you don't want to delete stuff
> out from under a running task. A reasonable time-based check is
> sufficient to avoid this.
>
> In my koji instance we keep content under work/ for two weeks and
> content under scratch/ for one week.
>
> Using tmpwatch is a another possibility. I believe it looks at access
> times by default.

Thank you very much for your help. With that assistance, I've scraped
together a "koji-maint" script & RPM that might be helpful for us
small-timers to automate Koji maintenance and mash repo creation:

http://messinet.com/rpms/browser/koji-maint

Feedback is welcome! Again, thanks! -A

--
Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
 
Old 01-27-2011, 04:06 PM
 
Default Cleaning scratch & work directories

tmpwatch works well for this. I've been using it for a few days now and
it works well.

- Steve

On Thu, 20 Jan 2011, Mike McLean wrote:

> On 01/20/2011 05:05 PM, Anthony Messina wrote:
>> I'm continuing work a on a private Koji instance (Kerberos) and am now
>> working on automating cleanup, etc.
>>
>> It is mentioned here
>> http://lists.fedoraproject.org/pipermail/buildsys/2010-December/003415.html
>> that "Koji leaves it to the administrator to clean out the scratch build
>> directory."
>>
>> Is there anything special about cleaning up scratch builds or the
>> work/{cli-build,cli-import,tasks} directories for that matter?
>>
>> Do I need to interact with the Koji database in any way to let koji know
>> these files are gone? What I mean is, can I simply remove the files and
>> directories without any Koji integration?
>
> The contents of work and scratch are not tracked in the database. The
> mechanism I use is a simple cron job that simply looks at modification
> times.
>
> The main things you want to watch out for is deleting things too early.
> Scratch builds aren't much use to anyone if you delete them before the
> builder gets a chance to poke at them. The work directory is used as a
> temporary holding zone by some tasks, so you don't want to delete stuff
> out from under a running task. A reasonable time-based check is
> sufficient to avoid this.
>
> In my koji instance we keep content under work/ for two weeks and
> content under scratch/ for one week.
>
> Using tmpwatch is a another possibility. I believe it looks at access
> times by default.
> --
> buildsys mailing list
> buildsys@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/buildsys
>

--
Steve Webb | System Administrator
Beatport | Play With Music
------------------------------------------
2399 Blake Street, Suite 170
Denver, Colorado USA 80205
tel: +1.720.932.9103
fax: +1.720.932.9104
noc: +1.303.565.2710
mobile: +1.303.564.4269
--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
 
Old 02-05-2011, 03:27 AM
Thomas Guthmann
 
Default Cleaning scratch & work directories

Hey,

> tmpwatch works well for this. I've been using it for a few days now and
> it works well.

Can you share your tmpwatch rules ? So we don't have to reinvent the
wheel

Thanks
--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
 
Old 02-05-2011, 12:19 PM
Anthony Messina
 
Default Cleaning scratch & work directories

On 02/04/2011 10:27 PM, Thomas Guthmann wrote:
> Can you share your tmpwatch rules ? So we don't have to reinvent the
> wheel


Steve is using tmpwatch; I am not using tmpwatch, but here is the part
of the cron job that does it for me:

http://messinet.com/rpms/browser/koji-maint/koji-maint/koji-maint#L13

The variables koji_scratch_remove and koji_work_remove are defined as a
number of days.

Also, if you really trust 'find', you can change the action from -print
to -delete and simplify it even more.

-A

--
Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
 
Old 02-07-2011, 02:17 PM
 
Default Cleaning scratch & work directories

I added to /etc/cron.daily/tmpwatch:

/usr/sbin/tmpwatch "$flags" 30d /data/koji/scratch
/usr/sbin/tmpwatch "$flags" 30d /data/koji/work

- Steve

On Sat, 5 Feb 2011, Thomas Guthmann wrote:

> Hey,
>
>> tmpwatch works well for this. I've been using it for a few days now and
>> it works well.
>
> Can you share your tmpwatch rules ? So we don't have to reinvent the
> wheel
>
> Thanks
> --
> buildsys mailing list
> buildsys@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/buildsys
>

--
Steve Webb | System Administrator
Beatport | Play With Music
------------------------------------------
2399 Blake Street, Suite 170
Denver, Colorado USA 80205
tel: +1.720.932.9103
fax: +1.720.932.9104
noc: +1.303.565.2710
mobile: +1.303.564.4269
--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
 

Thread Tools




All times are GMT. The time now is 08:21 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org