Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Fedora Build System (http://www.linux-archive.org/fedora-build-system/)
-   -   Koji 1.7.0 release (http://www.linux-archive.org/fedora-build-system/673344-koji-1-7-0-release.html)

Mike McLean 05-31-2012 09:20 PM

Koji 1.7.0 release
 
On 05/31/2012 04:21 PM, Dan Horák wrote:

could you be more specific what does the split storage means? Will I be
able to easily combine 2 or more storages (NFS mounts, local disks, etc)
and they will create one big storage for koji usage?


It does not quite work that way, and 'easy' is not a word I would use
here. I DO NOT recommend that folks use this feature unless they
absolutely have to. It is much better to simply buy a bigger disc if at
all possible.


WARNING. Using split storage is potentially dangerous. If you do not
manage your mount points carefully you could cause difficult to debug
problems or even CORRUPT your file store.


That being said. Here is how it works.

Each build now has a volume attribute that indicates which volume it is
stored on. The default volume is named 'DEFAULT' and corresponds to the
same storage paths under /mnt/koji that we have always used.


Additional volumes can be set up by creating the directory
/mnt/koji/vol/NAME (where NAME is the name of the volume). The
expectation is that this will be a mount point, but it doesn't have to be.


The new volume directory should initially contain a packages/
subdirectory, and the permissions should be the same as the default
packages directory.


Assuming you do use a mount for a vol/NAME directory, you will want to
ensure that the same mounts are created on any builders in the
createrepo group, any hosts running kojira or similar maintenance, and
any hosts that rely on the topdir option rather than the topurl option.


Once you have the directory set up, you can tell Koji about it by
running 'koji add-volume NAME' (which will fail if the hub can't find
the directory). You can get a list of known volumes with the 'koji
list-volumes' command, and you can move a build to a different volume
with the 'koji set-build-volume' command. Like all koji subcommands,
these have a --help option.


Moving a build across volumes will cause kojira to trigger repo
regenerations, if appropriate. When the volume is not DEFAULT, koji will
create a relative symlink to the new build directory on the default
volume. Moving builds across volumes may immediately break repos (until
the regen occurs), so use caution.


Policies involving builds (e.g. gc policy, tag policy), can test a
build's volume (by name) with the 'volume' test.


That's pretty much it. It is a very simplistic system. There is no
automation. It is up to the administrator to manage the mount points and
to manage which builds are stored on which volumes. All new builds are
stored on the default volume until they are moved elsewhere.

--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Mike McLean 05-31-2012 09:20 PM

Koji 1.7.0 release
 
On 05/31/2012 04:21 PM, Dan Horák wrote:

could you be more specific what does the split storage means? Will I be
able to easily combine 2 or more storages (NFS mounts, local disks, etc)
and they will create one big storage for koji usage?


It does not quite work that way, and 'easy' is not a word I would use
here. I DO NOT recommend that folks use this feature unless they
absolutely have to. It is much better to simply buy a bigger disc if at
all possible.


WARNING. Using split storage is potentially dangerous. If you do not
manage your mount points carefully you could cause difficult to debug
problems or even CORRUPT your file store.


That being said. Here is how it works.

Each build now has a volume attribute that indicates which volume it is
stored on. The default volume is named 'DEFAULT' and corresponds to the
same storage paths under /mnt/koji that we have always used.


Additional volumes can be set up by creating the directory
/mnt/koji/vol/NAME (where NAME is the name of the volume). The
expectation is that this will be a mount point, but it doesn't have to be.


The new volume directory should initially contain a packages/
subdirectory, and the permissions should be the same as the default
packages directory.


Assuming you do use a mount for a vol/NAME directory, you will want to
ensure that the same mounts are created on any builders in the
createrepo group, any hosts running kojira or similar maintenance, and
any hosts that rely on the topdir option rather than the topurl option.


Once you have the directory set up, you can tell Koji about it by
running 'koji add-volume NAME' (which will fail if the hub can't find
the directory). You can get a list of known volumes with the 'koji
list-volumes' command, and you can move a build to a different volume
with the 'koji set-build-volume' command. Like all koji subcommands,
these have a --help option.


Moving a build across volumes will cause kojira to trigger repo
regenerations, if appropriate. When the volume is not DEFAULT, koji will
create a relative symlink to the new build directory on the default
volume. Moving builds across volumes may immediately break repos (until
the regen occurs), so use caution.


Policies involving builds (e.g. gc policy, tag policy), can test a
build's volume (by name) with the 'volume' test.


That's pretty much it. It is a very simplistic system. There is no
automation. It is up to the administrator to manage the mount points and
to manage which builds are stored on which volumes. All new builds are
stored on the default volume until they are moved elsewhere.

--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

John Morris 07-19-2012 05:35 AM

Koji 1.7.0 release
 
As an additional datum, I'm happy to report that the shop's koji
instance seems to be running perfectly for several days after an update
to the new Fedora 16 RPMs (from shop-built RPMs of HEAD from a few
months ago) and a reasonably easy switch to the mod_wsgi configuration.


We'll be migrating our build system over to EL6 soon. Crossing my
fingers that everything goes smoothly there too!


John


On 05/31/2012 03:07 PM, Mike McLean wrote:

I've just tagged and posted the 1.7.0 release of Koji. The most
significant change is the move from mod_python to mod_wsgi, but there
are a number of other changes also (it's been a while since 1.6.0).

I've written a document for migrating from 1.6.0 to 1.7.0. Look under:

docs/Migrating_to_1.7.txt

Here is the changelog entry:
* Thu May 31 2012 Mike McLean <mikem at redhat.com> - 1.7.0-1
- mod_wsgi support
- mod_python support deprecated
- kojiweb configuration file (web.conf)
- split storage support (build volumes)
- configurable resource limits (hub, web, and kojid)
- drop pkgurl in favor of topurl
- better approach to web themes
- more helpful policy errors
- clearer errors when rpc args do not match function signature
- avoid retry errors on some common builder calls
- don't rely on pgdb._quoteparams
- avoid hosts taking special arch tasks they cannot handle
- kojid: configure yum proxy
- kojid: configure failed buildroot lifetime
- kojid: literal_task_arches option
- support for arm hardware floating point arches
- maven build options: goals, envs, extra packages
- store Maven build output under the standard build directory
- make the list of files ignored in the local Maven repo configurable
- add Maven information to taginfo
- make kojira more efficient using multicalls and caching
- speed up kojira startup
- kojira: configurable sleep time
- kojira: count untracked newRepo tasks towards limits
- kojira: limit non-waiting newRepo tasks
- gssapi support in the messagebus plugin
- grant-permission --new
- improved argument display for list-api command
- moshimoshi
- download task output directly from KojiFilesURL, rather than going
through getfile
- option to show buildroot data in rpminfo command
- show search help on blank search command
- wait-repo: wait for the build(s) to be the latest rather than just
present
--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

--
buildsys mailing list
buildsys@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys


All times are GMT. The time now is 01:34 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.