FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 07-26-2012, 06:40 PM
Michael Mol
 
Default Portage FEATURE suggestion - limited-visibility builds

On Thu, Jul 26, 2012 at 2:26 PM, Rich Freeman <rich0@gentoo.org> wrote:
> I've been messing around with namespaces and some of what systemd has
> been doing with them, and I have an idea for a portage feature.
>
> But before doing a brain dump of ideas, how useful would it be to have
> a FEATURE for portage to do a limited-visibility build? That is, the
> build would be run in an environment where the root filesystem appears
> to contain everything in a DEPEND (including @system currently) and
> nothing else? It might be useful both in development/testing, and
> also in production use (not sure how performance would work in the
> real world - I was able in a script to get it to build an enviornment
> in a few seconds for a few packages).

I very much like this, as it'd greatly simplify identifying any
unintended or unrecognized dependencies in my code. Furthermore, if
the mechanism for identifying and declaring specified-required content
can be generalized, this would make distributed builds potentially
more efficient by allowing the dispatching host to distribute the
necessary header and library files to the machines doing the building.
(Really, this observation is more about simply making the information
available; distcc could consume that information if someone chose to
do the work to add that functionality.)

> I really crazy idea would be to try to run packages in a similar
> environment, but I think that needs better kernel/etc level support
> since the performance hit would be much more noticeable, except for
> things like daemons that only start once.
>
> Implementing it wouldn't necessarily be hard - just create a tmpfs
> under /var/tmp/portage, unshare off a new mount namespace, and
> read-only bind-mount everything needed from the root filesystem
> (including /var/tmp/portage/...), and chroot into it. When the build
> is done the process governing it terminates and the kernel wipes out
> all the mounts and then portage unmounts the tmpfs. You wouldn't need
> to use a tmpfs for the build - it would actually be zero-size as
> reported by df since it just contains a bazillion bind mounts, though
> all those mounts would consume slab memory.
>
> Thoughts?

You've done 90% of the conceptual work needed for an idea I had; I've
wanted to do something similar at the 'make' level, to make
identifying and fixing broken parallel build systems easier. If I
could limit a make instance to only be able to see consequences of
jobs it declared as dependencies, that'd go a long way. I was going to
go by way of FUSE for this, though...

--
:wq
 
Old 07-26-2012, 06:59 PM
Michael Orlitzky
 
Default Portage FEATURE suggestion - limited-visibility builds

On 07/26/12 14:26, Rich Freeman wrote:
> I've been messing around with namespaces and some of what systemd has
> been doing with them, and I have an idea for a portage feature.
>
> But before doing a brain dump of ideas, how useful would it be to have
> a FEATURE for portage to do a limited-visibility build? That is, the
> build would be run in an environment where the root filesystem appears
> to contain everything in a DEPEND (including @system currently) and
> nothing else? It might be useful both in development/testing, and
> also in production use (not sure how performance would work in the
> real world - I was able in a script to get it to build an enviornment
> in a few seconds for a few packages).

The Cabal build system for Haskell packages does this and it's extremely
useful. It prevents me from forgetting dependencies like 'directory',
'time', etc. that I use without thinking.

runghc Setup.hs build
Building lwn-epub-0.0...
Preprocessing executable 'lwn-epub' for lwn-epub-0.0...

src/LWN/Article.hs:12:8:
Could not find module `System.Directory'
It is a member of the hidden package `directory-1.1.0.2'.
Perhaps you need to add `directory' to the build-depends in your
.cabal file.
Use -v to see a list of the files searched for.
 

Thread Tools




All times are GMT. The time now is 12:59 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org