FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 12-10-2010, 01:02 PM
"Daniel P. Berrange"
 
Default hosted reproducible package building with multiple developers?

On Wed, Dec 08, 2010 at 01:07:32PM -0500, seth vidal wrote:
> On Wed, 2010-12-08 at 13:03 -0500, James Ralston wrote:
> > Riddle me this.
> >
> > We want to provide a server for developers within our organization to
> > build RPM packages for use within our organization.
> >
> > These are our requirements:
> >
> > 1. The developers must not be able to leverage the package build
> > process to obtain root access on the server.
> >
> > 2. If a package has a build dependency that is not explicitly
> > specified, the build must fail.
> >
> > 3. If two developers are building packages simultaneously, their
> > builds must not conflict.
> >
> > The only way satisfy requirements #2 and #3 is to use a chroot'ed
> > build environment.
> >
> > mock(1) uses a chroot'ed build environment, but mock fails requirement
> > #1, as anyone in the "mock" group can trivially root the box.
> >
> > I think that koji would satisfy all three requirements, because koji
> > uses mock to build, but doesn't allow developers to interface with
> > mock directly. But setting up a koji infrastructure seems like a
> > highly non-trivial task.
> >
> > Is there really no way to meet all three of these requirements without
> > going the full-blown koji route?
> >
>
> the mock chroots that koji uses could still be rooted by someone who can
> submit their own build-requirement-providing packages.
>
> in order to protect the builders they must be:
> 1. disposable
> 2. in a vm
>
> or possibly both.

I'm not familiar with what attacks you can do on mocks'
chroot setup offhand, but perhaps it is possible to
avoid them by also leveraging some of the new kernel
container features which allow you to build stronger
virtual root, without going to the extreme of a full
VM.

Regards,
Daniel
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 01:17 PM
seth vidal
 
Default hosted reproducible package building with multiple developers?

On Fri, 2010-12-10 at 14:02 +0000, Daniel P. Berrange wrote:
> On Wed, Dec 08, 2010 at 01:07:32PM -0500, seth vidal wrote:
> > On Wed, 2010-12-08 at 13:03 -0500, James Ralston wrote:
> > > Riddle me this.
> > >
> > > We want to provide a server for developers within our organization to
> > > build RPM packages for use within our organization.
> > >
> > > These are our requirements:
> > >
> > > 1. The developers must not be able to leverage the package build
> > > process to obtain root access on the server.
> > >
> > > 2. If a package has a build dependency that is not explicitly
> > > specified, the build must fail.
> > >
> > > 3. If two developers are building packages simultaneously, their
> > > builds must not conflict.
> > >
> > > The only way satisfy requirements #2 and #3 is to use a chroot'ed
> > > build environment.
> > >
> > > mock(1) uses a chroot'ed build environment, but mock fails requirement
> > > #1, as anyone in the "mock" group can trivially root the box.
> > >
> > > I think that koji would satisfy all three requirements, because koji
> > > uses mock to build, but doesn't allow developers to interface with
> > > mock directly. But setting up a koji infrastructure seems like a
> > > highly non-trivial task.
> > >
> > > Is there really no way to meet all three of these requirements without
> > > going the full-blown koji route?
> > >
> >
> > the mock chroots that koji uses could still be rooted by someone who can
> > submit their own build-requirement-providing packages.
> >
> > in order to protect the builders they must be:
> > 1. disposable
> > 2. in a vm
> >
> > or possibly both.
>
> I'm not familiar with what attacks you can do on mocks'
> chroot setup offhand, but perhaps it is possible to
> avoid them by also leveraging some of the new kernel
> container features which allow you to build stronger
> virtual root, without going to the extreme of a full
> VM.


Since the pkgs have to be installed in the chroot as root if a user can
specify their own dependencies then they can buildrequire a pkg which
has a %pre or %post script which changes of the chroot and can then get
to the real system root. The 'easy' solution was to have throw away vms
so even if they got out they couldn't get far and the system wouldn't
last long.


-sv


--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 01:17 PM
seth vidal
 
Default hosted reproducible package building with multiple developers?

On Fri, 2010-12-10 at 14:02 +0000, Daniel P. Berrange wrote:
> On Wed, Dec 08, 2010 at 01:07:32PM -0500, seth vidal wrote:
> > On Wed, 2010-12-08 at 13:03 -0500, James Ralston wrote:
> > > Riddle me this.
> > >
> > > We want to provide a server for developers within our organization to
> > > build RPM packages for use within our organization.
> > >
> > > These are our requirements:
> > >
> > > 1. The developers must not be able to leverage the package build
> > > process to obtain root access on the server.
> > >
> > > 2. If a package has a build dependency that is not explicitly
> > > specified, the build must fail.
> > >
> > > 3. If two developers are building packages simultaneously, their
> > > builds must not conflict.
> > >
> > > The only way satisfy requirements #2 and #3 is to use a chroot'ed
> > > build environment.
> > >
> > > mock(1) uses a chroot'ed build environment, but mock fails requirement
> > > #1, as anyone in the "mock" group can trivially root the box.
> > >
> > > I think that koji would satisfy all three requirements, because koji
> > > uses mock to build, but doesn't allow developers to interface with
> > > mock directly. But setting up a koji infrastructure seems like a
> > > highly non-trivial task.
> > >
> > > Is there really no way to meet all three of these requirements without
> > > going the full-blown koji route?
> > >
> >
> > the mock chroots that koji uses could still be rooted by someone who can
> > submit their own build-requirement-providing packages.
> >
> > in order to protect the builders they must be:
> > 1. disposable
> > 2. in a vm
> >
> > or possibly both.
>
> I'm not familiar with what attacks you can do on mocks'
> chroot setup offhand, but perhaps it is possible to
> avoid them by also leveraging some of the new kernel
> container features which allow you to build stronger
> virtual root, without going to the extreme of a full
> VM.


Since the pkgs have to be installed in the chroot as root if a user can
specify their own dependencies then they can buildrequire a pkg which
has a %pre or %post script which changes of the chroot and can then get
to the real system root. The 'easy' solution was to have throw away vms
so even if they got out they couldn't get far and the system wouldn't
last long.


-sv


--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 02:06 PM
"Daniel P. Berrange"
 
Default hosted reproducible package building with multiple developers?

On Fri, Dec 10, 2010 at 09:17:27AM -0500, seth vidal wrote:
> On Fri, 2010-12-10 at 14:02 +0000, Daniel P. Berrange wrote:
> > On Wed, Dec 08, 2010 at 01:07:32PM -0500, seth vidal wrote:
> > > On Wed, 2010-12-08 at 13:03 -0500, James Ralston wrote:
> > > > Riddle me this.
> > > >
> > > > We want to provide a server for developers within our organization to
> > > > build RPM packages for use within our organization.
> > > >
> > > > These are our requirements:
> > > >
> > > > 1. The developers must not be able to leverage the package build
> > > > process to obtain root access on the server.
> > > >
> > > > 2. If a package has a build dependency that is not explicitly
> > > > specified, the build must fail.
> > > >
> > > > 3. If two developers are building packages simultaneously, their
> > > > builds must not conflict.
> > > >
> > > > The only way satisfy requirements #2 and #3 is to use a chroot'ed
> > > > build environment.
> > > >
> > > > mock(1) uses a chroot'ed build environment, but mock fails requirement
> > > > #1, as anyone in the "mock" group can trivially root the box.
> > > >
> > > > I think that koji would satisfy all three requirements, because koji
> > > > uses mock to build, but doesn't allow developers to interface with
> > > > mock directly. But setting up a koji infrastructure seems like a
> > > > highly non-trivial task.
> > > >
> > > > Is there really no way to meet all three of these requirements without
> > > > going the full-blown koji route?
> > > >
> > >
> > > the mock chroots that koji uses could still be rooted by someone who can
> > > submit their own build-requirement-providing packages.
> > >
> > > in order to protect the builders they must be:
> > > 1. disposable
> > > 2. in a vm
> > >
> > > or possibly both.
> >
> > I'm not familiar with what attacks you can do on mocks'
> > chroot setup offhand, but perhaps it is possible to
> > avoid them by also leveraging some of the new kernel
> > container features which allow you to build stronger
> > virtual root, without going to the extreme of a full
> > VM.
>
> Since the pkgs have to be installed in the chroot as root if a user can
> specify their own dependencies then they can buildrequire a pkg which
> has a %pre or %post script which changes of the chroot and can then get
> to the real system root. The 'easy' solution was to have throw away vms
> so even if they got out they couldn't get far and the system wouldn't
> last long.

Hmm, it sounds very much like you ought to be able to prevent
this kind of attack with clone+pivot_root, though it is likely
going to be more work than a traditional chroot() based.

The theory is as follows though

1. clone() with the CLONE_NEWNS set
2. Remount / with MS_PRIVATE|MS_REC flags

These two steps ensure the new process has a totally private
filesystem hiearchy where mount/unmount changes cannot leak
back into the host OS, and also new mounts in host can't
propagate down into the container process. Then the complex
bit is to setup a new root

3. Create $MOCKROOTPATH
4. Create $MOCKROOTPATH/.oldroot
5. Mount tmpfs on $MOCKROOTPATH/.oldroot
6. Create $MOCKROOTPATH/.oldroot/new
7. Bind mount $MOCKROOTPATH $MOCKROOTPATH/.oldroot/new
8. chdir $MOCKROOTPATH/.oldroot/new
9. pivot_root(".", ".oldroot")

At this point the container process has a new / filesystem
that is based on what the main OS sees as $MOCKROOTPATH.
The main OS' root filesystem is still (temporarily) visible
to the container procss as /.oldroot.

Now you'd need to bind the directory containing the RPMs
to install & any other data you want, and also setup
the generic special mounts you might need

10. Bind mount /.oldroot/path/to/rpms /tmp/rpms
11. Mount /proc, /dev, /sys, /dev/pts etc as normal

NB, devpts should have the 'newinstance flag set to ensure
there's no access to /dev/pts/N nods from the main OS.
Also the main OS should have been using 'newinstance' for
its own mount of devpts (/me wonders if Fedora is doing
that yet...it should be to ensure LXC containers have a
properly isolated /dev/pts).

Finally, iterate killing off all the mount points under
/.oldroot, from longest path first

12. foreach mountpoint under /.oldroot
umount $mountpoint

The process now has a private root filesystem where it
can only see files from $MOCKROOTPATH, the magic
filesystems proc, devfs, sysfs, etc a private devpts,
and the specific data directories you bind mount for it.
Kernel bugs aside, there should be no means of escape
from this private filesystem as you could with chroot.

Adding CLONE_NEWPID would be worthwhile to stop the
mock process seeing any other PIDs on the machine.

Using cgroups device ACL controller could be used to
block the cloned mock process from doing mknod in /dev
to access other device nodes (ie whitelist just /dev/null
/dev/zero, /dev/urandom and any /dev/pts/*).

There are various other CLONE flags that lock down more
things if desired, eg to hide all host network interfaces.

Of course alot of this setup complexity could be avoided
by using either libvirt's LXC driver or the alternative
LXC commandline tools. Both of these would require you to
copy some of the host yum/rpm/mock binaries inside the
chroot because they'd require you to exec() an intial
binary when starting the container, instead of just doing
a clone() and continuing to run your existing process
address space.

Regards,
Daniel
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 04:41 PM
James Ralston
 
Default hosted reproducible package building with multiple developers?

On 2010-12-08 at 21:00+00 "Richard W.M. Jones" <rjones@redhat.com> wrote:

> To the original poster: even a VM isn't a completely robust way of
> preventing root escalations.

This is a certainly true. If an attacker manages to gain control of a
VM guest, he can attempt to attack the VM host. (In fact, depending
on the attack, escalated privileges on the VM guest may not even be
necessary.)

But this is simply another instance of the generic problem of security
vulnerabilities providing an avenue for privilege escalation, which
isn't a problem that is unique to VMs.

> If the developers are all in your "organization", how about using a
> cluestick-based method to prevent them doing this?

Because a security solution that trusts users not to deliberately or
accidentally acquire access that they aren't [administratively]
permitted to have is no solution. It would be no different than
hanging a sign on the front door of my house that said, "even though
my door is unlocked, since stealing is wrong, I trust that no one will
rob me."

Furthermore, intelligent users would also refuse such an arrangement,
because if the server that provides mock is rooted, the prime suspects
would immediately be the mock users who did not otherwise have root
privileges on the box.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 04:43 PM
Matt McCutchen
 
Default hosted reproducible package building with multiple developers?

On Fri, 2010-12-10 at 15:06 +0000, Daniel P. Berrange wrote:
> Adding CLONE_NEWPID would be worthwhile to stop the
> mock process seeing any other PIDs on the machine.

It's critical, or mock could ptrace some process running as root on the
host and inject arbitrary code.

--
Matt

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 04:57 PM
"Daniel P. Berrange"
 
Default hosted reproducible package building with multiple developers?

On Fri, Dec 10, 2010 at 12:43:04PM -0500, Matt McCutchen wrote:
> On Fri, 2010-12-10 at 15:06 +0000, Daniel P. Berrange wrote:
> > Adding CLONE_NEWPID would be worthwhile to stop the
> > mock process seeing any other PIDs on the machine.
>
> It's critical, or mock could ptrace some process running as root on the
> host and inject arbitrary code.

That is true. I forgot to mention that you'd probably need to block
a large number capabilities while the 'root' part of mock were
executing. eg while mock needs things like CAP_DAC_OVERRIDE,
CAP_FOWNER, CAP_MKNOD, etc to put down files during RPM install,
you don't want it having SYS_ADMIN, MAC_ADMIN, AUDIT_CONTROL,
SYS_BOOT, SYS_MODULE, or SYS_TIME and some others (PTRACE if not
using CLONE_NEWPID).

Regards,
Daniel
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 05:01 PM
James Ralston
 
Default hosted reproducible package building with multiple developers?

On 2010-12-10 at 14:02+00 Daniel P Berrange <berrange@redhat.com> wrote:

> I'm not familiar with what attacks you can do on mocks' chroot setup
> offhand

<http://fedoraproject.org/wiki/Projects/Mock> describes an easy one:

$ /usr/bin/mock --init -r fedora-10-i386
$ /usr/bin/mock --shell -r fedora-10-i386
mock-chroot> chmod u+s bin/bash
$ /var/lib/mock/fedora-10-i386/root/bin/bash -p
# cat /etc/shadow

> but perhaps it is possible to avoid them by also leveraging some of
> the new kernel container features which allow you to build stronger
> virtual root, without going to the extreme of a full VM.

There are two challenges here.

First, you must be able to prevent the root user from breaking out of
the chroot jail.

But second, you must also prevent unprivileged users outside of the
chroot jail from being able to interact with things inside the chroot
jail in a manner that they can use to escalate their privileges.

Setting up a setuid bash shell within the chroot jail and then
invoking it via a normal user outside of the jail is the obvious
example, but there are undoubtedly other avenues of attack that must
be defended.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 05:06 PM
"Richard W.M. Jones"
 
Default hosted reproducible package building with multiple developers?

On Fri, Dec 10, 2010 at 03:06:59PM +0000, Daniel P. Berrange wrote:
> The theory is as follows though
>
> 1. clone() with the CLONE_NEWNS set
[...]
> There are various other CLONE flags that lock down more
> things if desired, eg to hide all host network interfaces.

I don't think CLONE_* can stop them creating a /dev/hda-equivalent
device node and then editing files on your real hard disk.

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-10-2010, 05:08 PM
"Daniel P. Berrange"
 
Default hosted reproducible package building with multiple developers?

On Fri, Dec 10, 2010 at 06:06:47PM +0000, Richard W.M. Jones wrote:
> On Fri, Dec 10, 2010 at 03:06:59PM +0000, Daniel P. Berrange wrote:
> > The theory is as follows though
> >
> > 1. clone() with the CLONE_NEWNS set
> [...]
> > There are various other CLONE flags that lock down more
> > things if desired, eg to hide all host network interfaces.
>
> I don't think CLONE_* can stop them creating a /dev/hda-equivalent
> device node and then editing files on your real hard disk.

That's what the cgroups device ACL I mentioned is for. You
set it up to only allow /dev/null, /dev/zero & similar
nodes.

Daniel
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 11:07 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org