FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 07-29-2010, 12:47 AM
Albert Hopkins
 
Default Facilities for creating Gentoo Linux based virtual appliances

Hello,

For a while I have been assembling methods for myself to be able to
easily create Gentoo-based virtual appliances. I have worked with
Ubuntu's vmbuilder scripts and basically wanted the same or similar ease
of use with Linux.

To make a long story short, I threw together a Makefile which basically
gives me this ability. I have it at a state where it's pretty much
useful to me for my purposes. But I thought someone else may find use
for it as well, so I put it up on Bitbucket:

https://bitbucket.org/marduk/virtual-appliance

The Makefile pretty much does an install into a chroot, pretty much by
the handbook, with a few alterations where I saw fit. It hast some
variables and a couple of hooks to allow for customization. I've
already created a few appliances (included) such as:

base: A base install of Gentoo Linux
gnome: GNOME stable appliance (no GUI console)
kde: KDE stable appliance (no GUI console)
lodgeit: Lodgeit pastebin appliance[1]
teamplayer: TeamPlayer appliance

The "no GUI console" means that by default these appliances don't have
an X console. I usually log into them via XDMCP or ssh. So they can be
thought of as "headless" desktop virtual appliances. The "base"
appliance is pretty much the handbook, with dhcpd and a syslog and
nothing else. TeamPlayer is a "democratic Internet radio station" web
app that I have been developing. To date I have not distributed it,
though eventually plan to. You can contact me directly if interested.

Right now there is no documentation or copyright or anything. It's
pretty much just the code. Like I said this has been primarily for my
personal use so there is nothing fancy like packaging or anything, but
it works. Some things can probably be done better. I'm no expert at
writing Makefiles.

So the way you basically use it is like this:

$ sudo make APPLIANCE=gnome VIRTIO=YES qcow

This will create a gnome virtual appliance image configured for virtio
and supply a qcow (qcow2) disk image. You can also just "sudo make" for
which the default is to create a "base" appliance not configured with
VIRTIO and only supply an image file in raw format. There are other
make variables that can be supplied. Check out the Makefile.

You can "easily" create your own VM appliaces by just mkdir
<appliance-name>. Copy the files from base/* into it and edit them as
needed. The supplied appliances should be good references.

One warning: the Makefile must be run as root, and does things inside a
chroot. I have been careful to make sure everything is done in the
chroot that needs to be. It is possible for you (or me) to forget to
put chroot before a command and cause irreversible damage to your host
system. Just giving you this caveat. I'm sure the appliances I
provided are safe, but I can't guarantee it and if you make your own
appliance just be careful. Ideally one would build VM appliances inside
a VM appliance itself

Well, I probably left some stuff out. Feel free to reply (to this
thread I guess) if you have any questions.

Oh, when you are finished building your disk image just

$ sudo make clean

Which will unmount things and remove the temporary files.

A FEW MORE IMPORTANT THINGS:
In the Makefile, I have PORTAGE=/portage and
DISTFILES=/var/portage/distfiles. These are my setup and are not the
defaults. You will need to change these values appropriately. Also,
you may need to edit the USEPKG variable. I have the Makefile use my
binary packages to speed up the creation process (allows me to build a
base appliance in about 12 minutes). You may not want this, especially
if you have your CFLAGS tuned to a specific processor that your
hypervisor does not support. But I use generic CFLAGS so I have no
problem using binpkgs.

Oh, I use kvm/virt-manager and use virtio for disks and network devices.
I tried to configure the VM kernel so that it will support other
formats. I am curious as to whether or not the appliances will work
with VMWare as I have not actually tried it, but you should be able to
create a VM image with

$ sudo make vmdk

I'm probably leaving something out. You should take a look at the
Makefile first if you are going to use it. It shouldn't be that foreign
(it's basically handbook stuff but tailored for the use case).

Anyway it's out there for you to reference.

-a

[1] http://dev.pocoo.org/projects/lodgeit/
 
Old 07-29-2010, 05:15 PM
Tanstaafl
 
Default Facilities for creating Gentoo Linux based virtual appliances

Top posting because I can't contribute anything meaningful inline...

Wow, Albert, this looks very, very cool. I have heard of using 'make'
and creating your own make files to do things like this, but after a few
minutes of perusing these files I realize this is just way over my head,
at least without some kind of tutorial (I'm just a lowly user whose
(lack of) bash skills would make most of you guys laugh).

Do you know of any generic tutorials on using creating/using your own
make files to do repetitive tasks (not necessarily like building an
entire system as you are doing here (but if there is something like that
even better)?

If not, no worries.

Anyway, thanks for sharing what you've done here...

Charles

Albert Hopkins wrote:
> Hello,
>
> For a while I have been assembling methods for myself to be able to
> easily create Gentoo-based virtual appliances. I have worked with
> Ubuntu's vmbuilder scripts and basically wanted the same or similar ease
> of use with Linux.
>
> To make a long story short, I threw together a Makefile which basically
> gives me this ability. I have it at a state where it's pretty much
> useful to me for my purposes. But I thought someone else may find use
> for it as well, so I put it up on Bitbucket:
>
> https://bitbucket.org/marduk/virtual-appliance
>
> The Makefile pretty much does an install into a chroot, pretty much by
> the handbook, with a few alterations where I saw fit. It hast some
> variables and a couple of hooks to allow for customization. I've
> already created a few appliances (included) such as:
>
> base: A base install of Gentoo Linux
> gnome: GNOME stable appliance (no GUI console)
> kde: KDE stable appliance (no GUI console)
> lodgeit: Lodgeit pastebin appliance[1]
> teamplayer: TeamPlayer appliance
>
> The "no GUI console" means that by default these appliances don't have
> an X console. I usually log into them via XDMCP or ssh. So they can be
> thought of as "headless" desktop virtual appliances. The "base"
> appliance is pretty much the handbook, with dhcpd and a syslog and
> nothing else. TeamPlayer is a "democratic Internet radio station" web
> app that I have been developing. To date I have not distributed it,
> though eventually plan to. You can contact me directly if interested.
>
> Right now there is no documentation or copyright or anything. It's
> pretty much just the code. Like I said this has been primarily for my
> personal use so there is nothing fancy like packaging or anything, but
> it works. Some things can probably be done better. I'm no expert at
> writing Makefiles.
>
> So the way you basically use it is like this:
>
> $ sudo make APPLIANCE=gnome VIRTIO=YES qcow
>
> This will create a gnome virtual appliance image configured for virtio
> and supply a qcow (qcow2) disk image. You can also just "sudo make" for
> which the default is to create a "base" appliance not configured with
> VIRTIO and only supply an image file in raw format. There are other
> make variables that can be supplied. Check out the Makefile.
>
> You can "easily" create your own VM appliaces by just mkdir
> <appliance-name>. Copy the files from base/* into it and edit them as
> needed. The supplied appliances should be good references.
>
> One warning: the Makefile must be run as root, and does things inside a
> chroot. I have been careful to make sure everything is done in the
> chroot that needs to be. It is possible for you (or me) to forget to
> put chroot before a command and cause irreversible damage to your host
> system. Just giving you this caveat. I'm sure the appliances I
> provided are safe, but I can't guarantee it and if you make your own
> appliance just be careful. Ideally one would build VM appliances inside
> a VM appliance itself
>
> Well, I probably left some stuff out. Feel free to reply (to this
> thread I guess) if you have any questions.
>
> Oh, when you are finished building your disk image just
>
> $ sudo make clean
>
> Which will unmount things and remove the temporary files.
>
> A FEW MORE IMPORTANT THINGS:
> In the Makefile, I have PORTAGE=/portage and
> DISTFILES=/var/portage/distfiles. These are my setup and are not the
> defaults. You will need to change these values appropriately. Also,
> you may need to edit the USEPKG variable. I have the Makefile use my
> binary packages to speed up the creation process (allows me to build a
> base appliance in about 12 minutes). You may not want this, especially
> if you have your CFLAGS tuned to a specific processor that your
> hypervisor does not support. But I use generic CFLAGS so I have no
> problem using binpkgs.
>
> Oh, I use kvm/virt-manager and use virtio for disks and network devices.
> I tried to configure the VM kernel so that it will support other
> formats. I am curious as to whether or not the appliances will work
> with VMWare as I have not actually tried it, but you should be able to
> create a VM image with
>
> $ sudo make vmdk
>
> I'm probably leaving something out. You should take a look at the
> Makefile first if you are going to use it. It shouldn't be that foreign
> (it's basically handbook stuff but tailored for the use case).
>
> Anyway it's out there for you to reference.
>
> -a
>
> [1] http://dev.pocoo.org/projects/lodgeit/
>
>
 
Old 07-29-2010, 06:45 PM
Albert Hopkins
 
Default Facilities for creating Gentoo Linux based virtual appliances

On Thu, 2010-07-29 at 13:15 -0400, Tanstaafl wrote:
[...]
> Wow, Albert, this looks very, very cool. I have heard of using 'make'
> and creating your own make files to do things like this, but after a few
> minutes of perusing these files I realize this is just way over my head,
> at least without some kind of tutorial (I'm just a lowly user whose
> (lack of) bash skills would make most of you guys laugh).
>
Charles,

Well, looks like you are looking for a make tutorial. There are a few
make tutorials on the internet (a simple Google search should return
something). I am by no means an expert on make or Makefiles, but for my
makefile specifically:

The first part is simply assigning variables, much like bash. The rest
are "recipes" for creating targets (files usually).

So we can look at the 2nd recipe which is probably the simplest. I'll
past it here and replace the variables with actual values for clarity:

base.img:
qemu-img create -f raw base.img 60G

So our "target" here is the file called "base.img". This recipe has no
dependencies (after the colon is where you list dependencies). The
TABed in line(s) are the instructions, so to create the base.img file we
simply execute that qemu-img command.

In the makefile the "default" target is the first one, so the default is
"all" in my case. "all" requires "image" and has no instructions. The
"image" target requires the RAW_IMAGE we saw above, and a few other
dependencies. So the make command will try to satisfy those
dependencies before it runs the recipe for "image"

> Do you know of any generic tutorials on using creating/using your own
> make files to do repetitive tasks (not necessarily like building an
> entire system as you are doing here (but if there is something like that
> even better)?
>
Makefiles may not be what you need if you are wanting to do "repetitive
tasks", but it depends on what you want to do. For example, the virtual
appliance makefile isn't really doing "repetitive" tasks, but a series
of tasks that need to be run in a certain order (a recipe).

Someone asked me why I chose to use a Makefile instead of just writing a
script. The reason is that Makefiles can do "checkpointing" and it makes
it much easier to debug the appliance creation process. For example, if
your script did the chroot and unpacking but then it failed on "emerge
world", well if you fix that and run your script again then it will
start from the beginning, which takes a long time. But with Makefiles
make will see that the first parts are "up to date" and continue from
where it failed. It also lets you test certain parts (targets) without
having to run the entire thing.

But if you really want a script all you need to do is

# make -n > script.sh


> If not, no worries.

http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=makefile
+tutorial


> Anyway, thanks for sharing what you've done here...

NP. I will be committing a fix soon that will use the default portage
directories instead of my funny ones. Oh, and I confirmed that you can
create virtual appliances inside a virtual appliance, which is safer.
So I will be adding a virtual appliance creating virtual appliance soon.

-a
 
Old 08-01-2010, 02:14 AM
"Walter Dnes"
 
Default Facilities for creating Gentoo Linux based virtual appliances

On Wed, Jul 28, 2010 at 08:47:48PM -0400, Albert Hopkins wrote

> https://bitbucket.org/marduk/virtual-appliance
>
> The Makefile pretty much does an install into a chroot, pretty much by
> the handbook, with a few alterations where I saw fit. It hast some
> variables and a couple of hooks to allow for customization. I've
> already created a few appliances (included) such as:
>
> base: A base install of Gentoo Linux
> gnome: GNOME stable appliance (no GUI console)
> kde: KDE stable appliance (no GUI console)
> lodgeit: Lodgeit pastebin appliance[1]
> teamplayer: TeamPlayer appliance

Do you know of any active end-user mailing lists or forums for
QEMU-KVM? I've found a few forums that seem to have a post every other
month. There's a developers' mailing list, but I don't see one for end
users. I got into QEMU-KVM "appliances" by accident...

- old PC started dying on me

- I backed up the data to an even older PC, and got a shiny new one,
with 8 gigs of ram, wheeeeeeee

- I installed pure 64-bit Gentoo *WITHOUT* 32-bit support

- and discovered that WINE will not install on a system without 32-bit
support (that's my version of "oops programing"; nothing to do with
object orientation)

- I play around a lot with 4NEC2, a Windows-only antenna modelling app
that runs nicely under WINE

- to avoid a wipe and re-install on my main system, I created a small
32-bit Gentoo guest inside QEMU-KVM, which allowed me to install WINE
and 4NEC2

I did have to do quite a bit of trial-and-error before I got the thing
working, and I'll probably re-do, so I can do it right this time. An
active QEMU-KVM mailing list or forum would be helpful.

--
Walter Dnes <waltdnes@waltdnes.org>
 
Old 08-01-2010, 04:25 AM
Albert Hopkins
 
Default Facilities for creating Gentoo Linux based virtual appliances

On Sat, 2010-07-31 at 22:14 -0400, Walter Dnes wrote:

> Do you know of any active end-user mailing lists or forums for
> QEMU-KVM? I've found a few forums that seem to have a post every other
> month. There's a developers' mailing list, but I don't see one for end
> users. I got into QEMU-KVM "appliances" by accident...

I use kvm but I'm not on/aware of any mailing lists. I'm subscribed to
the libvirt mailing list today, but only temporarily as I'm trying to
get help with an issue.

I did a Google search for "kvm mailing list" and came up with
http://www.linux-kvm.org/page/Lists,_IRC. The first list seems to be
pretty active from what I saw from the Gmane archives.

Not to single you out, but since my original post there have been two
replies and both of them seem to be questions that are solved by simple
Google searches. Is Google not working for you? The first questions
was basically "is there a tutorial on how to write makefiles?".

[other stuff completely unrelated to creating Gentoo virtual appliances
removed]

Well, if anyone actually wants to talk about virtual appliances or has
some cool ideas or knows how I can improve on the stuff I'm done then
let me know.

-a
 
Old 08-02-2010, 03:53 AM
"Walter Dnes"
 
Default Facilities for creating Gentoo Linux based virtual appliances

On Sun, Aug 01, 2010 at 12:25:37AM -0400, Albert Hopkins wrote
> On Sat, 2010-07-31 at 22:14 -0400, Walter Dnes wrote:
>
> > Do you know of any active end-user mailing lists or forums for
> > QEMU-KVM? I've found a few forums that seem to have a post every other
> > month. There's a developers' mailing list, but I don't see one for end
> > users. I got into QEMU-KVM "appliances" by accident...
>
> I use kvm but I'm not on/aware of any mailing lists. I'm subscribed to
> the libvirt mailing list today, but only temporarily as I'm trying to
> get help with an issue.
>
> I did a Google search for "kvm mailing list" and came up with
> http://www.linux-kvm.org/page/Lists,_IRC. The first list seems to be
> pretty active from what I saw from the Gmane archives.

I did that search before asking. The link for the first list's
archives is http://news.gmane.org/gmane.comp.emulators.kvm.devel (as in
developers). I glanced through their GMANE archives, and it's full of
developers submitting patches. The second list's link is
http://news.gmane.org/gmane.comp.emulators.kvm.scm and it seems to be
related to patch commits. Neither of the lists are end-user oriented.
The 3rd and 4th lists are specific to Itanium and PPC, respectively.

--
Walter Dnes <waltdnes@waltdnes.org>
 
Old 08-02-2010, 04:43 AM
Albert Hopkins
 
Default Facilities for creating Gentoo Linux based virtual appliances

On Sun, 2010-08-01 at 23:53 -0400, Walter Dnes wrote:

> > I did a Google search for "kvm mailing list" and came up with
> > http://www.linux-kvm.org/page/Lists,_IRC. The first list seems to be
> > pretty active from what I saw from the Gmane archives.
>
> I did that search before asking. The link for the first list's
> archives is http://news.gmane.org/gmane.comp.emulators.kvm.devel (as in
> developers). I glanced through their GMANE archives, and it's full of
> developers submitting patches. The second list's link is
> http://news.gmane.org/gmane.comp.emulators.kvm.scm and it seems to be
> related to patch commits. Neither of the lists are end-user oriented.
> The 3rd and 4th lists are specific to Itanium and PPC, respectively.
>

The page I posted says that the lists are for both development and user
queries. I for one think it's nice that the developers will take user
queries directly (the libvirt mailing list is the same way). Have you
actually tried asking your question on the list?

In addition to the mailing lists, the page lists resources from blogs,
wikis, an irc channel, and web forums. Pretty much everything you asked
is on that page in addition to the documentation that already exists at
linux-kvm.org.

If the developers are inviting you to ask questions of them directly, I
would go that route. I have a feeling they may know a thing or two.
 

Thread Tools




All times are GMT. The time now is 02:23 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org