FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora/Linux Management Tools

 
 
LinkBack Thread Tools
 
Old 04-17-2008, 08:07 PM
David Lutterkort
 
Default ANNOUNCE: Augeas - a configuration API

I am pleased to announce a new configuration management project: Augeas,
a low-level configuration API and editing tool.

Augeas' main goal is to make programmatic changes of configuration data
on Linux/Unix systems simple and safe. The main stumbling stone for this
is that configuration data is stored in numerous files in widely varying
formats. This is both next to impossible to change and is valuable in
many situations. Rather than demanding a radical change, Augeas adapts
to this reality.

Augeas parses configuration files in their native formats and transforms
them into a tree. Configuration changes are made by manipulating this
tree and saving it back into native config files
(/etc/hosts, /etc/grub.conf, ...).

The tree abstracts away all the pesky details that make editing config
files with grep/sed/awk so challenging, and exposes config data
uniformly, regardless of the format in which it is actually stored. With
this approach, the programs/daemons relying on config files continue to
work as they do today, and editing config files with Augeas can be
interleaved with changing them by other means, from vi to existing perl
scripts.

What is it ?
============

Augeas consists of a C library and API, a command line tool to
manipulate configuration from the shell, and language bindings for Ruby
and Python. The file <-> tree transformations are driven by schema
descriptions, one for each file format. Schemas essentially consist of
regular expressions describing the file structure and instructions on
how to map matches into the tree. The hope is that over time, the set of
schemas becomes both complete and sufficiently widespread to provide a
canonical tree view of common configuration data.

Where can I find Augeas
=======================

Augeas' website is http://augeas.net/
Downloads can be found at http://augeas.net/download/
To learn more, start with the introductory tour at http://augeas.net/tour.html
Discussions around Augeas happen on augeas-devel,
https://www.redhat.com/mailman/listinfo/augeas-devel

Help wanted
===========

Augeas is a reasonable proof-of-concept as of now; there are lots of
things that need to be done, and that I would love for others to pitch
in. Some of them are listed in the todo list at
http://augeas.net/todo.html

The most pressing issues right now though are

* Enable your favorite config mgmt tool to use Augeas as the
low-level config editing tool
* Cover more config files and write schemes for them; even without
writing actual schema, a discussion on augeas-devel on how
config file X would best be mapped into the tree would be really
useful.
* Anything to improve the quality of the current implementation; I
know I cut some corners in the initial implementation, and any
review/patches to improve it would be useful.

David

_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-18-2008, 02:02 AM
Mike MacCana
 
Default ANNOUNCE: Augeas - a configuration API

On Thu, 2008-04-17 at 13:07 -0700, David Lutterkort wrote:


I am pleased to announce a new configuration management project: Augeas,
a low-level configuration API and editing tool.

Augeas' main goal is to make programmatic changes of configuration data
on Linux/Unix systems simple and safe. The main stumbling stone for this
is that configuration data is stored in numerous files in widely varying
formats. This is both next to impossible to change and is valuable in
many situations.


The amount of effort spent creating and re-creating tools to parse, edit and transform a variety of unnecessary, unstructured data formats over the last 30 years, and to continue doing this for the next 10 years, is less than that required to:



* write an RFC for a standard format



* create patches to applications to support that format



* create an editor for that format (which handles data - settings, values, parents and children, rather than presentation related info like lines and paragraph)



* package those changes in a distribution



Using 'widely varying formats' is not 'valuable'. It's an unfortunate accident that wastes everyone's time with various horrible bandaid solutions, and occasionally makes destroying user data an 'accepted limitation' of tools like system-config-named.



Mike







Rather than demanding a radical change, Augeas adapts
to this reality.

Augeas parses configuration files in their native formats and transforms
them into a tree. Configuration changes are made by manipulating this
tree and saving it back into native config files
(/etc/hosts, /etc/grub.conf, ...).

The tree abstracts away all the pesky details that make editing config
files with grep/sed/awk so challenging, and exposes config data
uniformly, regardless of the format in which it is actually stored. With
this approach, the programs/daemons relying on config files continue to
work as they do today, and editing config files with Augeas can be
interleaved with changing them by other means, from vi to existing perl
scripts.

What is it ?
============

Augeas consists of a C library and API, a command line tool to
manipulate configuration from the shell, and language bindings for Ruby
and Python. The file <-> tree transformations are driven by schema
descriptions, one for each file format. Schemas essentially consist of
regular expressions describing the file structure and instructions on
how to map matches into the tree. The hope is that over time, the set of
schemas becomes both complete and sufficiently widespread to provide a
canonical tree view of common configuration data.

Where can I find Augeas
=======================

Augeas' website is http://augeas.net/
Downloads can be found at http://augeas.net/download/
To learn more, start with the introductory tour at http://augeas.net/tour.html
Discussions around Augeas happen on augeas-devel,
https://www.redhat.com/mailman/listinfo/augeas-devel

Help wanted
===========

Augeas is a reasonable proof-of-concept as of now; there are lots of
things that need to be done, and that I would love for others to pitch
in. Some of them are listed in the todo list at
http://augeas.net/todo.html

The most pressing issues right now though are

* Enable your favorite config mgmt tool to use Augeas as the
low-level config editing tool
* Cover more config files and write schemes for them; even without
writing actual schema, a discussion on augeas-devel on how
config file X would best be mapped into the tree would be really
useful.
* Anything to improve the quality of the current implementation; I
know I cut some corners in the initial implementation, and any
review/patches to improve it would be useful.

David

_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools







Cheers,



Mike



________________________________________________

Mike MacCana

Technical Specialist

Linux Services



IBM Global Services

Level 14, 60 City Rd

Southgate Vic 3000



Phone: +61-3-8656-2138

Fax: +61-4-8656-2423

Email: mmaccana@au1.ibm.com





_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-18-2008, 03:17 AM
"Stephen John Smoogen"
 
Default ANNOUNCE: Augeas - a configuration API

On Thu, Apr 17, 2008 at 8:02 PM, Mike MacCana <mmaccana@au1.ibm.com> wrote:
>
> On Thu, 2008-04-17 at 13:07 -0700, David Lutterkort wrote:
> I am pleased to announce a new configuration management project: Augeas,
> a low-level configuration API and editing tool.
>
> Augeas' main goal is to make programmatic changes of configuration data
> on Linux/Unix systems simple and safe. The main stumbling stone for this
> is that configuration data is stored in numerous files in widely varying
> formats. This is both next to impossible to change and is valuable in
> many situations.
>
> The amount of effort spent creating and re-creating tools to parse, edit
> and transform a variety of unnecessary, unstructured data formats over the
> last 30 years, and to continue doing this for the next 10 years, is less
> than that required to:
>

Then it should have happened.. because we have had several million
monkeys doing various things. I have seen multiple attempts at what
you outline, but because they are expressing what a person wants
something to be you end up with the effects of the tower of Babyl when
few people can understand what the other person is saying.


> * write an RFC for a standard format

You forgot the step: Argue about every 3rd line in the RFC and end up
with various versions that all state they meet the RFC but do not
interact with each other.. HTML-2.0 wasn't hard to write to.. but no
one ever did it the same way.

>
> * create patches to applications to support that format
>
> * create an editor for that format (which handles data - settings, values,
> parents and children, rather than presentation related info like lines and
> paragraph)
>
> * package those changes in a distribution
>
> Using 'widely varying formats' is not 'valuable'. It's an unfortunate
> accident that wastes everyone's time with various horrible bandaid
> solutions, and occasionally makes destroying user data an 'accepted
> limitation' of tools like system-config-named.
>



--
Stephen J Smoogen. -- CSIRT/Linux System Administrator
How far that little candle throws his beams! So shines a good deed
in a naughty world. = Shakespeare. "The Merchant of Venice"

_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-18-2008, 03:39 AM
Mike MacCana
 
Default ANNOUNCE: Augeas - a configuration API

On Thu, 2008-04-17 at 21:17 -0600, Stephen John Smoogen wrote:


On Thu, Apr 17, 2008 at 8:02 PM, Mike MacCana <mmaccana@au1.ibm.com> wrote:
>
> On Thu, 2008-04-17 at 13:07 -0700, David Lutterkort wrote:
> I am pleased to announce a new configuration management project: Augeas,
> a low-level configuration API and editing tool.
>
> Augeas' main goal is to make programmatic changes of configuration data
> on Linux/Unix systems simple and safe. The main stumbling stone for this
> is that configuration data is stored in numerous files in widely varying
> formats. This is both next to impossible to change and is valuable in
> many situations.
>
> The amount of effort spent creating and re-creating tools to parse, edit
> and transform a variety of unnecessary, unstructured data formats over the
> last 30 years, and to continue doing this for the next 10 years, is less
> than that required to:
>


(create a standard format, proactively patch apps to support a standard format, make an editor for that format, and start shipping packages)



Then it should have happened.. because we have had several million
monkeys doing various things. I have seen multiple attempts at what
you outline,




When? The closest thing I've ever seen has been gconf and dconf, which are limited to GNOME apps (albeit quite successful).



Mike






Cheers,



Mike



________________________________________________

Mike MacCana

Technical Specialist

Linux Services



IBM Global Services

Level 14, 60 City Rd

Southgate Vic 3000



Phone: +61-3-8656-2138

Fax: +61-4-8656-2423

Email: mmaccana@au1.ibm.com





_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-18-2008, 02:01 PM
"Daniel P. Berrange"
 
Default ANNOUNCE: Augeas - a configuration API

On Fri, Apr 18, 2008 at 01:39:06PM +1000, Mike MacCana wrote:
> On Thu, 2008-04-17 at 21:17 -0600, Stephen John Smoogen wrote:
>
> > On Thu, Apr 17, 2008 at 8:02 PM, Mike MacCana <mmaccana@au1.ibm.com> wrote:
> > >
> > > On Thu, 2008-04-17 at 13:07 -0700, David Lutterkort wrote:
> > > I am pleased to announce a new configuration management project: Augeas,
> > > a low-level configuration API and editing tool.
> > >
> > > Augeas' main goal is to make programmatic changes of configuration data
> > > on Linux/Unix systems simple and safe. The main stumbling stone for this
> > > is that configuration data is stored in numerous files in widely varying
> > > formats. This is both next to impossible to change and is valuable in
> > > many situations.
> > >
> > > ???The amount of effort spent creating and re-creating tools to parse, edit
> > > and transform a variety of unnecessary, unstructured data formats over the
> > > last 30 years, and to continue doing this for the next 10 years, is less
> > > than that required to:
> > >
>
> (create a standard format, proactively patch apps to support a standard
> format, make an editor for that format, and start shipping packages)

Back in the real world people also want to be able to manage existing deployed
production applications today. Even if you convince upstream to adopt your
grand unified configuration scheme, it'll take a seriously long time before
all applications you care about managing are changed & filter down into
real world deployed OS. And then there are all the closed source applications
which you have no control over.

So even if everyone agreed to use a standard configuration format, exceedingly
unlikely, there'd still be a huge set of existing applications to managed
with a tool like Augeas.

Dan.
--
|: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-18-2008, 05:17 PM
David Lutterkort
 
Default ANNOUNCE: Augeas - a configuration API

On Fri, 2008-04-18 at 12:02 +1000, Mike MacCana wrote:
> * create an editor for that format (which handles data - settings,
> values, parents and children, rather than presentation related info
> like lines and paragraph)

At its heart, Augeas is a tool to make writing such editors easy, easier
than starting from scratch for every format. Of course, there's also
benefit for users if they can edit many config files with the same tool
instead of having a separate tool for each config file.

> Using 'widely varying formats' is not 'valuable'.

I was a little oblique .. I meant 'that config data is stored in text
files is valuable', mostly to contrast that with some other approaches
that have been proposed in the past.

> It's an unfortunate accident that wastes everyone's time with various
> horrible bandaid solutions

I think we're all in agreement that the situation is far from the best
imaginable. That's why I chose "Augeas"[1] as the name for this

David

[1] http://en.wikipedia.org/wiki/Augeas

"In Greek mythology, Augeas ... was King of Elis ... He is best known for his
stables, which housed the single greatest number of cattle in the country and
had never been cleaned"

_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-18-2008, 05:22 PM
David Lutterkort
 
Default ANNOUNCE: Augeas - a configuration API

On Fri, 2008-04-18 at 15:01 +0100, Daniel P. Berrange wrote:
> Even if you convince upstream to adopt your grand unified configuration scheme

And therein lies the rub. I've been very careful to make sure that
Augeas' usability does not depend on any upstream adopting it - they
have perfectly valid reasons not to do that, starting with that they
can't tell which proposed scheme will actually work out (and therefore
deserve their support)

Augeas (or any similar config scheme) _has_ to be workable without
upstream support - it would be great if we get some eventually [1], but
we first need to prove that Augeas is worth their effort.

David

[1] And for now, it would really only require them shipping a schema
with their software, i.e. plonking one text file
into /usr/share/augeas/lenses


_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-20-2008, 09:04 AM
"Richard W.M. Jones"
 
Default ANNOUNCE: Augeas - a configuration API

On Fri, Apr 18, 2008 at 12:02:15PM +1000, Mike MacCana wrote:
> Using 'widely varying formats' is not 'valuable'. It's an unfortunate
> accident that wastes everyone's time with various horrible bandaid
> solutions, and occasionally makes destroying user data an 'accepted
> limitation' of tools like system-config-named.

Different formats suit different uses, and in any case I wouldn't
trust the people who would develop this new "super-format" not to do
something stupid like using XML.

Rich.

--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-20-2008, 01:31 PM
Jason Edgecombe
 
Default ANNOUNCE: Augeas - a configuration API

Richard W.M. Jones wrote:

On Fri, Apr 18, 2008 at 12:02:15PM +1000, Mike MacCana wrote:


Using 'widely varying formats' is not 'valuable'. It's an unfortunate
accident that wastes everyone's time with various horrible bandaid
solutions, and occasionally makes destroying user data an 'accepted
limitation' of tools like system-config-named.



Different formats suit different uses, and in any case I wouldn't
trust the people who would develop this new "super-format" not to do
something stupid like using XML.

Rich.



This sounds a lot like what gconfd was made for. How is it different?

Jason

_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 
Old 04-20-2008, 06:58 PM
David Lutterkort
 
Default ANNOUNCE: Augeas - a configuration API

On Sun, 2008-04-20 at 09:31 -0400, Jason Edgecombe wrote:
> This sounds a lot like what gconfd was made for. How is it different?

gconfd addresses slightly different uses in a different environment: its
main focus is on dektop apps, and because of its origin and focus on the
Gnome desktop, it's reasonable to expect applications to be built to
gconf's API and use it as a 'master' data storage.

Augeas is more focused on all the files that are _not_
managed/manageable by gconfd, especially all the sundry config files
in /etc; there it's unreasonable that the programs using those config
files will adapt to some grand unified API and storage scheme. It is
also unrealistic, as attempts at introducing that have failed --- and
uptream has good reasons for not jumping at such a cheme: for one, it's
not clear if any of those schemes will find wide enough acceptance to
make supporting it worthwhile. The way a daemon processes config files
to get its config out if it is also different from how an editing tool
like Augeas has to process it: the daemon can just ignore comments,
space etc. and does never have to worry about writing the file. These
are two very important concerns for Augeas.

There are two other approaches to programmatic config file editing
besides the grand unified API/storage approach that people commonly
take:

* Grep/sed/awk etc. your way through to make the change you need.
Works for simple file formats/situations, but everybody winds up
reinventing that for their specific situation, and there's
usually not enough effort put into that to make sure that every
possible variation of a file is processed correctly. Augeas
addresses that by focusing just on the processing/editing
aspect, so that for each file the description can be adapted and
perfected over time (assuming it wasn't perfect to begin with
A lot of this is about corner cases like not getting thrown off
by whitespace at the end of line etc.
* Templating gets you out of parsing files and worrying about the
above issues. The downside is that it introduces an ad-hoc
canonical store of config data that is not the real config file;
you can't edit those files anymore by hand or with a second tool
(or lose your edits the next time the template engine runs) You
usually also lose the flexibility to express everything that can
be expressed in the 'real' config file.

The limitations of these three approaches were very important in the
design of Augeas, and avoiding them lead to what it is and how it works.

David


_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools
 

Thread Tools




All times are GMT. The time now is 03:05 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org