FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 02-17-2008, 03:20 PM
namnd
 
Default Request for comment: a new software to manage linux networking features

Dear Debian developers,
My team have been developing a software for my company since 2004, now I
have plan to release it to the public, hopefully in an open source
license if the management board doesn't object. So I need the peer
reviews from the community of the quality of the software, if it's good
enough for the community, I'll convince the company to release it, if
the quality is not so good, it won't worth the effort to do necessary
works to publish it, so I'll just keep it for private use.


To be short, I consider my software "an evolution of ifupdown", so the
main part is called "netupdown", the purpose of this software is to use
on corporate internet gateway, the main features of the software:


* The configuration file is in XML so netupdown can handle sophisticated
configuration. Editing the XML the configuration by hand or by software
will be easy and comfortable.

<?xml version="1.0" encoding="utf-8"?>
<config version="1.0" logLevel="debug">
...
</config>
* The configuration syntax is unified and consistent, for example, I
need a VPN tunnel to run on a pppoe connection, the computer has more
than one ppp interfaces. You know, ppp numbers are automatically
allocated so it's quite troublesome, netupdown can solve this problem
because interfaces are mentioned by a fixed name, netupdown will
translate it into a name that the kernel can understand when necessary.
When the pppoe interface is down, the openvpn process is killed, when
the pppoe interface is up again, netupdown will generate the new openvpn
option file based on a template, and bind the local ip address to the ip
address of pppoe.

In this case, the configuration is as easy as:
<interface name="fpt1" type="ppp">
<ppp type="pppoe" options="" depend_on="tap1" username="***"
password="***"/>

<network id="11" name="FPT1" auto="1"/>
</interface>
<interface name="tap7" auto="0" type="ethernet" sub_type="openvpn">
<openvpn remote="210.245.87.151" rport="19817" comp="comp-lzo"
depend_on="FPT1"/>

<network id="7" name="TAP" auto="1" config="dhcp"/>
</interface>

* netupdown has a strong dependency system, much like Debian's, when the
system is operating, an internet connection stops working, all the
virtual interfaces depends on it is killed if there is no alternative,
and interfaces depends on the ones got killed is killed as well. When
the internet connection comes up again, netupdown will rebuilt the
configuration file and start the virtual interface again.


* netupdown is created to serve all the need of Linux networking. In one
file, people configure not just ordinary ethernet interfaces but also
VLAN, VPN, ppp, bridges, bonding interfaces, static routes, multipath
routes, firewall, traffic shaping ...
This example show how to bridge tap11 with eth0, so we have two
interfaces (br0 and tap1) connected to the broadcast domain of eth0,
instead of just one (eth0), in my network, this technique is quite useful:

<interface name="br0" auto="1" type="ethernet" sub_type="bridge">
<depend_on name="eth0" value="1"/>
<depend_on name="tap11" value="1"/>
<network id="3" name="DHCP" auto="0" config="dhcp"/>
</interface>
<interface name="eth0" auto="1" type="ethernet">
</interface>
<interface name="tap1" auto="1" type="ethernet" sub_type="openvpn"
hw_addr="xx:xx:xx:xx:xx:xx">

<openvpn remote="127.0.0.1" rport="19821" lport="19811"/>
</interface>
<interface name="tap11" auto="1" type="ethernet" sub_type="openvpn">
<openvpn remote="127.0.0.1" rport="19811" lport="19821"/>
</interface>

* netupdown is made for computer with multiple connections to the
internet (for example, two DSLs to two different ISPs). Actually, the
most notable case is that it serve networking on an enterprise internet
gateway with 10 DSLs and 1 fiber connection, and the bit rate is excelent.
This rule make a simple multipath routing on two interfaces, the
internal mechanism of netupdown also do job of iptables connmark rules
and iproute2 policy routing to make multipath routing works correctly.

<mroute id="60" name="default">
<group>
<nexthop route="FPT1" weight="2"/>
<nexthop route="DHCP" weight="1"/>
</group>
</mroute>

* netupdown pairs with routeskeeper, a daemon using Perl POE
non-blocking IO framework to check the availability of connections, each
route can has multiple tests (using ping and TCP connect to remote
host), when a defined percentage of tests fail, the daemon bring down
the route so new connections don't go to the black hole, but going to
still funtional connections. This feature keep the user happy because
the downtime is minimal.
This sample configuration define a set of 3 tests, used for all internet
connections of the computer, the number of tests can be many, lower the
false negative. We can also define the interval of each test, and many
parameters, but I don't mention them all in here.

<route name="*">
<test name="PING-google" type="connect" location="gg.gg.gg.gg"/>
<test name="CONNECT-yahoo" type="connect" location="yy.yy.yy.yy:80"/>
<test name="CONNECT-kernel" type="connect" location="kk.kk.kk.kk:80"/>
</route>

* There is another component of my software that is under development is
to utilise the netfilter netlink socket to watch conntrack table, and
log all connection to a database, it's useful for security audit.


* The internal variables are stored in /var/run/netupdown/runtime.env so
other programs and scripts can read it easily, each state such as
.up_wanted or .up has a timestamp to remember the moment when it happened.

.Initialized=1203133517
i.br0.configured=1203133518
i.br0.dev=br0
i.br0.explicit=1
i.br0.hw_addr=00:0f:xx:xx:xx:xx
i.br0.up=1203133518
i.br0.up_wanted=1203133518
i.fpt1.configured=1203133596
i.fpt1.dev=ppp0
i.fpt1.explicit=1
i.fpt1.hw_addr=ppp0
i.fpt1.up=1203133596
i.fpt1.up_wanted=1203133575
i.eth0.busy=1203133518
i.eth0.configured=1203133517
i.eth0.dev=eth0
i.eth0.hw_addr=00:0f:xx:xx:xx:xx
i.eth0.up=1203133518
i.eth0.up_wanted=1203133517
i.tap1.configured=1203133553
i.tap1.dev=tap1
i.tap1.explicit=1
i.tap1.hw_addr=00:13:xx:xx:xx:xx
i.tap1.up=1203133553
i.tap1.up_wanted=1203133551
i.tap11.busy=br0
i.tap11.configured=1203133552
i.tap11.dev=tap11
i.tap11.hw_addr=00:ff:xx:xx:xx:xx
i.tap11.up=1203133552
i.tap11.up_wanted=1203133517
mr.default.working_group=1
n.FPT1.dns1=210.245.0.11
n.FPT1.dns2=210.245.86.11
n.FPT1.gw_addr=210.245.0.45
n.FPT1.interface_depend_on=1
n.FPT1.ip_addr=58.187.xx.xx
n.FPT1.mask=255.255.255.255
n.FPT1.mask_len=32
n.FPT1.network_addr=58.187.xx.xx/32
n.FPT1.route_list=FPT1;FPT1,dns
n.FPT1.up=1203133596
n.FPT1.up_wanted=1203133596
r.FPT1,dns.network_name=FPT1
r.FPT1,dns.to=210.245.0.11
r.FPT1,dns.up=1203133596
r.FPT1,dns.via=210.245.0.45
r.FPT1.network_name=FPT1
r.FPT1.up=1203133596
r.FPT1.up_wanted=1203133596
r.FPT1.via=210.245.0.45

After reading the brief overview of my software, please give me your
opinions:

1. What is the quality of idea (1 star to 5 star)
2. What is the quality of the source code (1 to 5)
3. Your other comments
I also attach the source code in this email so you can review it if
interested. It's not yet approved to release under an open source
license, but it's available for education/research/review usage.


Best regards,
Nam

PS: I have sent an email to this mailing list but it doesn't appear,
perhaps because of the attachment, so I post the links instead

http://routeskeeper.com/namnd/generalgateway/generalgateway_2.1.2_i386.deb
http://routeskeeper.com/namnd/generalgateway/generalgateway_2.1.2.tar.gz
Example of an "/etc/network/config.xml"
http://routeskeeper.com/namnd/generalgateway/config.xml



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-17-2008, 06:38 PM
Hendrik Sattler
 
Default Request for comment: a new software to manage linux networking features

Am Sonntag 17 Februar 2008 schrieb namnd:
> In this case, the configuration is as easy as:
> <interface name="fpt1" type="ppp">
> * <ppp type="pppoe" options="" depend_on="tap1" username="***"
> password="***"/>
> * <network id="11" name="FPT1" auto="1"/>
> </interface>

The ppp stanza should not contain authentication information. Especially since
username/password only covers the most simple authentication method. And
other parts could use a auth tag, too (ethernet, wlan, etc.).
Additionally, the options attribute sounds like a work-around.

HS
 
Old 02-17-2008, 06:45 PM
Roberto C. Sánchez
 
Default Request for comment: a new software to manage linux networking features

On Sun, Feb 17, 2008 at 11:20:48PM +0700, namnd wrote:
>
> * The configuration file is in XML so netupdown can handle sophisticated
> configuration. Editing the XML the configuration by hand or by software
> will be easy and comfortable.

Forgive me, but "yeah right!" Have you ever actually edited XML by
hand? Try tweaking a build.xml or some other suitably complex XML
document and you will think differently. Basically, if your XML is so
simple that it can be edited by hand, your needs are such that it is
better handled without XML. If your needs are such that you actually
require XML, then what you end up with is too complex to edit by hand.

> <?xml version="1.0" encoding="utf-8"?>
> <config version="1.0" logLevel="debug">
> ...
> </config>
> * The configuration syntax is unified and consistent, for example, I
> need a VPN tunnel to run on a pppoe connection, the computer has more
> than one ppp interfaces. You know, ppp numbers are automatically
> allocated so it's quite troublesome, netupdown can solve this problem
> because interfaces are mentioned by a fixed name, netupdown will
> translate it into a name that the kernel can understand when necessary.
> When the pppoe interface is down, the openvpn process is killed, when
> the pppoe interface is up again, netupdown will generate the new openvpn
> option file based on a template, and bind the local ip address to the ip
> address of pppoe.
> In this case, the configuration is as easy as:
> <interface name="fpt1" type="ppp">
> <ppp type="pppoe" options="" depend_on="tap1" username="***"
> password="***"/>
> <network id="11" name="FPT1" auto="1"/>
> </interface>
> <interface name="tap7" auto="0" type="ethernet" sub_type="openvpn">
> <openvpn remote="210.245.87.151" rport="19817" comp="comp-lzo"
> depend_on="FPT1"/>
> <network id="7" name="TAP" auto="1" config="dhcp"/>
> </interface>
>
This is supposed to be easy to read and easy to edit by hand? It seems
to me that a properly commented normal text configuration file would be
much more appropriate to the task. I will not even comment on the rest
of your examples, except to say that the "Swiss Army Knife" approach is
certainly not in the *nix tradition.

> After reading the brief overview of my software, please give me your
> opinions:
> 1. What is the quality of idea (1 star to 5 star)
> 2. What is the quality of the source code (1 to 5)
> 3. Your other comments

I do not have time to review the code right now. However, the abuse of
XML that you call a configuration file really needs to be fixed.

Regards,

-Roberto

--
Roberto C. Sánchez
http://people.connexer.com/~roberto
http://www.connexer.com
 
Old 02-17-2008, 07:30 PM
namnd
 
Default Request for comment: a new software to manage linux networking features

Roberto C. Sánchez wrote:
* The configuration file is in XML so netupdown can handle sophisticated
configuration. Editing the XML the configuration by hand or by software
will be easy and comfortable.


Forgive me, but "yeah right!" Have you ever actually edited XML by
hand? Try tweaking a build.xml or some other suitably complex XML
document and you will think differently. Basically, if your XML is so
simple that it can be edited by hand, your needs are such that it is
better handled without XML. If your needs are such that you actually
require XML, then what you end up with is too complex to edit by hand.

I agree that a "suitably complex XML" is hard to edit by a simple
editing tool such as vim-tiny, but even a sophisticated networking
configuration isn't as complex as build.xml, I hope. And this software
package is intended to handle computers with quite sophisticated network
configuration, so it should be configured using a web based software,
and I will release the configurator as well. This software doesn't have
the ambition to replace ifupdown, but to solve the problems ifupdown can
not, and stays being compatible with ifupdown.

In this case, the configuration is as easy as:
<interface name="fpt1" type="ppp">
<ppp type="pppoe" options="" depend_on="tap1" username="***"
password="***"/>

<network id="11" name="FPT1" auto="1"/>
</interface>

This is supposed to be easy to read and easy to edit by hand? It seems
to me that a properly commented normal text configuration file would be
much more appropriate to the task. I will not even comment on the rest
of your examples, except to say that the "Swiss Army Knife" approach is
certainly not in the *nix tradition.
I do not have time to review the code right now. However, the abuse of
XML that you call a configuration file really needs to be fixed
In this software, XML is just a tool, it's totally separated from the
main code. There is a script parsing the configuration and generating an
equivalent Perl structure, and store it /var/run (caching mechanism),
the primary code eval() that piece of code to import the configuration
before running, so I may throw away the XML file and replace it with
traditional conf format.



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-17-2008, 07:45 PM
namnd
 
Default Request for comment: a new software to manage linux networking features

Tzafrir Cohen wrote:

It is part of the base system. And people will have to manipulate it
even when the web interface is not available. Think of using globs and
stats. Also think of creative usages of symlinks.

Well, I guess that it is up to me to "think" (demonstrate fisibility)
here. Otherwise I'm just talking :-(

BTW: perl-base is part of the base system, but perl and perl-modules
isn't.

Thank you, Cohen, for your many comments, I'm likely to replace XML
format by traditional conf format after receiving comments from several
people. I'll try to inherit the format of /etc/network/interfaces if
possible.


My idea of using the package is not to replace ifupdown in the normal
installer because many system requires a very simple network config, and
ifupdown is much more compact. netupdown should be installed later by
people having a complex network, like many people install /usr/bin/less
to replace /bin/more, and vim to replace vim-tiny after finish
installing the base Debian.


Best regards,
Nam


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-17-2008, 08:05 PM
namnd
 
Default Request for comment: a new software to manage linux networking features

Hendrik Sattler wrote:

Am Sonntag 17 Februar 2008 schrieb namnd:


In this case, the configuration is as easy as:
<interface name="fpt1" type="ppp">
<ppp type="pppoe" options="" depend_on="tap1" username="***"
password="***"/>
<network id="11" name="FPT1" auto="1"/>
</interface>

The ppp stanza should not contain authentication information. Especially since
username/password only covers the most simple authentication method. And
other parts could use a auth tag, too (ethernet, wlan, etc.).

Additionally, the options attribute sounds like a work-around.

Some people may want to put everything related to network config in 1
file to ease the management, other people may put it in
/etc/ppp/chap-secrets or anywhere else, they are all supported.


No doubt, each interface type may has its own authentication
configuration (such as wlan interface will support WPA). The idea is
that there is a template for external daemon (such as pppd, openvpn,
vtun, ...) lying somewhere else, netupdown takes the template and fill
the place holders with the content in the main config file, save the
final conf file in /var/run, and starts the daemon. This template
mechanism solves many problems fixed conf file can not do, such as
binding local IP address to the dynamic IP address of a ppp interface.


The options attribute is used for people who need to configure
parameters aren't defined in the template, this work arround is
necessary for some of my cases.


Best regards,
Nam


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-18-2008, 03:15 AM
Brian May
 
Default Request for comment: a new software to manage linux networking features

>>>>> "namnd" == namnd <namnd@fpt.com.vn> writes:

namnd> To be short, I consider my software "an evolution of ifupdown", so the
namnd> main part is called "netupdown", the purpose of this software is to
namnd> use on corporate internet gateway, the main features of the software:

Have you looked at netconf? It also claims it will solve problems with
ifupdown.

http://netconf.alioth.debian.org/

Also see presentation at LCA2008:

http://mirror.linux.org.au/pub/linux.conf.au/2008/slides/072-netconf_lca2008_2008.01.31.tar.gz
http://mirror.linux.org.au/pub/linux.conf.au/2008/Thu/mel8-072.ogg
http://mirror.linux.org.au/pub/linux.conf.au/2008/Thu/mel8-072.spx
--
Brian May <bam@snoopy.debian.net>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-18-2008, 05:50 AM
namnd
 
Default Request for comment: a new software to manage linux networking features

Hello Martin,

Hello Nam,

I am the author of netconf (http://netconf.alioth.debian.org), which
Brian has already introduced in another post to this thread. I am
very interested in your work because all the issues you highlight in
your mail are core to my motivations for writing netconf.

You are asking us to evaluate the quality of a software based only
on the descriptions you provide. This is really difficult. While
others have picked up on shortcomings, such as the use of XML,
I wonder about certain other things:

I have just read netconf wiki pages, and I think that our ideas are
exactly the same. I've finished my work in 2004 and used it on some
mission critical enterprise gateways of different companies we provide
service to. The gateways are all have multiple internet connections
(ranging from 4 to 11 connections). In 2005, I rewrote a new code from
scratch to make the design better. Since then, I've been occupied by a
lot of management works so I could not revive it. Now I have time again
any continue developing it seriously.

- what language are you using?


- netupdown is in Perl.

- is this a daemon-based design?

- netupdown is not daemon based, it reads the config, read the current
runtime state, does the job, save the runtime state, and exit. It flock
the state so only one process can do the job at a given time. But there
is a daemon to support it, mostly monitor the availability of
connections and log the counters into a Round Robin Database, and log
the conntrack events using netlink socket as well.

- how much internal state do you keep?

- I have posted a full example of internal state on the first email.
Runtime state contains info about interfaces, about networks running on
those interfaces, routes running on those networks, and multipath routes
using the available routes.

- how extensible is the software?

- The software is made to be extensible, otherwise it can't survive
itself because of 6 different types of interface. In order to release
it, I am trying to improve it further. Each feature will be stored in a
separated file.

- how large is the code base?


Very compact, about 1500 lines in Perl, other stuff doesn't count.

- how do you interface with e.g. openvpn? Do you wrap it (and thus
limit the possible configuration), or do you feedthrough to it
(and thus provide the full configuration abilities)?

I generate configuration file for them from specified template, and run
the daemons (pppd, openvpn, vtun, ...) with the newly generated
configuration file. Full configuration abilities are guranteed.

- does your software handle wireless networks, including those
configured by wpa_supplicant?

My laptop use the software itself to manage connectivity, of course, the
laptop uses wireless. But currently, it's not so nice because it doesn't
integrate very well with wifi-radar. It's not my first priority because
I care about internet gateways more than personal computers.

I think it's always a good idea to release software to the public.
As such, I encourage you to convince your management to slap a good
licence onto your work.

I agree. But it will take some effort so I will only try to release it
if some conditions are met:

- It attracts enough developers
- The quality is good, so it is worthy to continue developing

On the other hand, you and your team may also be interested in
netconf. You surely have a lot of experience in the domain, and
netconf could greatly benefit from your input. While netupdown seems
designed to override shortcomings in ifupdown, netconf aims to fix
those at the root.


I'm sure I will study it.

netconf is currently implemented in Python, but that only really
makes it easier for others to read the code and contribute to it.
Unfortunately, netconf is not yet ready for use, but it's not very
far either.

Even I am not familiar with python, I'll try to learn from netconf. My
software have been in production use for years so many bugs are fixed,
the users are very happy with it.

Please release your software to the public and then consider joining
the netconf team and getting your colleagues interested too. We can
then reuse your work on netupdown and integrate it into netconf to
add advanced network configuration abilities to Debian by default!

There is a challenge here, until Larry Wall does some breakthrough with
his perl6, I don't know how Perl developers and Python developers can
code together



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 10:31 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org