Hi!

I've had this idea in my head for long, but as never found the time to
work on it, didn't feel appropriate to throw it to the wall and expect
someone else to implement it. Anyway, it seems to me it might be a nice
GSoC project, and not necessarily too complex. As I've my plate already
full, I'm not volunteering myself for mentoring, though. I'm CCing
deity@l.d.o as they should be in the loop and Petter as he was working
on integrating package data into discover-data at some point in the past,
which might be interested in mentoring.

Problem
~~~~~~~

I've always found it annoying that it's become very difficult to hunt
all packages that might be needed for or useful with the current
hardware on the system, and usually you tend to miss some. It might
be even trickier for non-technical users who might not even know they
need specific packages for something to work.

Proposal
~~~~~~~~

Ideally the package manager front-ends would propose for installation to
the user all hardware related packages for currently detected hardware
in the system, or removal once such hardware is not present (although
that might need to be disabled for pluggable hardware). This could
include drivers, firmware, tools and applications [0]. There's a
distinction between packages that are required for something to work,
which could be handled somehow as Recommends, and packages which might
have additional functionality if the hardware is present, which could
be handled as Suggests.

Each package would provide a list of patterns for the hardware it
supports. Depending on their size, they could be provided in a new field
(for example Hardware-Id), or on a new control file (but then that would
need to get extracted from binary packages and collected into a foo-data
package for example) or something else. Those patterns would match
against the device IDs of cpu, pci, pnp, usb, eisa, dmi, pcmcia, acpi,
ieee1394, etc.

For some packages this information is already known, and can be
automatically extracted from some files (and possibly converted to the
chosen pattern format). For example X.Org drivers have an internal
list of patterns, not sure if those can be easily extracted, for Linux
kernel modules one can extract those with something like:

,---
|PATH=/sbin:$PATH
|
|find -name '*.ko' | xargs modinfo -F alias |
| egrep '^(pci|pnp|eisa|pcmcia|usb|acpi|dmi|ieee1394|ssb|w mi):'
`---

[0] An incomplete list from when I looked into this could include:

Drivers
-------

xserver-xorg-video-*, xserver-xorg-input-*, *-source, firmware-*

Tools
-----

Graphic cards: gatos, radeontool, rovclock, atitvout, s3switch,
i810switch, matroxset, nvclock, nvtv, libglide3.
Sound cards: awesfx, ld10k1.
Webcams: qcam, qpcr1k, pencam.
CPUs: athcool, set6x86.
Laptops: i8kutils, fnfxd, fnfx-client, toshset, toshutils, tpb,
spicctrl.
Printers: foomatic-db-hpijs, hpijs, hplip, hpoj.
SCSI: scsitools, sg-utils.

Design and Implementation
~~~~~~~~~~~~~~~~~~~~~~~~~

Things to decide and work on, would include:

* The format of the patterns for each hardware type. There's
existing examples that could be either reused or based for
inspiration.

* A tool to extract at package build time as much of the IDs as
possible from existing files and convert them to the normalized
form, if need be. (Ideally independent of any packaging helper.)

* Consider and decide where to ship such information.

* It might be a good idea to be able to have a global override, for
testing purposes, and a faster initial deployment, once a working
implementation is in place those could be moved to each package.

* Decide how to make the front-ends use the information, for example
by creating a shared library that does the detection and matching,
and just returns the list of packages, or through an external
program (say a new hwsel, or maybe just adapting and/or extending
disover or any of the other hardware detectors to be easily integrated
with the front-ends), etc.

* The actual integration with the front-ends.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100329215154.GA30406@gaara.hadrons.org">http://lists.debian.org/20100329215154.GA30406@gaara.hadrons.org

On Mon, 2010-03-29 at 23:51 +0200, Guillem Jover wrote:
> Hi!
>
> I've had this idea in my head for long, but as never found the time to
> work on it, didn't feel appropriate to throw it to the wall and expect
> someone else to implement it. Anyway, it seems to me it might be a nice
> GSoC project, and not necessarily too complex. As I've my plate already
> full, I'm not volunteering myself for mentoring, though. I'm CCing
> deity@l.d.o as they should be in the loop and Petter as he was working
> on integrating package data into discover-data at some point in the past,
> which might be interested in mentoring.
>
> Problem
> ~~~~~~~
>
> I've always found it annoying that it's become very difficult to hunt
> all packages that might be needed for or useful with the current
> hardware on the system, and usually you tend to miss some. It might
> be even trickier for non-technical users who might not even know they
> need specific packages for something to work.
[...]

There are also some hardware-specific drivers and tools that may be
useful to some users but should not be recommended because they depend
on out-of-tree kernel drivers that conflict with the in-tree drivers.
Any solution should avoid conflating 'works with this device' and
'recommended on systems with this device'.

Ben.

--
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

> Ideally the package manager front-ends would propose for installation to
> the user all hardware related packages for currently detected hardware
> in the system, or removal once such hardware is not present (although
> that might need to be disabled for pluggable hardware).

The implementation would IMO also to some extend need to be aware of the
environment into which the installation is taking place. On a Gnome
desktop system it should propose Gnome packages, on a KDE desktop system
it should propose equivalent KDE packages.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201003300219.27189.elendil@planet.nl">http://lists.debian.org/201003300219.27189.elendil@planet.nl

On Mon, Mar 29, 2010 at 11:51:54PM +0200, Guillem Jover wrote:
> Hi!
>
> I've had this idea in my head for long, but as never found the time to
> work on it, didn't feel appropriate to throw it to the wall and expect
> someone else to implement it. Anyway, it seems to me it might be a nice
> GSoC project, and not necessarily too complex. As I've my plate already
> full, I'm not volunteering myself for mentoring, though. I'm CCing
> deity@l.d.o as they should be in the loop and Petter as he was working
> on integrating package data into discover-data at some point in the past,
> which might be interested in mentoring.
>
> Problem
> ~~~~~~~
>
> I've always found it annoying that it's become very difficult to hunt
> all packages that might be needed for or useful with the current
> hardware on the system, and usually you tend to miss some. It might
> be even trickier for non-technical users who might not even know they
> need specific packages for something to work.
>
> Proposal
> ~~~~~~~~
>
> Ideally the package manager front-ends would propose for installation to
> the user all hardware related packages for currently detected hardware
> in the system, or removal once such hardware is not present (although
> that might need to be disabled for pluggable hardware). This could
> include drivers, firmware, tools and applications [0]. There's a
> distinction between packages that are required for something to work,
> which could be handled somehow as Recommends, and packages which might
> have additional functionality if the hardware is present, which could
> be handled as Suggests.
Ubuntu developed a tool called 'jockey' for installing hardware
drivers. There is an ITP for it in Bug #436722. Jockey is in my
opinion the best place to do something like this. I CCed Martin Pitt,
the developer of jockey (and quoted the remaining parts of the email
in case he did not read the original one).

>
> Each package would provide a list of patterns for the hardware it
> supports. Depending on their size, they could be provided in a new field
> (for example Hardware-Id), or on a new control file (but then that would
> need to get extracted from binary packages and collected into a foo-data
> package for example) or something else. Those patterns would match
> against the device IDs of cpu, pci, pnp, usb, eisa, dmi, pcmcia, acpi,
> ieee1394, etc.
>
> For some packages this information is already known, and can be
> automatically extracted from some files (and possibly converted to the
> chosen pattern format). For example X.Org drivers have an internal
> list of patterns, not sure if those can be easily extracted, for Linux
> kernel modules one can extract those with something like:
>
> ,---
> |PATH=/sbin:$PATH
> |
> |find -name '*.ko' | xargs modinfo -F alias |
> | egrep '^(pci|pnp|eisa|pcmcia|usb|acpi|dmi|ieee1394|ssb|w mi):'
> `---
>
> [0] An incomplete list from when I looked into this could include:
>
> Drivers
> -------
>
> xserver-xorg-video-*, xserver-xorg-input-*, *-source, firmware-*
>
> Tools
> -----
>
> Graphic cards: gatos, radeontool, rovclock, atitvout, s3switch,
> i810switch, matroxset, nvclock, nvtv, libglide3.
> Sound cards: awesfx, ld10k1.
> Webcams: qcam, qpcr1k, pencam.
> CPUs: athcool, set6x86.
> Laptops: i8kutils, fnfxd, fnfx-client, toshset, toshutils, tpb,
> spicctrl.
> Printers: foomatic-db-hpijs, hpijs, hplip, hpoj.
> SCSI: scsitools, sg-utils.
>
> Design and Implementation
> ~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Things to decide and work on, would include:
>
> * The format of the patterns for each hardware type. There's
> existing examples that could be either reused or based for
> inspiration.
>
> * A tool to extract at package build time as much of the IDs as
> possible from existing files and convert them to the normalized
> form, if need be. (Ideally independent of any packaging helper.)
>
> * Consider and decide where to ship such information.
>
> * It might be a good idea to be able to have a global override, for
> testing purposes, and a faster initial deployment, once a working
> implementation is in place those could be moved to each package.
>
> * Decide how to make the front-ends use the information, for example
> by creating a shared library that does the detection and matching,
> and just returns the list of packages, or through an external
> program (say a new hwsel, or maybe just adapting and/or extending
> disover or any of the other hardware detectors to be easily integrated
> with the front-ends), etc.
>
> * The actual integration with the front-ends.
>
> regards,
> guillem
>
>
> --
> To UNSUBSCRIBE, email to deity-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> Archive: http://lists.debian.org/20100329215154.GA30406@gaara.hadrons.org
>

--
Julian Andres Klode - Debian Developer, Ubuntu Member

See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.

[Guillem Jover]
> I've had this idea in my head for long, but as never found the time
> to work on it, didn't feel appropriate to throw it to the wall and
> expect someone else to implement it. Anyway, it seems to me it might
> be a nice GSoC project, and not necessarily too complex. As I've my
> plate already full, I'm not volunteering myself for mentoring,
> though. I'm CCing deity@l.d.o as they should be in the loop and
> Petter as he was working on integrating package data into
> discover-data at some point in the past, which might be interested
> in mentoring.

Yeah. Note that part of your propopsal is already implemented in
discover/discover-data, and the /sbin/discover-pkginstall script will
install hardware dependent packages. It can for example install raid
monitoring software when a supported raid is detected. It might
provide a starting point.

Keeping the mapping from hardware to packages up to date is a hard
problem without cooperation from the various package maintainers, and
some hardware is hard to detect, but for the easy stuff (PCI and USB),
the implementation is trivial.

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100403210921.GC14960@login2.uio.no">http://lists.debian.org/20100403210921.GC14960@login2.uio.no


Sun Apr 4 00:30:02 2010
Return-path: <bounce-debian-kernel=tom=linux-archive.org@lists.debian.org>
Envelope-to: tom@linux-archive.org
Delivery-date: Sat, 03 Apr 2010 23:55:38 +0300
Received: from liszt.debian.org ([82.195.75.100]:51393)
by s2.java-tips.org with esmtps (TLSv1:AES256-SHA:256)
(Exim 4.69)
(envelope-from <bounce-debian-kernel=tom=linux-archive.org@lists.debian.org>)
id 1NyANW-0007VN-24
for tom@linux-archive.org; Sat, 03 Apr 2010 23:55:38 +0300
Received: from localhost (localhost [127.0.0.1])
by liszt.debian.org (Postfix) with QMQP
id AC80913A61E2; Sat, 3 Apr 2010 21:27:21 +0000 (UTC)
Old-Return-Path: <agimenez@sysvalve.es>
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on liszt.debian.org
X-Spam-Level:
X-Spam-Status: No, score=-5.8 required=4.0 tests=FOURLA,RCVD_IN_DNSWL_LOW,
RDNS_NONE,STATIC_RIMA_TDE autolearn=failed version=3.2.5
X-Original-To: lists-debian-kernel@liszt.debian.org
Delivered-To: lists-debian-kernel@liszt.debian.org
Received: from localhost (localhost [127.0.0.1])
by liszt.debian.org (Postfix) with ESMTP id 21A9E2D0F19
for <lists-debian-kernel@liszt.debian.org>; Sat, 3 Apr 2010 21:11:21 +0000 (UTC)
X-Virus-Scanned: at lists.debian.org with policy bank en-ht
X-Amavis-Spam-Status: No, score=-7.8 tagged_above=-10000 required=5.3
tests=[BAYES_00=-2, FOURLA=0.1, RCVD_IN_DNSWL_LOW=-1, RDNS_NONE=0.1,
STATIC_RIMA_TDE=-5] autolearn=ham
Received: from liszt.debian.org ([127.0.0.1])
by localhost (lists.debian.org [127.0.0.1]) (amavisd-new, port 2525)
with ESMTP id 6qff5GfPqo9L for <lists-debian-kernel@liszt.debian.org>;
Sat, 3 Apr 2010 21:11:13 +0000 (UTC)
X-policyd-weight: using cached result; rate: -6.1
Received: from eul0600410.eu.verio.net (unknown [81.19.98.74])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client CN "smtp.capside.com", Issuer "int-W000026-CA" (not verified))
by liszt.debian.org (Postfix) with ESMTPS id A97862D0F16
for <debian-kernel@lists.debian.org>; Sat, 3 Apr 2010 21:11:13 +0000 (UTC)
Received: (qmail 21438 invoked by uid 89); 3 Apr 2010 21:11:09 -0000
Received: from 152.Red-88-2-254.staticIP.rima-tde.net (HELO [192.168.1.30]) (88.2.254.152)
(smtp-auth username lagimenez@capside.com, mechanism cram-md5)
by eul0600410.eu.verio.net (NEVRiON/smtpd) with (AES256-SHA encrypted) ESMTPSA; Sat, 03 Apr 2010 23:11:09 +0200
Message-ID: <4BB7AEEA.4080401@sysvalve.es>
Date: Sat, 03 Apr 2010 23:11:06 +0200
From: =?ISO-8859-1?Q?=22L=2E_Alberto_Gim=E9nez=22?=
<agimenez@sysvalve.es>
User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090706)
MIME-Version: 1.0
To: Paul McEnery <pmcenery@gmail.com>
CC: Diego Giagio <diego@giagio.com>, Daniel Borca <dborca@yahoo.com>,
Julien BLACHE <jblache@debian.org>,
libimobiledevice-devel@lists.libimobiledevice.org,
Debian kernel team <debian-kernel@lists.debian.org>
Subject: Re: [libimobiledevice-devel] ipheth
References: <1270127260.12516.31.camel@localhost> <p2yb4892f8e1004010630q941d7cfere161b15ab4d98a8c@m ail.gmail.com> <1270142630.6437.169.camel@mirell> <g2jb4892f8e1004011320h278ff4eeh7ea64aa595136ab1@m ail.gmail.com> <4BB5FB10.1090704@sysvalve.es> <y2qb4892f8e1004021309l27c086dex1fbe1d7db74df725@m ail.gmail.com>
In-Reply-To: <y2qb4892f8e1004021309l27c086dex1fbe1d7db74df725@m ail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
X-Rc-Spam: 2008-11-04_01
X-Rc-Virus: 2007-09-13_01
X-Rc-Spam: 2008-11-04_01
Resent-Message-ID: <wVN2aEqbjFM.A.egB.5K7tLB@liszt>
Resent-From: debian-kernel@lists.debian.org
X-Mailing-List: <debian-kernel@lists.debian.org> archive/latest/57791
X-Loop: debian-kernel@lists.debian.org
List-Id: <debian-kernel.lists.debian.org>
List-URL: <http://lists.debian.org/debian-kernel/>
List-Post: <mailto:debian-kernel@lists.debian.org>
List-Help: <mailto:debian-kernel-request@lists.debian.org?subject=help>
List-Subscribe: <mailto:debian-kernel-request@lists.debian.org?subject=subscribe>
List-Unsubscribe: <mailto:debian-kernel-request@lists.debian.org?subject=unsubscribe>
Precedence: list
Resent-Sender: debian-kernel-request@lists.debian.org
Resent-Date: Sat, 3 Apr 2010 21:27:21 +0000 (UTC)
Content-Transfer-Encoding: quoted-printable

On 04/02/2010 10:09 PM, Paul McEnery wrote:
[...]
> Without wanting to say anything on
> Bradley's behalf, it appeared as if he was in support of the tethering
> driver being implemented in kernel space. That said, and given the
> maturity of ipheth, would it be fair to say that ipheth is the way
> forward in terms of i<device> tethering?

Hi Paul,

Reading the thread that you posted, it seems like the original author
agreed to "kind of" support a kernel-space driver.

And about your question, I *personally* think that it's the way forward.
We have a hardware device that does networking. Si it needs a driver.
And I'm not the best friend of userspace drivers, except for very
specific tasks and to play a little bit (look ma! I implemented a
filesystem with FUSE!).


> Where do the udev rules and pairing utility reside?
>=20
> I see two options here:
>=20
> 1. Keep the ipheth-utils package and drop ipheth-dkms when ipheth
> makes it into the mainline kernel. Given that mainline inclusion could
> take a while, users (of Debian at least) could start to benefit almost
> immediately since all of the packaging work has already been done.

I think that this is the one that would fit my view (which may not be
the correct one, but it's my view). Then we can handle everything else
with package dependencies (usbmuxd, libimobiledevice, ...).

There are still some issues that are pending to fix over the first
driver submission. I've already posted a couple of messages regarding
those issues (I don't have the knowledge to decide about the low-level
aspects of the device or the kernel API itself) and contacted with
Daniel and Diego, so =CD think that it's close to get in (linux-next, I
guess) if the list members don't have further comments.

If you need help packaging (I think that you are the "future" maintainer
for those Debian packages, am I right?), I have some basic experience
with it, so don't feel alone, I can help if required

Best regards,
--=20
L. Alberto Gim=E9nez
JabberID agimenez@jabber.sysvalve.es
GnuPG key ID 0x3BAABDE1


--=20
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian=
.org
Archive: http://lists.debian.org/4BB7AEEA.4080401@sysvalve.es

[Julian Andres Klode]
> Ubuntu developed a tool called 'jockey' for installing hardware
> drivers. There is an ITP for it in Bug #436722. Jockey is in my
> opinion the best place to do something like this. I CCed Martin
> Pitt, the developer of jockey (and quoted the remaining parts of the
> email in case he did not read the original one).

Yes, it was a nice tool. I tested it for the first time a few weeks
ago.

I noticed in #436722 that Martin Pitt was positive to shared
maintenance. What is holding you from uploading the package to
Debian? The ITP haven't seen any progress for a long time.

Note that I GUI tool would be nice for the non-free stuff, but would
be a bad idea for other hardware specific packages like RAID
administration tools. For this, I would like to integrate something
like discover-pkginstall into the hwsetup package in debian-installer,
to make sure hardware specific packages are automatically installed
when it make sense.

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 2flhbnrxxkm.fsf@login2.uio.no">http://lists.debian.org/2flhbnrxxkm.fsf@login2.uio.no

Hi,

If you actually want a student to look at your proposal, please put it
on the wiki
http://wiki.debian.org/gsoc

Cheers

Arthur

On Sun, Apr 4, 2010 at 1:32 PM, Petter Reinholdtsen <pere@hungry.com> wrote:
>
> [Julian Andres Klode]
>> Ubuntu developed a tool called 'jockey' for installing hardware
>> drivers. There is an ITP for it in Bug #436722. Jockey is in my
>> opinion the best place to do something like this. I CCed Martin
>> Pitt, the developer of jockey (and quoted the remaining parts of the
>> email in case he did not read the original one).
>
> Yes, it was a nice tool. *I tested it for the first time a few weeks
> ago.
>
> I noticed in #436722 that Martin Pitt was positive to shared
> maintenance. *What is holding you from uploading the package to
> Debian? *The ITP haven't seen any progress for a long time.
>
> Note that I GUI tool would be nice for the non-free stuff, but would
> be a bad idea for other hardware specific packages like RAID
> administration tools. *For this, I would like to integrate something
> like discover-pkginstall into the hwsetup package in debian-installer,
> to make sure hardware specific packages are automatically installed
> when it make sense.
>
> Happy hacking,
> --
> Petter Reinholdtsen
>
>
> --
> To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> Archive: http://lists.debian.org/2flhbnrxxkm.fsf@login2.uio.no
>
>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: u2vc09ddae71004040437g619dacddq65461829b49fb919@ma il.gmail.com">http://lists.debian.org/u2vc09ddae71004040437g619dacddq65461829b49fb919@ma il.gmail.com