FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 08-06-2008, 08:42 PM
Axel Thimm
 
Default ANNOUNCE: rpmgrok - a web-based tool for tracking a full distribution of RPMs

On Wed, Aug 06, 2008 at 03:35:45PM -0400, David Malcolm wrote:
> I've been working on a new web-based tool for analyzing Fedora: rpmgrok
>
> It digests built RPMs, analysing the metadata and payload, and stores
> the results in a database. There's a web UI for viewing the data, an
> XML-RPC interface for querying it, and a command-line tool for using the
> XML-RPC interface.
>
> I've got a prototype running on:
> http://publictest7.fedoraproject.org/rpmgrok
>
> More info (e.g. source code) can be seen at
> https://fedorahosted.org/rpmgrok
>
> The idea is to provide a new way for Fedora developers, testers, and
> other enthusiasts to track various things across the entire
> distribution, without having to have a full tree installed. It's
> probably usable by other Linux distributions.

I like the part with the "Errors and warnings from rpmlint". I can
imagine that someone that's say an expert on "invalid-desktopfile"
issues could now dig into this much easier. Very nice!

> rpmgrok is Free/Open Source software (licensed under the LGPLv2.1)
>
> == What does it track? ==
> - all symbols in binaries/libraries, and the dependencies between
> them, so that you can see e.g. exactly what calls a particular function.
> This can also be used to locate instances of static linkage. See e.g.
> http://publictest7.fedoraproject.org/rpmgrok/elffile/258085 (details
> of /lib/libexpat.so.1.5.2 from a built RPM)
> - manifests of all RPMs, so that you can browse the files in packages
> via a web UI. See http://publictest7.fedoraproject.org/rpmgrok/files
> The file view is only interesting at the moment for ELF files (binaries)
> and for .desktop files.
> - all shared objects names, and the dependencies between them. See
> e.g.
> - http://publictest7.fedoraproject.org/rpmgrok/sonames (browsable
> view of all sonames in the distro)
> - http://publictest7.fedoraproject.org/rpmgrok/elffile/739167
> (details of /usr/lib/libxml2.so.2.6.32 from within a built libxml2 rpm)
> - Everything implementing or linking against libpcre.so.0 down to
> the level of individual binaries:
> http://publictest7.fedoraproject.org/rpmgrok/soname/libpcre.so.0
> - results of rpmlint of all rpms. See
> http://publictest7.fedoraproject.org/rpmgrok/rpmlint_messages for a UI
> to browse by error message, and e.g.
> http://publictest7.fedoraproject.org/rpmgrok/rpmlint/dangerous-command-in-%25post for an example of all error messages of a particular kind. It may be worth fixing some rpmlint errors (though others look like false positives, and others are probably not worth it)
> - all .desktop files and their fields so that you can e.g. find
> applications that can handle PDF files. See e.g.
> http://publictest7.fedoraproject.org/rpmgrok/mimetype/application/pdf
> for a view of all desktop files that can handle "application/pdf", and
> e.g. http://publictest7.fedoraproject.org/rpmgrok/desktopfile/253580
> showing a specific desktop file
> - SLOCcount stats (see http://www.dwheeler.com/sloccount/ ) for
> prepped source trees (e.g "what % of Fedora is in C/C++/Python?" etc).
> Don't have the data prepped yet.
> - any other kind of thing we want to add (provided there's a sane way
> to gather it in a script and slurp it into the database, of course...)
> - sizes of packages; why is package foo so big?
> - report on all fonts in the distro, and what packages provide them
> etc
>
> Note that due to my poor css there are lots of links that don't show up
> as such in the various table views. You may need to explore with the
> mouse to find all of the cross-referencing that the web UI has.
>
> == What's it currently showing? ==
> I queued up an analysis of all of rawhide as of 2008-07-25 on i386; a
> little over 10000 built packages. It took about a week to process, and
> about 200 of these jobs failed for one reason or another. See
> https://fedorahosted.org/rpmgrok/ticket/9 for more info.
>
> So the db is currently just showing a snapshot in time of rawhide two
> weeks ago, on one architecture (and missing 2% of the packages due to
> errors).
>
> Ultimately I want to build things up so that we can show time-based
> trend reports e.g. the size of a minimal install over time (or
> whatever).
>
> == Help Needed! ==
> Hopefully this looks of interest to people.
>
> I need help with coding, with sysadmin work, with making the UI better,
> and with things I probably haven't thought of yet etc. I hope this can
> be a useful tool for Fedora.
>
> If you're interested in hacking on rpmgrok, get in touch. The README
> file is hopefully of interest; see
> http://git.fedorahosted.org/git/rpmgrok.git?p=rpmgrok.git;a=blob_plain;f=README.tx t;hb=HEAD README.txt
>
> It's implemented using TurboGears and SQLAlchemy (specifically,
> sqlalchemy 0.4, since it uses polymorphic inheritance features from that
> version).
>
> It also has a somewhat general-purpose task scheduler, used to control a
> pool of worker hosts that do the actual analysis. It ought to be
> pluggable to do other types of task.
>
> == Source Code ==
> Git URLS are:
> git://git.fedorahosted.org/rpmgrok.git
> ssh://git.fedorahosted.org/git/rpmgrok.git
> http://git.fedorahosted.org/git/rpmgrok.git
> (you need to be in the gitrpmgrok of the Fedora Accounts System to have
> git push privileges; talk to me if you want to get involved)
>
> == Related work ==
> Inspiration includes
> - the OpenGrok project (see
> http://opensolaris.org/os/project/opengrok/ though that appears to focus
> on source trees, whereas rpmgrok focuses on built packages)
> - the Debian project's Lintian tool (see http://lintian.debian.org )
>
> Enjoy!
> Dave
>

--
Axel.Thimm at ATrpms.net
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 08-06-2008, 09:26 PM
David Malcolm
 
Default ANNOUNCE: rpmgrok - a web-based tool for tracking a full distribution of RPMs

On Wed, 2008-08-06 at 23:42 +0300, Axel Thimm wrote:
> On Wed, Aug 06, 2008 at 03:35:45PM -0400, David Malcolm wrote:
> > I've been working on a new web-based tool for analyzing Fedora: rpmgrok
> >
> > It digests built RPMs, analysing the metadata and payload, and stores
> > the results in a database. There's a web UI for viewing the data, an
> > XML-RPC interface for querying it, and a command-line tool for using the
> > XML-RPC interface.
> >
> > I've got a prototype running on:
> > http://publictest7.fedoraproject.org/rpmgrok
> >
> > More info (e.g. source code) can be seen at
> > https://fedorahosted.org/rpmgrok
> >
> > The idea is to provide a new way for Fedora developers, testers, and
> > other enthusiasts to track various things across the entire
> > distribution, without having to have a full tree installed. It's
> > probably usable by other Linux distributions.
>
> I like the part with the "Errors and warnings from rpmlint". I can
> imagine that someone that's say an expert on "invalid-desktopfile"
> issues could now dig into this much easier. Very nice!
Yes, this is:
http://publictest7.fedoraproject.org/rpmgrok/rpmlint/invalid-desktopfile

Unfortunately the db doesn't capture the errors from
desktop-file-validate itself. I wonder if there's a way to do this.
IIRC rpmlint doesn't emit this output anywhere.

(fwiw all these tests were done with desktop-file-utils-0.10-7)

[snip]

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 08-07-2008, 02:14 AM
Mamoru Tasaka
 
Default ANNOUNCE: rpmgrok - a web-based tool for tracking a full distribution of RPMs

David Malcolm wrote, at 08/07/2008 06:26 AM +9:00:
> On Wed, 2008-08-06 at 23:42 +0300, Axel Thimm wrote:
>> I like the part with the "Errors and warnings from rpmlint". I can
>> imagine that someone that's say an expert on "invalid-desktopfile"
>> issues could now dig into this much easier. Very nice!
> Yes, this is:
> http://publictest7.fedoraproject.org/rpmgrok/rpmlint/invalid-desktopfile
>
> Unfortunately the db doesn't capture the errors from
> desktop-file-validate itself. I wonder if there's a way to do this.
> IIRC rpmlint doesn't emit this output anywhere.
>
> (fwiw all these tests were done with desktop-file-utils-0.10-7)

Unfortunately I guess your list contains not a few false-positive errors.

For example, I checked fedora-alexandria.desktop with desktop-file-utils-0.10-7
(this is Fedora core 6 desktop-file-utils) and actually I get the error:
--------------------------------------------------------------------------
$ ./desktop-file-validate /usr/share/applications/fedora-alexandria.desktop
/usr/share/applications/fedora-alexandria.desktop: error: required key "Encoding" not found
--------------------------------------------------------------------------

However the key "Encoding" is now deprecated and with Fedora 10 desktop-file-utils
(0.15-3.fc10) including Encoding key is warned like:
--------------------------------------------------------------------------
$ desktop-file-validate ./fedora-alexandria.desktop
./fedora-alexandria.desktop: warning: key "Encoding" in group "Desktop Entry" is deprecated
---------------------------------------------------------------------------

Regards,
Mamoru

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 

Thread Tools




All times are GMT. The time now is 07:49 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org