Grant schrieb am 26.01.2009 18:35:
>
> I suppose you and Jesus are right, but what about cruft removal? Are
> you saying Gentoo is 100% cruft-free? I've got a lot of junk in /etc
> and especially ~/.*
>
No it isn't, but this is very hard to automate cruft detection via
scripts as you still have to decide yourself if a file is really needed
or not.
It is very easy to compare all the files on the filesystem with the list
of files controlled by portage. But everything that is left over is not
cruft, because there are files you put somewhere yourself or files that
are not controlled by portage but are essential to the system. So you
have to use some kind of ignorelist to exclude files that are not under
portage control but needed. As you see it is still you who has to decide
if a file is cruft or not.
I have done something minimalistic that evolved over time and that fits
my needs. I long refrained from attaching it here, but as this question
comes up so often I think I will share it. I am not a programming guru
so there might be bugs or other stuff done in a poor way so don't come
here and complain if the script has killed your kitten.
I will attach the cruft script written in perl and as an example the
ignorelist I use for my system and the list of cruft files that remain
as false positives. You have to customize the ignorelist yourself so it
does a proper job on your system. I have tried to make the script user
friendly but it surly isn't so you may have to tinker with it a bit. The
script actually does not delete anything it just presents you a list of
files that contains all files on the filesystem minus the files
controlled by portage and minus the files you have ignored.
There is also another cruft script that you may get from overlays but I
don't remember where. If you search the archives you may find it as it
has already been mentioned here.
Regards,
Daniel
#!/usr/bin/perl
#
#
#
################################################## #############################
# Script: cruft-control.pl
# Version: 0.19
# Autor: Daniel Pielmeier
################################################## #############################
use strict;
use diagnostics;
use warnings;
use File::Find;
#use Text:
iff;
use Getopt::Long qw (:config posix_default gnu_compat pass_through
no_ignore_case_always);
# Directories
my $destdir = ();
my $confdir = ();
my $searchdir = ();
my $vdb_path = ();
# Files
my $cruft = "cruft.txt";
my $oldcruft = "oldcruft.txt";
my $diffcruft = "diffcruft.txt";
my $negcruft = "negcruft.txt";
my $systemfile = "systemfile.txt";
my $contentlist = "contentlist.txt";
my $packagefile = "packagefile.txt";
my $ignorelist = "ignorelist.conf";
# Variables
my $uid = `id -u`; chomp $uid;
my $debug = ();
my $ignoreitem = ();
my @systemfile = my @ignorelist = ();
my @contentlist = ();
my @packagefile = ();
my $prev = ();
my @cruft = ();
my %packagefilecheck = ();
my @negcruft =();
my %systemfilecheck = ();
my $help = "
DESCRIPTION:
This script tries to find files not under the control of the package manager.
The resulting list of files should normally not called cruft as proper
detection of cruft is almost impossible. So this script does not offer
functions to delete any files reported.
In general it searches all files in the filesystem. It uses an
$ignorelist where files and directories can be specified which will be
ignored from the search to avoid obvious false positives.
Then it gets the files which are listed in CONTENTS from vdb_path where the
files installed by the package manager are stored. This two resulting lists
are compared with each other to get the files which are not under the control
of the package manager. The resulting list is stored in the $cruft file.
Additionally if available it compares the results with an older cruft file
called $oldcruft and creates $diffcruft out of it to find out changes
which occuring when merging and unmerging programs. This is useful when you
are not sure about the files listed in an initial search so one can just
track the changes.
Carefully examine $cruft and/or $diffcruft. Deleting wrong files which
are essential for a working system may break it. You are responsible for your
system. This script will just make it easier to to detect files which are
probably not needed on the system.
Optionally it writes debug info in form of $systemfile, $contentlist,
$packagefile and $negcruft.
$systemfile lists all files found in the filesystem excluding the files
and directories from $ignorelist.
$contentlist lists all paths to the CONTENT files only useful to see which
packages are installed.
$packagefile lists all files under the control of the package manager.
$negcruft this are the files which are in $packagefile but not in
$systemfile because they are not present in the filessystem or ignored by
the ignorefile.
CONFIGURATION:
For $0 you configure the path to configuration files which
for now is only $ignorelist. and the path to generated output ($cruft,
$oldcruft, $diffcruft) and debug files ($systemfile, $packagefile,
$contentlist, $negcruft).
By default the generated output and debug files go to the current directory
and configuration files are read from /etc/cruft-control
This paths can be overridden by the environment variables CRUFT_DESTDIR for
generated output and debug files and CRUFT_CONFDIR for configuration files
or on the commandline using the relevant switches.
Order of preference = default path < environment variable < command line
DEPENDENCIES:
dev-lang/perl
modules: File::Find, Getopt::Long
sys-apps/portage: portageq for getting the vdb_path
sys-apps/diffutils: diff for comparing old and new results
alternative to sys-apps/diffutils (Text:
iff (Algorithm:
iff)) currently
deactivated
TODO:
- Implement POD::USAGE.
- Define proper paths for all files read and written.
- Decide wether to include searchpath and pm database in the available
options or not.
- Decide wether to use sys-apps/diffutils or Text:
iff in combination with
Algorithm:
iff
";
# Configurable paths
if (defined $ENV{"CRUFT_DESTDIR"}) {
$destdir = $ENV{"CRUFT_DESTDIR"};
} else {
$destdir = ("./");
}
if (defined $ENV{"CRUFT_CONFDIR"}) {
$confdir = $ENV{"CRUFT_CONFDIR"};
} else {
$confdir = ("/etc/cruftcontrol");
}
# Hardcoded paths
# Path to searchdir. Normally this is the root-dir if you want to search the
# whole system.
$searchdir = ("/");
# Path to portage database which holds information of all installed ebuilds.
$vdb_path = qx (portageq vdb_path);
chomp ($vdb_path);
# Checking for permissions and correct usage so the program can access all
# files and directories.
if ($uid ne "0") {
print "e[1;31m[ERROR] You need to be root to use $0 for accessing all
";
print "[ERROR] directories and files!e[m
";
&usage();
}
GetOptions ('destdir|d=s' => $destdir, 'confdir|c=s' => $confdir,
#'searchdir|s=s' => $searchdir, 'vdb_path|v=s' => $vdb_path,
'debug|b' => $debug, 'help|h|?' => sub {usage()});
&usage() if ($#ARGV > -1);
foreach ("$destdir", "$confdir") {
unless (-d $_) {
print "e[1;31m[ERROR] The directory $_ does not existe[m
";
&usage();
}
}
# Creating a list of all files in the filesystem excluding files and
# directories which belong to an ignorelist.
print "[INFO] Searching files on the filesystem ...
";
if (-f "$confdir/$ignorelist" && -s _ && -T _) {
open (IGNORELIST,"<$confdir/$ignorelist") || die ("$ignorelist: $!");
print "[INFO] Excluding files from $confdir/$ignorelist
";
foreach (<IGNORELIST>) {
unless ( $_ eq "
" || $_ =~ "#" ) {
push (@ignorelist, $_);
}
}
chomp (@ignorelist);
find(&sysignore, $searchdir);
sub sysignore() {
my $count = 0;
foreach $ignoreitem (@ignorelist) {
if ("$File::Find::name" eq "$ignoreitem") {
splice(@ignorelist,$count,1);
($File::Find:
rune = 1);
return;
}
$count++;
}
push (@systemfile, "$File::Find::name
");
}
} else {
print "e[1;33m[WARN] The file $ignorelist does not exist in
[WARN] $confdir
";
print "[WARN] or has no content. This may result in a big and probably
";
print "[WARN] useless cruft list. ;-)e[m
";
find(&sys, $searchdir);
sub sys() {
push (@systemfile, "$File::Find::name
");
}
}
@systemfile = sort (@systemfile);
# Searches the package database for the CONTENTS-files of all installed
# packages. This is the base for getting the list of controlled files.
print "[INFO] Searching installed packages ...
";
find(&content, $vdb_path);
sub content() {
push (@contentlist, "$File::Find::name
") if (/CONTENTS/);
}
@contentlist = sort (@contentlist);
# Creating a list of all files owned by all installed packages.
print "[INFO] Assigning files to installed packages ...
";
foreach (@contentlist) {
open (CONTENT, "$_") || die("$_: $!");
foreach (<CONTENT>) {
if ((/^sym/) || (/^dir/)) {
push (@packagefile, "$2
") if (/(w+s)(S+)/);
}
if (/^obj/) {
push (@packagefile, "$2
") if (/(w+s)(.+)(sw+sd+$)/);
}
}
}
@packagefile = sort (@packagefile);
$prev = "not equal to $packagefile[0]";
@packagefile = grep($_ ne $prev && ($prev = $_, 1), @packagefile);
# Compares the list of the files in the system with the files controlled by
# portage to get unmanaged files.
print "[INFO] Comparing to get unmanaged files ...
";
open (CRUFT,">$destdir/$cruft") || die ("$cruft: $!");
@packagefilecheck{@packagefile} = ();
foreach $_ (@systemfile) {
push (@cruft, $_) unless exists $packagefilecheck{$_};
}
@cruft = sort (@cruft);
print CRUFT @cruft;
print "[INFO] List containing cruft written to $destdir/$cruft
";
# Compares the list of found cruft files with a list of files found in an
# older search.
if (-f "$destdir/$oldcruft" && -s _ && -T _) {
print "[INFO] Comparing cruft with old results ...
";
#open (OLDCRUFT,"<$destdir/$oldcruft") || die ("$oldcruft: $!");
#my @oldcruft = <OLDCRUFT>;
#print "[INFO] Using previous results from $destdir/$oldcruft
";
#open (DIFFCRUFT,">$destdir/$diffcruft") || die ("$diffcruft: $!");
#diff @oldcruft, @cruft, {OUTPUT => *DIFFCRUFT};
system "diff -u $destdir/oldcruft.txt $destdir/cruft.txt >$destdir/diffcruft.txt";
print "[INFO] Differences to previous run written to $destdir/$diffcruft
";
} else {
print "[INFO] The file $oldcruft does not exist in
";
print "[INFO] $destdir
[INFO] or has no contents.
";
print "[INFO] Note: For later comparisons you can move $cruft to $oldcruft
";
print "[INFO] to catch the differences in the file $diffcruft.
";
print "[INFO] Skipping comparison with old results ...
";
}
# Write debug files.
&debug() if defined $debug;
# Subroutines
# Print usage message.
sub usage() {
print "USAGE: $0 [options]
";
print " -d|--destdir=DIR directory for analysis and debug output
";
print " [$destdir]
";
print " -c|--confdir=DIR directory containing $ignorelist
";
print " [$confdir]
";
#print " -s|--searchdir=DIR base directory for searching
";
#print " [$searchdir]
";
#print " -v|--vdb_path=DIR path to the portage database
";
#print " [$vdb_path]
";
print " -b|--debug generate debug output
";
print " -?|-h|--help print this lot out
";
print $help;
exit;
}
# Subroutine to generate debug output.
sub debug() {
open (SYS,">$destdir/$systemfile") || die ("$systemfile: $!");
print SYS @systemfile;
print "e[1;35m[DEBUG] All files from the system excluding the ignored files written to $destdir/$systemfilee[m
";
open (CONTENT,">$destdir/$contentlist") || die ("$contentlist: $!");
print CONTENT @contentlist;
print "e[1;35m[DEBUG] Path to files containing package contents written to $destdir/$contentliste[m
";
open (PACKAGE,">$destdir/$packagefile") || die ("$packagefile: $!");
print PACKAGE @packagefile;
print "e[1;35m[DEBUG] All files controlled by the package manager written to $destdir/$packagefilee[m
";
# For reference list the files which are controlled by portage but not in the
# system maybe because they are deleted or ignored by the ignorefile.
print "e[1;35m[DEBUG] Comparing to get negcruft files ...
";
@systemfilecheck{@systemfile} = ();
foreach $_ (@packagefile) {
push (@negcruft, $_) unless exists $systemfilecheck{$_};
}
@negcruft = sort (@negcruft);
open (NEGCRUFT,">$destdir/$negcruft") || die ("$negcruft: $!");
print NEGCRUFT @negcruft;
print "e[1;35m[DEBUG] Negative list of cruft written to $destdir/$negcrufte[m
";
}
# Directories and files which should always be ignored.
/boot
/dev
/home
/lost+found
/media
/mnt
/proc
/root
/sys
/tmp
/usr/src
/lib/modules
/usr/tmp
/var
# Directories and files created by the user or in pkg_post_install plus other
# non cruft.
/etc/portage
/etc/runlevels
/etc/ssl/certs
/lib/rc/console
/lib/rc/init.d
/lib/splash/cache
/usr/i686-pc-linux-gnu/bin
/usr/i686-pc-linux-gnu/lib
/usr/include/GL
/usr/lib/openoffice/share/dict
/usr/share/mime
/usr/lib/perl5/site_perl/5.8.8/i686-linux
/usr/lib/gentoolkit/pym
/usr/lib/gimp/2.0/plug-ins
/usr/lib/portage/pym
/usr/lib/scons-1.0.0/SCons
/usr/share/dia/python
/usr/share/hplip
/usr/share/javatoolkit/pym
/usr/lib/python2.5/site-packages
/
/bin/sh
/etc/.pwd.lock
/etc/adjtime
/etc/asound.state
/etc/blkid.tab
/etc/blkid.tab.old
/etc/conf.d/lm_sensors
/etc/conf.d/net
/etc/csh.env
/etc/cups/ppd/HP_Business_Inkjet_1000.ppd
/etc/cups/printers.conf
/etc/cups/printers.conf.O
/etc/eix-sync.conf
/etc/env.d/02ccache
/etc/env.d/02locale
/etc/env.d/03opengl
/etc/env.d/05binutils
/etc/env.d/05gcc-i686-pc-linux-gnu
/etc/env.d/20java
/etc/env.d/90games
/etc/env.d/99local
/etc/env.d/binutils/config-i686-pc-linux-gnu
/etc/env.d/gcc/.NATIVE
/etc/env.d/gcc/config-i686-pc-linux-gnu
/etc/fstab
/etc/gre.d/gre.conf
/etc/group
/etc/group-
/etc/gshadow
/etc/gshadow-
/etc/gtk-2.0/gdk-pixbuf.loaders
/etc/gtk-2.0/gtk.immodules
/etc/init.d/net.eth0
/etc/java-config-2/current-system-vm
/etc/ld.so.cache
/etc/ld.so.conf
/etc/locale.gen
/etc/localtime
/etc/make.conf
/etc/make.profile
/etc/modprobe.conf
/etc/mtab
/etc/openldap
/etc/openldap/ssl
/etc/openldap/ssl/ldap.crt
/etc/openldap/ssl/ldap.csr
/etc/openldap/ssl/ldap.key
/etc/openldap/ssl/ldap.pem
/etc/pango/pango.modules
/etc/passwd
/etc/passwd-
/etc/printcap
/etc/profile.csh
/etc/profile.env
/etc/resolv.conf
/etc/sgml
/etc/sgml/catalog
/etc/sgml/xml-docbook-4.4.cat
/etc/shadow
/etc/shadow-
/etc/splash/default
/etc/ssh/ssh_host_dsa_key
/etc/ssh/ssh_host_dsa_key.pub
/etc/ssh/ssh_host_key
/etc/ssh/ssh_host_key.pub
/etc/ssh/ssh_host_rsa_key
/etc/ssh/ssh_host_rsa_key.pub
/etc/timezone
/etc/udev/rules.d/10-local.rules
/etc/udev/rules.d/70-persistent-cd.rules
/etc/udev/rules.d/70-persistent-net.rules
/etc/xml/catalog
/etc/xml/docbook
/lib/cpp
/lib/libgcc_s.so.1
/lib/rc/cache
/lib/rc/cache/depconfig
/lib/rc/cache/deptree
/lib/rc/cache/nettree
/lib/rc/cache/softlevel
/lib/rcscripts/awk
/lib/rcscripts/awk/fixlafiles.awk
/lib/splash/sys
/opt/Adobe/Help/de_DE/Adobe Reader
/opt/Adobe/Help/de_DE/Adobe Reader/8.0
/opt/Adobe/Help/de_DE/Adobe Reader/8.0/images
/opt/Adobe/Help/en_US/Adobe Reader
/opt/Adobe/Help/en_US/Adobe Reader/8.0
/opt/Adobe/Help/en_US/Adobe Reader/8.0/images
/sbin/fix_libtool_files.sh
/usr/bin/addr2line
/usr/bin/ar
/usr/bin/as
/usr/bin/c++
/usr/bin/c++filt
/usr/bin/c89
/usr/bin/c99
/usr/bin/cc
/usr/bin/cpp
/usr/bin/ctags
/usr/bin/erb
/usr/bin/esd
/usr/bin/ex
/usr/bin/g++
/usr/bin/gcc
/usr/bin/gcov
/usr/bin/gfortran
/usr/bin/gprof
/usr/bin/i686-pc-linux-gnu-addr2line
/usr/bin/i686-pc-linux-gnu-ar
/usr/bin/i686-pc-linux-gnu-as
/usr/bin/i686-pc-linux-gnu-c++
/usr/bin/i686-pc-linux-gnu-c++filt
/usr/bin/i686-pc-linux-gnu-cpp
/usr/bin/i686-pc-linux-gnu-g++
/usr/bin/i686-pc-linux-gnu-gcc
/usr/bin/i686-pc-linux-gnu-gfortran
/usr/bin/i686-pc-linux-gnu-gprof
/usr/bin/i686-pc-linux-gnu-ld
/usr/bin/i686-pc-linux-gnu-nm
/usr/bin/i686-pc-linux-gnu-objcopy
/usr/bin/i686-pc-linux-gnu-objdump
/usr/bin/i686-pc-linux-gnu-ranlib
/usr/bin/i686-pc-linux-gnu-readelf
/usr/bin/i686-pc-linux-gnu-size
/usr/bin/i686-pc-linux-gnu-strings
/usr/bin/i686-pc-linux-gnu-strip
/usr/bin/idle
/usr/bin/irb
/usr/bin/ld
/usr/bin/nm
/usr/bin/objcopy
/usr/bin/objdump
/usr/bin/pydoc
/usr/bin/python
/usr/bin/python-config
/usr/bin/python2
/usr/bin/ranlib
/usr/bin/rdoc
/usr/bin/readelf
/usr/bin/ri
/usr/bin/ruby
/usr/bin/size
/usr/bin/strings
/usr/bin/strip
/usr/bin/testrb
/usr/bin/vi
/usr/bin/view
/usr/bin/yacc
/usr/include/ansidecl.h
/usr/include/bfd.h
/usr/include/bfdlink.h
/usr/include/db.h
/usr/include/db_185.h
/usr/include/dis-asm.h
/usr/include/libiberty.h
/usr/include/qt4/Gentoo/gentoo-qconfig.h
/usr/include/symcat.h
/usr/kde/3.5/share/applications/mimeinfo.cache
/usr/kde/3.5/share/services/ksycoca
/usr/lib/cracklib_dict.hwm
/usr/lib/cracklib_dict.pwd
/usr/lib/cracklib_dict.pwi
/usr/lib/gconv/gconv-modules.cache
/usr/lib/graphviz/config
/usr/lib/libGL.la
/usr/lib/libGL.so
/usr/lib/libGLcore.so
/usr/lib/libXvMCNVIDIA_dynamic.so.1
/usr/lib/libdb.a
/usr/lib/libdb.so
/usr/lib/libdb_cxx.a
/usr/lib/libdb_cxx.so
/usr/lib/libdb_java.a
/usr/lib/libdb_java.so
/usr/lib/libperl.so
/usr/lib/libruby.so
/usr/lib/locale/locale-archive
/usr/lib/moo/plugins/lib/insert_date_and_time.pyc
/usr/lib/moo/plugins/lib/insert_date_and_time.pyo
/usr/lib/moo/plugins/lib/medit/__init__.pyc
/usr/lib/moo/plugins/lib/medit/__init__.pyo
/usr/lib/moo/plugins/lib/medit/runpython.pyc
/usr/lib/moo/plugins/lib/medit/runpython.pyo
/usr/lib/moo/plugins/lib/pyconsole.pyc
/usr/lib/moo/plugins/lib/pyconsole.pyo
/usr/lib/moo/plugins/pycmd.pyc
/usr/lib/moo/plugins/pycmd.pyo
/usr/lib/moo/plugins/python.pyc
/usr/lib/moo/plugins/python.pyo
/usr/lib/moo/plugins/terminal.pyc
/usr/lib/moo/plugins/terminal.pyo
/usr/lib/mozilla-firefox/defaults/autoconfig
/usr/lib/nsbrowser/plugins/javaplugin.so
/usr/lib/perl5/5.8.8/i686-linux/Encode/ConfigLocal.pm
/usr/lib/perl5/vendor_perl/5.8.8/XML/SAX/ParserDetails.ini
/usr/lib/xorg/modules/extensions/libglx.so
/usr/lib/xorg/modules/extensions/libwfb.so
/usr/qt/3/etc/settings/.qt_plugins_3.3rc.lock
/usr/qt/3/etc/settings/qt_plugins_3.3rc
/usr/share/X11/xkb/compiled
/usr/share/applications/mimeinfo.cache
/usr/share/binutils-data/i686-pc-linux-gnu/2.18/info/dir
/usr/share/config
/usr/share/dia/python-startup.pyc
/usr/share/dia/python-startup.pyo
/usr/share/fonts/100dpi/encodings.dir
/usr/share/fonts/100dpi/fonts.dir
/usr/share/fonts/75dpi/encodings.dir
/usr/share/fonts/75dpi/fonts.dir
/usr/share/fonts/Type1/encodings.dir
/usr/share/fonts/Type1/fonts.dir
/usr/share/fonts/Type1/fonts.scale
/usr/share/fonts/misc/encodings.dir
/usr/share/fonts/misc/fonts.dir
/usr/share/gcc-data/i686-pc-linux-gnu/4.1.2/info/dir
/usr/share/icons/Rodent/icon-theme.cache
/usr/share/icons/Tango/icon-theme.cache
/usr/share/icons/hicolor/icon-theme.cache
/usr/share/info/dir
/usr/share/man/man1/ctags.1.bz2
/usr/share/man/man1/ex.1.bz2
/usr/share/man/man1/python.1.bz2
/usr/share/man/man1/vi.1.bz2
/usr/share/man/man1/view.1.bz2
/usr/share/man/whatis
/usr/share/qt4/mkspecs/qconfig.pri
/usr/share/scite/locale.de.properties
/usr/share/services/ksycoca
/usr/share/timidity/current
/usr/share/vim/vim72/doc/gentoo-syntax.txt
/usr/share/xfce4/panel-plugins/mimeinfo.cache
Installing outside of Portage & cruft removal
