FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 03-21-2011, 11:07 AM
"Paweł Hajdan, Jr."
 
Default FEATURES=test, sys-devel/gcc ignored test failures

sys-devel/gcc runs tests, but the results are ignored and I remember the
tests fail most of the time.

Because the tests take long time to run and fail anyway (I understand
it's non-trivial to fix those on Gentoo side), I wonder whether it makes
sense to run them at all:

toolchain.eclass:

gcc_src_test() {
cd "${WORKDIR}"/build
emake -j1 -k check || ewarn "check failed and that sucks "
}

My suggestion is to make the src_test empty (I think the default one
still calls make). I can produce a patch if needed.

What do you think?
 
Old 03-21-2011, 11:29 AM
Amadeusz Żołnowski
 
Default FEATURES=test, sys-devel/gcc ignored test failures

Excerpts from Paweł Hajdan, Jr.'s message of Mon Mar 21 13:07:33 +0100 2011:
> My suggestion is to make the src_test empty (I think the default one
> still calls make). I can produce a patch if needed.
>
> What do you think?

Maybe restrict? https://bugs.gentoo.org/298014
--
Amadeusz Żołnowski

PGP key fpr: C700 CEDE 0C18 212E 49DA 4653 F013 4531 E1DB FAB5
 
Old 03-21-2011, 09:02 PM
Ryan Hill
 
Default FEATURES=test, sys-devel/gcc ignored test failures

On Mon, 21 Mar 2011 13:07:33 +0100
"Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> wrote:

> sys-devel/gcc runs tests, but the results are ignored and I remember the
> tests fail most of the time.

s/most/all

> Because the tests take long time to run and fail anyway (I understand
> it's non-trivial to fix those on Gentoo side), I wonder whether it makes
> sense to run them at all:

It does to me, I use them all the time. The important part is that we
install the test results, which can then be used for regression testing when
rolling patchsets.

> toolchain.eclass:
>
> gcc_src_test() {
> cd "${WORKDIR}"/build
> emake -j1 -k check || ewarn "check failed and that sucks "
> }
>
> My suggestion is to make the src_test empty (I think the default one
> still calls make). I can produce a patch if needed.
>
> What do you think?

I think that glibc and gcc tests and other testsuites that nearly always
fail shouldn't be run for the average user but should still be easily
accessible in a standard way. I think we need a more finely grained test
setup, where we can say tests are "expensive" or "interesting only to
developers" or "known to fail", and let people opt-in to these on a
per-package basis. Right now you always have to opt-out using
package.use.mask which "works" but is unintuitive.


--
fonts, gcc-porting, it makes no sense how it makes no sense
toolchain, wxwidgets but i'll take it free anytime
@ gentoo.org EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662
 
Old 03-22-2011, 11:09 AM
"Paweł Hajdan, Jr."
 
Default FEATURES=test, sys-devel/gcc ignored test failures

On 3/21/11 11:02 PM, Ryan Hill wrote:
> It does to me, I use them all the time. The important part is that we
> install the test results, which can then be used for regression testing when
> rolling patchsets.

I see, it makes sense. I guess you're comparing the test results before
and after rolling patchsets and look for regressions.

> I think that glibc and gcc tests and other testsuites that nearly always
> fail shouldn't be run for the average user but should still be easily
> accessible in a standard way. I think we need a more finely grained test
> setup, where we can say tests are "expensive" or "interesting only to
> developers" or "known to fail", and let people opt-in to these on a
> per-package basis. Right now you always have to opt-out using
> package.use.mask which "works" but is unintuitive.

My main point is that the developer profile has FEATURES=test, and also
arch testers and developers run with FEATURES=test. Being able to
quickly rebuild gcc, glibc and others is a win.

I'm trying to understand the problem better - do you know what causes
those test failures? I don't expect a "complete" answer because that'd
probably be a half of actually fixing the failures.
 
Old 03-22-2011, 02:01 PM
Matt Turner
 
Default FEATURES=test, sys-devel/gcc ignored test failures

On Tue, Mar 22, 2011 at 12:09 PM, "Paweł Hajdan, Jr."
<phajdan.jr@gentoo.org> wrote:
> I'm trying to understand the problem better - do you know what causes
> those test failures? I don't expect a "complete" answer because that'd
> probably be a half of actually fixing the failures.

They both have huge test suites covering massive amounts of code. It's
only natural that some of the tests fail from version to version.
 
Old 03-22-2011, 10:25 PM
Ryan Hill
 
Default FEATURES=test, sys-devel/gcc ignored test failures

On Tue, 22 Mar 2011 13:09:46 +0100
"Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> wrote:

> On 3/21/11 11:02 PM, Ryan Hill wrote:
> > It does to me, I use them all the time. The important part is that we
> > install the test results, which can then be used for regression testing when
> > rolling patchsets.
>
> I see, it makes sense. I guess you're comparing the test results before
> and after rolling patchsets and look for regressions.
>
> > I think that glibc and gcc tests and other testsuites that nearly always
> > fail shouldn't be run for the average user but should still be easily
> > accessible in a standard way. I think we need a more finely grained test
> > setup, where we can say tests are "expensive" or "interesting only to
> > developers" or "known to fail", and let people opt-in to these on a
> > per-package basis. Right now you always have to opt-out using
> > package.use.mask which "works" but is unintuitive.
>
> My main point is that the developer profile has FEATURES=test, and also
> arch testers and developers run with FEATURES=test. Being able to
> quickly rebuild gcc, glibc and others is a win.

Yes, I'm agreeing with you. I'd like these off by default too. We need a
standard way of enabling them however. USE="test-dev" or something. I
complain about this about once a year or so. Maybe I should just do it.

In the meantime:

echo -e 'sys-libs/glibc test
sys-devel/gcc test'
>> /etc/portage/profile/package.use.mask

> I'm trying to understand the problem better - do you know what causes
> those test failures? I don't expect a "complete" answer because that'd
> probably be a half of actually fixing the failures.

The GCC testsuite isn't designed to pass. It's designed to be a
regression test. Check before and after you apply a patch, make sure you
don't cock things up. From http://gcc.gnu.org/install/test.html: "It is
normal for some tests to report unexpected failures. At the current time the
testing harness does not allow fine grained control over whether or not a test
is expected to fail." Look at http://gcc.gnu.org/buildstat.html and you'll
see this in practice.

One thing I know of that causes a bunch of failures is the fact that we
enable -Wformat, -Wformat-security, and -Wtrampolines by default. Any
additional output during a test = fail. I patched these recently for 4.5
though so they shouldn't be a problem going forward.


--
fonts, gcc-porting, it makes no sense how it makes no sense
toolchain, wxwidgets but i'll take it free anytime
@ gentoo.org EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662
 

Thread Tools




All times are GMT. The time now is 12:53 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org