FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Desktop

 
 
LinkBack Thread Tools
 
Old 04-19-2012, 03:59 PM
Bjoern Michaelsen
 
Default Quality, testability for the desktop components

On Thu, Apr 19, 2012 at 11:17:21AM +0200, Sebastien Bacher wrote:
> I agree that libreoffice would be a good fit, especially if it has a
> test suite already. Bjoern, do you think it's feasible?

Yes, I had a call about that with the QA team already. There are multiple ways
this could be done. Im in a hurry right now, I will reply back with more
details later tonight.

Best,

Bjoern

--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop
 
Old 04-20-2012, 12:19 AM
Bjoern Michaelsen
 
Default Quality, testability for the desktop components

Hi all,

On Thu, Apr 19, 2012 at 11:17:21AM +0200, Sebastien Bacher wrote:
> I agree that libreoffice would be a good fit, especially if it has a
> test suite already. Bjoern, do you think it's feasible?

So, we have 3 possible sources of bugs/regressions/test failures for
LibreOffice on Ubuntu:
- LibreOffice Upstream (http://cgit.freedesktop.org/libreoffice/core/)
- Ubuntu platform (that is: all the packages that LibreOffice depends on
directly or indirectly)
- LibreOffice packaging itself (http://anonscm.debian.org/gitweb/?p=pkg-openoffice/libreoffice.git;a=summary)

As for test suites, at LibreOffice we have multiple ones:
1) unittests
- run during the build
- most cant run against an installation, because they are depending on
symbols that are not exported
- easy to debug
2) smoketest
- basic integration test
- runs against an installation
- hard to debug
3) complex tests
- manually written tests in junit that specific scenarios over
the UNO-API
- runs against an installation
- unusally good test code, but limited coverage
- harder to debug (test and tested code in different processes, UNO-bridge)
4) unoapi tests
- automatically created tests that test every UNO-interface
- rather mechanical test code, but better code coverage
- runs against an installation
- harder to debug (test and tested code in different processes, UNO-bridge)
5) testtool tests (defunct)
- testing LO/OOo by hooking into the toolkit and creating synthetic event
- fragile
- unreliable
- unmaintainable
- runs against an installation
- hard to debug
- defunct now at LibreOffice (good riddance!)
3+4 together are also called "subsequenttests"
see http://permalink.gmane.org/gmane.comp.documentfoundation.libreoffice.devel/8746
for details.

So what do we do now?
During each build we run the tests 1,2,3 and 4, but 2-4 against an installation
_before_ we pack the product. That helps us against bugs/regressions caused by changes in:

- LibreOffice Upstream
- Ubuntu platform at the time of the build

it does not find all bugs in LibreOffice packaging as these tests are run _before_ packing.

What can we do additionally?
1) just build the latest LibreOffice package available daily or weekly
This would show us if there is a bug/regression/test failure caused by an
updated package. LibreOffice has been ftbfs multiple times already by
innocent looking updates to its dependencies (which are a lot).
I consider building the LibreOffice package a testcase in itself, esp. since
a lot of tests are run during that.
2) build the latest LibreOffice package available with a brand new checkout of
the upstream sources. This would help detecting breakages early from
upstream changes, but I think that would not make sense to do before the
release branch is branched off (too instable, too much adjustment in
packaging needed before branchoff). see
http://wiki.documentfoundation.org/ReleasePlan/3.6 for details:
branchoff is scheduled for Week 23, 2012 (starting June 4th, 2012) around Alpha1
3) Run tests against a version packed and unpacked/installed into the system.
This would be the tests 3+4 from above. This would finally also cover the
LibreOffice packaging itself. This would still need a build and an
installation.
4) Provide a tinderbox to LibreOffice upstream:
http://tinderbox.libreoffice.org/MASTER/status.html
This would esp. make sense for example for armhf or somesuch to see nasty
platformspecific breakages early (those are mostly independant from
packaging, which this would not test).

What do I think to be sensible? IMHO:
1) until branchoff of the release branch
2) from branchoff
3) all the time
4) all the time

I think we will find a lot of stuff just by building LibreOffice regularly
(1+2) -- it is a rather good test case and easy/trivial to implement, even
though it is not exactly ressourcefriendly (although when using ccache and not
building all l10ns, it shouldnt be too hard on modern hardware).

3 would be a bit more work to implement, and might be questionable in the
long term as the Junit/UNO-Tests are not expected survive the
LibreOffice-4.0-API change without much work. Still, if we do 1 and 2, it
should not be too much additional work.

4 is rather indepedant of 1-3, still worth considering. I would have to choose
just one because of hardware limitations, I would go for: 1 up until branchoff,
2 from there -- both with ccache(*) and limited l10n. I believe that would give us
already some very good coverage compared to what we have now.

I will create a blueprint for UDS-Q this.

Best,

Bjoern

(*) ccache cleared weekly to catch errors by compiler updates

--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop
 

Thread Tools




All times are GMT. The time now is 09:59 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org