Looking for application testcases
 

As part of the precise cycle, the ubuntu QA team has been looking to
increase manual application testing. As part of this, I have
extended checkbox to serve up manual tests to testers to test ubuntu
applications post installation. We need your help! If your an
application developer who wants testing on his application I would
like your testcases included in the checkbox application tests for
beta1.*



If your interested in contributing a a testcase, please do so by
adding a checkbox job to this repository and submitting a merge
request. I plan on placing this in a ppa to be distributed to those
folks who are willing to help test applications during beta1.





bzr branch*lp:~nskaggs/+junk/checkbox-app-testing




Not sure how to make a checkbox test? Have a look at the following
wiki documents in addition to the example.txt.in file in the jobs
directory inside the repository. Create a new file named
yourapp.txt.in, and submit a merge request.




https://wiki.ubuntu.com/Testing/Automation/Checkbox


https://wiki.ubuntu.com/Testing/Automation/Checkbox/WritingTestsHowTo


https://wiki.ubuntu.com/Testing/Automation/Checkbox/Walkthrough



The format is very simple, and there are several examples inside the
repository. If you have any questions or issues, let me know. I
would be happy to help make sure your tests are included. Thanks
everyone for helping make the ubuntu experience better!



Nicholas



P.S. If your interested in helping test applications by running
these tests, instructions for doing so will go out next week with
the beta1 release. Thanks!



--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Looking for application testcases
 

Le 24/02/2012 01:33, Nicholas Skaggs a écrit*:


As part of the precise cycle, the ubuntu QA team has been looking
to increase manual application testing. As part of this, I have
extended checkbox to serve up manual tests to testers to test
ubuntu applications post installation. We need your help! If your
an application developer who wants testing on his application I
would like your testcases included in the checkbox application
tests for beta1.*


Hey,



Thanks for that, it's always great to have people looking at
improving the desktop ;-)



I've some question though:



- do you have any idea of what applications you would like to see
tested?

- who will run those tests and when?

- who will deal with the feedback, when and in which way?



Having things tested is great but I think we should figure how we
deal with the feedback before starting doing lot of testing this
way.



I've been working a bit with unity-checkbox to help Didier in the
previous unity update round, and dealing with the infos collected is
quite some work. It's useful for unity where we are upstream and
have resources to deal with the issues raised, I'm less sure we can
do an useful job of it on the application with our current structure
and "workforce"... we don't have the people to do upstream work and
to be fair we already know about quite some issues that we should
fix and didn't yet through bug report.



Could you picture how you would see the feedback loop work? Would
the QA team read those reports and turn issues in bugs for those
which are not already known? Or...?



Cheers,

Sebastien Bacher



--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Looking for application testcases
 

Am Freitag, den 24.02.2012, 11:24 +0100 schrieb Sebastien Bacher:
> Le 24/02/2012 01:33, Nicholas Skaggs a écrit :
> > As part of the precise cycle, the ubuntu QA team has been looking to
> > increase manual application testing. As part of this, I have
> > extended

Sorry, but: Who or what is the ubuntu QA team ?
What kind of people are here else ? Users ? Developers ?
What kind of discussion takes place here ?

I´m new here and BTW: Is my subscription correct ?

Even a look at the archives didn´t help dissolve my confusion :-(
Regards from
--
Thomas Prost <t@prosts.info>
ProstsInfo


--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Looking for application testcases
 

Hello Sebastien. Great questions; let me give you some answers!

- do you have any idea of what applications
you would like to see tested?


Yes, I am targetting the default installed desktop applications.
However, this is in no way limited to the default applications.
Ultimately it would be awesome to see testcases for all of the
"popular" applications people like to use on ubuntu.

- who will run those tests and when?


These tests will be run during the beta1 and beta2 cycles. That's
this week, and the last week in March for beta2. The tests will be
run by "normal" users; varying from our normal set of testers
(awesome work you guys do, thank you!)* who do iso and sru testing,
as well as those folks who just want to try out the new release and
are the more casual tester/user.

- who will deal with the feedback, when and
in which way?
Right now I plan on gathering the feedback and submitting it for
everyone to publicly see. All of the results will be on the results
tracker in launchpad so they can be seen as they are submitted by
anyone. I plan to share aggregate details on my blog, and hopefully
a public facing website showing the top testers, etc. In addition, I
am happy to share additional details with anyone upon request :-)



As far as your concerns about sorting thru all the testing and
getting good bug reports, etc, you are concerns are valid. This
approach doesn't scale to thousands of tests and users, but for the
moment the volume will be low enough to allow manual processing.
Even if thousands of cases are submitted, we can look at the
aggregate data simple enough and get useful information out of it.
The testers themselves will be encouraged to submit bug reports, but
the aggregated data will be collected by me and I will share with
the upstreams involved to ensure any anomalies are researched and
potential bugs filed if found.



Finally I will add that this entire testing loop is still a work in
progress. We want to refine this loop and the tools we use as we go
into next cycle. Expect to see some sessions at UDS around this
topic. I hope to get some feedback and ideas we can use to help
shape the Q cycle testing.



Thanks!



Nicholas



On 02/24/2012 05:24 AM, Sebastien Bacher wrote:


Le 24/02/2012 01:33, Nicholas Skaggs a écrit*:


As part of the precise cycle, the ubuntu QA team has been
looking to increase manual application testing. As part of this,
I have extended checkbox to serve up manual tests to testers to
test ubuntu applications post installation. We need your help!
If your an application developer who wants testing on his
application I would like your testcases included in the checkbox
application tests for beta1.*


Hey,



Thanks for that, it's always great to have people looking at
improving the desktop ;-)



I've some question though:



- do you have any idea of what applications you would like to see
tested?

- who will run those tests and when?

- who will deal with the feedback, when and in which way?



Having things tested is great but I think we should figure how we
deal with the feedback before starting doing lot of testing this
way.



I've been working a bit with unity-checkbox to help Didier in the
previous unity update round, and dealing with the infos collected
is quite some work. It's useful for unity where we are upstream
and have resources to deal with the issues raised, I'm less sure
we can do an useful job of it on the application with our current
structure and "workforce"... we don't have the people to do
upstream work and to be fair we already know about quite some
issues that we should fix and didn't yet through bug report.



Could you picture how you would see the feedback loop work? Would
the QA team read those reports and turn issues in bugs for those
which are not already known? Or...?



Cheers,

Sebastien Bacher











--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Looking for application testcases
 

Thomas, this list is for the desktop team. The ubuntu QA team is
involved with QA activities inside ubuntu. See more about what we do
and who we are here:

https://wiki.ubuntu.com/QATeam/



Hope that helps,



Nicholas



On 02/25/2012 10:37 AM, Thomas Prost wrote:

Am Freitag, den 24.02.2012, 11:24 +0100 schrieb Sebastien Bacher:


Le 24/02/2012 01:33, Nicholas Skaggs a écrit :


As part of the precise cycle, the ubuntu QA team has been looking to
increase manual application testing. As part of this, I have
extended




Sorry, but: Who or what is the ubuntu QA team ?
What kind of people are here else ? Users ? Developers ?
What kind of discussion takes place here ?

I´m new here and BTW: Is my subscription correct ?

Even a look at the archives didn´t help dissolve my confusion :-(
Regards from






--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop

Looking for application testcases
 

Am Montag, den 27.02.2012, 13:26 -0500 schrieb Nicholas Skaggs:
> Thomas, this list is for the desktop team. The ubuntu QA team is
> involved with QA activities inside ubuntu. See more about what we do
> and who we are here: https://wiki.ubuntu.com/QATeam/
>
> Hope that helps,
... helped me see, what the QA-Team does :-)

Didn't help about the DesktopTeam.
Maybe it's too busy to provide infos to curious people like me ?
--
Peace,
Nosy Parker


--
ubuntu-desktop mailing list
ubuntu-desktop@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-desktop