FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Releng

 
 
LinkBack Thread Tools
 
Old 02-01-2008, 07:58 AM
Rémi Cardona
 
Default Checklists for testers and QA

Chris Gianelloni a écrit :

Is it really that nobody cares or is everybody just being lazy and
saying "yeah, that's good enough" when I intentionally didn't list lots
of things that I'd came up with when writing the original list. I
honestly don't know which it is here.


Do we know how many people actually use the software on the live media?
Do they care if it's not 100% functional?


<personal experience> When I install Gentoo, I'll use which ever live
CD/DVD/USB I can find, if not, I'll download the minimal install CD and
the only pieces of software I'll use is dhcpcd and a web browser to read
the handbook. I'll occasionally use screen and ssh{,d} if the box is too
far from my desk.


So once you release a beta, that's what I'll be trying out first and
foremost.


But, wouldn't it be interesting to find out the usage patterns for our
users before making a big list of things to check?


Maybe not enough users do care to make it block QA and releng's time
when making new releases.



/me is confused


Don't be

Cheers,

Rémi
--
gentoo-releng@lists.gentoo.org mailing list
 
Old 02-01-2008, 07:58 AM
Rémi Cardona
 
Default Checklists for testers and QA

Chris Gianelloni a écrit :

Is it really that nobody cares or is everybody just being lazy and
saying "yeah, that's good enough" when I intentionally didn't list lots
of things that I'd came up with when writing the original list. I
honestly don't know which it is here.


Do we know how many people actually use the software on the live media?
Do they care if it's not 100% functional?


<personal experience> When I install Gentoo, I'll use which ever live
CD/DVD/USB I can find, if not, I'll download the minimal install CD and
the only pieces of software I'll use is dhcpcd and a web browser to read
the handbook. I'll occasionally use screen and ssh{,d} if the box is too
far from my desk.


So once you release a beta, that's what I'll be trying out first and
foremost.


But, wouldn't it be interesting to find out the usage patterns for our
users before making a big list of things to check?


Maybe not enough users do care to make it block QA and releng's time
when making new releases.



/me is confused


Don't be

Cheers,

Rémi
--
gentoo-releng@lists.gentoo.org mailing list
 
Old 02-01-2008, 09:18 AM
"Markus Hauschild"
 
Default Checklists for testers and QA

Hi,
I think I've come up with some more questions:
* did you use the graphical installer or did you install manually (and why)
* did everything with the graphical installer work as expected (if
used, and explain what did not work as expected)

--
Cheers, Markus
--
gentoo-releng@lists.gentoo.org mailing list
 
Old 02-01-2008, 09:37 AM
Nelson Batalha
 
Default Checklists for testers and QA

*
-Does your computer run other OS/distributions without the same problems? If so, please list them (optionally along with drivers/modifications you installed/made in it).







For the LiveCD/DVD:




-Did you start all applications? In any case,*are any of them failing? If so, what is reported when you run them on the command line?

*


> Subject: [gentoo-releng] Checklists for testers and QA
> From: wolf31o2@gentoo.org
> To: gentoo-dev-announce@lists.gentoo.org
> CC: gentoo-releng@lists.gentoo.org
> Date: Thu, 24 Jan 2008 16:38:22 -0800
>
> Well, I'm sure all of you are still reading over the summary and meeting
> logs, but when you're done, I'd like to get everyone's involvement in
> helping create the three checklists we plan on distributing. The lists
> are Beta, QA, and Release. For the most part, they will overlap quite
> heavily, but I still see the need for keeping them separate.
>
> In the case of Beta and Release, we'll also want to ask people for
> specific information about their environment (hardware, software, etc.)
> that will help us to troubleshoot the issues and focus more energy on
> the worst problems.
>
> I'll start us off:
>
> - Which media did you download? (Minimal, Universal, LiveCD, LiveDVD)
> - Did the media pass MD5/SHA1 checks? (If you didn't check, say so.)
>
> - Does the media boot on your system?
> - If not, what errors were reported?
> - Did you use any command line options?
> - If so, which ones?
>
> - After boot, were all of your devices detected?
> - If not, what was not detected?
> - What driver does this device use?
> - Is the driver on the media? (Say "don't know" if you're unsure.)
>
> - Did previous releases work where this one is not?
>
> - What chipset is your motherboard? IDE? SATA?
> - Do you use external controllers for SATA/SAS/SCSI/etc.?
> - If so, which ones?
> - How much RAM is in your system?
> - Did you try to boot with "docache" enabled?
> - What video card(s) do you have?
> - Was it properly detected?
> - Did X start properly?
>
> This should be a pretty decent start and give everyone an idea of what
> we would be looking for with these checklists. What things do you think
> need to be checked for and what information do you think we would need
> to properly diagnose any issues?
>
> --
> Chris Gianelloni
> Release Engineering Strategic Lead
> Games Developer
> --
> gentoo-releng@lists.gentoo.org mailing list
>


Express yourself instantly with MSN Messenger! MSN Messenger
 
Old 02-01-2008, 12:53 PM
Pacho Ramos
 
Default Checklists for testers and QA

El jue, 31-01-2008 a las 17:40 -0800, Chris Gianelloni escribió:
> On Thu, 2008-01-24 at 16:38 -0800, Chris Gianelloni wrote:
> > In the case of Beta and Release, we'll also want to ask people for
> > specific information about their environment (hardware, software, etc.)
> > that will help us to troubleshoot the issues and focus more energy on
> > the worst problems.
>
> So nobody has *any* ideas, at all for this?
>
> It's been a week and there hasn't been a *single* response.
>
> This is everyone's chance to give their input on what we should have
> checked on the release media. If you do not speak up now, it isn't
> likely that it will get checked.
>
> --
> Chris Gianelloni
> Release Engineering Strategic Lead
> Games Developer

I don't know what could I say:

1. On my laptop I still use ipw3945 driver as I am unable to make
iwl3945 work with my 3945 card

2. On my desktop I have to stay in nvidia-drivers-100.14.19 because
169.07 and 169.09 makes my monitor be set to unproper resolutions with
fullscreen games

But I dubt gentoo could do something with nvidia problem, and I know
that a lot of people also have problems with iwlwifi driver even with
other distributions :-/



--
gentoo-releng@lists.gentoo.org mailing list
 
Old 02-01-2008, 01:53 PM
"M. Edward (Ed) Borasky"
 
Default Checklists for testers and QA

Rémi Cardona wrote:
Do we know how many people actually use the software on the live media?
Do they care if it's not 100% functional?


I don't know about new users, but I suspect many veteran users use
Gentoo media only in the extremely unlikely event that they've blasted
their system to the point where it's necessary, or when they've acquired
a new box. The other major use case, of course, is giving media to
friends who are unfamiliar with Gentoo.


There have been occasions where I would boot the Gentoo (and others')
LiveDVD on my Windows workstation at my day job and use it as a Linux
workstation, but VMware Server pretty much eliminated that use case.
Corporate IT policy forbids me from running beta software, at least that
of vendors, because of licensing issues, but that may not apply to open
source betas.


But, wouldn't it be interesting to find out the usage patterns for our
users before making a big list of things to check?


Maybe not enough users do care to make it block QA and releng's time
when making new releases.


I can't speak for either QA or releng, but you may have a point. Veteran
users most likely will have the tools and motivation to file accurate
bugs without a checklist, and newbies probably will be looking for
something simpler, like,


a. Does it come up to a desktop, find my network, can I browse the web,
read email and edit a document in OpenOffice?


b. Why is the start button in the upper left corner?

c. Does the wireless work? Does the sound work? Where's the &$^#& virus
scanner?


d. Well, I've had enough of Linux -- did it trash my Windows install, or
can I eject the CD and give it to my 13-year-old gamer cousin and get
back to work?



--
gentoo-releng@lists.gentoo.org mailing list
 
Old 02-01-2008, 03:56 PM
Chris Gianelloni
 
Default Checklists for testers and QA

On Fri, 2008-02-01 at 09:49 +0100, Christian Faulhammer wrote:
> * Did you try a lot of the software on the live CD/DVD? Did they work
> fine?

This is good, but I'd prefer we list specific pieces of software, since
the CD/DVD is also used for networkless installs, the software needs to
work on the CD/DVD for it to work on their machines.

> * Are all links on the desktop working (terminal, Handbook etc.)?

Good one. Thanks!

> * Was splash image displayed correctly?

I'll add this one if we have a splash. Current catalyst/genkernel
doesn't really support the new splash stuff properly, so I'll either
need to hack in the code for it in a very short time, or I'll need to
drop the splash. I'm definitely going to dump it for the minimal CD, if
only to reduce space usage further.

Thanks for the ideas!

--
Chris Gianelloni
Release Engineering Strategic Lead
Games Developer
 
Old 02-01-2008, 04:01 PM
Chris Gianelloni
 
Default Checklists for testers and QA

On Fri, 2008-02-01 at 09:58 +0100, Rémi Cardona wrote:
> Chris Gianelloni a écrit :
> > Is it really that nobody cares or is everybody just being lazy and
> > saying "yeah, that's good enough" when I intentionally didn't list lots
> > of things that I'd came up with when writing the original list. I
> > honestly don't know which it is here.
>
> Do we know how many people actually use the software on the live media?

Yes. Every single person who uses the Installer uses the software on
the disk.

> Do they care if it's not 100% functional?

Yes, since anything non-functional ends up non-functional on their
machines.

Now, could we focus less on whether checking the software is a valid use
of time or not and instead look at *what* we should test within that
software?

> <personal experience> When I install Gentoo, I'll use which ever live
> CD/DVD/USB I can find, if not, I'll download the minimal install CD and
> the only pieces of software I'll use is dhcpcd and a web browser to read
> the handbook. I'll occasionally use screen and ssh{,d} if the box is too
> far from my desk.

We don't really care what you use. I mean no offense by this, but it
doesn't help *this* conversation any.

> So once you release a beta, that's what I'll be trying out first and
> foremost.

Sure. That's the absolute minimal amount of testing that we normally
perform. All we gain is knowing it works on your specific hardware and
not much else, since you're planning on testing the parts we test
extensively.

> But, wouldn't it be interesting to find out the usage patterns for our
> users before making a big list of things to check?

Not really. The quality should be as perfect as possible on everything,
not just the major things. The whole point here is to get community
involvement, which means many more eyes so we can cover much more
ground.

> Maybe not enough users do care to make it block QA and releng's time
> when making new releases.

We *want* to know what to test. That's the whole point. We've only
been testing certain things due to time constraints. If we had a more
comprehensive list, it would make a better release. Plain and simple...

--
Chris Gianelloni
Release Engineering Strategic Lead
Games Developer
 
Old 02-01-2008, 09:46 PM
Vlastimil Babka
 
Default Checklists for testers and QA

Chris Gianelloni wrote:

This should be a pretty decent start and give everyone an idea of what
we would be looking for with these checklists. What things do you think
need to be checked for and what information do you think we would need
to properly diagnose any issues?


Well quite obvious questions but not mentioned in the e-mail:

- what installation method did you use? (manual/installer)
- if manual, did following the instructions exactly work, or were any
extra steps needed or some steps performed differently? any missing
information? also which optional parts you performed, which not (I guess
that might be about doc issues but still)

- if installer, what method (networkless/advanced), what settings
- if installer, did all work or failed, where, post the log?

VB
 

Thread Tools




All times are GMT. The time now is 09:58 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org