FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS Development

 
 
LinkBack Thread Tools
 
Old 04-07-2011, 04:27 PM
Karanbir Singh
 
Default what / how could test reports be published

On 04/07/2011 04:14 PM, Les Mikesell wrote:
> It's not clear what the test should show.

That is quite strange. When one tests for functionality, the assumption
is that the test would go through the mechanics needed to prove that the
functionality exists. Either as conditional 'OK' or as 'Must Fail' as OK.

So I am not sure what you are looking at.

- KB

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2011, 04:28 PM
Karanbir Singh
 
Default what / how could test reports be published

On 04/07/2011 02:06 PM, Athmane Madjoudj wrote:
> I need this to prepare a similar environment for testing.

Ok, lets get a git repo together with whatever is needed to put together
a harness that people can deploy locally to help them with this process.
A fair bit of stuff is there already, lets see how much of that is
usable outside the centos-qa-environ

- KB
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2011, 04:35 PM
Les Mikesell
 
Default what / how could test reports be published

On 4/7/2011 11:27 AM, Karanbir Singh wrote:
> On 04/07/2011 04:14 PM, Les Mikesell wrote:
>> It's not clear what the test should show.
>
> That is quite strange. When one tests for functionality, the assumption
> is that the test would go through the mechanics needed to prove that the
> functionality exists. Either as conditional 'OK' or as 'Must Fail' as OK.
>
> So I am not sure what you are looking at.

What sort of test would SL binaries fail that would keep them from being
accepted in CentOS? Isn't that the kind of problem that has to be
solved to speed up a release?

--
Les Mikesell
lesmikesell@gmail.com


_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2011, 04:40 PM
Jerry Amundson
 
Default what / how could test reports be published

On Thu, Apr 7, 2011 at 11:35 AM, Les Mikesell <lesmikesell@gmail.com> wrote:
> On 4/7/2011 11:27 AM, Karanbir Singh wrote:
>> On 04/07/2011 04:14 PM, Les Mikesell wrote:
>>> It's not clear what the test should show.
>>
>> That is quite strange. When one tests for functionality, the assumption
>> is that the test would go through the mechanics needed to prove that the
>> functionality exists. Either as conditional 'OK' or as 'Must Fail' as OK.
>>
>> So I am not sure what you are looking at.
>
> What sort of test would SL binaries fail that would keep them from being
> accepted in CentOS? *Isn't that the kind of problem that has to be
> solved to speed up a release?

Umm, not applicable.
And, no.

jerry
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2011, 04:41 PM
Karanbir Singh
 
Default what / how could test reports be published

On 04/07/2011 05:35 PM, Les Mikesell wrote:
> What sort of test would SL binaries fail that would keep them from being
> accepted in CentOS? Isn't that the kind of problem that has to be
> solved to speed up a release?

No, the two things are orthogonal. Assurance that a package does what it
says its going to do is something that needs to be addressed - not
directly connected to what the input to the testing harness is or what
comes out from the other end.

- KB
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-08-2011, 12:55 AM
Athmane Madjoudj
 
Default what / how could test reports be published

On 04/07/2011 05:28 PM, Karanbir Singh wrote:
> On 04/07/2011 02:06 PM, Athmane Madjoudj wrote:
>> I need this to prepare a similar environment for testing.
>
> Ok, lets get a git repo together with whatever is needed to put together
> a harness that people can deploy locally to help them with this process.
> A fair bit of stuff is there already, lets see how much of that is
> usable outside the centos-qa-environ
>

Hi Karanbir

Here's my thoughts that I'm trying to implement, let me know if it's OK
so I can start by now (in my spare time):

- Setup a CentOS-based web-server $SERVER for hosting centos yum repo,
kernel+initrd ,kickstarts, test-scripts.

- Write a python script, that:
1. run virt-install with -l http://$SERVER/centos -x
http://$SERVER/kickstars/ks --nographics (serial console)

2. Connect to the system (serial console) download tests and run them
('pexpect' module), the output should be text only.


Regards.

--
Athmane Madjoudj
RHCE
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-08-2011, 01:35 AM
Les Mikesell
 
Default what / how could test reports be published

On 4/7/11 7:55 PM, Athmane Madjoudj wrote:
> On 04/07/2011 05:28 PM, Karanbir Singh wrote:
>> On 04/07/2011 02:06 PM, Athmane Madjoudj wrote:
>>> I need this to prepare a similar environment for testing.
>>
>> Ok, lets get a git repo together with whatever is needed to put together
>> a harness that people can deploy locally to help them with this process.
>> A fair bit of stuff is there already, lets see how much of that is
>> usable outside the centos-qa-environ
>>
>
> Hi Karanbir
>
> Here's my thoughts that I'm trying to implement, let me know if it's OK
> so I can start by now (in my spare time):
>
> - Setup a CentOS-based web-server $SERVER for hosting centos yum repo,
> kernel+initrd ,kickstarts, test-scripts.
>
> - Write a python script, that:
> 1. run virt-install with -l http://$SERVER/centos -x
> http://$SERVER/kickstars/ks --nographics (serial console)
>
> 2. Connect to the system (serial console) download tests and run them
> ('pexpect' module), the output should be text only.

Is this only going to handle one test instance on one machine at a time or
should it be able to deal with a bunch of concurrent changes across a group of
test machines? For the latter, you need a way to collate the responses and tie
them back to the source of the change each runs against. Something like jenkins
(http://jenkins-ci.org/) might be a good place to start and it already has
plugins for most build/test related things you might want to do
(https://wiki.jenkins-ci.org/display/JENKINS/Plugins). It does expect a JVM to
be working on the slaves (but is very platform-agnostic otherwise) so the
bare-metal installs would need to run the script driving the job on a controlled
slave with the real action happening in a VM or another target. You still need
to write the specific tests, but the jenkins framework can automate the work of
distributing them to the targets triggered by changes and collecting the results
for central viewing.

--
Les Mikesell
lesmikesell@gmail.com


_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-08-2011, 09:25 PM
Charlie Brady
 
Default what / how could test reports be published

On Thu, 7 Apr 2011, Karanbir Singh wrote:

> On 04/07/2011 05:35 PM, Les Mikesell wrote:
> > What sort of test would SL binaries fail that would keep them from being
> > accepted in CentOS? Isn't that the kind of problem that has to be
> > solved to speed up a release?
>
> No, the two things are orthogonal. Assurance that a package does what it
> says its going to do is something that needs to be addressed

I don't think so. I think that is upstream's problem. We need assurance
that what is produced is functionally the same as what upstream produces
(except for the very few areas where CentOS functionality differs).

> - not directly connected to what the input to the testing harness is or
> what comes out from the other end.
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-08-2011, 09:58 PM
Les Mikesell
 
Default what / how could test reports be published

On 4/8/2011 4:25 PM, Charlie Brady wrote:
>
> On Thu, 7 Apr 2011, Karanbir Singh wrote:
>
>> On 04/07/2011 05:35 PM, Les Mikesell wrote:
>>> What sort of test would SL binaries fail that would keep them from being
>>> accepted in CentOS? Isn't that the kind of problem that has to be
>>> solved to speed up a release?
>>
>> No, the two things are orthogonal. Assurance that a package does what it
>> says its going to do is something that needs to be addressed
>
> I don't think so. I think that is upstream's problem. We need assurance
> that what is produced is functionally the same as what upstream produces
> (except for the very few areas where CentOS functionality differs).

And more relevant to my question - I thought that the time consuming
part of the process had to do with the binary compatibility checking and
the iterations to make that succeed.

Not that there is anything wrong with a fully automated and publicly
visible build/regression test facility, but if that's not the slow part,
how much can it help?

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-09-2011, 07:24 AM
Marian Marinov
 
Default what / how could test reports be published

On Thursday 07 April 2011 16:07:33 Karanbir Singh wrote:
> Hi David,
>
> On 04/07/2011 02:02 PM, David Hrbáč wrote:
> > I will add this. Karanbir, you can't expect to find a lot of people to
> > contribute testing scripts. Firstly, this is something that not many
> > companies/people are doing, they are just deploying. And secondly, you
> > can't expect positive feedback and mailbox full of scripts after many
> > weeks of "if you don't like it, go aways".
>
> ok, I take your point onboard. However, at this stage its not a case of
> test-scripts, this is mostly orthogonal.
>
> > Thomas is very right, if you want the results, you must take it
> > seriously and keep it as project. It really seem to me like "go and
> > search for RH trademarks" issue again.
>
> That is mostly what I am trying to do here. But based on these comments,
> I feel it would be easier to buyinto once there is something people can
> see. Let me go do that and come back.
>
> In the mean time : the tests scripts are very welcome, keep them coming.
>
> - KB

I want to propose something here. While I'm setting up my servers with
kickstarts, after the kickstart finishes I have a very simple collection of
scripts that tests if everything is fine.

So armed with that experience I propse that we build a system that:

1. Has a big collection of kickstart files
2. Can provide different kickstart file per every request
3. Will know which ks is send to which client
4. Receive status reports from each server

On the servers, after install, in the post part of the ks we can have a simple
download of a test "framework" that will handle the testing.

For example one directory with scripts:
tests/0000_apache.sh
tests/0001_mysql.sh
tests/0002_ftp.py
......
And then you only need to run the tests:
for i in $(find tests -type f -perm 700|sort); do
if ( $i ); then
chmod 600 $i
else
fi
done

This works quite well for me, and it is very easy to add more tests. We can
have different collections of scripts for different ks installs.

I would love to help with something like this.

Best regards,
Marian Marinov
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 

Thread Tools




All times are GMT. The time now is 04:05 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org