FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS Development

 
 
LinkBack Thread Tools
 
Old 04-13-2011, 05:05 PM
Johnny Hughes
 
Default Test-scripts for Dovecot POP3/IMAP

On 04/13/2011 03:07 AM, David Hrbáč wrote:
> Dne 13.4.2011 8:39, Fabian Arrotin napsal(a):
>> Hi,
>> Thanks a lot for your submissions, i'll integrate them in the scripts
>> repository.
>
> Fabian,
> Would you mind opening the process slightly? I mean it'd be helpful
> having the test list to avoid people doing the same thing. Also it would
> really help to codify the return values, etc.
>
>> One simple remark : we launch the tests on a minimal setup (from a
>> minimal ks) so the first thing to do in the script itself is to install
>> the required package, verify that it installs correctly and then test
>> the feature that you want to test.
>
> This is something that should be written on Wiki page, so that people know:
> - what are the current scripts
> - expected return values
> - testing environment and requirements
> - etc.
>
>> I'm working on redesign quickly something that can be used for the 6.0
>> tests (now that 5.6 is out of the door)
>>
>> Fabian
> Thanks,
> DH

Do you have access to the Wiki?

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-14-2011, 08:34 AM
Mister IT Guru
 
Default Test-scripts for Dovecot POP3/IMAP

On Wed, 2011-04-13 at 09:50 +0100, Mister IT Guru wrote:
> On Wed, 2011-04-13 at 08:39 +0200, Fabian Arrotin wrote:
> > On 04/13/2011 07:38 AM, Athmane Madjoudj wrote:
> > > Hello,
> > >
> > > I've attached two test-script for testing IMAP and POP3 services
> > > provided by Dovecot (default IMAP/POP3 server in CentOS/RHEL).
> > >
> > > Best regards.
> > >
> >
> > Hi,
> >
> > Thanks a lot for your submissions, i'll integrate them in the scripts
> > repository.
>
> Sorry for jumping into this thread. I'm making it my personal mission to
> learn as much as I can about CentOS. Please excuse this noob question...
> What is the purpose of these scripts?
>
>
> > One simple remark : we launch the tests on a minimal setup (from a
> > minimal ks) so the first thing to do in the script itself is to install
> > the required package, verify that it installs correctly and then test
> > the feature that you want to test.
>
> The above only further deepens the mystery for me! I want to learn!
>
>
> >
> > I'm working on redesign quickly something that can be used for the 6.0
> > tests (now that 5.6 is out of the door)
>
> Tell me more of these tests!
>
> >
> > Fabian


Hmmm -- I don't mind waiting, I'm trying to hold onto my enthusiasm
here. I can guess, or try and piece together an answer for myself, but
just some inkling that I was noticed, or something.
--
Mister IT Guru
Blog http://solosysad.blogspot.com/
Twitter @misteritguru
Follow me, I follow you - it's only fair

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-14-2011, 09:03 AM
Michael Simpson
 
Default Test-scripts for Dovecot POP3/IMAP

On 14 April 2011 09:34, Mister IT Guru <misteritguru@gmx.com> wrote:
> Hmmm -- I don't mind waiting, I'm trying to hold onto my enthusiasm
> here. I can guess, or try and piece together an answer for myself, but
> just some inkling that I was noticed, or something.
> --
> Mister IT Guru


I wondered the same thing so i channeled my inner "misc" and opened up
the tests in my favourite text editor.
They test for basic functionality
ie create a basic configuration, switch it on and check that it is
working and give an exit status of pass or fail.
The conversation above suggests that it would be gold standard for the
test for each service to be able to successfully install the service
onto a minimal set up as well rather than having separate install and
test scripts.

open them up and have a look

regards

mike
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-14-2011, 10:27 AM
Mister IT Guru
 
Default Test-scripts for Dovecot POP3/IMAP

On Thu, 2011-04-14 at 10:03 +0100, Michael Simpson wrote:
> On 14 April 2011 09:34, Mister IT Guru <misteritguru@gmx.com> wrote:
> > Hmmm -- I don't mind waiting, I'm trying to hold onto my enthusiasm
> > here. I can guess, or try and piece together an answer for myself, but
> > just some inkling that I was noticed, or something.
> > --
> > Mister IT Guru
>
>
> I wondered the same thing so i channeled my inner "misc" and opened up
> the tests in my favourite text editor.

I understand the phrasing of my question assumed that I hadn't done that
- I did the same, and when I noticed the purpose of the scripts, and
what they do, I wondered why is this being "repeated"

> They test for basic functionality
> ie create a basic configuration, switch it on and check that it is
> working and give an exit status of pass or fail.

This I understand - what I would like to know, is this a new thing? Or
is this part of the normal procedure? Are these scripts uploaded to a
SVN repo where we can get to it, and test against our own builds if we
are so inclined? etc etc

> The conversation above suggests that it would be gold standard for the
> test for each service to be able to successfully install the service
> onto a minimal set up as well rather than having separate install and
> test scripts.

If this is standard - then I'm surprised that the distribution mechanism
is the list. (I'm sure I'm wrong, but understand, this is all I can see
as a newbie, or even someone who was just browsing to see how projects
are run) - I just assumed that the devs used automated script deployment
mechanisms to do this kind of thing.

I'd use nagios as a front end, with NRPE on my target machine with a
minimal install, and push the scripts from an SVN repo to the test box,
and then look in nagios to see what the results of the script is. That
way, I know that when I get green across the board - Everything is all
good!


> open them up and have a look

Way ahead of you! hehe!


I can see what the tests do, I am asking why are we doing these tests?

I'm assuming that this is done as part of the mechanism to test that
certain packages are working as expected after install, once they have
passed binary compatibility. (I would assume that if they are binary
compatible, that they will behave the same way as the RHEL packages)
which if this is the case, then why are we testing them again? Which
leads me to think my assumption is wrong - SO .... why are we doing
these tests?

I use nagios to run scripts and get results from environments, along
with other deployment methods to create clean environments to do this
automagically. Does the centos devs do something like this, (I know I
inferred this earlier, bare with me).

Something like this - the writing of the scripts, can that be opened up
a bit? Maybe thrown open to the community? Get them cracking on the
current release, see what comes back - it's just their time and effort,
you just have to ask for it.

Should I be starting a new thread for this by the way, I know how fickle
people can be!


> regards
>
> mike

Thanks for the response Mike!

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-15-2011, 11:17 AM
Johnny Hughes
 
Default Test-scripts for Dovecot POP3/IMAP

On 04/14/2011 05:27 AM, Mister IT Guru wrote:
> On Thu, 2011-04-14 at 10:03 +0100, Michael Simpson wrote:
>> On 14 April 2011 09:34, Mister IT Guru <misteritguru@gmx.com> wrote:
>>> Hmmm -- I don't mind waiting, I'm trying to hold onto my enthusiasm
>>> here. I can guess, or try and piece together an answer for myself, but
>>> just some inkling that I was noticed, or something.
>>> --
>>> Mister IT Guru
>>
>>
>> I wondered the same thing so i channeled my inner "misc" and opened up
>> the tests in my favourite text editor.
>
> I understand the phrasing of my question assumed that I hadn't done that
> - I did the same, and when I noticed the purpose of the scripts, and
> what they do, I wondered why is this being "repeated"
>
>> They test for basic functionality
>> ie create a basic configuration, switch it on and check that it is
>> working and give an exit status of pass or fail.
>
> This I understand - what I would like to know, is this a new thing? Or
> is this part of the normal procedure? Are these scripts uploaded to a
> SVN repo where we can get to it, and test against our own builds if we
> are so inclined? etc etc
>
>> The conversation above suggests that it would be gold standard for the
>> test for each service to be able to successfully install the service
>> onto a minimal set up as well rather than having separate install and
>> test scripts.
>
> If this is standard - then I'm surprised that the distribution mechanism
> is the list. (I'm sure I'm wrong, but understand, this is all I can see
> as a newbie, or even someone who was just browsing to see how projects
> are run) - I just assumed that the devs used automated script deployment
> mechanisms to do this kind of thing.
>
> I'd use nagios as a front end, with NRPE on my target machine with a
> minimal install, and push the scripts from an SVN repo to the test box,
> and then look in nagios to see what the results of the script is. That
> way, I know that when I get green across the board - Everything is all
> good!
>
>
>> open them up and have a look
>
> Way ahead of you! hehe!
>
>
> I can see what the tests do, I am asking why are we doing these tests?
>
> I'm assuming that this is done as part of the mechanism to test that
> certain packages are working as expected after install, once they have
> passed binary compatibility. (I would assume that if they are binary
> compatible, that they will behave the same way as the RHEL packages)
> which if this is the case, then why are we testing them again? Which
> leads me to think my assumption is wrong - SO .... why are we doing
> these tests?
>
> I use nagios to run scripts and get results from environments, along
> with other deployment methods to create clean environments to do this
> automagically. Does the centos devs do something like this, (I know I
> inferred this earlier, bare with me).
>
> Something like this - the writing of the scripts, can that be opened up
> a bit? Maybe thrown open to the community? Get them cracking on the
> current release, see what comes back - it's just their time and effort,
> you just have to ask for it.
>
> Should I be starting a new thread for this by the way, I know how fickle
> people can be!
>

The CentOS Project is building a suite of automated tests that we are
using to help us with QA after we get our tree ready.

This test suite will run on our tree after we get it moved to our QA
servers.

This is being run by people other than Karanbir, Tru, and me ... so it
is already "opened up".

As to how it is being run, that is up to the people who we have "opened
it up to".



_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-15-2011, 12:26 PM
Mister IT Guru
 
Default Test-scripts for Dovecot POP3/IMAP

On Fri, 2011-04-15 at 06:17 -0500, Johnny Hughes wrote:
>
> The CentOS Project is building a suite of automated tests that we are
> using to help us with QA after we get our tree ready.
>
> This test suite will run on our tree after we get it moved to our QA
> servers.
>
> This is being run by people other than Karanbir, Tru, and me ... so it
> is already "opened up".
>
> As to how it is being run, that is up to the people who we have "opened
> it up to".


See this here? This should be documented and on the web in a very
prominent position. This is awesome to know this.

There are thousands of people who use CentOS who build automated testing
suites, using numerous tools and methodologies.

So, so far I've learnt that we use Mock to build the centos packages,
and we actually do have a test suite in development.

Great - Now, we just needs those two steps up, online, and for the
community to see. Hopefully with hundreds more eyes on it, we can grow
even more. Personally, I think I'll start helping you guys with your PR,
because getting info feels like getting blood from a stone. Call me
impatient of whatever you wish, (not aimed at Johnny Huges or anyone
person,) I will continue to pester you guys, and you can hate seeing
Mister IT Guru in your inbox!! But I truly believe CentOS needs to come
into the light - don't want to CentOS to go the debian root (geddit!)

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-16-2011, 05:07 PM
Fabian Arrotin
 
Default Test-scripts for Dovecot POP3/IMAP

On 13/04/11 10:07, David Hrb?? wrote:
> Dne 13.4.2011 8:39, Fabian Arrotin napsal(a):
>> Hi,
>> Thanks a lot for your submissions, i'll integrate them in the scripts
>> repository.
>
> Fabian,
> Would you mind opening the process slightly? I mean it'd be helpful
> having the test list to avoid people doing the same thing. Also it would
> really help to codify the return values, etc.

Hi David,

Sorry for the late answer but due to some illness on my side (and so
being very tired when going back home from $work) i wasn't able to
answer that question. Yes it's really a need to open that part of the
tests so that people can contribute to those tests and also open the QA
process (and also the status) to the community


>
>> One simple remark : we launch the tests on a minimal setup (from a
>> minimal ks) so the first thing to do in the script itself is to install
>> the required package, verify that it installs correctly and then test
>> the feature that you want to test.
>
> This is something that should be written on Wiki page, so that people know:
> - what are the current scripts
> - expected return values
> - testing environment and requirements
> - etc.

Yes, and so first a need to design it before documenting it. Let me
clarify this a little bit : actually we (Karan and myself) designed
quickly something that could be used directly during the 5.6 process (we
did that during Fosdem, between discussions with people at the CentOS
booth ...)
We started 'thin' and so actually it's just a file with different
functions declared and some bash scripts (one per VM/type) sourcing that
first file and using the functions.
Those 'functions' contain CentOS QA hardware informations (like on which
machine to deploy a vm, which type of vm (xen, kvm, virtualbox, etc ...)
disk-size, memory etc ..

So we'd like first to separate maybe those informations from the tests
themselves. The tests scripts would be available in a git repo and we'd
use them during QA and post the results somewhere.

I would like to redesign our QA subnet/vlan a little bit and reflect
that in our existing QA script. (I don't think that this part interests
the Community as it also involves public ip, etc ...)

We'd try to have a small meeting with KB about that part and then a
public discussion (an IRC meeting maybe) for people wanting to
contribute their tests.

Part of the discussion would be also about what to test. For example,
testing if dovecot imap/pop3 is working is good, but my personal
approach would be to test if centos packages for example are also
'centosified'.
Let me explain that with a real example : if we have to change a logo
(and that we use the same logo during the whole distro lifetime) a basic
script would be to retrieve that logo, compare it with the known md5sum
and so guess if the logo was replaced or not correctly. Stupid example,
but believe me it's worth writing such stupid script :-)

Fabian

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 

Thread Tools




All times are GMT. The time now is 12:27 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org