Fedora package maintainers, want test results?

For a some time now, automated tests have been running against Fedora package builds thanks to the AutoQA post-koji-build watcher.  The current tests are by no means exhaustive, but they do provide some level of automated feedback for those interested in a quick sanity check of their package builds.  As a proof-of-concept, and before the spiffy results database is online, all test results are sent to the autoqa-results mailing list.

A few weeks back, Seth Vidal proposed several changes to allow package maintainers to sign-up for test results by mail.  We all get a lot of email as it is, so this certainly isn’t the long-term option.  However, it is an option for package maintainers who want test results emailed directly to them for review.

So, if you want to sign-up for email notification of test results (includes rpmlint, rpmguard and, if applicable, initscripts) whenever you build new packages:

1. Using your FAS credentials, login to fedorapeople.org  (for assistance, see access instructions)
2. Sign-up by running the command  autoqa-optin with appropriate options.  For example, to enable test emails for pykickstart for all rawhide and Fedora 14 builds, you would run:

   # autoqa-optin pykickstart devel F-14
   Opted into autoqa results for pykickstart of devel release
   Opted into autoqa results for pykickstart of F-14 release

To query which releases are configured for test result notification, you can run the autoqa-optin script with just a package name.  For example:

# autoqa-optin yum
yum opted in for releases:
 devel
 F-12
 F-13
 EL-6
 F-11
 F-14

Available Releases to be opted into:
 EL-4
 EL-5

Enjoy!

AutoQA help wanted — initscript verification

In Fedora QA, we’ve been looking to simplify integration of existing tests into AutoQA.  With thanks to Petr Muller, a commonly used test library called beakerlib is now available in Fedora.  Beakerlib provides a nice API to make test development simpler for testers.  If you are familiar with other test frameworks (e.g. unittest, nose, junit, xunit etc…), you should feel right at home.  Since it’s no fun to have a test library without tests, folks from the Red Hat Enterprise Linux QA and Fedora QA teams have been integrating tests designed to automate aspects of the fedora package update acceptance test plan.

One batch of tests are designed to validate LSB compliance for SysV initscripts.  As a proof-of-concept, Josef Skladanka committed several tests for a subset of initscripts.  The good news, as new packages were built the tests behaved properly (see results for samba and openssh).  The bad news, we need more tests!  Josef answered the call and committed another batch of initscript tests.

While discussing the effort, we both felt it was a good time to open this up to fellow contributors.  The idea is simple.  We need help making sure the recent initscript tests behave as expected.  One just needs to …

1. install the appropriate packages
2. run the test
3. report your findings

Pretty easy, right?  I know, famous last words.  We’re hoping to offer more events of this style in the future.  The hope, of course, is that by validating, or contributing, automated tests, you are helping to increase package acceptance test coverage and expose more people to the possibilities using AutoQA.

If you’d like to help, the procedure is documented at https://fedorahosted.org/autoqa/wiki/initscripts.