Monthly Archives: February 2011

FUDCon Tempe trip report

I arrived in Phoenix, AZ on Friday afternoon after an uneventful flight.  Which, when it comes to flying, is a good thing.  At the airport, I came to the realization that not only is Adam Williamson always online, but he’s also there whenever you go to pick-up your bags.  Which was convenient, since we were able to catch-up over the shuttle to the hotel and lunch.  Later in the day, I met up with Lucas Rodrigues (upstream maintainer for autotest).  We were trying to get Lucas to FUDCon, and it didn’t work out until the last-minute.  I’m glad he was able to make the long trip, turns out … it was instrumental to some hacking.

Day#1 – Barcamp

Catching up with fenzi …

During the first time slot, I spent some time catching up with familiar nicks/faces.  I spoke with Kevin Fenzi, who is always extremely helpful+approachable on irc and in-person, on the current state of the AutoQA project.  We discussed some of the challenges we’ve had soliciting contributors.  The consensus was that once we have tests+results that directly impact maintainers, we can expect more participation.  It’s been a long time since the last release of autoqa, but once depcheck and upgradepath start posting results to bodhi, I’m sure maintainers will get excited (or at least agitated).

What gives mockbuild?

One test idea that Kevin suggested was to write a new rpmlint test to ensure that no files in a package are owned by the user: mockbuild.  You’ve probably seen such errors when installing a package.

warning: user mockbuild does not exist - using root
warning: user mockbuild does not exist - using root
warning: user mockbuild does not exist - using root

It looks sloppy, and it’s something we should be able to detect and prevent.

AutoQA BarCamp talk

Next, I hosted an AutoQA talk.  I prepared some slides to provide talking points, however I was hoping for a more interactive discussion.  I think the talk went well.  Both Chris Lumens and Will Woods were in attendance and were able to provide more in-depth details on the installer unittests and depcheck test that will be landing in the next release (autoqa-0.4.4).

Several good test ideas surfaced, including Dave Malcolm’s ideas on artificial ignorance.  Dan Walsh asked if we could detect whenever dependencies might unintentionally change, and pull in undesired packages.  For example, accidentally including xorg-x11-server-Xorg on a Server SIG image, or increasing the size of Live images beyond the desired 700 Mib (or 1Gib).  Andrew Overholt discussed an eclipse test suite that he’d like run for him after koji builds.  It’s a fairly comprehensive test suite, but early discussions seem like it shouldn’t be too much trouble to wrap and run using AutoQA. .  Finally, some of the installer folks were asking about resultsdb, which is planned for a future autoqa release.  Hopefully more on that later.

Other sessions…

After the AutoQA talk, I attended the Cloud: The future of computing talk by Mike McGrath.  Good talk, with an active demonstration and screenshots.  We have some preliminary support in a branch of AutoQA for provisioning virtual test systems on the fly, so it’s neat to review other ways of solving this problem.  We’re always interested in any off-the-shelf solutions we can integrate with to dynamically allocate and connect to virtual test systems.  Obviously, for tests that need bare metal hardware … this wouldn’t work.  However, a large portion of our tests don’t have this requirement and work just as nice on virtual systems.

Next, I attended the Meet the Anaconda team talk by David Cantrell.  In Fedora QA, we work with them pretty closely during installation validation, so it was really nice to get to know them a bit better.  Throughout the talk, each member of the team presented their current projects.  There is a lot of exciting work planned for the next 3-4 Fedora releases, and the good news is that the team documented their ideas on the wiki.

After a few diversions, I attended Using the SELinux Sandbox by Dan Walsh.  SELinux sandbox allows administrators to take untrusted content, run it through one or more SELinux filters, and have confidence that the content won’t damage to their system/files.  Since Dan’s initial blog post on the subject, a lot has changed.  Dan demonstrated running an entire desktop session in a sandbox, including firefox, openoffice, evince and other desktop applications.  Pretty cool stuff.

Day#2 – More Barcamp, and some hackfest

UI Mockups…

I’ve always been slow when it comes to doing mock-ups.  The process usually involves the gimp (not the one that’s sleeping) and copy+pasting from screenshots to assemble something that looks somewhat decent.  It takes me a long time to make anything reasonable, which means … I don’t do it often.  Of course, this means it takes even longer the next time I try.  Designing UI mockups in Inkscape by Máirín Duffy was really helpful to demonstrate an easier way to rapidly generate UI mock-ups.  I don’t think this was the first time she gave this talk, but it was the first time I attended.  Great demos and tons of examples.  I’ll start with inkscape next time.  And remember, d saves time!

I skipped the next session, and instead spent time catching up with fellow home brewer Chris Lumens.  His keg has a leak, and this must be stopped.

What’s next for Fedora Engineering?

The last talk of the day was The Next Big Fedora Engineering Project by spot.  Similar in structure to the anaconda team talk, members of Tom’s group presented ideas for future engineering projects.  The good news, was that all the ideas were interesting, and they all seemed to have wide support (by a show of hands in the room).  The bad news, was that all the ideas were interesting and seemed to have wide support.  Two that stood out to me were the Fedora message bus and several discussions on how to identify and present contributor metrics.  The most notable idea, was the Fedora RPG.  I’m sure many different groups have this challenge, but in QA, we have no shortage of activities that generate data.  It’s impossible to manual monitor all the data points (bugs, TRAC tickets, bodhi feedback, mailing list, IRC and wiki contributions) so you can  give kudos where deserved.  The RPG concept offers a new twist on this metrics challenge.  I’d love to take advantage of this for QA so we have better insight on community contributions.

Let the hacking begin …

Lucas and I cornered spot for some ideas on how we can resolve the packaging challenges in autotest.  The current autotest package installs all content into the directory /usr/share/autotest.  This really works well for the autotest scheduler since it rsync’s /usr/share/autotest/client to all test client systems.  The directory has all it needs to run jobs and report results and works well across all distributions.  Unfortunately, it’s not likely we’ll get an exception for this format during Fedora package review.  Lucas began the difficult work of adjusting autotest python and configuration files so it would function with content in FHS friendly locations (e.g. /etc, /var,  site-packages etc…).  I worked on the easier task of updating our existing autotest.spec to accommodate the new layout.  We made a good bit of progress, but there is still a lot of work remaining to ensure that autotest will continue operating.

Day#3 – Hackfest

More autotest please…

After settling in, Lucas and I picked up right where we left off.  Autotest relies on an older version of Django that is no longer in Fedora.  Upstream plans to update to the latest Django release, but that’s not on the short-term horizon.  In the meantime, I package Django-1.0.4 privately.  However, if we want autotest accepted into Fedora, I need to formally package this older Django in a manner where it can co-exist with the current Django.  Toshio and Luke shared ideas they’ve used for packaging compat versions of CherryPy.  Those tricks were just what the doctor ordered.

I learned about using setuptools (instead of distutils) to build and install the egg.  The egg installation doesn’t interfere with the regular Django package, and allows callers to use the compat version by specifying:

__requires__ = 'Django==1.0.4'
import pkg_resources

Compat-Django-1.0.4 is packaged and about ready for review, my remaining task is to review the current Django security advisories and patch as necessary.  Thanks to Steve Milner for the security guidance.

GNOME Shell discussion

I had a quick talk with Chris Aillon, J5 and Adam Williamson regarding test ideas for the upcoming GNOME Shell Test Days.  This was a really good conversation, and one that can only take place in person.  Always amazing how much you can cover in person, as opposed to IRC or email.  For more details, read Adam’s blog post on the subject.

Back to autotest…

Throughout the day, Lucas continued hacking on autotest.  Like I said, this is a large change to the autotest code-base and will take some time to get right.  I helped where I could with small patches and brainstorming, but Lucas ran the show.  His changes are available for testing in github.  I plan to revisit and test his branch once F15Alpha is behind us.  Once again, this would be nearly impossible to do over IRC/email.  Or at least, we wouldn’t have made this much progress in such a short time.  Kudos to Lucas for his work!

Nitrate

I posted earlier about my plans for FUDCon Tempe.  I planned to talk with Samuel Greenfeld to understand his test case management system (TCMS) needs and use cases.  Time was running short on Monday, so we only managed to talk briefly.  Samuel has a good grasp on what other open-source TCMS offerings are available, and the benefits and drawbacks to each.  We talked about the current status of nitrate, and Hurry’s plans to host a demo instance so we can further explore modelling test plans and gathering data.  If anyone is interested in helping, please ask questions on the nitrate-devel list and take a look at the TCMS requirements Hurry is tracking.

I had a fun and productive time at FUDCon Tempe.  Too often, I under-estimate the value of face-to-face interaction with people I interact with on a regular basis.  You know, the whole networking thing.  Of course, when you call it that, I don’t want to do it.  Anyway, with FUDCon Tempe behind me, it’s time to get Fedora 15 Alpha out the door.

Advertisements

Fedora 15 Alpha release candidate in sight

If you’ve been following the Fedora 15 schedule, you probably are aware that the Alpha is looming.  Getting the Alpha to line-up with the release criteria has always been a challenge.  There is only so much you can cram into a six month release schedule.  The QA team starts testing candidate builds before F15 has been branched from rawhide.  In addition, due to the gcc-4.6 feature, all packages must be rebuilt.  Testing a moving target is … difficult.

In addition to daily rawhide/branched updates testing, we attempt to reduce the risk with several small early test runs (called rawhide acceptance test), a simulated release candidate ISO drop (called the test compose), and we pepper the weeks leading up to a release candidate testing (F-15-Alpha.TC1), and blocker bug reviews.  The first review was held last Friday, and another is planned for this Friday.

F15Alpha blocker bugs

Proposing a blocker bug is easy.  Simply add the text F15Alpha to the Blocks field of any bugzilla.redhat.com bug. It’s also helpful to provide some impact statement as to what users or use cases are hindered by the bug. Often, we cut and paste from the release criteria to keep things simple.  You can find the full details on the wiki.  If you aren’t sure … send a message to test@lists.fedoraproject.org with details.

The F15Alpha blocker list is very manageable at the moment.  However, we have another test compose (TC2), and release candidate (RC) yet to test.  Fedora contributors perform miracles on a regular basis, so I wouldn’t be surprised if we manage to line up all the moving targets in time. If you’d like to help get Fedora 15 Alpha released on time (QA sign-off is February 23), please contribute test feedback or keep approved blocker bugs updated and accurate.

The current approved, but unresolved, blocker bugs are listed below for convenience …

  • 676485 (distribution) – repoclosure fails on 15-Alpha.TC1
  • 674978 (gdm) – GDM in autologon error loop
  • 676904 (livecd-tools) – osmin.img is sometimes not a valid squashfs on livecd images
  • 672265 (livecd-tools) – “Install to harddrive” fails with “not a live image”
  • 670379 (firstboot) – Unknown lvalue ‘ValidNoProcess’ in section ‘Service’.
  • 579838 (binutils) – glibc not compatible with AMD Geode LX

Image by tanakawho via flickr used under a Creative Commons license