Not too much trouble traveling to FUDCon Toronto. The flight out of Raleigh experienced some mechanical issues. Will Woods and I were transferred to Cleveland for a long layover. That gave us plenty of time to catch up on AutoQA. Since I like lists … some might say too much … so here it goes.
Day#1 – Barcamp
- boot.kernel.org (or killing CD ISOs w/ the network) — see log — Interesting talk on removing the need for media-based installs. Basically, you download a small image and store it on a USB key, tftpserver or optical media (so not really getting rid of the CD’s). When booted, that smaller image boots off the network and offers installation of just about anything (Fedora, CentOS, Ubuntu, Gentoo, etc…). Mike McGrath was keen on implementing this within Fedora infrastructure to provide another easy method for users to get involved and help test.
- Fedora QA: What we do, and how you can help — see log — Great talk by Adam on what the Fedora QA team does, and how to get involved. Some participants were surprised that the blocker bug process was so open. That’s certainly a strength with our current process in that it doesn’t require l33t permissions to engage and add value. A big ‘hooray!‘ went out for Kevin Fenzi .. who helped make it even easier to get involved by providing non-destructive nightly live images of rawhide.
- AMQP Qpid — session not active — I was hoping to catch this session. However, it seems most of the discussion took place during the previous amqp talk. Instead, I headed over to …
- Designing the Future of Free Software Operating System User Experiences – GNOME Shell / 3.0 — see log — Jon McCann and Colin Walters demonstrated the rationale behind the GNOME shell and gave a quick demonstration (see screencasts) along with screenshots. They’re focus on the user experience is interesting.
- AutoQA Beaker Automated Testing — see log — Bill Peck and Will Woods discussed recent improvements and the rationale behind the different solutions. Beaker has come a long way since it first jumped into the fedorahosted project scene. Will talked about the initial design goals for the AutoQA project, as well as some upcoming milestones. Bill discussed how beaker is being used now and testing possibilities for Fedora. This is a big area of growth for the QA team. AutoQA was designed for testing that does not require complicated test environments (either dedicated test systems or virtual guests). As our testing needs mature, beaker will likely be involved in helping transition to more complex test infrastructure (multiple test systems, system provisioning, etc…).
Day#2 – Hackfest
- Fedora release criteria discussion with John Poelstra, Adam Williamson, Bill Nottingham and Tim Burke. Our initial thought was the discussion would simply put a bow on the existing pages and wrap-up with the feedback provided leading up to FUDCon (wiki and mailing lists). However, many hours later, we arrived at something that we can all agree with and build on for future releases. For the most part, the release criteria simply restate what we’ve always done each Fedora release. This time, Adam suggested a clever revamp of the requirements as to not leave too much of a burden on QA for defining policy using our test plans. The current result is something the QA team can execute on for Fedora 13, has the thumbs up from key stake-holders in Desktop, Release Engineering and Development, and leaves room for future growth. I’m pretty jazzed with the results … https://fedoraproject.org/wiki/Fedora_Release_Criteria.
Day#3 – Hackfest
- Discussion around AutoQA Use Cases with John Poelstra I’ve been struggling to make sense of the use cases. My intent was that by completing the use cases, we could highlight gaps in AutoQA documentation or process.. What John suggested was keeping them short, sweet and task focused. I cleaned up the initial ‘Getting started’ cases accordingly. Next step, catch up with AutoQA developer and rpmguard creator, Kamil Paral, to help flesh out more details.
- Spoke with the installer team on their plan for build-time unit tests. David Cantrell and team are working on building unit tests to validate anaconda dependencies at build-time. These are dependencies that normally manifest as install-time failures. This includes libraries such as udev and dbus, binaries like iscsiadm and mdadm, directory locations (/etc/sysconfig/network-scripts) and config files/formats that it depends on. Following a model similar to ‘is rawhide broken?‘, a new AutoQA milestone will likely come from this. The milestone will include a post-git-push watcher to initiate tests whenever anaconda changes are pushed into anaconda.git and another turbogears front end for display and review of test results.
- Happy recipient of guidance from Toshio Kuratomi and Mike McGrath on packaging and deploying autoqa. The previous package was for EL5, after moving to F12 several bugs surfaced related to config files and the reliance on python-cherrypy2 (not the latest cherrypy). With those issues out of the way, next stop … package review process.
- Discussion with Adam Miller on the possibility of using dogtail to automate tests inside a running live image. Adam, Will (and possibly others) discussed how to automate GUI testing on a live image. Some of the challenges involved getting the test code and dependencies installed on the live image … without of course changing the environment under test. One clever idea I heard was to use an overlay partition. The other challenge is our general lack of knowledge around dogtail testing. Dogtail has been around for some time, but it’s unclear in what state upstream is and how robust of a system it is for GUI testing. Hopefully Adam is able to shed some light on that untapped resource.
- Finalizing the Fedora 13 QA schedule with John Poelstra and Denise Dumas (with input from Jesse Keating). I’m happy with the current QA schedule that John has asked for feedback on. It continues the F-12 trend of having 3 test events leading up to each major milestone (Alpha, Beta, Final). Each milestone will be preceded by the following test runs … 1) pre-$MILESTONE rawhide acceptance test run, 2) test-compose test run, and 3) release candidate test run. Additionally, the Alpha and Beta will have additional planned rawhide acceptance test runs. I’m confident that the QA team can
- Discussed techniques for using AutoQA to automate the installation test matrix. At different times during the event, Adam, Will and I outlined different techniques for integrating AutoQA and the install test matrix that Liam maintains. I’ll catch up withLiam and Hurry to bounce some more ideas around. However, I expect we’ll need to get a small group of folks together to collaboratively move things forward so that we have something in place for Fedora 13.
All told, I’m pretty happy with FUDCon Toronto. With one exception, I completed about everything on my todo list. I had hoped to leave FUDCon with a deployed instance of autoqa. However, time was not my friend in that regard. The details of package review are still ahead. Anyone interested in reviewing?