Expanding wiki templates inside <pre>

I tend to do a fair amount of wiki work these days. Whether it’s scripting against the mediawiki api, or just manually editing pages … you tend pick up little tips here and there. One such gem I’ve been using for a while is to get mediawiki templates to expand when used inside preformatted <pre> tags.

Why bother? Well, when writing test cases, or related pages, I frequently provide command-line samples or output. I like to ensure that the output remains relevant over time. Even more, I dislike having to frequently update a page release after release to ensure the commands or output are still relevant. For some common QA-related <pre> tag examples, checkout the following links …

Okay, you get the idea. The fun part is hidden in the first example. What if you want to take advantage of some awesome wiki templates, such as {{FedoraVersion}}? If you try that … the <pre> tags will show you just that … pre-formatted text as shown below.

mock -r fedora-{{FedoraVersion||previous}}-x86_64 --init 

If you like puzzles, and are hell-bent on having templates expansion inside <pre> tags, let me save you a tremendous waste of time searching for the correct solution. … To expand mediawiki templates inside <pre> tags … you can use a miscellaneous parser function. The function is a bit strange, but works as follows …

{{#tag:pre| 
mock -r fedora-{{FedoraVersion||previous}}-x86_64 --init 
}}

The above syntax will produce the following wiki output …

mock -r fedora-14-x86_64 --init 

All that to emit two characters! I know, but I’m lazy and this solution helps me to never update that particular wiki page ever again. It will always point to the proper release thanks to Template:FedoraVersion. Also, thanks to a post from Das Gurke on the Customizing-MediaWiki mwusers forum for pointing out the right answer.

If you have other creative uses for the #tag:tagname parser function, please share!

Fedora 15 QA retrospective

It’s that time again.  Time to review, discuss and learn from any hiccups encountered while testing the previous release.  No matter how successful a release is, there is always something to improve on.  Fedora 15 was no exception.

First, some ground rules … we’re not trying to solve the world problems with this retrospective.  It’s focused on Fedora QA, our interactions with other teams, and what we can improve on to better accomplish our goals.  For a more general release retrospective, Jared already has an app for that.  Next, this isn’t the place to point fingers and taunt other teams/people.  Use your own blog for that :).  Let’s keep the retrospective as objective and actionable as possible.  For example, when recording issues, it’s important to know the bug(s) involved and what factors contributed to the problem(s).  I’m not interested in what Steve thinks Mary said that Jillian heard after passing out at 31 flavors last night.

Next, a bit of history.  The process starts early each release when I draft a retrospective wiki page to track issues as they happen.  I break the page down into three groups; 1) what worked (sometimes we get things right!), 2) what didn’t work (learn from our mistakes etc…), 3) wishlist (aka pony).  It’s just my nature to want to know what went wrong, and try to fix it.  Accordingly, most of the content focuses on problems we encounter during the release.  Also, I wanted to have some concept of scope … which is where the wishlist/pony group comes in.  I still don’t know if it really fits, but I find it interesting to see what comments people have without constraints.

The retrospective concept doesn’t really work if only one person provides feedback.  So, I try to get the word out so anyone can add issues.  This is an area where I could certainly do better, probably by blogging more about it.  As the release progresses, I beg/plead/nag people closest to the issues to add some notes to the wiki.  I’m sure most people get tired of my nagging for content, but I’ve seen too often where we forget the causes of a critical problem, or don’t properly record the conditions leading up to a failure.  Without accurate data on the cause of problems, any corrective measures are just busy work.  So, the release chugs along, and hopefully the list of good/bad/wishlist issues matures.

mental note … I need to document this in a SOP

Okay … so all that preamble just to let folks know I completed a draft of recommendations based on the Fedora 15 QA retrospective feedback.  I’ll be sending this out to the test@ list for review shortly.  Check it out, and tell me what you think.  See something missing, don’t agree with something … just let me know.

Image by Camilo via flickr used under a Creative Commons license.

 

{{Key_press}}

If you haven’t heard me say it before … “I really like media wiki.”  It’s tremendously flexible, clearly very well-used and maintained, and as there is an abundance of useful content on mediawiki.org. If you are trying to do something on the wiki, chances are someone has already done it.

For example, displaying test results on the wiki (refer to {{Result}}).  While this was started by Will Woods, it later adapted based on experiences with templating on OLPC’s wiki.  Also adapted from OLPC’s wiki is our use of {{QA/Test_Case}}.  Finally, as Athmane recently pointed out, I borrowed {{!}} and {{=}} from mediawiki so you can make use of those characters when using a template.  Who cares?  Well, if you’ve ever attempted to write an admonition (e.g. {{admon/warning}}), you may have noticed it doesn’t format properly when you include certain characters (notably = and | which have special meaning in mediawiki template syntax).

Without “escaping” the special characters, your template will render improperly as seen below …

{{admon/warning|Warning!|Sometimes a=b and b=c. 
In that case, a=c. :-| }}
{{admon/warning|Warning!|Sometimes a=b and b=c.  In that case, a=c. :-| }}

This doesn't look correct.

This can be corrected by using the {{!}} and {{=}} templates, as shown below.

{{admon/warning|Warning!|Sometimes a{{=}}b and b{{=}}c. 
In that case, a{{=}}c. :-{{!}} }}
{{admon/warning|Warning!|Sometimes a{{=}}b and b{{=}}c. In that case, a{{=}}c. :-{{!}} }}

Much better!

Today I found another really helpful series of templates for something I’ve wanted for a long time.  In Fedora QA, we write and maintain a lot of test cases on the wiki.  Often in a test case, one describes a series of steps, or procedure, to perform a particular function in a specific manner.  We extensively use <pre>, lists (both forms # and <ol>) and helper templates such as {{filename}}, {{package}} and {{command}}.

It’s always bugged me that we didn’t have a consistent, and visually striking manner for representing keyboard events.  For example, telling a tester to … “Hit <code>Ctrl-Alt-Backspace to stop Xorg” ([1]) just doesn’t stand-out to me enough.  What can I say … I’m a visual person.  I stumbled upon an awesome series of templates from mediawiki.org for displaying key combinations.  Enter … {{Key_press}}.  With the help of many smaller templates, I’ve added this support, along with the rock solid “upstream” documentation, to the fedoraproject.org Wiki.  I’m not sure who to thank for the template, as there are a significant number of editors.

So instead of something like  …

# Activate the overview by pressing the Windows key,
 pressing <code>Alt-F1</code>, or by moving the mouse
 to the top-left corner of the screen.

One can write …

# Activate the overview by pressing the 
 {{key press|Win}} key, pressing {{key press|Alt|F1}},
 or by moving the mouse to the top-left corner of the
 screen

Which results in something my brain responds to much better…

Much better!

From my docbook XML days, I appreciate structured ways to write content and separation between content and the look’n’feel.  However, I dislike editing content in XML.  Templates like {{command}}, {{filename}} and now {{Key_press}} offer a ways to accomplish the same thing with mediawiki.  I plan to use this more when documenting keyboard events in future test cases.  Hopefully, someone else finds this useful as well.

PreUpgrade Test Day this Thursday (2011-03-17)

Upgrade your system...

Have you seen this dialog?  It’s from a pretty handy utility called preupgrade.  Preupgrade takes some of the complexity out of upgrading your Fedora system.  But like any software, to keep running smoothly, it needs testing and maintenance.

For the past several releases, Fedora QA has hosted several test days (F14 and F13) to shake out bugs related to system upgrades using the preupgrade tool.  Those same tests are also included in our regular release validation matrix.  I’d like to think that regular, organized testing of preupgrade (and installer upgrades) contributes to a more reliable upgrade experience once Fedora GA’s.  Thanks to Hurry for helping to organize these events. :)

Testing involves performing system upgrades using the latest preupgrade package.  There are test instructions (including test cases) on the wiki pageFedora QA and Richard Hughes (preupgrade maintainer) will be in the IRC channel #fedora-test-day on Freenode all day to help with testing and answer any questions you might have.  If you’re not familiar with IRC, a web-based (WebIRC) client is available.

To participate, you’ll need a Fedora 14, or Fedora 13 system ready for upgrade.  Test systems can be bare metal or virtualized guests, it doesn’t matter.  If you don’t have a previous release handy, don’t worry … live images for previous releases are a great way to quickly install a working desktop.

Unfortunately, preupgrade doesn’t test itself, so we need your help.  This is a good chance to get a head start on Fedora 15 Beta testing, and help identify preupgrade-related bugs early.    See you on Thursday!

FUDCon Tempe trip report

I arrived in Phoenix, AZ on Friday afternoon after an uneventful flight.  Which, when it comes to flying, is a good thing.  At the airport, I came to the realization that not only is Adam Williamson always online, but he’s also there whenever you go to pick-up your bags.  Which was convenient, since we were able to catch-up over the shuttle to the hotel and lunch.  Later in the day, I met up with Lucas Rodrigues (upstream maintainer for autotest).  We were trying to get Lucas to FUDCon, and it didn’t work out until the last-minute.  I’m glad he was able to make the long trip, turns out … it was instrumental to some hacking.

Day#1 – Barcamp

Catching up with fenzi …

During the first time slot, I spent some time catching up with familiar nicks/faces.  I spoke with Kevin Fenzi, who is always extremely helpful+approachable on irc and in-person, on the current state of the AutoQA project.  We discussed some of the challenges we’ve had soliciting contributors.  The consensus was that once we have tests+results that directly impact maintainers, we can expect more participation.  It’s been a long time since the last release of autoqa, but once depcheck and upgradepath start posting results to bodhi, I’m sure maintainers will get excited (or at least agitated).

What gives mockbuild?

One test idea that Kevin suggested was to write a new rpmlint test to ensure that no files in a package are owned by the user: mockbuild.  You’ve probably seen such errors when installing a package.

warning: user mockbuild does not exist - using root
warning: user mockbuild does not exist - using root
warning: user mockbuild does not exist - using root

It looks sloppy, and it’s something we should be able to detect and prevent.

AutoQA BarCamp talk

Next, I hosted an AutoQA talk.  I prepared some slides to provide talking points, however I was hoping for a more interactive discussion.  I think the talk went well.  Both Chris Lumens and Will Woods were in attendance and were able to provide more in-depth details on the installer unittests and depcheck test that will be landing in the next release (autoqa-0.4.4).

Several good test ideas surfaced, including Dave Malcolm’s ideas on artificial ignorance.  Dan Walsh asked if we could detect whenever dependencies might unintentionally change, and pull in undesired packages.  For example, accidentally including xorg-x11-server-Xorg on a Server SIG image, or increasing the size of Live images beyond the desired 700 Mib (or 1Gib).  Andrew Overholt discussed an eclipse test suite that he’d like run for him after koji builds.  It’s a fairly comprehensive test suite, but early discussions seem like it shouldn’t be too much trouble to wrap and run using AutoQA. .  Finally, some of the installer folks were asking about resultsdb, which is planned for a future autoqa release.  Hopefully more on that later.

Other sessions…

After the AutoQA talk, I attended the Cloud: The future of computing talk by Mike McGrath.  Good talk, with an active demonstration and screenshots.  We have some preliminary support in a branch of AutoQA for provisioning virtual test systems on the fly, so it’s neat to review other ways of solving this problem.  We’re always interested in any off-the-shelf solutions we can integrate with to dynamically allocate and connect to virtual test systems.  Obviously, for tests that need bare metal hardware … this wouldn’t work.  However, a large portion of our tests don’t have this requirement and work just as nice on virtual systems.

Next, I attended the Meet the Anaconda team talk by David Cantrell.  In Fedora QA, we work with them pretty closely during installation validation, so it was really nice to get to know them a bit better.  Throughout the talk, each member of the team presented their current projects.  There is a lot of exciting work planned for the next 3-4 Fedora releases, and the good news is that the team documented their ideas on the wiki.

After a few diversions, I attended Using the SELinux Sandbox by Dan Walsh.  SELinux sandbox allows administrators to take untrusted content, run it through one or more SELinux filters, and have confidence that the content won’t damage to their system/files.  Since Dan’s initial blog post on the subject, a lot has changed.  Dan demonstrated running an entire desktop session in a sandbox, including firefox, openoffice, evince and other desktop applications.  Pretty cool stuff.

Day#2 – More Barcamp, and some hackfest

UI Mockups…

I’ve always been slow when it comes to doing mock-ups.  The process usually involves the gimp (not the one that’s sleeping) and copy+pasting from screenshots to assemble something that looks somewhat decent.  It takes me a long time to make anything reasonable, which means … I don’t do it often.  Of course, this means it takes even longer the next time I try.  Designing UI mockups in Inkscape by Máirín Duffy was really helpful to demonstrate an easier way to rapidly generate UI mock-ups.  I don’t think this was the first time she gave this talk, but it was the first time I attended.  Great demos and tons of examples.  I’ll start with inkscape next time.  And remember, d saves time!

I skipped the next session, and instead spent time catching up with fellow home brewer Chris Lumens.  His keg has a leak, and this must be stopped.

What’s next for Fedora Engineering?

The last talk of the day was The Next Big Fedora Engineering Project by spot.  Similar in structure to the anaconda team talk, members of Tom’s group presented ideas for future engineering projects.  The good news, was that all the ideas were interesting, and they all seemed to have wide support (by a show of hands in the room).  The bad news, was that all the ideas were interesting and seemed to have wide support.  Two that stood out to me were the Fedora message bus and several discussions on how to identify and present contributor metrics.  The most notable idea, was the Fedora RPG.  I’m sure many different groups have this challenge, but in QA, we have no shortage of activities that generate data.  It’s impossible to manual monitor all the data points (bugs, TRAC tickets, bodhi feedback, mailing list, IRC and wiki contributions) so you can  give kudos where deserved.  The RPG concept offers a new twist on this metrics challenge.  I’d love to take advantage of this for QA so we have better insight on community contributions.

Let the hacking begin …

Lucas and I cornered spot for some ideas on how we can resolve the packaging challenges in autotest.  The current autotest package installs all content into the directory /usr/share/autotest.  This really works well for the autotest scheduler since it rsync’s /usr/share/autotest/client to all test client systems.  The directory has all it needs to run jobs and report results and works well across all distributions.  Unfortunately, it’s not likely we’ll get an exception for this format during Fedora package review.  Lucas began the difficult work of adjusting autotest python and configuration files so it would function with content in FHS friendly locations (e.g. /etc, /var,  site-packages etc…).  I worked on the easier task of updating our existing autotest.spec to accommodate the new layout.  We made a good bit of progress, but there is still a lot of work remaining to ensure that autotest will continue operating.

Day#3 – Hackfest

More autotest please…

After settling in, Lucas and I picked up right where we left off.  Autotest relies on an older version of Django that is no longer in Fedora.  Upstream plans to update to the latest Django release, but that’s not on the short-term horizon.  In the meantime, I package Django-1.0.4 privately.  However, if we want autotest accepted into Fedora, I need to formally package this older Django in a manner where it can co-exist with the current Django.  Toshio and Luke shared ideas they’ve used for packaging compat versions of CherryPy.  Those tricks were just what the doctor ordered.

I learned about using setuptools (instead of distutils) to build and install the egg.  The egg installation doesn’t interfere with the regular Django package, and allows callers to use the compat version by specifying:

__requires__ = 'Django==1.0.4'
import pkg_resources

Compat-Django-1.0.4 is packaged and about ready for review, my remaining task is to review the current Django security advisories and patch as necessary.  Thanks to Steve Milner for the security guidance.

GNOME Shell discussion

I had a quick talk with Chris Aillon, J5 and Adam Williamson regarding test ideas for the upcoming GNOME Shell Test Days.  This was a really good conversation, and one that can only take place in person.  Always amazing how much you can cover in person, as opposed to IRC or email.  For more details, read Adam’s blog post on the subject.

Back to autotest…

Throughout the day, Lucas continued hacking on autotest.  Like I said, this is a large change to the autotest code-base and will take some time to get right.  I helped where I could with small patches and brainstorming, but Lucas ran the show.  His changes are available for testing in github.  I plan to revisit and test his branch once F15Alpha is behind us.  Once again, this would be nearly impossible to do over IRC/email.  Or at least, we wouldn’t have made this much progress in such a short time.  Kudos to Lucas for his work!

Nitrate

I posted earlier about my plans for FUDCon Tempe.  I planned to talk with Samuel Greenfeld to understand his test case management system (TCMS) needs and use cases.  Time was running short on Monday, so we only managed to talk briefly.  Samuel has a good grasp on what other open-source TCMS offerings are available, and the benefits and drawbacks to each.  We talked about the current status of nitrate, and Hurry’s plans to host a demo instance so we can further explore modelling test plans and gathering data.  If anyone is interested in helping, please ask questions on the nitrate-devel list and take a look at the TCMS requirements Hurry is tracking.

I had a fun and productive time at FUDCon Tempe.  Too often, I under-estimate the value of face-to-face interaction with people I interact with on a regular basis.  You know, the whole networking thing.  Of course, when you call it that, I don’t want to do it.  Anyway, with FUDCon Tempe behind me, it’s time to get Fedora 15 Alpha out the door.

Fedora 15 Alpha release candidate in sight

If you’ve been following the Fedora 15 schedule, you probably are aware that the Alpha is looming.  Getting the Alpha to line-up with the release criteria has always been a challenge.  There is only so much you can cram into a six month release schedule.  The QA team starts testing candidate builds before F15 has been branched from rawhide.  In addition, due to the gcc-4.6 feature, all packages must be rebuilt.  Testing a moving target is … difficult.

In addition to daily rawhide/branched updates testing, we attempt to reduce the risk with several small early test runs (called rawhide acceptance test), a simulated release candidate ISO drop (called the test compose), and we pepper the weeks leading up to a release candidate testing (F-15-Alpha.TC1), and blocker bug reviews.  The first review was held last Friday, and another is planned for this Friday.

F15Alpha blocker bugs

Proposing a blocker bug is easy.  Simply add the text F15Alpha to the Blocks field of any bugzilla.redhat.com bug. It’s also helpful to provide some impact statement as to what users or use cases are hindered by the bug. Often, we cut and paste from the release criteria to keep things simple.  You can find the full details on the wiki.  If you aren’t sure … send a message to test@lists.fedoraproject.org with details.

The F15Alpha blocker list is very manageable at the moment.  However, we have another test compose (TC2), and release candidate (RC) yet to test.  Fedora contributors perform miracles on a regular basis, so I wouldn’t be surprised if we manage to line up all the moving targets in time. If you’d like to help get Fedora 15 Alpha released on time (QA sign-off is February 23), please contribute test feedback or keep approved blocker bugs updated and accurate.

The current approved, but unresolved, blocker bugs are listed below for convenience …

  • 676485 (distribution) – repoclosure fails on 15-Alpha.TC1
  • 674978 (gdm) – GDM in autologon error loop
  • 676904 (livecd-tools) – osmin.img is sometimes not a valid squashfs on livecd images
  • 672265 (livecd-tools) – “Install to harddrive” fails with “not a live image”
  • 670379 (firstboot) – Unknown lvalue ‘ValidNoProcess’ in section ‘Service’.
  • 579838 (binutils) – glibc not compatible with AMD Geode LX

Image by tanakawho via flickr used under a Creative Commons license

FUDCon Tempe plan

Busy week!  Fedora 15 Test Days started with a fairly visible change to network device names.  A fair amount of autoqa testing and review in preparation for the upcoming release.  And of course … FUDCon Tempe kicks off tomorrow morning.  Leading up to each event, I always experience some anxiety trying to figure out how best to make use of my time, and Red Hat’s dollars.  Like clockwork, that anxiety returned on the weeks leading up to FUDCon Tempe.  I thought I’d share my goals for the event before the fun gets started.  No sense in delaying further, the fun starts in the morning …

  • AutoQA talk – I’m assuming most of the users+developers in attendance have heard about the autoqa project before.  It has occupied many cycles, especially on the lead up to autoqa-0.4.4.  I’d like to pitch a talk to discuss where we are, roadblocks/challenges, what’s coming next, and the shameless solicitation for help
  • Autotest packaging – After an in depth search for reasonable flights, Lucas Rodrigues, upstream autotest maintainer, arrived in Phoenix on Thursday.  With Lucas in town, we’d like to host a hackfest session to help package autotest so it is ready for package review.  The web front-ends in autotest are built using gwt, which has some java build requirements.  I’m no java expert, so one idea is package autotest without the web front-end.  A FUDCon hackfest session seems like a perfect place to work through the issues.
  • Test case management – A few weeks back, Hurry began identifying Fedora QA workflows and use cases in an effort to determine if the nitrate project would be a good fit for our test events.  Samuel Greenfeld (OLPC) provided some feedback on the list and has been experimenting with nitrate as well as other test case management systems.  I’m anxious to talk with Samuel, document his experiences and use cases in the hope we can better understand our own needs.
  • QA – Fedora 15 is heating up.  This week it was the Network Device renaming test day, next week, the first of several GNOME3 test days.  I’d like to sync up with Adam Williamson and offer any help preparing tests for the upcoming test day.  GNOME3 is a huge change from what you might be used to.  It’s going to frighten some folks, it’s already angered others, but hopefully some people will welcome the change.  Either way it’s just that … a big change, and it’s going to need a lot of help filing bugs and identifying any gaps.

Hope to see you at FUDCon Tempe!