Community-driven software QA. My quest for inspiration.
Scene-setter for blogs to come.
I haven’t mentioned since blog #1 that I’m the Test Lead at Symbian. Nine months into the job, it’s sunk in that what it must mainly be about is enabling. Enabling the Symbian community to forge the testing capability it needs.
Probably I’ve been slow to digest this insight because of my prior CV. Until I started this job, for going on 5 years I’d led the team that maintained the Developer Build & Test System (DABS) in Symbian’s corporate precursor, Symbian Software Ltd (call it SSL). DABS was the nightly build and test system shared by the 30 or so software development teams that made up the software engineering force in SSL. In SSL, naturally, it went without saying that we test our software to shippable quality, all by ourselves before anybody else gets a look at it. And DABS bore most of that testing effort.
Now, I can see that I my professional mindset needs reset. So I’ve been looking at Linux distros to see what I – and Symbian – can learn about successful test strategies for a non-profit provider of an open-source personal computing platform; foremost, to learn how these projects are able to harness community resources for their testing operations and the ways in which they do it.
I looked to Linux distributions because they are only sustained examples of successful non-profit providers of open-source personal computing platforms and are exemplary of community powered software production at its most accomplished and ambitious. Symbian by its raison d’être faces the same challenges that face a Linux distro. (But not only the same challenges; a qualification that might be crucial).
I use “non-profit” there in a relaxed sense. I mean that the platform is provided at no charge. All of the most popular Linux distros now – say the distrowatch top 5 – have some sort of corporate backing that turns a profit from professional services and support for the free platform, or by selling and supporting a value-added down-stream spin. Symbian has profit-motivated corporate backing too – our funding members – but our platform is provided at no charge, and nothing is success for Symbian but community-built success.
I’ve studied the web-presenses of leading Linux distros – Debian, Fedora, Mandriva, openSUSE, Ubuntu (impartially alphabetical order here). I’ve peered around the online resources and activities that lie behind. I’ve made some contacts who are in roles like mine, and been welcomed to question what they do, and how and why. (Debian, I must hasten to emphasise, has no corporate stakeholders: it is a pure community project).
I’ve taken away a rich load of perceptions and learning points about community-driven software testing and QA for which I want to get the attention of Symbian, its community and ecosystem. I’ll be blogging on this theme. Not all of my perceptions are of beckoning opportunities for Symbian to catch up. Some of them are of daunting impediments for Symbian catching up.
I’ve also taken away a sense of obligation to reciprocate the openess of the projects I’ve researched. I’m thinking: In the light of everything I’ve learned, what do I know about testing and QA that could be useful to these and other open-source projects? I’ll blog on that theme too.
Back to Syborg on Linux, with GCCE
In my “Syborg” article I made the point that Syborg – the Symbian kernel port that runs on QEMU – is a breakthrough for Symbian development on Linux. Symbian virtual devices with the Syborg kernel, running in the QEMU virtual machine environment, offer the same emulated Symbian development target under both Windows and Linux. The Symbian Emulator of yore was Windows-bound.
I did some experimental work to show that, with fairly superficial hacks, the Syborg kernel could be built, and a Symbian textshell ROM could be built and run on Ubuntu Linux using the code base of Symbian^2.0.0. I wrote up my findings and posted them on the Symbian developer wiki:
How to build the Syborg/QEMU baseport for Symbian^2.0.0 on Ubuntu Linux
How to build Symbian QEMU on Ubuntu Linux
How to build and run a Syborg/QEMU textshell ROM on Ubuntu Linux
Toward Symbian development by open source means
These efforts represent initial moves toward the goal of making Symbian development doable with wholly open source means.
That goal entails making Symbian development doable on Linux. So knowing that it’s already practical to build and run a rudimentary Symbian vm on Linux is an important encouragement.
Those efforts fell short, however, of demonstrating how to build and run a Symbian vm by wholly open source means. I cheated on three points:
- I didn’t have an open source compiler that could build Syborg. I used ARM’s proprietory RVCT compiler v2.2, for which Symbian has licenses.
- I didn’t build the necessary ROM-building tools for Linux. So far, Symbian kits provide only Windows executables for these tools and though they are open source, they haven’t been ported to Linux. I let Ubuntu execute the Windows executables via the Wine program loader.
- I didn’t have an open source installation of the Symbian build system, sbsv2 (a.k.a. “raptor”). Raptor is also open source, and runs on Linux, but the Linux installer for Raptor is not open source, yet. I didn’t know how to install it from first principles, so I used a binary preview of the installer, obligingly furnished by the Raptor team in Nokia.
This last cheat was unimportant. The Linux Raptor installer will emerge from the contribution pipeline in due course, and in any case I soon afterwards figured out how simple it is to build Raptor on Ubuntu by conventional GNU make methods: the apparatus is all already present and correct in the source.
But the first and second cheats side-stepped real obstacles. Not having an open source compiler that can build the Syborg kernel is hardly less of a problem than not having Syborg. And even when you have built Syborg, without the ROM tools you can’t create a QEMU vm image from it. I didn’t have any idea what size of a task porting the ROM tools for Linux might be.
GCC 4.3 for Symbian
The news from both of these fronts is good. Early in November CodeSourcery, the embedded GCC compiler specialists, finalised a free release of the GCCE (”E” for embedded) v4.3.3 toolchain supporting Symbian on ARM devices. It’s called “G++ Lite for ARM Symbian OS”, and you can get it here, for both Windows and Linux. This release has generated an upsurge of activity in the Symbian community directed at porting the platform to GCCE. The kernel, inevitably, is the prime objective of these efforts. To pull them together, we now have a landing page up on our developer Wiki, The GCCE Toolchain Initiative.
ROM tools for Linux
Building the ROM tools on Linux turns to be a light-to-middling porting job that I’m equal to. I cloned out the build tools package source from its Mercurial repo, http://developer.symbian.org/oss/MCL/sftools/dev/build/, and identified the components I’d need to build a ROM. (Strictly they’re not all ROM tools: there’s a couple of tools in there you need just to build the Syborg kernel – elf2e32, checklib). I studied their MMP files (the Symbian build system’s “build recipe” files), wrote equivalent GNU make files, and then started trying to build the stuff with the the standard GNU/Linux GCC 4.3 toolchain and libraries as shipped with Ubuntu 9.04, plus the Symbian public headers as we shipped them in PDK 3.0.b.
As I encountered failures, I devised fixes and carried on. Mostly these fixes are superficial enough that I can be apply them via the makefiles: they’re to do with GCC 4.3’s more up-to-date and rigorous C++ Standard compliance, versus compilers that the code has previously known; or they’re to do with the fact that the Linux/x86/GCC build configuration isn’t one that the code yet supports. I’ve come across just one component in the build tools thus far ( the sisutils library) that’s thoroughly wedded to Windows APIs and will need thoroughly reworked for portability. But it’s not a library I need to make a ROM.
I’ve built the stuff I need to build a ROM and am familiar enough now with the build tools source to be confident that it will not be difficult to bring up the whole package on Linux. That’s a project I’d naturally like to nudge along, and I’ll get back to it another blog. I’ve committed my work-in-progess to a Mercurial FCL at http://developer.symbian.org/oss/FCL/interim/sftools/linux_build/.
And then again, Syborg
Meantime, I still can’t build a ROM with wholly open source means. Armed with GCCE. QEMU and my ROM tools, I turned to building the Syborg kernel. But that proves not to be a walk in the park. In theory, GCCE will do it. In reality there is a thorny thicket of things that GCCE objects to about the kernel source and I’ve yet to see daylight at the other side of it.
But happily here I have common cause with the brains engaged in the aforementioned GCCE Toolchain Initiative. The thorny problems lie between that compiler and the kernel. Linux isn’t implicated. The Toolchain Initiative is well-connected with the Kernel & Hardware Services team in Nokia, and a GCCE shipshape kernel will surely get committed to our master codeline in the next couple of months, whether or not I do anything to help it. I’ve done a bit to help, by way of committing half-a-dozen “Compiler Compatibility” bugs based on my GCCE kernl-building experience.
Then it will be possible to build and run a Symbian vm by wholly open source means. Or at least it will be possible to start discovering the bugs in my Linux ROM tools.