planet.freedesktop.org
February 09, 2010
Through last minute travel approval, I got to come to FOSDEM again this year. I gave a short talk about cairo-gl. Openoffice presentation is here. But a few more words here since reading slides is failure.

I've been promising great 2D performance from open source graphics for years. It was reaching the point where I was feeling awfully bad about being wrong so frequently. So this summer I started playing in my free time with making a GL backend for cairo. There was a previous sort of GL backend in the form of glitz, but it made a big mistake in trying to abstract GL through a Render-like API. The problem with accelerating 2D is that Render is a bad match for hardware!

A native GL backend turned out to be shockingly easy, now that we have support for EXT_framebuffer_objects all over, non-power-of-two textures, and GLSL. Here's a comparison of 3 backends, normalized to the image backend. Bigger bars means faster.



This shows an accelerated backend beating the CPU rasterization backend on 3 tests. Note that things for the image backend are a little unfair in its favor -- we can't scan out from cached system memory buffers, so if you want to actually see the results you have to do an upload at some point, which isn't reflected in the cairo-perf-trace results. Being able to beat that with GPU rendering to something that could be scanned out is pretty awesome. But that's only 3 tests -- for most of them image is winning. I've got some ideas for hacks on the 965 driver that may fix up a bunch of those bars (it's hard to estimate, since it's all about cache effects, and fixing those has a tendency to improve by more than the amount of time spent according to sysprof).

Since comparing to image isn't too fair, and we're not using image today, I did a comparison to xlib. This looks awesome:



By replacing Xlib usage with GL, we get a speedup on almost all the testcases, and a huge speed up on one that Xlib is pathologically slow on (I haven't figured out why for xlib yet). We've got a good pass rate on the cairo test suite, so I think this stuff is ready for people to start experimenting with in apps.

There's much more to do for performance still. I've got a plan to work on the 965 driver to improve glyphs-heavy tests like firefox-talos-gfx (and ETQW and WoW as well). For firefox-talos-svg, right now we're hitting aperture full because of all the spans data we're sending out before the GPU gets done with things. If we speed up the GPU rendering just a little, for example by tuning the inefficient shaders we're using right now, we can probably avoid hitting aperture full and cut CPU further. I think we're missing throttling for non-swapbuffers apps in DRI2, and we might actually do better and avoid aperture full if we do some appropriate throttling. And there's a lot of room for people who'd like to experiment with GL shader and state optimizations to jump in and tear this code apart.

I'd say that the Linux 2D acceleration story is starting to finally look good after all these years.
I'll try and post these regularly when I make major additions/removals.

drm-radeon-testing is the cutting edge KMS radeon branch, it is going to be rebased and things will be added/removed as they are worked on by developers. So you can base patches on it but you should talk to the developer who owns the area first.

git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6.git drm-radeon-testing

I've just pushed a rebased tree now with the following:

latest i2c algo + hw i2c engine code + all fixes squashed: This adds support for hw i2c engines found on radeons and
exposes them + sw i2c buses to userspace so i2c tools can use them. (agd5f).

pll algorithm reworking + quirks: cleans up the code to allow for the selection of the old pll algorithm on some hardware. (agd5f)

pm support so far: Adds all the current PM patches - just does engine reclocking so far using the power tables from the BIOS. (Zajec/agd5f)

Evergreen (Radeon HD 5xxx) support: basic KMS support for the evergreen range of devices - no irqs or accel yet. (agd5f)

radeon unlocked ioctl support (airlied)

bad CS recording (glisse)

misc cleanups/fixes - Dell/Sun server support ported from userspace hopefully.

The tree did contain Jerome's r600 CS checker but I've dropped it for now at his request as he has newer patches
in testing.
February 08, 2010
You can grab my fosdem's slides here http://people.freedesktop.org/~glisse/glisse-fosdem2010.pdf hope it might highlight some of the thing i am doing :)
Lately we've been working a lot on Gallium but we haven't been publicly talking about it enough. I, personally, spent a considerable amount of time last week coming up with my strategy for the inevitable monster invasion. In general I always thought that if, sorry, "when" the monsters attack I'd be one of the very first ones to go on account of my "there's a weird noise coming from that pitch black spot over there, I better check it out" tendency. Obviously that's not ideal, especially if the need to repopulate the world arises; I just really feel like I should be present for that. In conclusion I'll quickly go over some things we've been doing with Gallium. Admittedly that wasn't one of my best transitions.

There are three new major APIs that we want to support in Gallium. OpenCL 1.0, DirectX 10.x and DirectX 11. DirectX 10.x was somewhat prioritized of late because it's a good stepping stone for a lot of the features that we wanted.

Two of my favorites were the geometry shaders and new blending functionality. I want to start with the latter which Roland worked on because it has immediate impact on Free Software graphics.

One of the things that drives people bonkers is text rendering. In particular subpixel rendering, or if you're dealing with Xrender component alpha rendering
Historically both GL and D3D provided fixed-function blending that provided means of combining source colors with the content of a render buffer in a number of ways. Unfortunately the inputs to the blending units were constrained to a source color, destination color or constants that could be used in their place. That was unfortunate because the component alpha math required two distinct values: the source color and the blending factor for destination colors (assuming the typical Porter&Duff over rendering). D3D 10 dealt with it by adding functionality called dual-source blending (OpenGL will get an extension for it sometime soon). The idea being that the fragment shader may output two distinct colors which will be treated as two independent inputs to the blending units.
Thanks to this we can support subpixel rendering in a single pass with a plethora of compositing operators. Whether you're a green ooze trying to conquer Earth or a boring human you have to appreciate 2x faster text rendering.

Geometry shaders introduce a new shader type, run after the vertices have been transformed (after the vertex shader), but before color clamping, flat shading and clipping.

Along the support for geometry shaders we have added two major features to TGSI ("Tokenized Gallium Shader Instructions", our low level graphics language).
The first one was support for properties. Geometry shaders in Gallium introduced the notion of state aware compile. This is because compilation of a geometry shader is specific to, at the very least, the input and output primitive they respectively operate on and output. We deal with it by injecting PROPERTY instructions to the program as so:
GEOM
PROPERTY GS_INPUT_PRIMITIVE TRIANGLES
PROPERTY GS_OUTPUT_PRIMITIVE TRIANGLE_STRIP
(rest of geometry shader follows)
The properties are easily extendable and are the perfect framework to support things like work-group size in OpenCL.
The second feature is support for multidimensional inputs in shaders. The syntax looks as follows:
DCL IN[][0], POSITION
DCL IN[][1], COLOR
which declares two input arrays. Note that the size of the arrays is implicit, defined by the GS_INPUT_PRIMITIVE property.

It's nice to see this framework progressing so quickly.

Summing up, this is what yin yang is all about. Do you know what yin yang is? Of course you don't, you know nothing of taoism. Technically neither do I but a) it sounds cool, b) I've been busy coming up with a "monster invasion" contingency plan, so couldn't be bothered with some hippy concepts, c) the previous two points are excellent.

February 07, 2010
My first FOSDEM is almost over. I've seen a lot of interesting talks, and it's great to finally meet some of the people I only knew by email or IRC. Brussels has not been so nice to me, on the other hand, plaguing me with some bad food. Oh well...

I gave a talk on the R300 shader compiler. Phoronix will probably have a video online some time tomorrow, together with videos of the other X.Org talks. In the meantime I am making my slides available. (The file contains some additional slides which I left out from the talk to save a little time.)

And not to forget: a big thank you to all the people who make this amazing event possible!

Me: "I think I'm done buying computers that I can't run my own code on."
Friend: "Just think of the iPad as being a pile of books. You can't run your code on those either."
Me: "Thinking of a computer as being a pile of books is like thinking of a guitar as being Abbey Road by the Beatles."

February 04, 2010

I’m glad to announce the first release of our fully managed Theora audio / video decoder implementation for the Silverlight platform! The Highgate media suite will bring installation-free support for HTML5 streaming video to an additional ~40% of web users overnight.

Theora

So, a few drinks will be in order to celebrate the release at the FOSDEM beer event, Friday — drop by! And of course, I’d like to invite anyone excited about making open codecs a first class citizen of the Silverlight / Moonlight ecosystem to visit the Mono dev room over the weekend for source code and some frivolous demos between sessions.

Technology

We’ll be releasing a high-performance decoder for Theora video / Ogg Vorbis audio streams that plugs into the Silverlight 3 streaming media abstraction, as well as a reference front-end player interface and JavaScript bridge layer providing basic compatibility with standard HTML5 media tags, adding support for the standard to Internet Explorer and extending the capabilities of WebKit-based browsers like Safari and Epiphany. A cunning plan, one might say!

Going to FOSDEM

I'm happy to go to Brussels again this year: it's been a long time since I didn't eat some really good waffles! Of course, I had to pretend I would do something useful there, so I'm participating in two talks, both in the distributions devroom:

  • Working with GNOME upstream: this will really be an interactive session, where upstream and downstream people for GNOME can meet, and discuss what can be improved to make the life of both upstream contributors and downstream contributors. It will probably be a good opportunity to also clarify how GNOME 3 might affect distributors, and to get a first overview of how it will be handled downstream. I hope it will be useful for everybody. I certainly hope we'll have a bunch of upstream people attending, and that we'll have representatives for various distributions there (make sure to tell the GNOME people in your distribution to come!).
  • Translations of package descriptions: as you can see, I don't have my name attached to this session, so I can pretend I didn't know about it ;-) But I'm supposed to help Anne there as we've discussed this a bit in the past few months. Translating package descriptions has obvious benefits for the end users, but it's a huge amount of work. And therefore we can seriously wonder if it's really worth duplicating this effort in each distribution (hint: probably not). So this session aims to look at what we can do to work together on this, and what it implies.

I'm also eager to have some insightful discussions with the great people that will come to FOSDEM. The hallway meetings have always been productive there!

GNOME

GNOME will have some strong presence again this year. There's the GNOME devroom on Saturday, that will become CrossDesktop devroom on Sunday, and during the whole week-end, we'll have a booth. If you want to help, that's definitely the place where you should go: we need volunteers to run the booth (see Lionel's mail for more details).

The booth should look great again, thanks to the event box, lovely stickers (they'll be free again!) and, hopefully, t-shirts (we're unsure they'll be ready in time, but let's be optimistic ;-)). On Saturday, you'll also find there some information about what is becoming a tradition: the GNOME Beer Event that will occur on Saturday evening.

Oh, apparently, I was volunteered to organize the GNOME group photo on Saturday, at 15:30. So make sure to be around the devroom at that time!

openSUSE

There will also be a good number of openSUSE people, with the usual booth (looking for some openSUSE DVD or sticker? You'll find them there!). This year, there's no openSUSE devroom because there's one big distributions devroom instead; I believe it's a good thing, though: it should help get more collaboration happen, and it's also a nice opportunity to steal good ideas ;-)

I'm happy that most of the boosters team will also attend; I heard it's a great group of people to hang out with!

In about 30h I will be on the ICE to Brussels to FOSDEM, to have 2 DevRooms there.

The Sportsbag with kit has been packed. The meeting at the customary restaurant tomorrow evening has been pretty much set up. Both my talks should be ready, and just need some studying. And I am now making the posters with the schedules and the fliers (used for possible status changes and directions) for the two devrooms so that they can be printed out later today.

The schedule for the Coreboot DevRoom (on saturday) is unchanged since my last post:

* 13:00 : Peter Stuge - coreboot introduction.
* 14:00 : Peter Stuge - coreboot and PC technical details.
* 15:00 : Rudolf Marek - ACPI and Suspend/Resume under coreboot.
* 16:00 : Rudolf Marek - coreboot board porting.
* 17:00 : Carl-Daniel Hailfinger - Flashrom.
* 18:00 : Luc Verhaegen - Flash Enable BIOS Reverse Engineering.

All seems well, and it seems that we will have everything filmed nicely too!

The schedule for the Xorg DevRoom (on sunday) has seen one last minute change with the addition of Nicolai Haehnle's talk:

* 12.00: Nicolai Hähnle : Towards GLSL in the r300 Gallium driver
* 13.00: Daniel Stone : Polishing X11 and making it shiny.
* 14.00: Luc Verhaegen : The free software desktop’s graphics driver stack.
* 15.00: Jerome Glisse : GPU Userspace - kernel interface & Radeon kernel modesetting status.
* 16.00: Mikhail Gusarov : X on e-Paper.

This addition was very last minute, and did not make the FOSDEM folder. Nicolai was quite lucky as the FOSDEM organisers had already assigned the DevRoom to the openMoko project in the morning, which luckily left a single slot free for Nicolai.

I've finished my slides for my own talk in the X.org devRoom, and it promises to be an interesting one, albeit controversial for some.

It examines some of what we can tell a few years down the road from the modular tree, and goes over the real needs. Then it explains the unification of graphics driver stacks, how to build it for the unichrome driver, and what infrastructural changes it could use.

But before i go into details i will have a small in room survey (Which will probably become a more widespread survey online). One of the questions i will ask is: "For those using, or those who have used, the nvidia or ati closed source drivers: how would you like it if nvidia or ATI told you that you needed the very latest upstream kernel, X, and mesa to be able to run the latest catalyst or nvidia driver, especially when you might need this version for its new hardware support or for bugfixes?"

(and urgh... the msi fuzzy cn700t motherboard i was using as a mailserver just died on me, even the port80 card says that no bios code is being run. Now i stuck in another cn700 board, and it has one of those really noisy fans :()
Here's a really badly shot video of GPU switching in action ;-0 - whiteouts are mostly be logging out and in ;-)

February 03, 2010
v6 of the patch + another patch which needs some work before I can merge it are available now.

This mainly cleans up the patch architecture a lot and allow for Matthew to put his nvidia code in easier hopefully. Its moves the ATPX specific code to the radeon driver.

The second patch is from an experiment that I videod on a webcam but am now failing to upload, I'll probably get a better video tomorrow, the lighting was fairly bad for it today.

It basically allows for a delayed gpu switch ( it changes the debugfs API ), and allows gpu drivers to block the switch.

The switch file now takes ON/OFF like always, but the PCI IDs input is gone. There are 4 commands

IGD - try and switch now to the integrated device - can fail if drm drivers block it (mainly if X has the device open)
DIS - try and switch now to the discrete device - can fail if drm drivers block it (X again).
DIGD - try a delayed switch to integrated device
DDIS - try a delayed switch to discrete device.

So with X running you can echo DDIS to the file and log off X, it'll then switch as soon as X closes the drm device, and when
gdm restarts X it'll be running on the discrete GPU. If we had a shiny GUI on top of this it'd be as close as MacOSX can do it.
When you select to do a delayed switch we power up the other GPU straight away so the switch is quicker.

It needs more debugging, some open issues include:

after a few switches it can die on its ass
powering up the Intel glitches the display even when running the AMD
there may be race conditions in the patch, probably need a mutex around device open + this stuff
suspend/resume - since we D3 the card, if you do an s/r cycle it'll resume it, we need a flag in the
driver to say its powered off by the mux and to ignore s/r cycles - I've started adding this to radeon.

mjg59 has access to an nvidia laptop and is looking closely at how to make that all work.
February 02, 2010

Went down to London yesterday for a meeting and I used the opportunity to visit the scandinavian deli and grocery store there. Turned out to be pretty nice, with a decent selection of food. I stocked up on Swedish liquorice, Norwegian lompe, Danish red hotdog sausages and a Norwegian brown cheese.

In other words I am well set for another week of scaring Abigail with Scandinavian food :)

Okay v4 of the patch is available at

http://people.freedesktop.org/~airlied/vgaswitcheroo/
http://people.freedesktop.org/~airlied/vgaswitcheroo/0001-vga_switcheroo-initial-implementation-v4.patch

First thing I added was power up/down methods. This calls the DRM suspend/resume methods + along with the cut power for discrete GPU. I'm not sure dynamic Windows ever does this as it seems to take a bit of work, I suspect they just run the second GPU in really low power modes. This is slow because we have to repost the ATI card after turning it back on.

I then talked to mjg59 and worked over the ATPX detection. I removed all the DMI code and it should detect *any* laptop that uses ATPX (i.e. ATI/ATI and Intel/ATI) from what I can see. I've tested on Lenovo W500 and T500 now and it appears to work on both. Would be nice if someone on a ATI/ATI and/or ASUS ATI/ATI or ATI/Intel machines could give it a whirl. I think the main problem with ATI/ATI is the poweroff methods have a hardcoded Intel PCI ID. I've no idea yet how to tell on an ATI/ATI which device is the IGD and which is discrete. Its probably more than likely the IGD is the one with the ATPX method on it.

It doesn't switch off at boot yet but I've added commands to let you do it.

echo "OFF" > switch - turns off the not in use card, so if Intel and ATI are on at boot, it will turn off ATI
echo "ON" > switch - turns back on not in use card
echo "PCIID" > switch - causes a switch with full off/on cycles.

nvidia combos appear to use a DSM method and in theory nouveau_acpi.c should be detecting that, so it might be possible for someone to hook that up.

I've also started looking at some desktop integration via gdm or logout - but its not my usual place to code so going is a bit slower ;-)
February 01, 2010

I’ve just pushed initial evergreen (Radeon HD5xxx) modesetting support to xf86-video-ati.  There are still some issues to be sorted out with multi-head, and displayport doesn’t quite work yet, but these will be worked out soon.  Initial evergreen KMS modesetting support is also just about done and will be released soon.  Acceleration is not available yet.

UPDATE: KMS support for evergreen is now working.  You can grab the patches here.  Dave should have them merged into the  drm-radeon-testing branch soon.

So someone thought it would be a good idea to make laptops with two graphics chips in them and switch betweem them to save power.

Now other OSes support this to varying degrees, I think XP + MacOSX require a logout cycle and Vista/Win7 can dynamically switch while running, while Linux basically falls over in a heap.

So I sat down today with a Lenovo W500 which has an Intel GM45 and AMD Radeon 3650 Mobility in it, and I wrote a patch to try and get closer to the XP/MacOSX level.

The result of one days straight hacking is at:
http://people.freedesktop.org/~airlied/vgaswitcheroo/

The patch is totally focused on the Lenovo W500, other switchers will need to add stuff to this codebase.

So what works?
Boot in switchable graphics - which boots with intel and radeon turned on
KMS drivers load for radeon and intel, radeon BIOS stored in start of VRAM (driver hacked to read it)
bind to both drivers + fbs for both.
mount debugfs - cat /sys/kernel/debug/vgaswitcheroo/switch
2
0 :0000:01:00.0
1+:0000:00:02.0
shows the 02.0 (intel) device is in charge of the MUX.
goto runlevel 5, play with X under the Intel driver, goto runlevel 3 kill X
at fbcon echo "0000:01:00.0" > /sys/kernel/debug/vgaswitcheroo/switch
barely glitches console and switches
goto runlevel 5, play with X under the ATI driver, goto runlevel 3 kill X
echo "0000:00:02.0" > /sys/kernel/debug/vgaswitcheroo/switch
goto runlevel 5, play with X under intel again.
wash and repeat.

What does it do?
So far its just switching the MUX using the ACPI method and remapping all the console to the other framebuffer device,
it also reset the bits that denotes which devices is the boot vga device which X uses to pick the primary GPU. This
means X doesn't need an xorg.conf to switch. (I think all those patches are in upstream X server).

What does it not do?
It doesn't powerdown the radeon when its not in use yet. I know the ACPI call to power it off/on, and since I have
the BIOS I should be able to repost it. So I'll try adding the callbacks into the KMS driver to do this soon.
It doesn't poewrdown the intel when its not in use yet. Not sure what I can do here, since there is no ACPI method to turn
it off. I think I can just D3 the GPU, and use the normal s/r paths to bring it back. Again requires more investigation.
The whole what ACPI + methods map to what device, how the mux ids match etc will probably all need to be stored in the DMI table.
Anything not a Lenovo W500 - probably not that hard to add other Intel/AMD variants to this, add DMI and mux switching method.
nouveau isn't hooked up - this could probably be done by some interested party - the driver hooks so far aren't very hard.
No idea about ATI/ATI or NV/NV ones either.

I'm really hoping interested community people can make this actually useful to them on other hw, I won't have permanent access to the W500 to keep this all tested in the future.

Can we do dynamic switch without restarting X?
No. X needs a lot of work, a lot more than the day it took to hack the kernel.

How do we go forward?
We probably need to add gdm support to move this forward. A logout button that is "Switch GPU", that gdm kills the X server,
then hits the switch port and starts a new X server. I'll try and talk to some gdm hackers over the next few days.
I'll try and push this into a git tree against Linus current, and we can add tested patches for other machines as they go in.
Also the DMI section is only imaginary of what I think others might need, we might have to rip it all out. Also I've no idea
if there are ACPI methods to query the switchable modes etc.

The Bluetooth DUN package has been well received, but was hardly a profound programming endeavour. So, I’ve been trying to find suitable inspiration for a more substantial project. This morning, someone made a comment on the Bluetooth DUN page that there’s no way to tell if the phone has a tethered data connection or not - and he’s right: there’s no visual feedback on the phone, unlike Nokia’s Symbian devices or, I imagine, many other phones in the world. With that as a motivation, I decided to try and write a status indicator for tethering.

My first decision was language, and I went with Python as I’ve wanted to use it more and I know how laborious it would be to write this kind of utility in C. I then had to dig around to find out how to write a status area plugin in Python, and luckily there is a way, and it’sfairly well documented.
The biggest source of confusion is that the get_dbus_connection method isn’t exposed in the Python bindings. So, after that took *way* too much time to work out, I had to try and achieve the same thing with direct DBus calls (get a private connection that doesn’t kill the app if it dies), which I reckon I’ve got right.

Once you’ve got the basic stuff sorted out, it becomes really easy to iterate and test - you replace the python file and move .desktop file in and out of a specific directory and Hildon will reload it. Debugging was a real pain because the phone components that I’m talking too don’t exist inside the scratchbox dev environment - so I had to play a trial and error game on the N900 itself, where my only feedback was the icon failing to appear - what fun.

The next challenge was investigating what DBus interfaces to use to find the necessary information. The most important one is com.nokia.csd.GPRS. It’s not documented anywhere, but it’s fortunately introspectable and has obviously named methods and signals, so I was able to establish when a connection is made, suspended or disconnected.

Unfortunately, you see the same set of signals whether the connection is made by the phone itself or a tethered client, so then I had to find a way to detect if the phone was using the connection. I eventually found a way by using com.nokia.icd and com.nokia.icd2 - the first is undocumented and unintrospectable while the second is actually documented. For com.nokia.icd, I was able to use dbus-monitor to find a useful status signal and the get_ipinfo() method I needed had been explored by others. So, now I can avoid false positives from phone initiated connections.

There is one problem that remains, however: It’s possible to tether through the phone at the same time that the phone is using the connection for itself - this is apparently not as amazing as it sounds; all my old phones could do it. In this case, there appears to be no way to notice the tethered connection, so the monitor will not report it. At the moment, I’ve got no good ideas for doing this cleanly - I might be able to poke sysfs or look for pnatd processes, but neither is particularly attractive. But it’s not that common a case, so I consider the program useful before this gets solved.

And what does the program actually do? It shows an icon in the status area that reflects the connection state: disabled, attached, or suspended. I actually think that showing an icon when there’s no connection is a bad use of real eastate, so I’m going to take that out of the next release, but it’s there for now and helpful for confirming that the plugin actually started.

If you’re interested, you can grab it from extras-devel.

Enjoy!

January 29, 2010

Progress

Wow it’s been awhile. Life in the land of Linux graphics has been exciting recently, and there have been a few interesting developments on the Linux PCI front as well.

Linux Graphics Maturing

The Linux graphics stack has really been maturing recently. The Intel and radeon KMS drivers are seeing a lot of bug fixing, and nouveau is getting into shape as well. I think the Intel driver is in better shape than the userland driver ever was at this point (though that’s not to say it’s without defects; our serious bug count is still way too high for my liking). It supports more hardware and features, including power saving, DisplayPort, new hardware, advanced rendering APIs, than ever, and has been shipping in Linux distros for quite awhile now.

We recently finished off the page flipping support, and landed it upstream (it’ll be part of 2.6.33). We also landed a new, core, buffer execution interface (creatively named execbuf2), that allows for more flexibility in the way we submit our command buffers. Specifically, it allows us to control whether a given buffer needs to be mapped with a fence register for operations performed by the commands in its parent execution buffer. This allows our command buffers to be larger, since we won’t exhaust our fence registers prematurely by mapping all objects unconditionally, and allows us to enable tiled texture rendering on pre-965 chips, which can improve performance significantly for some types of rendering.

To support the page flipping work, I had to extend the DRI2 protocol a bit to include support for a SwapBuffers request. While I was at it, I added support for the SGI_video_sync and OML_sync_control extensions, which meant adding support for a few more requests. The SGI_video_sync addition was an important one, since its absence was a regression relative to DRI1. All this new protocol meant new Mesa and X server code, new DRI2 interfaces between the server and DDX drivers, and a bunch of testing and reworking of the interfaces as I figured things out.

All these new features are landed now, and should be a part of Linux 2.6.33, Mesa 7.8, X server 1.9 and xf86-video-intel 2.11. See CompositeSwap for an overview of the features and how they’re implemented. With that out of the way I’ve been able to think more about how compositors and clients should interact, so I came up with CNP. It’s not implemented yet, since I’m still gathering feedback on it, but my hope is that it will help us reduce memory consumption and partial frames in composited environments, as well as address some of the undefined behavior of current GLX calls when drawables are redirected.

Finally, after some discussions with toolkit and compositor developers, I worked with Kristian and Ian to come up with the INTEL_swap_event GLX extension (note it’s definitely possible to implement this on non-Intel as well, but only Intel has support at the moment). This extension allows GLX clients to receive X events when previously queued buffer swaps complete. So rather than making another swap call before the previous one has completed, clients with mainloops can simply poll their X event queue and do other work if their last swap isn’t done yet, rather than wasting time blocked in the server or queuing another swap and getting too far ahead of the display.

Using it all

One side effect of the new DRI2 code is that glXSwapBuffers calls are now totally asynchronous. Previous versions of DRI1 and DRI2 would either block waiting for vblank, or only return after the blit to implement the swap had completed. With the new code, a DRI2SwapBuffers protocol request ends up in the X server, where it’s scheduled by the DDX driver to occur at some later time (though in some cases it will happen immediately, e.g. if the drawable is offscreen). This leaves more time for clients to do other work while their swap occurs; the INTEL_swap_event extension can help clients take advantage of this extra CPU time.

Some optimizations are present in the new code as well. For instance, if the drawable is the same size as the current root window pixmap and there’s no clipping to worry about, the DDX driver can queue a page flip instead. This saves a tremendous amount of memory bandwidth, and so can really increase performance, especially on high resolution and/or bandwidth starved configurations (e.g. most integrated and embedded graphics platforms). Similarly, if a simple back to front copy is requested for a window, if the back and front pixmaps are the same size (i.e. the window manager hasn’t reparented the front window to accommodate decorations and the like), the DDX can simply exchange the backing pixmap object pointers rather than blit. Again this is important on low memory bandwidth platforms (though note this code is currently disabled due to lack of testing; however it’s trivial to enable once I have some test cases).

New hardware

With our Core i7 parts launched, I can talk about some of the hardware feature work we’ve been doing. Zhenyu has been doing most of the bringup and hardware support work for this platform, but I’ve been busy with one of the more interesting hardware features in the Core i7-6xx series, called Intelligent Power Sharing (IPS). Core i7-6xx and 7xx chips are MCP (multi-chip packages); both the CPU and GPU/MCH are in the same physical processor package, but not on the same die. This means they share a thermal and power design domain. In many cases, only one of the components will be very busy, and thus generating much heat or drawing much power, and it would be a waste to let any extra thermal or power headroom go unused. IPS allows one component to use more than its share of power or thermal budget so long as the other component is idle enough to allow it. One of the key parts of this technology is so-called “graphics turbo", in other words the capability of the GPU to exceed its default frequency (and therefore thermal and power budget) when possible. I posted support for this at around launch time (latest patch here), and hope to be able to post the full IPS driver soon, since the potential graphics performance upside is fairly large (still collecting measurements but I’m hoping for something around 15% or maybe even a little higher). The code also allows the GPU to downclock when idle, saving power. The CPU already has its own opportunistic turbo mode which is very effective, but there may be cases where giving it extra power will be helpful (though I’ve yet to find a benchmark, again I’m still testing).

PCI

A recent thread highlighted an interesting design choice in Linux. All platforms supporting PCI (indeed pretty much every platform, PCI or no), splits its address space into multiple regions, allowing for memory mapped I/O (MMIO) from the CPU to different devices. Discovering which ranges belong to which devices is done in a number of different ways, from hard coded offsets (as is found on many embedded platforms), to firmware descriptor tables (as found in OpenFirmware or ACPI), to physically reading MMIO routing information from CPU host bridges down through the hierarchy.

There’s a drive in Linux to support the last option. After all, Linux is the operating system driving your hardware, it should do everything itself, right? Well, that’s where we get into trouble. Linux usually runs on platforms designed for Windows (either specifically for Windows or for Windows in addition to Linux). Windows generally uses the second option to make it easier to port to new platforms. For better or for worse (usually the latter) BIOS writers for new platforms generally consider their work done when Windows boots on their new platform and the Windows device manager doesn’t have any dreaded “yellow bang"s next to devices in the device tree. This usually means the ACPI tables used to describe MMIO layout need to be fairly accurate, or Windows may map a device into a location occupied by another or by a host bridge range with decode priority, causing hangs, corruption or the dreaded “yellow bang".

In October of last year, for arguably good reason, we tried to take Linux down the last path. Yinghai Lu added support for reading root bus resource ranges directly from the host bridge on Intel systems. The thought was that we’d be insulated from firmware bugs this way, and have a more accurate view of the system in general. Unfortunately, due to the above, bridge vendors like Intel have no reason to fully document all the decode windows of a given host bridge, which bits might enable or disable decode for a given region, or generally worry about providing the sort of info we’d need to make this approach tenable. So as of now, we’ve removed the supporting code, and are placing a bet that using the same information Windows does (and hopefully in the same way) will give us the same level of portability. We actually tried this back in 2.6.31 I believe, but had to disable it because our resource tracking code couldn’t handle all the resources handed us by some ACPI firmware implementations. We (well Bjorn hopefully) should fix that limitation for 2.6.34, and we’ll try again, and hopefully fix quite a few resource mapping related bugs in the process.

January 28, 2010

For the first time in life, Xorg is being used in a single platform and for a given device only (other devices have used an X11 implementation but using other non-canonical servers, such kdrive’s based – Tiny-X).

Previously Xorg was being packed to run in a huge amount of OSes – mostly Linux and Unix-like distributions – with the characteristic of be architecture portable and able to run on a huge set of video and input devices. In terms of software, this means an extensive amount of code able to cover all of this mentioned. But this is far from the needs of a small and single platform device.

Some days ago, at #xorg-devel, Alan mentioned the following:


06:59 < alanc> vignatti: the whole point of Xorg is to drive video output - where else would you possibly sanely put that code?
07:00 < alanc> I think you're going overboard in the drive to remove all code from Xorg

Alan was referring about my previous comment to remove some code of video memory mapping from server… I understand (and respect a lot) his concerns but lemme put this right here: it’s not about removal of code; I don’t even care if the code is in xserver or not; what I do care is about the customization – or more fancy, the true modularization [0] – of Xorg.

As discussed on the last X conference, we’re aiming to optionalize [1] lot of components inside Xorg: distros would build all components, satisfying all supported devices and drivers, whereas constrained environments (such as maemo + n900) would use a restrict set only.

So recently I’ve been confusing people’s mind trying to in fact optionalize several components of the server. There are some straightforward modifications on the code like turn off libdrm, vgahw or libxdmcp, but there’s also other more challenger like all the old-school mechanism to initialise cards, to remove cursor support or even to choose if we want or not all bus subsystem. Sometimes we’ll have to be careful to not run out of the protocol. But the truth is: the currently the code is _very_ tied all over the server. It’s not trivial to “get there”.

IMHO the plan traced at XDC looks perfectly clear and while other display systems seems not suitable enough for us yet, I’ll be keep digging on this direction.

[0] the modularization that happened in the version 1.0 was related with drivers going outside the server.

[1] what would be a good word here?

Ooh, the strain.

If you filed a bug against shared-mime-info in the past and wonder why your requested mime-type still isn't in, it's just a lack of time, and the fact that most of the bug reports require too much work on my side to be integrated.

If your bug doesn't include a test case, I won't look at it.
If your bug is a copy/paste of a stand-alone mime definition file, I won't look at it.
If your bug doesn't contain any reference information, I won't look at it.
If your patch isn't git-formatted, I won't look at it.
If your patch breaks the test suite, I won't look at it.

Given the requirements to compiling shared-mime-info (git, a C compiler, and glib), I don't think I'm setting the barrier too high. Furthermore, all those requirements are spelled out in the HACKING file.

Let me know if you have any questions, or want clarification on some points, so I can update the HACKING file with that information.
January 27, 2010
Remember this guy?

He knows a thing or two about SUSE and its community. He is used to travelling a lot, to give interviews, talking to the press, and to speaking in public, and on top, he knows how to organise events for a once big linux distribution. And thanks to a really fun group event at his previous company right after FOSDEM last year, he might actually be available for a new opportunity.

Plus, he lives like 200 meters away from the SUSE Nuernberg office, how lucky is that?

(oh, music is the stones, track 4 of 12x5, listed here as suse planet is horribly broken with respect to livejournal.)
January 26, 2010
Hello,

About LDTP:

Linux Desktop Testing Project is aimed at producing high quality test automation framework (using GNOME / Python) and cutting-edge tools that can be used to test Linux Desktop and improve it. It uses the Accessibility libraries to poke through the application's user interface. We strive to help in building a quality desktop.

Changes in this release:

Fixed ldtp binary name and now it spits out the version info
Updated README and AUTHORS file
Fixed OpenSolaris bug reported by Qinghua Cheng

Acknowledgement:

Ara Pulido[1] requested the above change for backward compatibility. Thanks to Ara, Conny.

Download RPM from http://download.opensuse.org/repositories/home:/anagappan:/ldtp2:/rpm
Will schedule deb build tomorrow

Download source http://download.freedesktop.org/ldtp/2.x/2.0.x/ldtp-2.0.2.tar.gz

Documentation references:

For detailed information on LDTP framework and latest updates visit http://ldtp.freedesktop.org

For information on various APIs in LDTP including those added for this release can be got from http://ldtp.freedesktop.org/user-doc/index.html

Report bugs - http://ldtp.freedesktop.org/wiki/Bugs

To subscribe to LDTP mailing lists, visit http://ldtp.freedesktop.org/wiki/Mailing_20list

IRC Channel - #ldtp on irc.freenode.net

Thanks
Nagappan

[1] - http://ubuntutesting.wordpress.com/

I always try to bring some Norwegian food back with me when I have visited Norway. This year I brought a leg of Fenalår which is a leg of cured mutton. So it is just like the cured ham people eat all over Europe, except from being made from sheep. Anyway I brought it to the Collabora office today to let people have a taste and Marco snapped this picture of me holding the leg.

Of course not all scandinavian food is equally appreciated by our southern neighbours.

January 24, 2010
So, you likely noticed i am a crazy blogger with a new post everydays ;)
Today amusement was finishing cleaning up the early state of the r600/r700
winsys API i want to base the r600/r700 gallium3d driver on. There is
still a lot of things to finish, mostly finishing the states splitup.
Once all the states are in a nice structure i will plug this winsys
with a r600/r700 pipe driver but don't hold your breath, first i am
doing this during my free time, second even when we reach this state
i don't think we will be able to walk on Mars or the on Moon.

What matter here, more than the beauty of the code, is the design of the
beast. It's a different approach from what we have been using so far and
i believe it brings enough interesting features. For instance, the pipe
driver won't have to worry about the cs buffer size. We can do advanced
command/states checking so that we don't try to program the GPU into a
stupid states. There is a bunch of others interesting things about this
but i need to save some of my tricks for my FOSDEM talks.

So if you bore and you an R7xx GPU with KMS working then you might want
to test this beautifull software and see a neat grey square on a blue
background, which, oddly enough, i find entertaining, well it's maybe
not as much fun as gears but it's good enough.

(It's a standalone KMS app which runs from a console without X running)
http://cgit.freedesktop.org/~glisse/r600winsys/
January 22, 2010

Bastien Nocera is known for a lot of things. Most people in Gnome probably knows him as the maintainer of the Totem media player. In the Fedora community he is known as one who deals with a lot of the challenges around multimedia. Here in the UK he is also known as the only frenchman to ever get fully naturalized.

Anyway his latest effort is to try to try and kickstart some work in Google Chromium to get it to use everyone favorite media framework GStreamer. Bastiens explains a lot of his rationale in the Chromium bug report, but I am hoping to get others in the community to chime in too, and even if you don’t have anything new to add, just let Google know you care by starring up this bug report.

I just finished pitching my fledgling GLU3 library to the ARB, and everybody loved it. Quite a few people said that they've hacked up something similar, but much less complete, to get simple demo apps running. A few of the people had functionality not in the library, and it sounds like I may get a few code contributions. It also sounds like it may be included in the OpenGL SDK. Yay me!

For those interested, the doxygen documentation is at:

http://people.freedesktop.org/~idr/glu3

The GIT tree for the code is available at:

git://anongit.freedesktop.org/~idr/glu3

At some point the tree will probably move somewhere that can allow direct commits by people other than me. I'll cross that bridge when I come to it.

Luis already unleashed the word: GNOME 2.28.2 will be released as an online update for openSUSE 11.2 (for reference, openSUSE 11.2 was initially released with 2.28.1). You can currently help testing that everything is fine with the packages by adding the 11.2-test repository and upgrading. Please go ahead and test it, and tell us if it breaks anything. Hopefully, it should work quite fine.

What is really exciting about this is of course not that we're delivering bug fixes to our users ;-) But with 11.2, openSUSE got a new maintenance team, with more community involvement. One of the amazing result is that it is (or at least, feels) much easier now to release online updates for packages, with a process that everybody can follow — it used to be restricted to Novell employees. Another welcome change is that we can finally release new upstream versions as updates, with some obvious restrictions: the new versions should only contain bug fixes, and should fix real important bugs for users.

And this is what enabled the release of GNOME 2.28.2 as an update for openSUSE 11.2: this version bump was lead by Dominique and Magnus. I must admit I'm really glad that I didn't have to do anything ;-) The GNOME policy to only do bug fixes (and updated translations, which is something we also care about!) on a stable branch, and the fact that we're doing a good job at being reliable on this upstream, certainly helped too.

But wait, there's more! If you're crazy about GNOME but still want a stable distribution, you can use GNOME 2.29 on openSUSE 11.2! The Build Service is really helping us here, making it easy to reuse our GNOME 2.29 packages that we have in Factory on 11.2, with nearly no work at all. We have some documentation on how to use GNOME 2.29 on openSUSE 11.2, and testers are welcome. It should work fine and not eat your computer. Testing Factory is also an option, and while it used to be hardly usable in the past, the community is now doing a good job at making sure it works fine most of the time, if not all the time.

Did I mention you can get the latest version of various applications on 11.2 by just adding the GNOME:Apps repository? No need to update the distribution. No need to update GNOME. This is getting insanely cool :-) And both for packagers (nearly no effort to backport packages) and users (latest versions of their preferred applications available on a stable distribution).

Two years ago, I wrote about my switch from my beloved xmms to audacious.

During this 2 years with Audacious, I suffered a bit. It was working quite fine, but I saw no big progress around it. Life happened, and I had to use a network system to play music. I started to use PulseAudio over TCP, but it does not work well, and does not work at all with Audacious (and even if the plugin is provided by upstream). So I decided to dump it.

And some days ago I discovered Sonata, a MPD client. I never liked MPD so far because all clients I found were lame.

But I really like Sonata. It allows me to listen music the way I still want: load everything in one playlist, listen everything randomly or type a song/artist to jump to it directly in the current playlist. It even has some nice feature (lyrics, so I'll be able to song out loud, covers, tag editing…) and is written in Python and GTK+ (some days I may even hack it!).

You can rest in peace x11amp :-p

So I originally was going to attend LCA 2010 for the week, but real life interjected and I couldn't abandon family commitments for that long, so I ended up doing a crazy cross-Tasman dash. As well as the change in flights, Isabel came down with a virus and Gia also got it, I think I got a milder version of it, but they both seem alright by the time I left but I had little sleep the previous two nights.

So I flew on Wednesday morning, got in Wed afternoon, met up with ppl, had a couple of beers, wrote slides, slept, finished slides, went to Thur morning, drugged myself up on Nurofen Plus to combat viral effects, gave my talk, went to see ajax talk, went to professional network dinner, went for beers with ajax + benh (listening to an American and a Frenchman speaking about wine while listening to drum n bass in a Wellington pub was a bit wierd). Decided to push on through, so got back to room at 3am or so, checked out/left for airport at 4:20am, flew at 6:20am, into BNE at 7am, home, bed, sleep for a few hours and mind Isabel for afternoon.

So my talk was "So you've put kernel graphics drivers in the kernel... what next? can I haz ponies?". My slide deck is off the 0-content style + lots of pictures of various ponies, which I've found, they'll be on the LCA site soon and I'll upload them when I plug that laptop in again.

(a) stop people reading ahead of your bullet points so they don't doze off while you are catching up
(b) gives them something to look at apart from me while they actually have to listen to me :-P

It seemed well received, the room was pretty packed out (ppl standing/sitting - LCA schedulers you listening?) and I don't think the sickness or lack of decent preparation made a big difference. I'd like to apologise for not even mentioning SGX/poulsbo, I'm not sure how but it totally slipped my mind, but the situation hasn't really changed in terms of how screwed it is.
coreboot DevRoom

This year, at FOSDEM, there is a coreboot DevRoom, and for the hardware-lover, there are some really interesting talks there. When putting together the final schedule, I actually had to spend some time on containing people's enthusiasm and limiting their ideas to something feasible for FOSDEM. A rather good sign, and i am sure that many people will enjoy the truly interesting and juicily in-depth talks there.

Here is a quick copy of the saturday afternoon schedule:
* 13:00 : Peter Stuge - coreboot introduction.
* 14:00 : Peter Stuge - coreboot and PC technical details.
* 15:00 : Rudolf Marek - ACPI and Suspend/Resume under coreboot.
* 16:00 : Rudolf Marek - coreboot board porting.
* 17:00 : Carl-Daniel Hailfinger - Flashrom.
* 18:00 : Luc Verhaegen - Flash Enable BIOS Reverse Engineering.

So yeah, i have a talk at 18:00 which promises to be fun.

Basically, for flashrom, we sometimes need to find out what write protection certain motherboards implement, so we can disable that from within flashrom. This usually means that someone (usually Michael Karcher or me) has to go and disassemble the BIOS to find out which GPIO line needs to be toggled.

For a single board, this is usually not a hard task (and actually fun), but the overal amount of work in flashrom is sizable, and the two of us are not available 24/7 to do this for every user in need. So this talk is meant to alleviate that.

I just wrote the slides for it, and while i might alter the notes and twiddle it a bit still in the next two weeks, it should be pretty much finished.

The structure of the talk is the following:
* quick overview of how flashes are usually wired up on motherboards.
* quick overview of how to handle Award, ASUS, AMI and phoenix BIOSes.
* example of disassembly of a BIOS.

The BIOS that is being disassembled is an especially crafted one, with an award style flash enable for a fictitious board. It's mostly filled with 0xFF, apart from the flash enable itself. And the talk itself is pretty accessible, you do not have to know much of assembly or hardware to be able to participate.

If you are visiting the talk, and you want to join in, then you might already want to download the image, and make sure that you have hexdump and ndisasm installed (as there are about 4000 others trying to use the wireless at FOSDEM).

It promises to be a blast!

X.org DevRoom

On sunday there is of course the X.org DevRoom, for which the schedule is also online.

It seems to be a tradition that I have to go and beg people to talk at truly great event like FOSDEM.

Last year, we had 7 talks in the brochure, while there was room for 13, and we eventually had 9 talks. All these talks were hugely popular, with almost full capacity usage most of the time, but the actual talk coverage itself left much to be desired.

This year I decided to split the usual two day X.org DevRoom into coreboot (saturday afternoon) and X.org (sunday), so that the X.org schedule could be tighter and more exciting.

Here is the result;
* 10.00: ...
* 11.00: ...
* 12.00: ...
* 13.00: Daniel Stone : Polishing X11 and making it shiny.
* 14.00: Luc Verhaegen : The free software desktop’s graphics driver stack.
* 15.00: Jerome Glisse : GPU Userspace - kernel interface & Radeon kernel modesetting status.
* 16.00: Mikhail Gusarov : X on e-Paper.

I, and a lot of fosdem visitors with me, thank Daniel, Jerome and Mikhail for their commitment. These talks seem very interesting and will no doubt draw in a lot of visitors and carry out the good name of X.org.

It does make one think. There seems to be a direct and inverse relationship between the size and status of a free software project, and the willingness of its members to go and promote this project on the biggest free software event on that continent that holds the majority of its current developers and the largest potential of future contributors.

In any case, I have been communicated that, as there are many very worthwhile and also very willing projects who get denied a DevRoom each year, a project with such a poor showing will not be given another DevRoom.

While I start putting together the slides for my X.org talk, I can only concede.
January 19, 2010

Here's the seventh and final part of my ongoing series.

One of the grandest sights in Delhi is Humayun's tomb, a predecessor of the greatest mausoleum of them all, the Taj Mahal:

Humayun's Tomb

A little bit further down a view on the garden:

Humayun's Tomb

From a different corner:

Humayun's Tomb

We'll finish with our last panorama that shows the courtyard the Jama Masjid of Old Delhi:

Jama Masjid

That's all panoramas from this trip. Thanks for your interest.

When I went to Norway this year for Christmas I had the most convenient travel route ever. I had a direct flight from Stanstead Airport to Rygge in Norway. Rygge is a tiny airport, but when you visit people on the southeast side of Oslo like I did, its perfect.

The ticket was also quite reasonably priced even though it was the Christmas holiday.

Yet, despite offering me this ideally set up trip between the two airports on both side which fits me the best, I struggle to dig up any kind of positive feelings for Ryanair.

Was discussing it a bit with Wim and Tim and I think a big part of the reason for this is the constant feeling you get when booking a trip with them that they are trying to make you think their fares are cheaper than they actually are or the feeling that they attempt to ’sneak’ in extra options.

For instance how they charge you for the use of your credit/debit card, a fee that got nothing to do with the actual cost to them and the fact that it is practically impossible to avoid it. As Tim mentioned when we discussed it, if they just included that 5£ fee in the price you wouldn’t think about it and just be happy you got a cheap ticket, instead you feel they are trying to pull a fast one on you and make their prices look even better than they are by pretending mandatory expenses aren’t mandatory.

Maybe Ryanair doesn’t care if I or anyone else actually likes them, I mean I still booked my flight to Norway with them because they where the only ones flying from Stanstead. Yet on the other side I always tend to fly with Easyjet instead of Ryanair if I can, both due to usually flying to better airports and a general feeling that if I search for a flight the price they list for that flight is the price I am actually paying.

Maybe the problem is simply that Ryanair doesn’t understand that even when you are travelling I don’t like the feeling of being taken for a ride :)


Apparently my little after work project was cool enough to make the news page over at ioquake3.org. :-) I just thought I'd mention here that you can read some more about it in my earlier (first actually) blog post, and there are more pictures over here.

Thanks again to id Software and ioquake3 for a great code base! And to the people at Nokia and Maemo Summit for support and encouragement.

Btw, if you do own an N900 and have played with the ioquake3 port, please leave your comments and suggestions for improvements in case I feel the urge for another weekend hacking session. (I am aware there have been problems installing the package in the past, but they should now be fixed.)


January 18, 2010

Here's the sixth part of my ongoing series.

Leaving Jodhpur we continued our journey to Jaisalmer, a sand castle of a town in the Thar desert:

Jaisalmer

In the vicinity of Jaisalmer you'll find cliche sand dunes like you'd expect from a grown-up desert:

Jaisalmer

Our next station after a long, cold and dusty train ride was Delhi. The principal mosque of Old Delhi is the Jama Masjid:

Jama Masjid

That's all for now, tomorrow I'll post the rest of my panoramas from this trip, all from Delhi.

Hello all,

LDTPv2 a complete rewrite of LDTPv1 in Python. This release is dedicated to Eitan Isaacson[1]. Eitan wrote the LDTPv2 framework and important API's in LDTPv2 !

My co-workers in VMware Ranjith Murugan, Gaurav Sharma and Anupa Kamath, did a wonderful job in verifying the compatibility of LDTPv1 and v2. Special thanks to them and my manager Greg McShea on supporting this effort.

Special thanks to Ara Pulido[2] for tracking the LDTPv2 status and pushing us to make the release at the earliest, as Ara wants to include LDTPv2 in Ubuntu Lucid, before feature freeze.

Following are the difference between LDTPv1 and v2:

* getlabel function is deprecated - you can use getobjectproperty('winodw', 'objectname', 'label') # To verify the display text
* Label in v2 doesn't return the accelerator key (eg: in v1 "_Find" will be returned on v2 just "Find" is returned)
* Strict data types are checked, in v1 most of the inputs are considered as string, if not they will be converted to string, but on v2 exception will be
thrown, if incorrect type is passed to any function * In v1 we have ldtp binary, on v2 we need to check ldtpd.sh for now, this doesn't return the version for now, it has to be implemented, if you check for "ldtp --version" in v1
* In v1 each action command was given 1 second sleep time internally before execution, but on v2 there is no delay unless its set in environment variable LDTP_COMMAND_DELAY. So, the script has to use appropriate wait time
* As Javier (from Ubuntu QA team) found, launchapp, argument name changed from 'arg' to 'args'

Some of missing API in v2:

* Calendar object
* logFailures in v1 is not implemented in v2
* LDTP logging methods
* appundertest
* launchapp2
* blackoutregion
* label object
* panel object
* ProcessStatistics

LTFX is completely removed in LDTP v2 in favor of wnck implmentation

Download LDTPv2 source from http://download.freedesktop.org/ldtp/2.x/2.0.x/ldtp2-2.0.1.tar.gz

New dependency:

python-twisted-web
python-pyatspi
python-gtk
python-gnome

Will schedule binary package building for different Linux distribution using openSUSE Build Service - http://download.opensuse.org/repositories/home:/anagappan:/ldtp2:/

Documentation references:

For detailed information on LDTP framework and latest updates visit http://ldtp.freedesktop.org

For information on various APIs in LDTP including those added for this release can be got from http://ldtp.freedesktop.org/user-doc/index.html

Report bugs - http://ldtp.freedesktop.org/wiki/Bugs

To subscribe to LDTP mailing lists, visit http://ldtp.freedesktop.org/wiki/Mailing_20list

IRC Channel - #ldtp on irc.freenode.net

Thanks
Nagappan

[1] - http://monotonous.org/
[2] - http://ubuntutesting.wordpress.com/
January 17, 2010

Here's the fourth part of my ongoing series.

After Udaipur the next stop on our trip was Jodhpur, the blue city. Which is called that way due of the blue colour of many of its houses:

Jodhpur

On a hill next to Mehrangarh Fort, one of the biggest Forts in India (the big sand castle on the hill in the panorama above), you find the Jaswant Thada, a memorial of the Maharajas of Jodhpur:

Jodhpur

Inside the fort you'll find highly decorated courtyards:

Jodhpur

That's all for Jodhpur, tomorrow I'll post more panoramas, from other stops of our trip.

January 16, 2010

Here's the fourth part of my ongoing series.

After Hampi we went to Bangalore to attend foss.in. (Fantastic conference, btw. The concerts at the venue are unparalleled.) From there we flew up to Udaipur, in Rajasthan. Udaipur is (among other things) famous for being the place where the central scenes of Octopussy were filmed. Octopussy's famous white palace is on Jagniwas Island in Lake Pichola:

Udaipur

This panorama was taken from another island in the lake, Jagmandir Island, which is visible in the following shot on the left:

Udaipur

Udaipur's scenery, seen from the Maharaja's City Palace down onto Pichola Lake:

Udaipur

That's all for Udaipur, tomorrow I'll post more panoramas, from other stops of our trip.

It's easy to get lost in /sys and not much fun typing long udevadm info command lines all the time. Today, when I had enough of that I sat down and spent an hour to write a little UI for exploring the udev/sysfs tree: udev-browse. I wrote it for my own use, but I am quite sure I am not the only one who wants a little bit simpler access to the device tree. So here you go.

And since everybody loves screenshots here you go:

udev-browse

Two usability hints: if you run udev-browse from a directory in /sys udev-browse will automatically present the device of that path on startup. And if you know the name of a device you can just type it into the device listbox (which is focussed by default). The usual Gtk+ live search will then find you the right entry right-away. It's pretty nifty.

It's written in Vala with minimal dependencies.

I want to keep the maintainership burden for this minimal. So no tarballs, no releases, and I won't reply to your emails regarding this tool, unless they include a good, clean, git formatted patch. Thank you for your understanding.

Anyone wants to package this for Fedora? I'd be very thankful if someone would pick it up.

Have fun.

January 15, 2010
Over Christmas, Matthias worked on the first pass at the long awaited user accounts tool.

I did my bit and committed this afternoon the new icon selection popup, which allows you to capture and crop a picture from your webcam (through my earlier cheese work). I also committed the ability to save your fingerprints, as was available in gnome-about-me.

Screenshots below. More information on the Fedora Features page.

The new icon selection popup

Fingerprint enrollment

Webcam capture and cropping
Jetway sells a cute VX800 mini-itx board which comes with a choice of VIA C7 and Nano cpu's: the JNF76. At the end of last year, i became the proud owner of one of the rare DVI modules for this board, and shortly thereafter there was an ebay auction which got me a brand new board at half the listed price. I now found the time to go and play with this board, to at least give the ebay seller the positive rating he deserves :)

What I found is that this board booted a debian old-stable (etch) just fine, but that debian stable (lenny), opensuse 11.1 and 9.04 ubuntu, all locked up shortly after initialising the ide controller. Some digging around revealed that this was an issue with the processor power state and recent kernels, and that this is only seen with an older bios version, and that version A03 and higher would fix this. Upon reboot, it was indeed confirmed that this was an old version, namely, A02.

Since sadly, the network controller on this board is not supported by the old-stable kernel, i had to use a usb stick. So I stuffed the latest bios and a copy of flashrom master on the stick, and stuck it in the jetway board. As i already had the libpci-dev package installed, a simple run of make got me a working flashrom binary. I then used flashrom -r jnf76.old.bios to dump the current bios, as a precaution. And then i ran flashrom -w on the new image.

When running flashrom, there is a relatively high probability (50% is my current estimate, but will improve to about 30% soon) that it will not work. Motherboard makers like to provide an extra bit of protection for their bios images, and often use a gpio line from either the southbridge or the superio chip to set write protection on the flash chip.

In flashrom, we keep a list of special board enables, which today mostly map which io line to set, and Michael Karcher and I spend quite a bit of our time digging out bioses for users to map this. So in this case, before i had tried to write to this chip, i had already done my 5 minute magic (which i will explain at the coreboot devroom at FOSDEM) and found out that there was no special protection present.

Flashrom then erased the chip correctly, and then subsequently wrote and verified the image. And i was then able to reboot into any of the installed distributions painlessly.

If there was protection present, (and we didn't handle it yet in flashrom) then this would've made the erase fail at the first byte that isn't 0xFF, and then nothing was lost. There is also another type of protection only found on LPC/FWH/Parallel chips, where the boot block can be locked, and this has the same effect, but only on the boot block part, where you just need to restore all other pages to be able to continue working. We will then happily help create the board enable with you on the flashrom mailing list, so that you still can get your bios updated correctly (now, and in future).

In any case, this was one of the easiest emergency bios updates i had ever done. No messing around with dos images and what not. I had already identified a working linux kernel when trying to nail down the issue, and then just needed to run flashrom to live happily ever after :)
January 14, 2010

Here's the third part of my ongoing series.

Still in Hampi here's another 360 from the Hills in Hampi down to the Achyutaraya Temple:

Matanga Hill

A little further down, before dawn, here's a shot from the rocky path leading up the hill:

Matanga Hill

Our last picture for today is a view down from Hemakuta Hill which is covered with old temples and other structures. In the middle you'll see the large Virupaksha Temple which is still in full use. In that temple you'll find an amazing camera obscura, a physics teacher's dream that projects the temple tower onto a wall (projection, subject, more interesting in reality. Really.)

Hemakuta Hill

That's all for Hampi, tomorrow I'll post more panoramas, from other stops of our trip.

January 13, 2010

As pointed out in the excellent Real-Time Rendering blog, Sony Pictures Imageworks has released the documentation and source code for their Open Shading Language compiler. It's a slightly different take on shading languages than GLSL / HLSL / Cg. The idea is that instead of generating final colors, as in other SLs, shaders compute information about how a surface / light / whatever interacts. These shaders are then connected into a network, and complex global illumination is then calculated by evaluating the network. At least, that's my understanding from readin the introduction for five minutes. :)

Pretty cool stuff.

January 12, 2010

Everyone and his dog seem to call libxml2's xmlCleanupParser() at inappropriate places. For example Empathy does it, and Abiword does it too. Google Code Search seems to reveal at least Inkscape and Dia do it as well.

So, please, if your project links against libxml2 verify that it calls xmlCleanupParser() only once, and right before exiting! And if it calls it more often or somewhere else, then please fix that!

For more information see my post on fedora-devel.

Thanks for your time.

Here's the second part of my ongoing series.

Climbing down the hills, on the banks of the Tungabhadra river you find people washing laundry and bathing, and coracles waiting to be used for a trip through the river.

Tungabhadra River

The greatest of the ancient temples in Hampi is the Vitthala Temple:

Vitthala Temple

Set in in lush green scenery you find the Achyutaraya Temple, which you already might have seen, from above, in yesterday's series:

Achyutaraya Temple

That's it for today, tomorrow I'll post more panoramas, both from Hampi and other stops of our trip.

January 11, 2010

Yes, I won't spare you my panorama shots from my recent trip to India. After arriving in Goa Badami was our next stop. It's a very pretty little town in northern Karnataka, and here's a panorama shot from the entrance of the town's famous caves:

Badami

Next step was one of the most amazing places on earth, Hampi in central Karnataka. It is definitely one of the greatest sights I have ever seen, and I guess I can say I have seen quite a few in my life. A vast landscape of hills covered in boulders, lush mango and banana plantations, rice fields, dotted with age-old temples and impressive ruins. Locals crossing the river in coracles that look like they belong in a time centuries ago. Women washing colourful laundry in the river, pilgrims wading across the river in their black clothes. An India that delivers every bit of that promise it makes to its visitors. The ruins rival the grand sites in Greece and the landscape sometimes looks like a Crysis in-game scene.

Taken from one of the hills in Hampi this is the sunset:

Hampi Sunset

And then, the next day at dawn make your way up the hills again and you can get an even greater view on the whole scenery:

Hampi Dawn

That's it for today, tomorrow I'll post more panoramas, both from Hampi and other stops of our trip.

Also, if you haven't seen them yet, don't miss my panoramas from my India trip the year before.

Introducing TeleMetrum v0.2

Bdale and I (mostly Bdale, of course) finished the TeleMetrum v0.2 design work in December, and this weekend we got boards made and parts ordered and Bdale sat down with his trusty electric skillet and built 3 new boards. The new design has an integrated GPS receiver and patch antenna, and is otherwise fairly similar in design to v0.1.

TeleMetrum v0.2 Hardware

Here’s the front side of the board:

From the left, you’ll see a connector for an external power switch and the two ejection charge circuits, a battery connector for a single 3.7V lipo cell, the GPS patch antenna, a 4-pin debug connector, the piezo buzzer and the new 8-pin companion board connector. We weren’t happy with the connectors used on the v0.1 board and finally found these Tyco Micro-MaTch parts which take up a modest amount of board space (more than pico-blade connectors, less than regular pin blocks), have a locking option and crimp on to standard ribbon cable. They’re also bright red and surprisingly low in profile.

And here’s the back side:

Elements on this side include the new 100μF cap in the upper left corner which sits on the 3.3V supply to try and keep the CPU alive through minor power glitches. Below that is a new package containing a pair of FETs for the ejection circuits. We used discrete FETs in v0.1, but this device has better specs for our needs (lower on resistance, etc). The USB connector was pulled in-board far enough to keep it from hanging over the edge. Right of that is the new data logging chip, and right of that is a U.FL connector in case you want to use an external GPS antenna. We supply power to that connector as most external GPS antennas include their own LNA. And, of course, to the right of that is the Skytraq Venus 634 GPS receiver.

Below and to the right of the GPS receiver is the cc1111, to the left lies the accelerometer and then the barometric pressure sensor above the 5V boost regulator which powers the accelerometer. We haven’t found any high-G accelerometers that run on 3.3V yet. Finally the two tiny 5-pin chips are the USB LiPo charger and the 3.3V regulator. What you can’t see easily are a pile of 0402 passive components scattered across the board. Even close up, they’re hard to pick out by eye.

The only hardware ‘bug’ was in the reset logic — the new board was designed with a much larger capacitor on the reset line than the old board. The debug code would only hold the reset line low for a brief instant, sufficient for the old capacitor value but not the new one. Instead of fixing the code, Bdale decided to try a smaller capacitor value and found that it worked just fine. After that, the board came up just fine and the updated firmware was flashed into the CPU.

TeleMetrum v0.2 Software

The only significant software change was that the data logging part changed from a 25LC1024 1Mbit eeprom to an AT45DB161D 16Mbit DataFlash. This required writing a new driver, but fortunately much of the code could be copied from the 25LC1024 driver. Because the AT45DB161D comes from a family of similar-but-different parts ranging from 1Mbit to 64Mbits, I decided to make the code automatically adapt to the installed part, detecting which one was attached and adjusting the driver.

The story here is that the configuration data didn’t appear to be getting preserved across reboots — we use the last block of the data logging part to hold configuration data, including call sign, sensor calibration values and flight parameters. A bit of testing and we found that the code to read/write the device worked perfectly. It turns out that a premature optimization in detecting which kind of flash part was installed had a race condition when multiple threads were trying to access storage at the same time, resulting in the configuration data being left uninitialized. Oops!

The TeleMetrum firmware has a clever hack for selecting between ground mode (for fetching data from the device or altering the configuration) and flight mode (prepared to fly the rocket). It switches between these by detecting whether the board is upright (flight mode) or not (idle mode). However, the accelerometer must be calibrated to tell the difference. What never occurred to us was that if the calibration data was broken enough, the device might always come up in flight mode. In that mode, it isn’t listing to either USB or the radio link, so it’s impossible to fix the accelerometer calibration data.

A bit of brainstorming led to a fairly simple hack — check to see if one of the pins on the companion connector was shorted to ground at power on time, if so, force the computer to enter idle mode. Pin 1 of the companion connector is ground, and fortunately, pin 2 was the SPI clock pin, normally output-only, so we could safely use that in this mode as any companion device shouldn’t ever pull that low.

Future Events

As of this evening, three boards are built and mostly tested; the radios appear to work, GPS tracks satellites and the beeper makes plenty of noise. Still to check is whether the deployment circuits will fire an ematch (we’ve tested the design before, just not this specific implementation).

Next weekend, we’re off to linux.conf.au in Wellington, New Zealand where we’re scheduled to give a presentation on the hardware and software in TeleMetrum. We’ll have v0.2 boards to show off, so come and see them in person.

With v0.1, we used the same board design for both flight computer and ground station, TeleDongle. For TeleDongle, we just left most of the components off of the board and loaded alternate firmware. For v0.2, we’re planning on building a separate TeleDongle board; that design is finished but no boards are made yet.

Once we’re happy with the design, we’ve got big plans to get more boards made so we can let a few friends buy them for use them in their own rocket projects. That should happen in the next month or so. Once we’ve gotten enough testing done, and made sure that other people can actually operate them without hand-holding from us, we’ll make them available for sale to the general rocket-flying public.

Beyond that, we’ve got plans to build more stuff:

  1. A stand-alone ground station, called TeleTerra, that would include an LCD readout and flight data recording so you wouldn’t need a laptop during the flight.

  2. A companion board, called TelePyro, to control 8 additional pyro channels. These could be used for almost anything from air starts to staging or any other whacky plans.

January 05, 2010

I just got back from watching Avatar in the cinema this time, in much higher quality and in anaglyph 3D; actually I started out at the wrong cinema, Finnkino in Kaisaniemi; luckily the one in Kamppi is only a few minutes walk away.

I've mentioned before that usually these anaglyph 3D effects don't work very well for me due to varying degrees of myopia in both of my eyes, but I thought the movie was visually cool enough to give it a try with contact lenses instead of my normal glasses. I think this helped the effect significantly, but I still get the feeling that my brain starts to adjust to the effect over time so it's most impressive at the start of the movie.

This peaked my interest in ioquake3's anaglyph rendering mode, but unfortunately they ask for the glasses back after the movie, so no chance of trying it out (at least until I order a pair of my own.)

Anaglyph 3D on the N900 might be quite interesting, but of course I would either need to make much more efficient use of vertex buffer objects to make up for halving the FPS, or do a port of XreaL...

I think the later option is better for several reasons, but we'll see; Python, SCons and Scratchbox is a whole bundle of pain that I really don't want to be getting into at 02:00 in the morning.


January 04, 2010

Discovered a blog post today from Philip Jägenstedt at Opera. The post describes the new HTML5 enabled Opera (alpha)( release and how it uses our favourite media framwork GStreamer. Great to see this available for easy download now and a big thanks to Opera for their work in supporting HTML5 and the Ogg codecs.
Philip got a lot of interesting details in that blog post so I really recommend reading it.

Opera also makes the source code of their copy GStreamer available as a git repo which should make merging any changes they do to GStreamer very easy, a big thanks for that.

In the last couple of weeks, Dan Nicholson and Julien Cristau have been working on revamping the X server configuration and hotplugging code, introducing a couple of new features. This work is now pulled into master and brings about a new way of configuring the X server.

The three new features are:

  • udev backend support

  • Support for an xorg.conf.d configuration directory

  • Support for matching rules in the xorg.conf



udev backend support


Back in 2006, support for hotplugged input devices was added and up to server 1.7, the incarnation worked with HAL as the provider of the input devices.
HAL is now deprecated and - at least under Linux - replaced with udev. Julien wrote a new backend that queries udev instead of HAL but otherwise follows the same principle:
Upon startup the X server queries the active backend for the list of input devices and initializes them accordingly. When a new device is added at runtime, the server is notified and adds it at runtime. HAL and udev are mutually exclusive, so enabling one disables the other. Currently, udev is disabled by default though I'd like to see it being default for 1.8.

On that note, users still confuse HAL and evdev though they have little to nothing to do with each other. The same is of course valid for the udev backend.

xorg.conf.d support


Over recent years, the xorg.conf support has moved towards two overlapping goals - support partial xorg.conf's with the server filling in the rest and removing the need for an xorg.conf entirely.
For most installations the server now has the right defaults and is usable without any configuration file. For the few cases where it is needed, a single section is often enough.

However, one remaining problem was that the xorg.conf is a single file and not easy to programmatically edit. Dan has now added support for a directory-based configuration. Again, the principle is simple: in addition to the xorg.conf file (if it exists), the /etc/xorg.conf.d/**.conf files are read and parsed as well. It performs like expected, with a section being added just as if it was present in the xorg.conf. Hence, a possible configuration may be:


:: whot@barra:~> ls -R /etc/xorg*
/etc/xorg.conf

/etc/xorg.conf.d:
10-synaptics.conf 97-wacom.conf 99-evdev.conf


Note that the xorg.conf is parsed before the xorg.conf.d, hence it always has precedence. This allows for distributions to populate the xorg.conf.d directory with various quirks, yet still have user-specific configuration in the good old xorg.conf.

To better support snippet-based xorg.conf, a new tag has been introduced: AutoServerLayout. Traditionally, any input device needs to be referenced from the ServerLayout section to become active. With AutoServerLayout, an input device may be present in any layout. Hence, your input configuration may look something like this:


Section "InputDevice"
Identifier "touchpad"
Driver "synaptics"
Option "AutoServerLayout" "on"
EndSection


Regardless of which layout is started, the touchpad will always be part of the input device list.

InputClass matching


With the hotplugging support, much of the configuration wandered into the HAL fdi files. This was never the best solution but at that time the only available one. With udev, the fdi files went away and instead of requiring users to port HAL-specific configuration to udev-specific configuration, Dan has stepped in and wrote a set of patches that allow similar functionality to the fdi configuration as part of the xorg.conf (or xorg.conf.d).


InputClasses are similar to InputDevices but can apply to multiple devices (and multiple classes may apply to one device). They work on match rules, with the following matches being available:

MatchProduct, MatchVendor, MatchDevicePath, MatchIsKeyboard, MatchIsPointer, MatchIsJoystick, MatchIsTablet, MatchIsTouchpad, MatchIsTouchscreen

Their meaning is what they describe, with the first three being substring matches, the others being binary matches. An input class is assigned to a device if all match rules apply. As an example, the following snippet would assign the synaptics driver to all touchpads:


Section "InputClass"
Identifier "synaptics-catchall"
Driver "synaptics"
MatchIsTouchpad "on"
EndSection


Or, the catchall evdev rule that applies to every device.

Section "InputClass"
Identifier "evdev-catchall"
Driver "evdev"
EndSection


Or, a more complicated example to set a specific option:

Section "InputClass"
Identifier "vendor-quirk"
MatchVendor "SomeVendor"
MatchDevicePath "/dev/magicdevice"
MatchIsTouchpad "on"
Option "Quirk" "on"
EndSection


Input classes are cumulative and only apply if a given option hasn't been set yet. Hence, the first Driver match will apply and the evdev catchall needs to be the last file matched. This is a bit odd at first, but it again guarantees that the user-defined xorg.conf takes precedence.

We may see some more match conditions before 1.8 is released, but for now the ones above seem to suffice for the general cases.
January 03, 2010
Hi guys,

Let me get straight to my problem. The turkmen keyboard has different letters where x, c, v and q are supposed to be. So in the turkmen keyboard layout, I used:

key.type[group1]="ALPHABETIC";

To get ctrl shortcuts (copy, paste, cut, quit) to work. However, after adding these, the altGr doesn't work, we use altGr to get x, c, v which we do need in international words, though not frequently.

I looked at some other layouts they have lines such as FOUR_LEVEL, FOUR_LEVEL_ALPHABETIC. I'm guessing these are the ones to be used, though, if I substitute these, the ctrl shortcuts stop working. I frankly don't understand the mechanism, I got this far with try & fail. If anyone sees my problem, please share some tips :)
December 31, 2009
Jodhpur Jodhpur Jodhpur

India is a weird and beautiful country. And I am too lazy to retouch my photos.


I'm sure this has all been said before, but people don't seem to get the message: Subversion sucks for external contributors. Occasionally I send some patches for projects using Subversion...

Subversion has no way to commit local patches so that you can test out your changes, split up commits functionally, generate patches easily so you can send them to a review list, or any of these great features. Git does not have this problem; in fact it has a very nice set of tools for working this way. I won't go into the details of that, as most readers probably know it already, other than saying that it's amazingly simple and easy.

The pain comes when you're an outside contributor for a Subversion project: you might be testing some experimental changes that you don't want to commit to the public repository. Okay, you can edit the files and svn diff locally but this pathetic compared to the power of git commit and git format-patch
  1. You're stuck doing one functional change, and only one functional change. Because you have no way to create a patch series. It's either everything in one commit, or nothing at all.
  2. Even when you have commit access, often this results in multiple functional changes being squashed into a single commit. This is very bad.
  3. Subversion provides no way to add commit messages or authorship to the diff generated by svn diff, so contributors are left out in the cold.
    • Yes, if you have commit access you can provide this meta-data, but you're still pushing directly to the repository with no review what-so-ever.
I could continue, but I think my point is made: Subversion sucks for distributed development (in the across the globe sense) and it also sucks for development in the "few developers distributed across the room" sense. It all adds up to pain and inefficiency: especially for external contributors, but even for people working with the project with commit access.

Thankfully Sourceforge does at least provide rsync access to the Subversion and CVS repositories so git svn clone can be both fast and bandwidth efficient. Unfortunately, however, it seems that Subversion chokes on anything but patches generated by Subversion itself (including standard GNU diff format.)

Everyone should be using Git (or at least some distributed version control system with similar features.)


December 30, 2009

Just in time for the new year, I’m pleased to be able to say that the Bluetooth DUN package is now in the Maemo Extras repository. This is the primary location for community packages that have been through a community QA process that tries to ensure the packages are safe for ‘normal’ users. If you don’t have the extras repository turned on, you can do so by following the instructions here.

New XMMS2-Scrobbler release. Better than ever: multi-scrobbling is supported now. libre.fm users rejoice! See README for the details.

December 29, 2009
Okay, technically it would be a backhoe scoping up snow/ice from the road and footpath, but seriously, at 09:45 and right outside my bedroom window!

I've had the alarm on my N900 fail to wake me several times, however this is much more effective; now if only they could do it at some reasonable time, like, oh, 12:00 or 13:00... and in keeping with the day of fail, my box at Nokia seems to have lost it's IP address, so no working remotely today.



I like UI and I’m not gay.

Some time ago someone posted a comment, complaining about the white fonts over the dark background in my blog, which makes the read be difficult. The guy teased me, recommending an ironic page where shows how hard is to read texts like this. He was right…

So now I changed wordpress to a totally opposite theme called “White as Milk”. Yay, total white and easy to read now! Let’s see how long I’ll get sick of that interface.

December 28, 2009
In the last few weeks, I've had a surprising number of discussions about commit messages. Many of them were with developers new to a project, trying to get them started. So here's a list of things you should do when committing, and why you should do it. Hint: the linux kernel mailing list gets it right, go there to learn.

Any software project is a collaborative project. It has at least two developers, the original developer and the original developer a few weeks or months later when the train of thought has long left the station. This later self needs to reestablish the context of a particular piece of code each time a new bug occurs or a new feature needs to be implemented.

Re-establishing the context of a piece of code is wasteful. We can't avoid it completely, so our efforts should go to reducing it to as small as possible. Commit messages can do exactly that and as a result, a commit message shows whether a developer is a good collaborator.

A good commit message should answer three questions about a patch:

  • Why is it necessary? It may fix a bug, it may add a feature, it may improve performance, reliabilty, stability, or just be a change for the sake of correctness.

  • How does it address the issue? For short obvious patches this part can be omitted, but it should be a high level description of what the approach was.

  • What effects does the patch have? (In addition to the obvious ones, this may include benchmarks, side effects, etc.)



These three questions establish the context for the actual code changes, put reviewers and others into the frame of mind to look at the diff and check if the approach chosen was correct. A good commit message also helps maintainers to decide if a given patch is suitable for stable branches or inclusion in a distribution.

A patch without these questions answered is mostly useless. The burden for such a patch is on each and every reviewer to find out what the patch does and how it fixes a given issue. Given a large number of reviewers and a sufficiently complex patch, this means many man-hours get wasted just because the original developer did not write a good commit message. Worse, if the maintainers of the project enforce SCM discipline, they will reject the patch and the developer needs to spend time again to rewrite the patch, reviewers spend time reviewing it again, etc. The time wasted quickly multiplies and given that a commit message only takes a few minutes to write, it is simply not economically viable to omit them or do them badly.

Consider this is a hint for proprietary software companies too - not having decent SCM discipline costs money!


How to do it better


There's no strict definition of the ideal commit message, but some general rules have emerged.
A commit should contain exactly one logical change. A logical change includes adding a new feature, fixing a specific bug, etc. If it's not possible to describe the high level change in a few words, it is most likely too complex for a single commit. The diff itself should be as concise as reasonably possibly and it's almost always better to err on the side of too many patches than too few. As a rule of thumb, given only the commit message, another developer should be able to implement the same patch in a reasonable amount of time.

If you're using git, get familiar with "git add -p" (or -i) to split up changes into logical commits.

The git commit format


If you're submitting patches for git, the format is mostly standardised. A short one-line summary of the change (the maximum length of the line differs between projects, it's usually somewhere between 50 and 78 characters). This is the line that'll be seen most often, make it count. Many git tools are in one way or another optimised for this format. After that one-line summary, an empty line, then multiple paragraphs explaining the patch in detail (if needed). Don't describe the code, describe the intent and the approach. And keep the log in a present tense.

Learn to love the log


I have used CVS (and SVN to a lesser extent) in the past and log was a tool that was hardly ever used. Mostly because it was pretty useless, both the tool and the information available. These days I look at git logs more often than at code. The git log tool is vastly superior to CVS log and the commit discipline in the projects I'm working on now is a lot better. I grep git logs more often than code files and I use git blame all the time to figure out why a particular piece of code looks the way it does. It's certainly saving me a lot of time and effort. It's come to the point where the most annoying X server bugs are the ones where the git history stops at the original import from XFree86. If you're not using your SCM's log tool yet, I recommend to get more familiar with it.

How not to do it


There's a bunch of common sins that are committed (yay, a pun!) regularly.


  • SCM is not a backup system! My personal pet hate. Developers who use it as such tend to do end-of-day commits, checking in everything at the end of the day. The result is useless, a random diff across the code with changes that are impossible to understand by anyone including the original author once a few months have passed. (On this note: universities, please stop teaching this crap).

  • Per-file commit. More often than not a logical change affects more than one file and it should not be split up into two commits.

  • Lazy commit messages, any commit labelled as "misc fixes and cleanups" or similar. I've seen my fair share of those on non-FOSS projects and they always come back to bite you. Impossible to find when a bug was introduced, hard to bisect and makes it harder for anyone else to keep track of what's happening in the project.

  • Two changes in one patch. Something like "Fixed bug 2345 and renamed all foo to bar". Unless bug 2345 required the renaming, fixes whould be split it up into multiple patches. Others may have to take one of those bug fixes and apply it to a stable branch but not the other one. Picking bad patches apart into useful chunks is one of the most time-consuming and frustrating things I've done since it doesn't actually add any value to the project.

  • Whitespace changes together with code changes. Needle in a haystack is a fun game, but not when you're looking at patches. It's a great way to introduce bugs, though because almost no-one will spot the bug hidden in hundreds of lines that got reindented for fun and profit.

  • The ever-so-lovely code drops. Patches with hundreds of lines of code to dump a new feature into the code while at the same time rewriting half the existing infrastructure to support this feature. As a result, those hundreds of lines of code need to be reviewed every time a bug is discovered that is somehow related to that area of code.
    It's easier and less time consuming to first rework the infrastructure one piece at a time, then plug the new feature on top. As a side-effect, if a project relies on code dumps too often it's discouraging outside developers. Would you like to contribute to a project where the time spent filtering the signal from the noise outweighs the actual contribution to the code?

  • Unrelated whitespace changes in patches. A reviewer needs to get the big picture of a patch into their brains. Whitespace-only hunks just confuse, a reviewer has to look extra hard to check if there's a real change or whether it can be ignored. That's not so bad for empty lines added or removed,it's really bad for indentation changes.



There's plenty of excuses for the above, with the favourite one being "but it works!". It may work, but code is not a static thing. In a few weeks time, that code may have moved, been rewritten, may be called in a different manner or may have a bug in it. At the same time, the original developer may have moved on and no-one knows why the code is that way. In the worst case, everyone is afraid of touching it because nobody knows how it actually works.

Another common excuse is the "but I'm the only one working on it". Not true, any software project is a collaborative project (see above). Assuming that there's no-one else is simply short-sighted. In FOSS projects specifically we rely on outside contributors, be it testers, developers, triagers, users, etc. The harder it becomes for them to join, the more likely the project will fail.

Another, less common excuse these days is that the SCM used is too slow. Distributed SCMs avoid this issue, saving time and by inference money.
December 27, 2009

Today, the applet code is dropped from gnome-applets. Since gnome-settings-daemon now includes the notification icon (automatically appearing/disappearing), the applet is no longer needed.

Back in 1999 (Nov 29) it was the gswitchit applet that was committed to SourceForge. After GUADEC in Dublin it got merged into GNOME, some bits and pieces went to gnome-control-center. The rest is history…

The technology goes on. The functionality stays and improves. The initial point, the applet is gone.

December 26, 2009
I don't have any plans for the Christmas holidays, and Helsinki is at just the right temperature to make doing anything outside really difficult. Some of the snow has melted and frozen into ice, making it feel a lot like walking on broken glass: physically and emotionally; I stopped counting the number of unintentional acrobatics after about the 5th or 6th time sliding in some way.

I might implement zlib compression for network traffic in my engine, which should only be a few dozen lines... or something a bit more interesting; stencil shadow volumes could still use more optimization, or some SIMD stuff...

There are at least a few dozen minor X server issues discovered by Coverity, but that's getting dangerously close to Nokia work over the holidays. The majority of the issues are error paths anyway, nothing really critical.

In the more being lazy and doing nothing area, Avatar looks like it could be an interesting movie...


December 25, 2009

I started looking into the projective light issues in XreaL and my forked engine again. It turned out the problem was setting up the light rotation via both the quaternion and light projection vectors.

The flashlight was working, but would point in the opposite direction you were looking. Right was left and left was right. Of course, it seems simple in hindsight, but these kind of bugs always annoy me until I find out what was going wrong.

Boring screenshot included for the fun of it...