I've had a great time working with Qt so far; it seems like a well-designed modern toolkit.  There are a few quirks; most notably, the pieces that Gecko/Cairo needs to render text aren't really there, but on the whole the integration is pretty good.  The big text piece that's missing is a drawGlyphs method on QPainter — something that would take a QFont and an array of glyphs and positions.  Additionally, methods to go from a string to an array of glyph indices/positions for a given QFont would also be helpful (though most of our font backend here is using FreeType directly, so we can obtain glyph indices directly).  In the QPainter Cairo backend that I wrote, I kind of cheat on this issue under X11 — I grab the X Drawable and pass on the show_glyphs work down to a wrapper xlib surface... but that doesn't work for Qt Embedded.  Anyone out there who knows Qt innards well want to implement this?

There are a few other minor annoyances with QPainter; Cairo's rendering code does a pretty good job of letting operations be specified in a general way (e.g. everything with paths), and it does whatever optimizations are possible internally (e.g. converting a rectangular path to a rectangle for clipping/painting purposes).  QPainter doesn't seem to have these optimizations for the general path methods, so the QPainter backend has to do a bunch of extra work to detect this case to call the faster filling functions.  Similar optimizations are missed for brushes — filling with a tiled pixmap QBrush is slower than calling drawTiledPixamp, even without transforms.  The Cairo backend tries to handle these cases, but there's more that needs to be done.

Overall, though, what's on the Qt branch now works pretty well under X11; under Qt Embedded, there are some problems with text rendering mentioned above, but things are in good shape otherwise.  I hope that in the future that the Qt port will be a toplevel supported Gecko port, alongside gtk2, Win32, and OSX, but there's still a bunch of work needed to make that happen.  If anyone's interested in helping out, drop in to #gfx on mozilla irc!

(Note that there aren't any plans on switching away from gtk2 for our official Linux/X11 builds; gtk2 will remain the default supported toolkit there.)

I've tested this build on OS X and Windows; I was unable to test in Linux under VMware — Firefox crashed with a GLXUnsupportedPrivateRequest X error.  (If anyone wants to check this under Linux with a real OpenGL driver, let me know how it goes.)

Looking around, though, I found iPartition, which claimed it could do exactly what I wanted.  So, I bought it, burned the CD.. went through the cute UI (grabbing a little circular slider and adjusting partition sizes while large numeric readouts adjust in real time), gulped, and hit Go.

An hour or two later, everything was resized, moved, and back in action.  I'm impressed (which is why I'm posting... usually these stories don't have happy endings).  The software's a bit pricy for something that you won't use all that often (I wish they'd included a full version of iDefrag along with it), but that's probably exactly why it's pricy -- when you need it, it does exactly what you want.

For the past two months, Antoine Azar and Frédéric Plourde have been working for Mozilla, cranking away at both Cairo and pixman.  They started out with a completely unfamiliar codebase and quickly producing useful enhancements.  Antoine was able to implement some missing support in pixman, allowing us to implement smooth scaling of images.  This was the final missing piece that makes Firefox 3's full-page zoom feature able to display a high quality version of the unscaled page.

Johnathan sent me some screenshots from Vista, comparing the rendering of a recent Firefox 3 beta to IE 8's beta.  He's running in Vista's high-DPI mode, so all the scaling was applied by default.  Firefox is on the left, IE's on the right... the difference is pretty striking (click for the full image, make sure you view it unscaled for all the gory details):

Linux users will have to wait a little longer to get higher quality image scaling; as we now rely on the X server to do image compositing through the Render extension, we'll have to wait until the pixman patches are integrated into the X server.  I'm hoping that will happen in time for the next X.org release.

Frédéric has also been working with pixman, specifically low-level optimization of the compositing routines (pixman is used on Windows for all non-trivial rendering, and used indirectly on Linux through the X server).  He cleaned up the existing MMX code so that it could build with Visual C++, and is now working on further optimization and extending the existing fast paths.  In Mozilla, we've enabled the MMX code everywhere.

At around the same time,  a rapidly-progressing effort to add SSE2-optimized paths to pixman began on the Cairo mailing list, and Frederic will be joining those guys shortly.  Given that most x86 computers in the past few years have full SSE2 support, it makes sense to focus work there. Early results are showing significant peformance gains, which should be a huge boon for authors taking advantage of SVG and Canvas features.  Much of this additional work will not ship with Firefox 3 at release time, but it may come shortly after in a minor release.  I'm pretty excited to have additional improvements already waiting in the wings... I think you'll see Gecko continuing the past few months' trend of rapid development even after Firefox 3's release.

One interesting bit of fallout from this is that Quartz/CoreGraphics also happens to be the native graphics layer for the iPhone: so, Cairo is trivially portable to the iPhone.  The major issue is that Cairo currently uses ATSUI for font access, whereas it needs to be using CGFont -- which is the native font API suppored under MacOS X 10.5 and the iPhone.  (However, the CGFont API is missing many public methods to make it entirely general, but I hope that will be remedied in time.)  I've made some last-minute changes to Cairo to allow us to switch to a CGFont-based text backend with no change to Cairo-using apps, and I have such a backend in progress for landing shortly after Cairo 1.6 is released.

There is also some exciting work going on with improving the pixman library that cairo uses for its software-only rendering -- I've often maligned it for being slow and hard to work with, but the newer cleaned-up version is much easier to deal with, and Soeren has great plans for its future.  A number of people are working on adding highly optimized MMX and SSE2 compositing routines to augment the little bit of MMX code that's present currently.  Initial benchmarks are pretty encouraging, and I'm hoping that we can do some optimization work targetted at ARM as well.  There is also some work in progress to implement accelerated backends using OpenGL (and ultimately Direct3D).

Gecko 1.9 (and thus Firefox 3) will be shipping with Cairo 1.6 (or, as close as possible, if the schedules don't quite line up): everything that Firefox 3 renders on the screen is going through Cairo, and is ultimately translated by Cairo into the native platform API.  Future work to expand Gecko's graphical capabilities will intimately involve Cairo.  It's safe to say that Cairo is ready for prime time.

Chris [Wilson]: The thing that I'm really passionate about is web standards. Silverlight is a part of the web just like PDF and plaintext are part of the web.

Uh, it's what now?  (Read the rest of the panel transcript, it's pretty good.)

I noticed a few odd things -- if I disabled drawing of the native theme (used for the scrollbar), and if I tested just a tall page with a solid colored background and a scrollbar, I saw the same performance issues.  There's a problem here, one that I initially blamed on the native theme drawing, because if I disabled native theme rendering, benchmark performance shot up, but there were still some inconsistencies.  I collected some numbers: for a given test, the minimum time that it can report is the number of iterations multiplied by the setTimeout delay used.  For example, with 200 iterations and a 50ms setTimeout, there's no way that test can finish faster than 10,000ms.  The difference between that time and the time the test actually reports is the time spent doing interesting things.  By running my test for different timing values, I ended up with this graph:

The horizontal axis is the setTimeout value, and the vertical axis is in milliseconds.  That plateau there?  That's not supposed to be there.  Looking more closely at that number, it's roughly 6500ms.  With 200 iterations, that's 6500/200 = ~32.5ms per iteration.  So, per second, that gives us 1000/32.5 ~= 30.77.  We're capping at 30.77 frames per second here.   That's way too round of a number.  Even more suspicious, if I disabled native theme rendering, the test hit a plateau at around 3300ms, which ends up being 60fps.

It's at this point that I start to get an idea of what's going on.  I was aware of Beam Sync on OSX, but assumed that each app had to opt-in to it, given that it didn't seem to affect Firefox 2.  Quartz Debug lets you disable Beam Sync on a global basis; I did that, and the benchmark numbers dropped -- the line above kept nicely following the blue line down, and I was able to peg the fps needle in Quartz Debug over to the right.  So, we're being throttled by the OS which is forcing us to wait for the next frame interval before allowing us to draw again.  This is a pretty serious problem, because at this point I thought that the only way to disable this was on a system-wide basis, which wouldn't be acceptable.  Firefox 2 didn't suffer from this, though, so I did some more digging.

Eventually I came to this tech note from Apple. The reason why Firefox 2 wasn't affected was that Fx2 was not a Cocoa app; it is a Carbon app, and as such was exempt from being subject to coalesced updates.  The key thing showed up in the "last resort" section of that tech note: how to disable coalesced updates for an individual app!  This seems to be available available only on 10.4.4 or later, but that was fine; OS minor updates are free.  I verified that adding the plist entry fixed the problem for me locally, and checked this in to become part of the build.  See if you can spot when this change hit our performance testing infrastructure:

Not bad for two lines of XML.

While figuring all this out, I noticed that Safari/WebKit didn't seem to be affected by this framerate cap -- the fps meter when Safari was running the same benchmark happily went up beyond 60fps.  After I found the plist entry, I checked Safari's plist and was surprised to discover that they didn't have this disabling in there.  Doing some more searching, I found this code in WebKit.  Apparently, there is a way to do this programatically, along with some other interesting things like enabling window update display throttling (though it's unclear what that means!) -- but only if you're Apple.

All these WK* methods are undocumented, and they appear in binary blobs shipped along with the WebKit source (see the WebKitLibraries directory).  There are now over 100 private "OS-secrets-only-WebKit-knows" in the library, many of which are referred to in a mostly comment-free header file.  Reading the WebKit code is pretty interesting; there are all sorts of potentially useful Cocoa internals bits you can pick up, more easily on the Objective C side (e.g. search for "AppKitSecretsIKnow" in the code), but also in other areas as a pile of these WK* methods used in quite a few places.  Would any other apps like to take advantage of some of that functionality?  I'm pretty sure the answer there is yes, but they can't.  It's not even clear under what license libWebKitSystemInterface is provided, so that other apps can know if they can link to it.

Despite my frustrations with Linux, this type of hiding isn't really possible in a real open source environment.  In the end, I really do hope that Linux can rise to the technical challenge and compete in desktop performance and features, but it's not there yet.  However, I'm glad that there was a workaround for this issue for us on OSX, because the performance benefits are huge --  Firefox 3 on the Mac (everywhere, really) is going to be a kick-ass release!

Edit: Slashdot seems to have picked up on this, and in typical style, has completely misunderstood the post.  To be clear, I do not think that Apple is in any way trying to purposely "cripple" non-Apple software.  I also do not think that undocumented APIs give Safari any kind of "significant performance advantage" (as Firefox 3 should show!).  However, as I said, the undocumented functionality could be useful for Firefox and other apps to implement things in an simpler (and potentially more efficient) manner.  I don't think this is malicious, it's just an unfortunate cutting of corners that is way too easy for a company that's not fully open to do.

The second script fixes something that's been annoying me for a while about the Mozilla development infrastructure --- all of our servers have an identical favicon, which means it's impossible to differentiate between different sites (or, most commonly, to find the tinderbox tab) when you have many tabs open.  I blogged about this a while back, before the crash of my blogging machine; I received some good feedback, but my requests to get the icons changed on the actual sites didn't go anywhere.  So, this script forces favicons for a couple of key mozilla sites, including tinderbox, bugzilla, wiki.m.o, and developer.m.o.  The icons come from the famfamfam Silk set and a bit of hackery on my end using Pixelmator.  Here's an image of what the result looks like:

After some sniffing around, using a very reduced testcase (that I was working with for another performance issue) I came across the Mac native theme code.  Disabling native theme entirely (commenting out all the calls) gave back all the performance.  So I start digging into it.  I collect a bunch of really weird data --- such as, if I replace a call to HIThemeDrawTrack with a loop calling HIThemeDrawTrack 5, 10, 50 times... the overall time spent in the benchmark doesn't change from 1 time.  At 100 times though, it changes drastically.  The delta between benchmark time and raw time spent in NSChildView's drawRect should remain mostly constant with the above changes, one would think.  But it doesn't.  Going from no native theme rendering to turning it on increases the "unknown" time significantly.

I'm still spending time with Shark (which is, I would argue, Apple's best developer tool -- maybe one of the best developer tools, ever), but the latest is completely baffling and what prompted this post.  I finally got around to doing a shark System Trace, which traces at the user/kernel boundary, recording syscalls, preemptions, etc.  In the syscall log, I noticed a bunch of calls to geteuid, getuid, etc., all being repeated.  Looking at the call stack...

... you guessed it.  HIThemeDrawTrack makes syscalls.  For every single call, it calls geteuid, geteuid, getuid, geteuid, stat64, stat64. That's 4 geteuid/getuids and 2 stats.  For every single call to the theme function. This is ludicrous.  The stack traces indicate that this is because this function seems to pull out user preferences on each call, and then has to check if the preferences should change because the euid may have changed (?!?)... the getuid call is coming from _CFUserName.  Because, you know, drawing a UI element needs to know the user name.  It's probably also allocating the CFString to hold it.

I wish this was a joke, but it's not.  I'm now looking into how to draw this crap faster; the NSCell API may be a better way to go -- we'll see if it has this same damage.

You're probably not going to see me do too many of these types of posts, mainly because I hardly ever finish games, but I figured I'd start blogging about the games that I've played and do a mini review of sorts.