CSS Sprites have been around for a while, and keep coming up as a way to get blazing fast speed to your site.  Steve Souders just showed off SpriteMe! at Velocity '09 (speaking of — why use the CSS Sprite Generator or other server-based tools when you can use <canvas> and toDataURL and generate sprites on the fly?).  However, there are a number of misconceptions about CSS Sprites, the main one being that they have no downsides.

The basic idea of CSS Sprites is to combine a number of images used on your site into a single image, thus reducing the number of HTTP requests that need to be made to your site.  The image is rendered using a CSS background and background-position (which, incidentally, means that your markup becomes more complex; the image to use is specified in CSS, not in a plain <img> tag).

The biggest problem with CSS sprites is memory usage.  Unless the sprite image is carefully constructed, you end up with incredible amounts of wasted space.  My favourite example is from WHIT TV's web site, where this image is used as a sprite.  Note that this is a 1299x15,000 PNG.  It compresses quite well — the actual download size is around 26K — but browsers don't render compressed image data.  When this image is downloaded and decompressed, it will use almost 75MB in memory (1299 * 15000 * 4).  If the image didn't have any alpha transparency, this could be maybe optimized to 1299 * 15000 * 3, though often at the expense of rendering speed.  Even then, we'd be talking about 55MB.  The vast majority of this image is blank; there is nothing there, no useful content whatsoever.  Just loading the main WHIT page will cause your browser's memory usage to go up by at least 75+MB, just due to that one image.

That's not the right tradeoff to make for a website.

Another (but much less important) downside is that if a sprite-using page is zoomed using the full-page zoom feature that's present in many browsers, the browser may need to do extra work to get the correct behaviour at the edges of these images — basically, to avoid adjacent images in the sprite from "leaking in."  This isn't a problem for small images, but can be a performance hit for bigger ones.

There are cases where CSS sprites are clearly beneficial; the main example is combining a bunch of small similarily-sized images into a single one.  For example, a bunch of 16x16 icons used as indicators for various things on your site, or a bunch of 32x32 icons used as category headers or similar.  But combining images with vastly different dimensions, especially when there are skinny and tall images in the same sprite as wide and short ones is almost never a good idea.

However, sprites do improve page load time (at least initial page load — on subsequent page loads, the browser hopefully has the images cached).  What's the alternative?  Unfortunately, there isn't one, yet.  Many browsers have support for offline manifests already; it might be possible to extend that to allow downloading one file (like a jar/zip file) that contains a manifest of resources and equivalent URLs that are contained inside it.  But any such work won't show up on the web for some time yet.

So, when deciding whether to sprite or not to sprite, keep in mind that there are factors in play other than raw page load performance.  As a general rule of thumb, if most of your sprite doesn't contain real image content, you should probably avoid using it.  Also, keep an eye out for future solutions that will preserve both the page load speed, as well as avoiding the negative memory and performance impact of sprites.

Shortly before our office move, we kicked off an effort to take a hard look at our startup time, to both understand what we all do, and to figure out how to improve it. zpao (Paul O'Shannessy), ddahl (David Dahl), and I have been working towards a few goals:

• Document how to reproducibly get a cold and warm startup on Windows (XP/Vista/7), MacOS X, and Linux
• Create tools to capture both JS execution during startup, as well as file IO
• Add instrumentation to firefox to identify "big blocks" of startup for timing
• Create tools to visualize the captured data in a way that's easy to analyze

One thing that's fairly obvious with playing with startup is that "warm" startup is significantly faster than "cold" startup; that is, when you've launched Firefox before, the OS caches a bunch of the data off the disk, and it doesn't have to hit the disk again. This directly points to IO being a major component of our startup time, which is why IO is part of the capture above.  This is a pretty big problem even on desktop systems; on my fairly beefy Windows 7 box, a cold startup takes upwards of 12 seconds (!); warm startup is also fairly slow if the system is under load.

We've fixed some bugs in our dtrace javascript provider along the way (bug 403345), so dtrace will actually give correct (and sane) data now. Also, I've been doing a lot of work with Microsoft's xperf (part of the Windows Performance Toolkit), which can capture much the same data. (In theory we should be able to create JS providers for xperf as well, but that's out of scope for this particular project.)

One example of the type of data we're capturing and tools that we're building is available here — this is just a quick io capture with xperf, with the data dumped into a Timeline widget from the SIMILE project. (The time scales are a bit off; the raw data is in microseconds, but SIMILE only handles milliseconds... so all times need to be divided by 1000, which becomes a problem when you go over 60 seconds -- which is actually just 60 ms! Something that we'll fix.)

Another example is the result of a startup trace; zpao is still working on the visualization and data capture, but you can see an early version here — the "Exclusive function elapsed times" view will provide the most accurate data, basically telling you "how long did we spend in a given function, ignoring all descendants". In this view, the "null" filename dominates, generally indicating native code. And within that, calls to "getService" also dominate, which indicates that much of the time is spent within getService, presumably initializing whatever the requested service is.

In the future, we hope to have hierarchy correctly represented in the inclusive view, as well as adding IO operations as part of that hierarchy. Also, these tools aren't really limited to analyzing startup; they will hopefully form the basis of a set of javascript performance analysis tools that we can apply to any browser operation.

Besides IO and JS, Taras Glek found in earlier examinations of startup that loading CSS/XBL/etc. was taking a significant amount of time.  We're working on instrumenting those parts of the code as well, so that we can capture it along with the raw js/io/etc. portions.

Is there any other data that we should be capturing?  Let us know, and we'll see if we can figure out how to add it in.  I'll keep posting updated data as we have it, and will probably create a web page to collect it all — at that point it'll be open season on any issues that can be identified.

Please direct comments followups to the dev.apps.firefox post.

There's been a lot of interest around the Web Storage spec (formerly part of whatwg HTML5), which exposes a SQL database to web applications to use for data storage, both for online and offline use.  It presents a simple API designed for executing SQL statements and reading result rows.  But there's an interesting problem with this; unlike the rest of the HtML5, this section defines a core piece of functionality in terms of an undefined chunk referenced as "SQL".

The initial implementations of Web Storage are both based on SQLite, and expose the dialect of SQL understood by SQLite to web content.  I'm actually a big fan of SQLite, and was one of the advocates for pulling it into the Gecko platform.  However, SQLite implements a variant of SQL, with a number of deviations from other SQL engines, especially in terms of the types of data that can be placed in columns.

Web content that is created to use database storage with SQLite as the backing is unlikely to work with any other backend database.  Similarly, if another database was chosen as a browser's backing implementation, web content that works with it is unlikely to work with anything else.  This is a serious interop problem, the root of which is that there really isn't a useful core SQL standard.  SQL92 is generally taken as a base, but is often extended or altered by implementations.  Even beyond the parser issues (which could be resolved by defining a strict syntax to be used by Web Storage), the underlying implementation details will affect results.

So, the only option is for the Web Storage portion of the spec to state "do what SQLite does".  This isn't specified in sufficient detail anywhere to be able to reimplement it from the documents, so it would be even worse — "do what this exact version of SQLite does", because there are no guarantees that SQLite won't make any incompatible changes.  For example, a future SQLite 4 may introduce some changes or some new syntax which wouldn't be supported by earlier versions.  Thus, it requires every single browser developer to accept SQLite as part of their platform.  This may not be possible for any number of reasons, not the least of which is it essentially means that every web browser is on the hook for potential security issues within SQLite.

Instead of all of this, I think it's worth stepping back and consider exactly what functionality web developers actually want.  It's certainly much easier to say "well, server developers are used to working with SQL, so let's just put SQL into the client", but it's certainly not ideal — most people working with SQL tend to end up writing wrappers to map their database into a saner object API.

So, I would propose stepping back from Web Storage as written and looking at the core pieces that we need to bring to web developers.  I believe that the solution needs to have a few characteristics.  First, it should be able to handle large data sets efficiently; in particular, it should not require that the entire data set fit into memory at one time.  Second, it should be able to execute queries over the entire dataset.  Finally, it should integrate well with the web, and in particular with JavaScript.

With these needs in mind, I think there are other options that should be considered, even beyond a subset of SQL; for example, an object-oriented database approach might serve those needs better.  A good prototype example of what such a system could look like is jLINQ, which implements client-side querying on JavaScript objects and arrays.  As such, a basic implementation is simple; more complex ones can have browser support for efficient indexing, triggers, rapid serialization to and deserialization from disk, etc.  An implementation could even map all of this on top of an underlying SQL engine.  Another option is something like CouchDB.  I was also just pointed at Persevere, which looks quite cool; much in the same way as CouchDB, the same API could be implemented both client-side and server-side, for efficient online/offline switching.  An approach such as one of these could well serve the web better than just throwing a SQL dialect over the web content fence.

This is a conversation that's worth having, both to figure out what could be done about the issues with directly exposing SQL/SQLite, and also to step back and explore alternate approaches to getting the same functionality in web developers' hands.  The mozilla.dev.platform group is as good a place as any for this conversation, so please post any thoughts you may have there. Hixie rightfully points out that public-webapps is probably a better place for this discussion (which I wasn't even subscribed to for some reason).

An updated version of the prototype Canvas 3D extension is finally available, with a good number of improvements.  Many of these came from Ilmari Heikkinen, who's done some great work in fixing bugs and plugging holes/errors in the implementation.  Also, with this release, support for the OpenGL ES 1.1-based context has been dropped, in order to focus on the more forward-looking ES 2.0-based work.  You can download the new release (0.4.2) here.

There are a few changes in this version from the API mapping and description I posted earlier.  One of the things I mentioned in that original post was changing some functions to try to make them more palatable to JS.  However, no matter how much you munge the GL API, it's still a low-level API.  So, with this version, a bunch of helpers went away, such as uniformf, uniformi, uniformMatrix, and vertexAttrib.  They added significant complexity on the native side, and just confused matters for those who are already familiar with the OpenGL family of APIs.  Instead of those helpers, the GL-like uniform[1234][if][v] and similar should be used.  This doesn't prevent a convenience wrapper from being made, but it should be done on the JS side, not hardcoded into the native implementation.  Additionally, some arguments to functions such as vertexAttribPointer were originally omitted; those are back, and follow the GL prototype.  There are, however, still some changes, particularly in getter functions — the full mapping list is available on the API mapping page.

This release of the extension also adds support for software rendering, using the Mesa GL library.  Software rendering shims are included for all 3 platforms, though only for the 32-bit Intel variants (which is why the extension download is bigger than previous versions).  Software rendering should be used automatically if hardware rendering can't be (due to lack of OpenGL or lack of specific extensions), but you can also set the extensions.canvas3d.software_render pref to true in about:config to force it.  Also, if the version of OSMesa included isn't being found, or if you'd like to try a different OSMesa renderer, extensions.canvas3d.osmesalib can be set to the full path of the library to load.  Note that the functions are assumed to be prefixed with "mgl" as opposed to "gl" when loading this library.

I also went ahead and ported most of the native code examples from the OpenGL ES 2.0 Programming Guide.  These are really API examples, and are not intended to demonstrate what can be done with OpenGL support in the browser.  You can view them here:

• Chapter 2, Hello Triangle
• Chapter 8, Simple Vertex Shader
• Chapter 9, Simple 2D Texture
• Chapter 9, Simple Cubemap Texture
• Chapter 10, Multitexturing

As the examples progress, they build up common functionality and move them into helper js libraries; if those are useful, feel free to use them in your own experiments.  A list of more interesting demos will be coming as soon as I compile one.

Along with this release, I've set up a Canvas 3D group at Google Groups, as the old labs forums are going away shortly.  Please use the forums there for reporting problems (soon to be Bugzilla, but it's not quite set up yet).  I'd like to hear any feedback, or find out about any demos/experiments that may be using Canvas 3D!

Note: I've identified a problem with 0.4.1 that will cause crashes on Windows systems when software rendering is used (either selected or forced).  An updated 0.4.2 with a fix for this problem will be available soon.

Note: 0.4.2 is available at the link above; it fixes the Windows crash and a number of other software rendering problems on other platforms.

I'm pretty excited by the Khronos announcement today.  We've been experimenting with bringing 3D capabilities to the web for some time now, in the form of Canvas 3D, an experimental binding of an OpenGL subset to a HTML5 Canvas context. This work has been going off and on for the past while, mainly as a side project of mine.

For a number of reasons, I think now is the time to bring this out into a wider audience, and to figure out what an initial take of 3D on the web should look like. First, the intense focus on Javascript performance over the past year has seen tremendous improvements across all browsers. Raw language performance is getting to the point where it can keep up with the raw computational requirements of 3D.  It will only continue to improve, spurred on by 3D and other use cases. Second, the hardware required for accelerated 3D is becoming pervasive; hardly any desktop computer ships without some form of hardware acceleration, and the latest crop of smartphones almost uniformly have at least OpenGL ES 1.1, if not 2.0 available.  Starting this work now ensures that a standard will be ready when web developers want to take advantage of the capabilities available in hardware.

Finally, people are doing more and more on the web, and are coming to expect more from the applications that they use.  Web applications already have access to features that have traditionally been reserved for desktop apps, including being able to work while offline, storing data locally, multiple choices for 2D graphics, and native audio & video support.  Adding 3D to this mix ensures that current web apps can experiment with new user experiences, while also enabling new classes of web applications.