WebGL introduces two interesting concepts that I think have application outside of WebGL, the CanvasArrayBuffer and CanvasArray WebGLArrayBuffer and WebGLArray. This is all subject to change, of course, though this is what the current Gecko (and others’) implementation looks like. In particular, the Canvas prefix in the names might change soon.
Edit 12/2: these types have changed names; they now have a WebGL prefix instead of a Canvas prefix.
It became clear that pure JS arrays are not a useful way of shoveling around lots of 3D data; their very flexibility makes them impractical for performance-critical uses. In particular, WebGL often wants to deal with arrays of a specific type — an array of integers, an array of floats, etc. Even more complicated is the need to manage multiple types within a single memory region; for performance, it’s often preferable to allocate one chunk of video memory, and place coordinates, colors, and other types in there, replacing them as necessary.
There are two portions to the solution: the WebGLArrayBuffer and a set of typed WebGLArray views. A WebGLArrayBuffer represents chunk of data. It can be allocated with a size in bytes, but it can’t be accessed in any way. To actually manipulate the data inside a WebGLArrayBuffer, a WebGLArray has to be created that references it. An example:
var buf = new WebGLArrayBuffer(3*4); var floats = new WebGLFloatArray(buf); floats = 12.3; floats = 23.4; floats = 34.5;
The above chunk of code allocates a 12-byte WebGLArrayBuffer, and then creates a float-typed view onto the buffer which can then be manipulated (almost) like a normal array. Of course, the above is cumbersome to write, so there are shorthands that will allocate a WebGLArrayBuffer, and optionally fill it with data from a JS array:
var f1 = new WebGLFloatArray(3); var f2 = new WebGLFloatArray([12.3, 23.4, 34.5]);
The size of each WebGLArrayBuffer is fixed; there is currently no way to change its size once allocated.
Multiple WebGLArrays can reference the same WebGLArrayBuffer. For example:
var buf = new WebGLArrayBuffer(12*3*4+12*4); var points = new WebGLFloatArray(buf, 0, 12*3); var colors = new WebGLUnsignedByteArray(buf, 12*3*4, 12*4);
This creates a buffer of 192 bytes, which is enough room for 12 3-coordinate float points followed by 12 RGBA colors, with each component represented as an unsigned byte. The arguments to the WebGLArray constructors are the offset from the start of the buffer (in bytes), and the length (in elements). The offset must always be a multiple of the element size (to preserve alignment), and the buffer must obviously be large enough for the given offset and length. If length is not given, the length is assumed to be “from offset until the end of the array buffer”; that value must be a multiple of the element size. If offset is not given, it’s assumed to be zero.
For extra complex use cases, WebGLArrays can reference overlapping regions of a WebGLArrayBuffer:
var buf = new WebGLArrayBuffer(192); // same value from above var points = new WebGLFloatArray(buf); var colors = new WebGLUnsignedByteArray(buf); points = 12.3; points = 23.4; points = 34.5; colors = 0xff; colors = 0xaa; colors = 0x00; colors = 0x00;
In the buffer, this writes 3 float values followed by 4 byte values. You’ll note that this use is significantly more complex, and requires the user to keep track of the current position in terms of whatever element they’re modifying (thus setting array elements 12, 13, 14, and 15 for the color).
If an attempt is made to store data in a WebGLArray that doesn’t fit within the right type, a C-style cast is performed. If the data is an entirely wrong type (e.g. trying to store a string or an object), Gecko currently throws an exception, but this might become a silent 0 or similar in the future.
Where does this fit in WebGL? Any API function that needs an array of data takes a WebGLArrayBuffer. This avoids placing costly JS array type conversion in a potential critical performance path, and simplifies a number of aspects of the API. So, VBOs, texture data (if not loaded from a DOM image element or from a CanvasImageData object), index array, etc. all use WebGLArrayBuffers/WebGLArrays for pulling data in and out. WebGLArrays also help manage memory usage — an array of byte color data now takes up exactly as much memory as needed, instead of getting expanded out to 4 bytes. Also, critically, floating point data can be stored as 32-bit single-precision floats instead of 64-bit doubles, taking up half as much space when the underlying graphics system can’t support 64-bit values.
This API is overall lacking in developer niceties, since the focus was on providing the necessary functionality. Higher level wrappers can be written in JS to simplify usage. In addition, by keeping it as bare-bones as it is, it allows for fast implementation on native hardware via the JITs in all the current-generation JS engines. The web currently fudges around the problem of binary data by passing it around either in strings (because JS strings are UCS2, therefore all 8-bit elements are valid, but with a performance and memeory cost), or often encoding as base64 (again going back to strings). I can see this type of dense/native type access being useful for both the File and WebSockets APIs as a way to exchange and deal with binary data.