CanvasArrayBuffer and Canvas*Array
6 Comments Published by vladimir November 6th, 2009 in Canvas 3D, Firefox, MozillaWebGL introduces two interesting concepts that I think have application outside of WebGL, the CanvasArrayBuffer and CanvasArray WebGLArrayBuffer and WebGLArray. This is all subject to change, of course, though this is what the current Gecko (and others') implementation looks like. In particular, the Canvas prefix in the names might change soon.
Edit 12/2: these types have changed names; they now have a WebGL prefix instead of a Canvas prefix.
It became clear that pure JS arrays are not a useful way of shoveling around lots of 3D data; their very flexibility makes them impractical for performance-critical uses. In particular, WebGL often wants to deal with arrays of a specific type -- an array of integers, an array of floats, etc. Even more complicated is the need to manage multiple types within a single memory region; for performance, it's often preferable to allocate one chunk of video memory, and place coordinates, colors, and other types in there, replacing them as necessary.
There are two portions to the solution: the WebGLArrayBuffer and a set of typed WebGLArray views. A WebGLArrayBuffer represents chunk of data. It can be allocated with a size in bytes, but it can't be accessed in any way. To actually manipulate the data inside a WebGLArrayBuffer, a WebGLArray has to be created that references it. An example:
var buf = new WebGLArrayBuffer(3*4); var floats = new WebGLFloatArray(buf); floats[0] = 12.3; floats[1] = 23.4; floats[2] = 34.5;
The above chunk of code allocates a 12-byte WebGLArrayBuffer, and then creates a float-typed view onto the buffer which can then be manipulated (almost) like a normal array. Of course, the above is cumbersome to write, so there are shorthands that will allocate a WebGLArrayBuffer, and optionally fill it with data from a JS array:
var f1 = new WebGLFloatArray(3); var f2 = new WebGLFloatArray([12.3, 23.4, 34.5]);
The size of each WebGLArrayBuffer is fixed; there is currently no way to change its size once allocated.
Multiple WebGLArrays can reference the same WebGLArrayBuffer. For example:
var buf = new WebGLArrayBuffer(12*3*4+12*4); var points = new WebGLFloatArray(buf, 0, 12*3); var colors = new WebGLUnsignedByteArray(buf, 12*3*4, 12*4);
This creates a buffer of 192 bytes, which is enough room for 12 3-coordinate float points followed by 12 RGBA colors, with each component represented as an unsigned byte. The arguments to the WebGLArray constructors are the offset from the start of the buffer (in bytes), and the length (in elements). The offset must always be a multiple of the element size (to preserve alignment), and the buffer must obviously be large enough for the given offset and length. If length is not given, the length is assumed to be "from offset until the end of the array buffer"; that value must be a multiple of the element size. If offset is not given, it's assumed to be zero.
For extra complex use cases, WebGLArrays can reference overlapping regions of a WebGLArrayBuffer:
var buf = new WebGLArrayBuffer(192); // same value from above var points = new WebGLFloatArray(buf); var colors = new WebGLUnsignedByteArray(buf); points[0] = 12.3; points[1] = 23.4; points[2] = 34.5; colors[12] = 0xff; colors[13] = 0xaa; colors[14] = 0x00; colors[15] = 0x00;
In the buffer, this writes 3 float values followed by 4 byte values. You'll note that this use is significantly more complex, and requires the user to keep track of the current position in terms of whatever element they're modifying (thus setting array elements 12, 13, 14, and 15 for the color).
If an attempt is made to store data in a WebGLArray that doesn't fit within the right type, a C-style cast is performed. If the data is an entirely wrong type (e.g. trying to store a string or an object), Gecko currently throws an exception, but this might become a silent 0 or similar in the future.
Where does this fit in WebGL? Any API function that needs an array of data takes a WebGLArrayBuffer. This avoids placing costly JS array type conversion in a potential critical performance path, and simplifies a number of aspects of the API. So, VBOs, texture data (if not loaded from a DOM image element or from a CanvasImageData object), index array, etc. all use WebGLArrayBuffers/WebGLArrays for pulling data in and out. WebGLArrays also help manage memory usage -- an array of byte color data now takes up exactly as much memory as needed, instead of getting expanded out to 4 bytes. Also, critically, floating point data can be stored as 32-bit single-precision floats instead of 64-bit doubles, taking up half as much space when the underlying graphics system can't support 64-bit values.
This API is overall lacking in developer niceties, since the focus was on providing the necessary functionality. Higher level wrappers can be written in JS to simplify usage. In addition, by keeping it as bare-bones as it is, it allows for fast implementation on native hardware via the JITs in all the current-generation JS engines. The web currently fudges around the problem of binary data by passing it around either in strings (because JS strings are UCS2, therefore all 8-bit elements are valid, but with a performance and memeory cost), or often encoding as base64 (again going back to strings). I can see this type of dense/native type access being useful for both the File and WebSockets APIs as a way to exchange and deal with binary data.
6 Comments to “CanvasArrayBuffer and Canvas*Array”
- 1 Pingback on Nov 9th, 2009 at 6:47 am
- 2 Pingback on Nov 13th, 2009 at 8:48 pm
Vlad,
Many thanks for the explanation, that clears up some discussion we’ve been having on the WebGL forums: http://www.khronos.org/message_boards/viewtopic.php?f=45&t=2163&p=5404
However, as one of the guys in the forum said, it looks like there’s still a little way to go in Minefield; if you run:
var f2 = new CanvasFloatArray([12.3, 23.4, 34.5]);
alert(f2[2]);
…the alert box says “undefined”. It works as you’d expect in Chrome.
(Happy to put in a bug report if that would help.)
Cheers,
Giles
I actually like the sound of this. One of my problems with the Canvas API (as a video game level editor author) is that using a canvas ImageData / CanvasPixelArray to fill a large pixel buffer with random pixels (like a complete SNES level will do) is much slower than it ought to be. Filling up a “CanvasUnsignedWordArray” or whatever the unsigned 32-bit data type is, would be much faster than writing 4 bytes to a CanvasPixelArray per pixel.
Wherever the CanvasArrayBuffer API goes, I hope (please! please!) that it will also be usable directly with putImageData() and friends.
This will be a great help to c3dl developers. Thanks.
The example about overlapping arrays to a single buffer makes a case for having a byteStride for the WebGLArrays. I mean, what you’d like to write is “points[i*3] = foo; normals[i*3] = bar; texcoords[i*2] = qux;” where points, normals and texcoords all point to the same ArrayBuffer.
Well, what I’d actually like to write is “points[i] = vec3(1,2,3); normals[i] = vec3(1,2,3); texcoords[i] = vec2(1,2);”, but that’s orthogonal to byteStride.