If you’ve been reading my blog, you probably know that one of the pieces of the web platform that I’m most involved in is the HTML5 Canvas. It’s a very direct way of getting 2D (and soon 3D) graphics into web pages, and removes many of the barriers between developers and graphics rendering. One of the biggest roadblocks towards adoption of modern web tech such as Canvas is the lack of support in Internet Explorer. ExCanvas was an early attempt at providing Canvas support in IE, but it did so by translating Canvas commands to VML. This worked for simple static stuff, but was pretty slow, and definitely too slow for interactive usage. Microsoft does not support Canvas in IE7, and has stated that it will not be supported in IE8.
For the past few weekends (and one frustrating afternoon trying to work around an IE bug), I’ve been working on a native Canvas implementation for IE based on the same rendering core that’s in Firefox. With an object tag, a bit of CSS, and (to work around another IE bug) a single line of script, <canvas> elements in HTML just work. I’m excited that this experiment is working out, because lack of Canvas support in IE is one of the reasons people skip Canvas and instead turn to Flash and other plugin technologies.
Canvas is just one piece of the full modern web platform, but because it’s so self-contained, it lets us experiment with pushing the web platform forward even for browsers that have fallen behind (or that might not be interested in an open web). However, in IE’s case, the rest of the browser is still IE — most Canvas apps and demos depend on more pieces of the web platform than just Canvas, and won’t run out of the box. (For example, many apps use ‘if (canvas.getContext) { … }’ to see if canvas is really supported; turns out you can’t do that in IE to check for methods on IDispatch-implemented interfaces.) Web developers are already (unfortunately) used to dealing with this, and know how to write code that runs in both modern browsers and in IE. (The ScreamingMonkey project, another experiment similar in spirit to Canvas for IE, is working on bringing a modern JavaScript engine to IE to solve this problem.)
I don’t have binary builds yet, but you can track development in an hg repository. One of the big unresolved questions is how to distribute this thing — it’s what’s called a “binary rendering behaviour” in IE parlance, and is packaged as an ActiveX component (well, it might or might not be ActiveX — I don’t really understand that part of the equation). Currently, the experience is pretty crappy: you have to click through an infobar to allow installation of this component, then you have to click “Yes” to say that you really want to run the native content, and then you have to click “Yes” again to allow the component to interact with content on the page. This sucks. In theory, with the right signatures, the right security class implementations, some eye of newt, and a pinch of garlic, it’s possible to get things down to a one-time install which would make the component available everywhere.
Implementation-wise, there are still many missing pieces — drawImage, get/put/createImageData, text, shadows, patterns, and gradients aren’t implemented — but with the exception of drawImage, it’s all pretty straightforward stuff, and the core path stroking/filling/compositing works. drawImage will take a bit of work, since I don’t think it’s possible to pull out the decoded image from IE (anyone know how to get a BITMAP or HDC from a IHTMLImgElement?)
We’ll see how this works out; if successful, other pieces of the modern web could be retrofitted onto IE; <video> and <audio> come to mind. I’ll be working on this as I have time, and will post updates here… don’t expect super fast progress, but I hope to have a full implementation by the end of the year.
52 Comments to “No Browser Left Behind”
- 1 Pingback on Jul 30th, 2008 at 2:07 pm
- 2 Pingback on Jul 31st, 2008 at 12:00 am
- 3 Pingback on Aug 1st, 2008 at 3:35 pm
- 4 Pingback on Aug 2nd, 2008 at 8:23 pm
- 5 Pingback on Aug 7th, 2008 at 10:14 pm
- 6 Pingback on Aug 8th, 2008 at 5:57 am
- 7 Pingback on Aug 11th, 2008 at 4:32 pm
- 8 Pingback on Aug 19th, 2008 at 3:31 pm
- 9 Pingback on Aug 20th, 2008 at 4:48 pm
- 10 Pingback on Aug 21st, 2008 at 2:18 am
- 11 Pingback on Aug 26th, 2008 at 7:07 pm
- 12 Pingback on Aug 27th, 2008 at 1:29 pm
- 13 Pingback on Sep 9th, 2008 at 11:38 am
- 14 Pingback on Sep 10th, 2008 at 5:09 am
- 15 Pingback on Sep 21st, 2008 at 2:21 pm
- 16 Pingback on Oct 8th, 2008 at 1:41 am
This seems amazing.
But still, I wonder, even if you get it to install in the optimal way (one approval for all time), how would a website developer use this (and screaming monkey, etc)? Would you get the control housed in a central site and developers would point to that when they use the CANVAS tag? Would you urge OEMs to install it? Install it whenever FF (or Safari, or Opera) are installed on Windows? Get the EU to force MS to bundle it :) ? It does seriously lower the barrier to use CANVAS, but the bar does remain high.
IHTMLElementRender?
http://msdn.microsoft.com/en-us/library/aa752274(VS.85).aspx
I’m a bit rusty on this stuff…
Nice job on this… I’ve been hoping for this to happen for a long time :)
@Jamie:
But still, I wonder, even if you get it to install in the optimal way (one approval for all time), how would a website developer use this (and screaming monkey, etc)?
In different ways — for this, they’d have to embed an object tag that references a GUID that would load the factory that would handle canvas (and other) elements. For ScreamingMonkey, they have info on their site — basically via different language in the script tag.
Would you get the control housed in a central site and developers would point to that when they use the CANVAS tag?
Yep, likely.
Would you urge OEMs to install it? Install it whenever FF (or Safari, or Opera) are installed on Windows? Get the EU to force MS to bundle it :) ? It does seriously lower the barrier to use CANVAS, but the bar does remain high.
Doubtful on any of those. If someone’s installing a modern browser, it doesn’t make any sense to enhance IE, no?
The only other possibility I can think of would be using IViewObject on the IHTMLDocument, but it could only capture a visible image. I don’t think that IHTMLImgElement supports this interface.
You might also be able to use OleLoadPicture, passing in stream you get from the URL from nameProp. This should resolve through the IE cache, avoiding a second load. The IPicture interface has lots of goodies on it.
Wow – so you’re actually working for Microsoft now. ;) Problem is if your plugin receives widespread deployment, then there isn’t any incentive for Microsoft to implement the feature themselves…
An open source IE browser plugin might be an interesting idea though. Particularly if, as you say, it will support more than just canvas.
I also remember Mark Finkle working on using Mozilla’s engine via XULRunner within IE to get XUL and SVG and maybe XHTML usable in IE. However, including the entire Mozilla rendering engine is a vastly different approach than yours.
Turns out grabbing IHTMLElementRender from the IHTMLImgElement and doing a DrawToDC seems to work great; that was easier than I thought! Amazingly, at least under IE7, transparency is even preserved, so transparent PNGs just work.
Oh, here’s that link to Mark Finkle’s Mozilla Plugin for IE
This is really exciting. I hope we can just do this one day to get , and :
What’s the current release-mode size of the DLL?
Nice.
Very nice work.
> Problem is if your plugin receives widespread deployment, then there isn’t any incentive for Microsoft to implement the feature themselves…
I think the opposite will happen, widespread deployment might guilt Microsoft into implementing canvas. Their browser will seem incomplete (I already think it’s incomplete, but it will just make it more obvious for the rest)
Coolness, Firefox still ROCKS.
JT
http://www.FIreMe.To/udi
Superb! I’m giving a talk on in less than a week (http://refreshbmore.org/), it’ll be nice to mention that this will hopefully exist.
despite your good intention, I think you should not help Micro$oft to improve their crappy browser.
Any chance of getting this thing into IE version of [Google] Gears?
Why only canvas? I’d like entire gecko as ActiveX!
Good luck with this, if you pull it off eternal love is yours :)
For Bomomo.com, which was originally written in Canvas, I started off by using ExCanvas, noticing it was much too slow, and then my friend played around trying to use the Flash API. This would have meant the JavaScript would not need to be rewritten, there would just be a script overriding the JS/ Canvas methods to forward to the Flash via the Flash JS API. Unfortunately, the bottleneck was the JS-Flash communication, so he ended up having to rewrite my whole JS code in ActionScript anyway. The upside with Flash though is that it’s widely deployed already… a working JS/ Canvas to Flash/AS converter might be neat :)
… oh, and why doesn’t Microsoft do the right thing and deliver Canvas support for upcoming IEs? I’d take it any time over improvements like hiding the IE menu bar (which I personally don’t like) or passing another Acid test (not that all these are mutually exclusive).
Sounds like the sort of thing that could be a nice addition to something like BrowserPlus from Yahoo (http://browserplus.yahoo.com/) – or at least benefit from a similar type of installation.
Great idea! I’m looking forward to your success!
Good news. I’ll be happy to update ExCanvas to correctly detect this plugin and use that instead of the VML emulation. Personally I would love if this could get into Gears so that there is only one plugin that needs to be installed.
Interesting. Is the clearRect method implemented? It is my missing feature of the excanvas.
I want to second Erik’s suggestion: this belongs in Gears. They’ve got a distribution mechanism, a growing user-base, and an open process.
Regards
This along with Screaming Monkey and the video/audio stuff is absolutely fantastic way to get some sanity into IE.
What would be ideal is for those three packages to be distributed in one package. Some kind of IE enhancement pack under the Mozilla or Google (or both) brand (I’m personally rather fond of Screaming Monkey :-) ).
What would be even more ideal would be for MS to actually bundle this work with IE by default (it’s all LGPL as far as I know) with support contracts with Mozilla corp. I don’t see any reason for them not to do that, other than short sightedness or emotional constipation on the part of MS managers. They are willing to bundle other licensed work, I don’t see why this would be any different.
Great job! Canvas is very userfull and this project can help.
IE/Microsoft sometimes blocks the evolution of WEB…
>I think the opposite will happen, widespread deployment might guilt Microsoft into implementing canvas.
Unlikely, as I imagine Microsoft sees this as a threat to their investment in Silverlight. Then again, if they saw it as another way to dethrone Flash…
>despite your good intention, I think you should not help Micro$oft to improve their crappy browser.
Most people continue to use IE, primarily because it’s the default browser in Windows. If IE enhancements like this make it easier for site authors to stick with open standards rather than resorting to Microsoft-specific solutions to address the majority of viewers out there (which I believe is one of the primary goals of Firefox) then this is a very good thing.
Vladimir, can you say some more about how canvas element will work in IE after implementing ScreamingMonkey project? I mean how Tamarin adds canvas support, via flash or native or other way? Thanks.
“Why only canvas? I’d like entire gecko as ActiveX!”
That’s already possible, I have it setup on my system so that when IE encounters a SVG (and XUL actually) file (via or opening it up normally) it loads Gecko and displays the SVG file properly.
It’s not the nicest thing to get working (you need to download the files and stash them somewhere, then install registry information), but a pre-packaged form would be a nice replacement for the old Adobe SVG viewer.
Is there a good reason why this wouldn’t just be better as a wrapper around Flash, as Manish Jethani points out?
It really seems like a huge hassle to ask users to take yet another binary plugin, when Flash 9 is already available on, what, 98% of the world’s PCs including all three of Win, Mac, Linux?
Dave
@Dave: It’s not just a wrapper around Flash because I don’t think it’s possible to do that in a transparent way (as far as handling the canvas element is involved), and because there are features that I plan on adding to this extension that Flash can’t support.
Hi Vladimir,
I have a very trivial question: had we autostereoscopic displays, would we see depth with all 3D graphics?
VMLR, X3D, OpenGL?
I have heard of 3D primitives that contain all 3D info necessary for stereoscopic display, no matter if due to the display we have we just see the monopictorial cues effect.
Can you help?
Thanks,
Margherita
God, it must suck to be an IE developer and read this article :)
I love this approach, let’s apply more openness to slow moving closed circuit software. Personally, I can’t weight to build canvas based applications.
Nice job,
Alex
I can answer how people would use this… I’ve been working on a 2D game engine written entirely in Javascript. Yeah, I know many others have done it, tried it, attempted it, but I intend to get it to a completion point around year’s end as well. Most of the stuff I’ve been able to pull off is rendered via the Canvas element. I abstracted the rendering context so that any accessible context could be used, but Canvas is (by far) the most accessible and complete in the majority of browsers.
People like gaming, they like web browsers, and they most certainly like what Canvas can do. I’ve tried to keep the engine as OO as possible so that it’s extendable and accessible on as many platforms as possible. It (was) running in Firefox 2.5 , Opera 9 , Safari 3 , on the iPhone, and now you’ve potentially made it possible to game in IE. If I could ever figure out the lib loading mechanism on Wii, then games could be developed there as well. A recent change has rendered it only operating in Firefox, but I’m working on that currently.
Being able to write a game once, without having to purchase, pirate, or learn Flash, would (I believe) make a lot of people happy. Then being able to play those games, without any changes, in any environment would most certainly be a great proposition.
Thanks for this work… I hope to see a binary version soon so I can get this working with the engine.
My website is the project site. Take a look, leave me a comment. =)
So, if I want to actually test this in my copy of IE, what would I have to do?
Vladimir i wish you all the luck with this, and really really hope you can get it to work without losing too much hair :)
It’s obviously a big deal, getting canvas in IE.
M$’ behaviour on this issue really pisses me off, but politics aside:
if u succeed it would be great for creativity.
i’m see great potential for canvas anims in my CMS, and have just begun intergrating canvas anims into my CMS with google’s excanvas. I’d hate to have IE ‘regress to animated gifs and less visual gimmicks overall.
hope you keep us informed of your progress, it would be good to know how hard it is to fix.
i’m working on my CMS (c link) but can make time available to work on this component too..
and i’d work 4 free, u keep the copyright completely..
Hey Vladimir!
I keep checking back hoping for more good news. Have you been able to make any progress? We’re all rooting for you!
One idea/suggestion I had was that the Flash runtime ships with a fairly elaborate javascript package that does all of the browser detection and embedding. Somehow, it is able to magically insert Flash movies into the DOM without forcing the user to click. Couldn’t you borrow and ultimately adapt the techniques used in this approach for use with your plugin?
It could be as simple as changing the name of the file it attempts to load.
Pete