HTML5 Canvas for mobile apps

Recently I’ve been getting up to speed with HTML5 Canvas. Along with other HTML5 technologies ( i.e. geolocation, database) I can see real potential for developing interactive mapping applications using just HTML5 and Javascript  technologies. I’ve already done some experiments that attempt to simulate feature control using Canvas. (An effort inspired by the pixastic project.)

I was very excited then to discover support for HTML5 Canvas has been growing for mobile browsers as well as desktop browsers.  Could this open the way forward for high fidelity mobile maps that do not rely a on super fast data connection? It is certainly the case that managing application state on the client is more scalable and  consistant with the stateless architecture of the web. And of course a web application is more portable across different devices, well in theory, at least.

So I set out to see if I could port a simple Canvas application that allows a user to draw freehand (or free finger?) on an image with Opera Mobile and iPhone Safari. The desktop version of the app works on Firefox 2, Safari, Opera and even recent versions of IE.  IE does not itself  support Canvas but the brilliant excanvas.js manages to port the Canvas API to IE VML, a reasonable workaround until Microsoft finally get fully onboard the HTML5 omnibus . Watch out though using the onmouseout event on the canvas object – IE triggers this event everytime you leave a VML cell. After a lot of head scratching I eventually found replacing onmouseout with Microsoft’s onmosueleave solved the problem.

As you can see viewing the page source, using the canvas API itself is straightforward. Simply add an HTML canvas tag to the body, being sure to supply alternative text for non compliant browsers.

<canvas id="canvasimg" width="250" height="250">
This demo uses the HTML canvas object. You'll need a browser such as Firefox 2 or Safari to use the demo.
</canvas>

Then initialize the canvas with the image you want to draw on.

var img = new Image(255,255) ;
img.src = "http://.../testimg/test1.png"  ;
var newCanvas = document.getElementById("canvasimg") ;
var canvasContext = newCanvas.getContext("2d") ;
var newimg = canvasContext.drawImage(img, 0, 0);

Then implement a method to handle the mousemove event on the canvas object.

if ( drawMode == true )
{
var xy = getxy(e, newCanvas ) ;
c.lineWidth = 1;
c.strokeStyle = '#000';
c.fillStyle = '#00A308';
c.tertStyle = '#DDD';
c.strokeFill = 1; //outline shapes
c.fillRect(xy.x,xy.y,5,5) ;
return ;
}

The only thing I struggled with was getting the mosue position relative to the Canvas object . The getxy() function achieves this using some code I lifted from the awesome CanvasPaint app. One thing I added though is yet another catch for IE. It seems IE only provides values for the clientX and clientY property on window.event. If you pass in another event such as mousemove you get null. The following line catches this ensuring the getxy() function works for IE aswell.

var ev = e || window.event;

Interestingly, getting the finger position using the iPhone touch API was much easier and I was able to drop the getxy() function. More on that below.

Getting the app to work on the mobile browsers did present a challenge, mostly due to the default handling of finger touch gestures and mapping these to standard Javascript events.  For standards compliant Opera Mobile at least, the code was exactly the same as it was for the desktop browser. Standard Javascript events such as mousemove and mouseup were interpreted as you would expect.  Well at least they would have been if it had not been for the infuriating Text Selection toggle. It seems that while the Text Selection mode is toggled Off the touch events perform default behaviour which is scrolling for mousemove and opening the context menu for mousedown. You would think the event.preventDefault() function would cancel this behaviour ( as it does for iPhone Safari) but that did not work for me. I tried some other techniques such as adding listeners to the window node and calling event.stopPropagation but eventually gave up figuring life was too short. This means that for the Opera the user has to manually switch on Text Selection mode before they can do drawing on the image. A similar problem seems to occur on the Android Webkit browser, except it is much worse as the “Text Select” toggle only lasts for one touch before reverting back to default behaviour. More investigation required, I think.

For iPhone the standard mosue events are replaced with iPhone touch events – e.g. touchmove for mousemove, touchstart for mousedown etc. The only problem was that there was no obvious way to simulate mouseout. Getting a fix on the finger position was much easier using the touch API. I could get rid of the bulky getXY() function and simply use the targetTouches array that API helpfully includes as a property of the event:

c.fillRect(e.targetTouches[0].pageX ,e.targetTouches[0].pa
geY,5,5) ;

Otherwise the code was similar to the desktop version with e.preventDefault() blocking scrolling and zoom behaviour when the user is drawing on the image. I found that the Android WebKit browser also worked better using Apple’s touch API rather than regualr mouse events, which was a surprise. However the e.preventDefault didn’t seem to help with the Text Select problem in Android.

The screenshot above illustrates the main problem with using HTML5 canvas on mobiles. Performance. Even running on relatively high spec iPhone 3G ( 600Mhz) processor the draw line is a bit grainy. In this example, the issue is more likely due to the way Safari implements touchmove (i.e the frequency of the event being triggered as the finger moves ) than clock speed. When the image is zoomed up the line is less grainy suggesting the event handling could be optimized. But on slower devices such as the 201Mhz XDA Orbit the processing speed is problematic even for this simple application. For more complex applications that manipulate every pixel in the image, such as the feature selection app shown below, even the iPhone struggles. To get reasonable performance on the iPhone 3G I had to switch from touchmove to touchend to reduce the frequency of redraws.

Summing up, I think canvas on mobile browsers is an exciting prospect, but not yet easy to port across devices with different browsers and specs. By the way, I never managed to get the IE version working on Windows mobile.