On Converting Flash to HTML

I received a question from Bob (no, really), who wrote:

I have a question about the newest version of Flash and its HTML publishing option using CreateJS. What do you think of that approach going forward?

I started to write an email response but figured I should probably post it here.

I haven’t been paying much attention to Flash, so I don’t know what the ‘HTML export’ is capable of these days. In general, I’m very wary of converting Flash-based projects to HTML. When Adobe Captivate first released a “publish to HTML5” feature, all it did was convert the SWF animation to a video file, losing all interactivity along the way.

The limitations of the browsers and the HTML5 spec mean you can’t expect a fully 1:1 conversion from Flash to HTML, regardless of libraries like CreateJS. Some of the features found in Flash are still not quite supported in browsers, or might not work quite the way you’d expect. For example, CSS transitions, CSS gradients, and SVG/Canvas support vary widely from browser to browser (though it’s getting better, and there are workarounds aka “polyfills”). Streaming video, which is a breeze in Flash, is not part of the HTML5 Video spec (yet) and is unsupported in browsers. Video and audio codec support is inconsistent. In some cases, devices add new limitations — last time I checked, iOS devices wouldn’t autoplay audio or video in Safari. ‘Play’ couldn’t be scripted, it required the user to press a button.

Publishing to HTML (with the aid of JavaScript libraries like CreateJS) is definitely the way of the future, but I would flip the workflow: use a tool that’s “HTML first”. For my workflow, I always start in HTML then only fall back to Flash if I absolutely have to. I hand-code, but if you want to stick to a WYSIWYG editor, maybe try some of the Adobe Edge products, or go with a third-party product such as Tumult Hype.

If you continue to use Flash as an HTML development tool, temper expectations and test widely, as some things might not work the way you expect when converted to HTML.

And regardless of whether you publish to Flash or HTML, always test the accessibility of your project. Just because it’s HTML doesn’t mean it’s accessible; HTML by nature is more accessible than Flash, but libraries like CreateJS add a lot of complexity to the page, which can easily impact accessibility.

Advertisements

HTML5, Flash, Silverlight, and your courseware

What a busy week.

Flash is dead. Sort of, but not really.

In case you haven’t heard, Adobe formally announced the discontinuation of Flash Player for mobile devices (“Flash to Focus on PC Browsing and Mobile Apps; Adobe to More Aggressively Contribute to HTML5“). Adobe employees struggled to come to grips with what has undoubtedly been a tough week for them — aside from the product news, they were also informed of massive layoffs (around Adobe 750 employees). Regardless of your feelings about Flash, your heart must go out to the families affected by a sudden job loss.

Flash critics were quick to declare Flash dead. But in the immortal words of Monty Python’s Eric Idle, “I’m not quite dead yet.”

Yes, Adobe is scaling back on Flash to focus more energy on so-called HTML5 technologies (“so-called” because many of the technologies involved are not actually part of the HTML5 spec.) However, their public announcement was that mobile Flash is dead, and Flash on the desktop will continue to thrive.

Flex is being abandoned

Then the “Official Flex Team Blog” dropped a bombshell: Adobe Flex is going open-source:

Yes. We know Flex provides a unique set of benefits for enterprise application developers. We also know that the technology landscape for application development is rapidly changing and our customers want more direct control over the underlying technologies they use. Given this, we are planning to contribute the Flex SDK to an open source foundation in the same way we contributed PhoneGap to the Apache Foundation when we acquired Nitobi.

Okay. Flex is being open-sourced. To be honest, this isn’t really shocking news to me since Flex has mingled with Eclipse for a long time and has had its toes in open source for ages. The part that made my jaw drop was in the next section:

Does Adobe recommend we use Flex or HTML5 for our enterprise application development?
In the long-term, we believe HTML5 will be the best technology for enterprise application development. We also know that, currently, Flex has clear benefits for large-scale client projects typically associated with desktop application profiles.

Given our experiences innovating on Flex, we are extremely well positioned to positively contribute to the advancement of HTML5 development, starting with mobile applications. In fact, many of the engineers and product managers who worked on Flex SDK will be moving to work on our HTML efforts. We will continue making significant contributions to open web technologies like WebKit & jQuery, advance the development of PhoneGap and create new tools that solve the challenges developers face when building applications with HTML5.

In the long-term, we believe HTML5 will be the best technology for enterprise application development. For those of you unfamiliar with Flex, it’s Adobe’s ‘enterprise-level’ Flash development path. Designers use Flash Professional and the timeline, while programmers use Flex and its pure ActionScript/MXML environment. This line is basically saying “see ya, Flex, was nice knowing you. We’re putting our best guys — the ones who weren’t laid off — on the ‘HTML5’ train.”

By extension, if Adobe is wiping its hands of Flex, it means they’re seriously scaling back support for the entire Flash ecosystem.

Whoa. The anti-Flash crowd really does have reason to celebrate. But not so quick, my friends: Support for the HTML5 environment is nowhere near Flash’s level of ubiquity, it will take a couple of years to get there, and even then, I doubt it will have the full capabilities Flash offers today.

But does it matter? Let me come back to that in a minute.

Silverlight is dead. Sort of, but not really

For now, lets turn our attention to our friends in Redmond, WA. While Adobe created this week’s loudest thunder, Microsoft made a few waves itself. If you recall, about a year ago one of Microsoft’s presidents (yes, they have more than one) caused a stir when he said “HTML is the only true cross platform solution for everything, including (Apple’s) iOS platform”. Microsoft quickly issued a statement backtracking from the implication that Silverlight was on its way out, and stressed their commitment to Silverlight.

A year later, the circus is back in town. According to some industry sources, Microsoft is planning to stop development of future editions of Silverlight; version 5 may be the last hurrah, except for security updates. Stop me if this sounds familiar.

However, this time around, the rumor sounds much more feasible because of Internet Explorer 10.

Microsoft’s Internet Explorer, that grand ol’ browser, is going through a major metamorphosis. In general, Internet Explorer 10 will behave less like its ancestors and more like Firefox and Webkit (Safari/Chrome). For example, IE 10 will not support Microsoft’s long-standing conditional comments. This is a big deal for web developers, as conditional comments have become a major crutch when dealing with Internet Explorer. But the biggest news I’ve heard about IE 10 thus far is that it will not support plugins — Flash, Silverlight, Quicktime, etc. — when running in “Metro” mode. Microsoft’s Windows 8 team believes that removing plugins from IE will result in improvements across-the-board: security, reliability, privacy, battery life in mobile devices, etc.

For the web to move forward and for consumers to get the most out of touch-first browsing, the Metro style browser in Windows 8 is as HTML5-only as possible, and plug-in free. The experience that plug-ins provide today is not a good match with Metro style browsing and the modern HTML5 web.

Running Metro style IE plug-in free improves battery life as well as security, reliability, and privacy for consumers. Plug-ins were important early on in the web’s history. But the web has come a long way since then with HTML5. Providing compatibility with legacy plug-in technologies would detract from, rather than improve, the consumer experience of browsing in the Metro style UI.

Plugins will still be supported in the non-Metro environment, but it appears Microsoft’s way of the future is Metro, and Metro is HTML5-based, with no plugins allowed, just like Apple iOS. Microsoft is clearly serious about leaving plugins behind, including its own Silverlight Player.

The analysis

Flash and Silverlight are not dead, but are being minimized by their owners. If you read the articles closely, you’ll see a pattern: both Adobe and Microsoft hope to port some of the technology from their plugins to the open web, aka the “HTML5 ecosystem”.

As it currently stands, browser technology and HTML5 ubiquity will take a number of years to catch up to the baseline capabilities of Flash and Silverlight. Even then, I doubt browsers will have some of the advanced capabilities Flash offers today.

The thing is, I don’t think most Flash developers take full advantage of Flash’s capabilities; they seem to use the same basics they’ve been using for years. The cutting-edge stuff is nowhere near as common as the mundane: video players, slideshows, and simple games.

Today’s editions of Firefox, Chrome, Safari, and even Internet Explorer 9 are massively capable browsers. JavaScript, canvas, and CSS3 can be used to replicate many of Flash’s features. The real question becomes: what are you trying to do?

Video

The vast majority of Flash usage appears to be for serving streaming video. HTML5’s video support is not quite ready to replace Flash in this regard, but it’s getting pretty close. The codec wars are the biggest stumbling block, followed by support for DRM and perhaps a handful of advanced features. If you have a simple video that doesn’t need DRM and can be served via two codecs (H264 and OGG or WebM), you can proceed directly to Go, collect $200.

Need to add some interactivity to your movie? Canvas and JavaScript can do an awful lot. Projects like WebGL and Mozilla’s popcorn.js are upping the ante. The future looks pretty bright for native online video.

Gaming

Flash-based gaming is another strong user base. My kids have been using NeoPets and similar sites for years, so I understand just how far and deep Flash gaming goes in our community. But some of these games are deceptively simple with their graphics, and I wouldn’t be surprised if Adobe’s HTML5 tools soon enable Flash game developers to port their games to the HTML5 ecosystem.

Apple’s iOS and Facebook have created big-money opportunities for game developers willing to switch gears to HTML5. We’ve already seen JavaScript/HTML ports of Nintendo games, including the best video game ever made: Super Mario Brothers. They work. They’re fun. Yes, it will take a long time to get market saturation, but HTML5 already has its foot in the door and a lot of money behind it — gaming is big business!

E-Learning

If you’re reading this blog, chances are good that you’re an e-learning developer. You probably also know that most major e-learning rapid development software outputs Flash SWFs. I can’t think of any industry aside from online gaming that is so utterly dependent on Flash at the moment. This has worried me for years — I’ve long preached that e-learning developers should be less reliant on Flash.

When Apple’s iPad was released without Flash support, the outcry from the e-learning industry was fierce. I also feel it was misplaced — it clearly showed the industry’s reliance on a plugin, and rather than focus on removing the Flash requirement barrier, many people chose to plant their feet in the ground and buy non-Apple devices as a form of protest. Some vendors, to their credit, modified their products (Raptivity, Lectora, Rapid Intake, etc.) to take steps away from plugin requirements and towards modern HTML5 experiences. (I admit I haven’t tried the HTML5 editions of these products yet, so I can’t speak to their quality.)

Don’t get me wrong, I’m not anti-Flash — anyone who follows this blog knows that I work with ActionScript and am a member of the SWFObject project team — I just think browsers plugins should be used sparingly, with decent fallbacks in place for people who don’t have Flash, such as video transcripts for people who can’t see the video.

My personal development approach has been to develop the course interface and base pages in HTML/JavaScript, and only use Flash when necessary, such as when playing a video or providing a software simulation experience. Now that native video playback is possible in modern browsers, it’s easy to go back and retrofit courses to use HTML5 video. Same goes for audio if you like to have narration in your course. Some situations, such as software simulations, are much trickier, but this is where we rely on tool vendors to improve their tools. For example, Adobe is working on enabling Captivate to output to HTML5 (it currently only outputs a non-interactive video version of the SWF).

Quizzes, fun interactions, and elegant slideshows are all possible with JavaScript today, even in lowly Internet Explorer 6. I have no idea why a simple course system like Articulate Presenter doesn’t provide an HTML5 publishing option yet… interactions can be handled via JavaScript. Narration can be handled via HTML5’s audio element. Quizzes can be handled by JavaScript (though I admit the security is not as strong when you can’t obfuscate answers via a SWF). Even Captivate should be farther along than it is, but at least they’re making progress.

Hopefully this week’s news about Adobe and Microsoft shifting gears away from plugins will help the more hard-headed among us in the e-learning industry to take a more active role in moving away from Flash and towards a true native web experience.

For Your Reading Pleasure: EasyCaptions

Introducing EasyCaptions: A simple system for adding captions and an interactive transcript to online videos. EasyCaptions uses progressive enhancement to provide the best possible experience for all visitors, regardless of their browser’s JavaScript, HTML5 or Flash support.
Demonstration

Background

I don’t produce much video these days, but as a web surfer I often encounter other people’s videos, and was recently impressed by two video implementations: a TED Talks video page, and an HTML5 video demo produced by Bruce Lawson.

The TED Talks page had a great feature I’d never really seen anywhere else: an interactive transcript of the video that you can read and click. For example, you can click the third sentence of the transcript and the video will jump to that point.

Bruce Lawson’s demo illustrated a dead-simple way to add captions to an HTML5 video using just a wee bit of HTML markup and JavaScript.

Both of these pages shared a unique attribute: they stored complete transcripts of the videos in the markup of the HTML page itself, NOT in an external XML file, as is most commonly seen. What’s the big deal about inline transcripts, you ask? Well, today’s most common captioning options require placing your caption text in an external XML file that loads via ActionScript or JavaScript/AJAX. Both the TED page and Bruce’s demo eschew that approach and place the full transcript in the HTML. This means the transcript is always available to the visitor, regardless of that browser’s support for HTML5, JavaScript or Flash Player.

In both cases, the transcript had been marked up as paragraphs, with extra markup denoting phrases that align with the video’s timecode. The TED site used a tags to mark each phrase, with inline JavaScript triggering the playback of the clicked link. This is functional when JavaScript is enabled, but fails when JavaScript is disabled, leaving you with a ton of bloated markup that doesn’t work:


<p>
<a href="#" class="transcriptLink" onclick="seekVideo(0); return false;">One way to change our genes is to make new ones,</a> 
<a href="#" class="transcriptLink" onclick="seekVideo(2000); return false;">as Craig Venter has so elegantly shown.</a>
</p>

Bruce’s span tags, on the other hand, are semantically sound — a span is a neutral, unobtrusive inline element meant to be a child of a block element such as p. Perfect. But wait, there’s more! Since this was an HTML5 demo, Bruce took advantage of the new data attribute that can be used on just about any HTML element. (The short version is: you can create any custom attribute you want, so long as the name begins with data-.) Bruce decided to create two attributes for each span: data-begin, which indicates (in seconds) when the phrase starts in the video, and data-end, which indicates when that phrase has ended in the video. Simple as can be, and extremely efficient:


<p>
<span data-begin=1 data-end=6>Hi, my name's Dr Archimedes Einstein 
and I'm a Dctor of Science at the University of Science</span>
</p>

Enter: EasyCaptions

I was incredibly inspired by these two pages and decided to combine their features, creating a new system I call EasyCaptions. The goal of EasyCaptions is to make it as simple as possible to add useful captions to your online videos. I want to eliminate the headaches of captioning as well as the excuses (too hard, confusing, etc.).

What EasyCaptions does:

  • Dynamically generates a div under your video that will display caption text (the div‘s style and positioning is completely configurable via CSS).
  • Dynamically makes each span in your transcript clickable, jumping to that point in the video. This is done via progressive enhancement and event delegation, leaving your markup clean and avoiding heavy-handed use of onclick.
  • Works out-of-the-box with HTML5.
  • Includes HTML5 video support detection, enabling you to use a Flash-based fallback if you desire.
  • Provides a hook for Flash-based fallback systems, enabling the captions and transcript to behave identically to the HTML5 version.

What you’ll need to do:

  • Type up the transcript of your video, using <span> tags to wrap each phrase, just like Bruce’s example above.
  • Place the transcript in a container element with a unique ID (such as div id="transcript").
  • Add a teeny bit of JavaScript (about 4 lines).

The final product. Successfully tested in Firefox, Safari, Opera, and Internet Explorer (IE and older versions of the other browsers all use a Flash Fallback if provided).

Peruse the test suite to get an idea of how flexible the system is.

The documentation and downloads are located on the EasyCaptions page.

HTML5 Video, minus Ogg

As you’ve probably read somewhere on the interwebs, HTML5 is bringing native video support to browsers. This will enable us to embed a video file directly in our HTML, much like a SWF or image.

Background

You may have also heard that there’s currently a big controversy over what kind of video files will be supported. The defacto standard is MP4/H.264, which is used by Adobe in their Flash video format, and by huge media sites like YouTube. Mozilla, the makers of Firefox, refuse to support the MP4/H.264 standard because it isn’t open-source and free from licensing constraints.

Turns out H.264 is not public domain. Although the company that owns the H.264 patent has temporarily agreed to waive royalty fees for the next decade or so, they reserve the right to charge fees later on. Mozilla says no way, we will only support a video format that is free from licensing issues and has no patent holders (because patent holders can decide to sue some day). Mozilla supports the completely free/open-source Ogg format.

Apple and Adobe, already knee-deep in MP4/H.264 with their Quicktime and Flash video products, vow to press on with H.264.  Google also supports H.264 because YouTube relies on it, and because Google’s new Chrome browser is based on the WebKit project, which has Apple as a main code contributor. In case you haven’t noticed, Apple, Adobe and Google have pretty much cornered the internet video market the past few years, so if they’re throwing their support behind H.264, you can count on it being around for a while. Not to mention that many mobile devices, including the iPhone and most Android phones, have hardware that is designed specifically to support H.264 video, enabling smoother playback and less battery drain than non-dedicated hardware.

(For what it’s worth, Opera is in agreement with Mozilla and supports Ogg. However, not many people seem to pay attention to Opera these days, so they don’t appear to have much influence in this race. Microsoft has endorsed H.264 with it’s upcoming IE9 browser, but it won’t be available for some time.)

The problem

Firefox and Opera are essentially forcing websites to offer two versions of each video: an Ogg version and an MP4 version. In my opinion — and the opinion of many others — this simply will not do. Providing two different video files is not realistic, Ogg’s quality is inferior to H.264, and many computers and mobile devices have direct hardware support for H.264 but not Ogg. In reality, without MP4 support, HTML5 video is rendered useless for most site developers in Firefox and Opera.

The most logical workaround is to code <video> elements to work for MP4 and have a Flash Player-based fallback for older browsers and browsers that only support Ogg. Since the <video> element is designed to allow for fallback content, just like the <object> element, we can do this:


<video id="myvideo" width="480" height="360" controls>
    <source src="/video/file.m4v" type="video/mp4"></source>
    <object data="flash-video-player.swf" type="application/x-shockwave-flash" width="480" height="360">  
        <param value="flash-video-player.swf" name="movie"/>  
        <param value="file=/video/file.m4v" name="flashvars"/>
    </object>
</video>

This works fine in Safari, Chrome, Internet Explorer and older versions of Firefox and Opera that don’t support the <video> element. However, Firefox 3.6 and Opera 10.5 do something very irritating: even though they KNOW their <video> doesn’t support “video/MP4”, they load the <video> element anyway. Which is … like … OMG so duh … because the video can’t possibly play!

If the <video> element is loaded, Firefox and Opera will never load the fallback <object> containing the Flash-based fallback.

Because this behavior cannot be fixed with markup, we’re forced find a scripted workaround (notice that we haven’t used a single bit of JavaScript yet). Thankfully, there’s a pretty straightforward solution: Delete the <video> element in Firefox and Opera.

A Solution

Here’s a simple script that will detect whether HTML 5 video is supported, and if it is, will check to see if MP4 is supported. If HTML5 video is supported but MP4 is NOT supported, the script deletes the specified <video> element but leaves the Flash fallback in its place.


var detectVideoSupport = function (){
    var detect = document.createElement('video') || false;
    this.html5 = detect && typeof detect.canPlayType !== "undefined";
    this.mp4 = this.html5 && (detect.canPlayType("video/mp4") === "maybe" || detect.canPlayType("video/mp4") === "probably");
    this.ogg = this.html5 && (detect.canPlayType("video/ogg") === "maybe" || detect.canPlayType("video/ogg") === "probably");
    return this;
};

var replaceVideoWithObject = function (video_id){    
    if(!video_id){ return false; }
    var video = document.getElementById(video_id);
    if(video){
        var obj = video.getElementsByTagName("object")[0];
        if(obj){
            var obj_copy = obj.cloneNode(true);
            video.parentNode.insertBefore(obj_copy, video);
            video.parentNode.removeChild(video);
        }
    }
};

window.onload = function (){
    var video = detectVideoSupport();
    //Both Opera and Firefox support OGG but lack MP4 support
    if(video.ogg && !video.mp4){
        replaceVideoWithObject("myvideo");
    }
};
</script>

Functioning demo.

A few notes

Tested successfully in:

  • Windows XP: Firefox 3.0, Firefox 3.5.8, Internet Explorer 6, Internet Explorer 8, Google Chrome 4.1.2
  • Windows Vista: Internet Explorer 7
  • Mac OS X (10.6): Firefox 3.6, Safari 4.01, Chrome 5 (beta), Opera 10.1, Opera 10.5b

(Note: IE6, IE7 & IE8 give an unexplained “object required” error in the demo page, but everything works fine. I will investigate as time permits.)

This demo uses the JW Media Player as the Flash fallback video player, but you can use any Flash-based video player on your page.

This demo doesn’t do any Flash Player version detection to ensure the visitor has a supported version of Flash Player. If you need to add Flash Player version detection, you can use SWFObject to embed your Flash file.

Update April 3, 2010: This post was updated to add Opera 10.5 to the list of non-behaving browsers and remove Firefox user agent sniffing.

Changes to pipwerks.com, part 2

In case you hadn’t heard, pipwerks.com was hacked last week. The entire database was erased. Bastages. Luckily, I had a recent backup.

While going through the pains of a new WordPress install (with new plugins, extra security, and imported posts/comments), I decided “why not throw a new layout in the mix, too?” I mean, if I’m going to make changes, I may as well do them all in one shot, eh?

I had been working on an HTML 5-based layout for some time, but had been reluctant to publish it because of HTML 5’s newness and uncertainty; where do I use ‘section’ versus ‘article’, etc.? Do I really want to let WordPress continue to abuse <ol> and <ul>, or does it make more sense to use <dl> in some cases?

The outcome of the argument is what you see today. I didn’t want to fight WordPress too much (esp. the plugins), so there are still a bunch of legacy <div>, <ul> and <ol> elements hanging around. I’m sure I’ve neglected a number of things, but for the most part I’m satisfied.

Here is a short list of fun stuff i implemented in this theme:

  • <dl> for comments (<dt> for author, <dd> for comment body). I feel this is more appropriate as it maintains a relationship between the comment author and the comment body. It also allows for interesting styling (though it makes threaded comments more difficult).
  • <nav> for page header
  • <section> to demarcate where the sidebars go, “related posts” lists, etc.
  • <article> to contain the post itself
  • <footer> to hold footer-ish stuff (though the definition of footer in HTML 5 is still a work in progress)
  • Widgets for sidebars and the footer, which make editing the template SO much easier!
  • Myriad Pro as the body font. Yes, many people won’t have it, and they’ll see Arial instead. But many of my readers probably have Adobe products, which include Myriad. So there you go. No @font-face required! 😛
  • Malarkey‘s Universal IE 6 CSS for the laggards who still use IE 6. No offense — we still have IE 6 at the office, too — but I simply am not interested in bending over backwards to support IE 6 anymore. Kthxbai, IE 6!
  • Alpha transparency in images to allow overlays (such as the cartoon head). Easy in every browser except IE 6 (see above)
  • CSS-based rounded corners in browsers that support them (Safari and Firefox at the moment)
  • The comments styling uses a CSS trick to create the triangle that gives the impression of a cartoon ‘talk’ bubble
  • Icons for external links via CSS3
  • Meyer’s CSS Reset
  • Gravatars
  • Sociable links
  • Related Posts displayed for each post
  • …and more!

I must say I really enjoyed working on this theme, though it took me way too long to get to this point. Please have a look around and let me know how you like the site (or, heaven forbid, you find any problems/errors). And of course, a big thank you to everyone who contributed to the CSS and plugins that I’m using! Your hard work makes my life easier and I appreciate it.

HTML 5: The strong element

I just saw something interesting I thought I’d pass along. In the new HTML 5 proposal, the strong element is being modified to represent “importance rather than strong emphasis.”

The WHATWG gives the following example:


<strong>Warning.</strong> This dungeon is dangerous. 
<strong>Avoid the ducks.</strong> Take any gold you find. 
<strong><strong>Do not take any of the diamonds</strong>, 
they are explosive and <strong>will destroy anything within 
ten meters.</strong></strong> You have been warned.

The b element is supposed to represent “a span of text to be stylistically offset from the normal prose without conveying any extra importance, such as key words in a document abstract, product names in a review, or other spans of text whose typical typographic presentation is boldened.”

The WHATWG gives the following example:

The <b>frobonitor</b> and <b>barbinator</b> components are fried.

To me, this is both exciting and confusing. Knowing we can nest the strong element opens up some new styling possibilities while keeping the markup valid and semantic. But using both the b and the strong elements, with different conditions for their usage, will ultimately be confusing for most people, including me.

Wonder how WYSIWYG editors will handle this.