On Converting Flash to HTML

I received a question from Bob (no, really), who wrote:

I have a question about the newest version of Flash and its HTML publishing option using CreateJS. What do you think of that approach going forward?

I started to write an email response but figured I should probably post it here.

I haven’t been paying much attention to Flash, so I don’t know what the ‘HTML export’ is capable of these days. In general, I’m very wary of converting Flash-based projects to HTML. When Adobe Captivate first released a “publish to HTML5” feature, all it did was convert the SWF animation to a video file, losing all interactivity along the way.

The limitations of the browsers and the HTML5 spec mean you can’t expect a fully 1:1 conversion from Flash to HTML, regardless of libraries like CreateJS. Some of the features found in Flash are still not quite supported in browsers, or might not work quite the way you’d expect. For example, CSS transitions, CSS gradients, and SVG/Canvas support vary widely from browser to browser (though it’s getting better, and there are workarounds aka “polyfills”). Streaming video, which is a breeze in Flash, is not part of the HTML5 Video spec (yet) and is unsupported in browsers. Video and audio codec support is inconsistent. In some cases, devices add new limitations — last time I checked, iOS devices wouldn’t autoplay audio or video in Safari. ‘Play’ couldn’t be scripted, it required the user to press a button.

Publishing to HTML (with the aid of JavaScript libraries like CreateJS) is definitely the way of the future, but I would flip the workflow: use a tool that’s “HTML first”. For my workflow, I always start in HTML then only fall back to Flash if I absolutely have to. I hand-code, but if you want to stick to a WYSIWYG editor, maybe try some of the Adobe Edge products, or go with a third-party product such as Tumult Hype.

If you continue to use Flash as an HTML development tool, temper expectations and test widely, as some things might not work the way you expect when converted to HTML.

And regardless of whether you publish to Flash or HTML, always test the accessibility of your project. Just because it’s HTML doesn’t mean it’s accessible; HTML by nature is more accessible than Flash, but libraries like CreateJS add a lot of complexity to the page, which can easily impact accessibility.

Advertisements

Flash support is increasingly a minefield

Back in 2011, I mentioned that Microsoft was about to halt development of the Silverlight plugin, that Flash mobile was being discontinued, and that Adobe recommended HTML5 for enterprise RIA development instead of Flex, which was being open-sourced. My post was a little long-winded, but the short version was: whoa, the times-are-a-changin’, it’s getting dangerous to rely on browser plugins.

Over the last year, the situation has evolved in an interesting way — browser support for plugins (especially Flash Player) has been considerably restricted by browser vendors due to repeated security vulnerabilities in Flash Player and Java.

Automatically disabling Flash Player

In May 2012, Apple’s Safari browser began automatically disabling outdated versions of Flash Player: “Out-of-date versions of Adobe Flash Player do not include the latest security updates and will be disabled to help keep your Mac secure.” If a webpage contains a SWF but the installed edition of Flash Player is deemed out of date, Safari will display a “blocked plugin” message and inform the user they need to download the latest edition of Flash Player at adobe.com. This change came with Adobe’s blessing.

In January 2013, Mozilla introduced a global “click to play” mechanism that disables ALL plugins in Firefox by default, except the latest edition of Flash Player:  “Our plan is to enable Click to Play for all versions of all plugins except the current version of Flash.”

To Adobe’s credit, Flash Player updates are being released at a fast clip to address known security vulnerabilities. Unfortunately, this has a nasty side effect: you’re very likely to have an outdated edition of Flash Player when you try to view Flash content on a website. (By my unofficial count, there have been at least 13 updates over the past calendar year, averaging about once a month.)

At a recent job, I managed a small network of Macs in classrooms. The Macs were set up to use Adobe’s ‘automatic updates’ feature for Flash Player, but they never seemed to update fast enough — we received numerous complaints from classroom users who couldn’t view Flash content because Safari blocked it.

Internet Explorer’s on-again, off-again relationship with Flash Player

I previously mentioned that Microsoft’s Windows 8 ‘Metro’ mode disabled all plugins, including Flash Player; Microsoft said Internet Explorer in Windows 8 Metro mode “provides an add-on–free experience, so browser plugins don’t load and dependent content isn’t displayed“.

In May 2012 Microsoft changed their minds and enabled Flash in Metro mode. BUT… Microsoft will ship Flash Player as a component of IE 10 (much like Google Chrome does), and will restrict which sites are allowed to run Flash! “While any site can play Flash content in IE10 on the Windows desktop, only sites that are on the Compatibility View (CV) list can play Flash content within Metro style IE.”

In other words, if you don’t have Microsoft’s blessing, your Flash site will not work when viewed in Metro mode.

Update: @chris_sage pointed me to a post by Microsoft written just three days ago where they apparently changed their minds again — almost a year after saying they’d require a whitelist, they now say they support Flash Player by default in Metro mode without requiring sites to be whitelisted.

What it boils down to…

If you develop Flash-based content, it will be more and more of a challenge to provide a smooth, problem-free user experience. For e-learning developers, one of the original lures of Flash was the ubiquity of Flash Player; Flash made it easy to provide the same e-learning experience across browsers and platforms. Due to fragmentation in Flash support, this no longer appears possible.

  • Adobe says: No Flash Player for mobile devices.
  • Microsoft says: No Flash Player on Surface tablets (or other Windows 8/RT tablets) unless the user switches to desktop mode OR gets on our whitelist for Metro mode. We love us some Flash! But we’ll manage the security updates ourselves, thank-you.
  • Mozilla says: Only the latest edition of Flash Player for Firefox.
  • Apple says: No Flash Player on Apple iOS devices, and only the latest edition of Flash Player for desktop Safari.
  • Opera says: Flash Player on desktop editions of Opera? No problem. Flash Player in Opera Mobile? Don’t get mad at us, Adobe stopped providing Flash Player for mobile devices!
  • Google says: We control Flash Player for Chrome (desktop) ourselves. No worries. Flash Player in Chrome on mobile devices? Don’t get mad at us, Adobe stopped providing Flash Player for mobile devices!

The browser vendors are enforcing their will. You don’t have to be a Flash-hater to see that building Flash-dependent sites is a minefield.

For those of you in e-learning who depend almost completely on Flash-based courseware, it’s a good idea to start looking for alternatives.

 

 

Introducing SWFRightClick

Adobe Captivate currently ships with a 3rd-party JavaScript utility named RightClick.js, which enables the Captivate SWF to detect when a user right-clicks the SWF. While upgrading the Captivate publishing templates, I realized RightClick.js wasn’t built to work with SWFObject 2.x and suffered from a few shortcomings. I modified the Captivate template’s SWFObject code to get around the issue, but marked it down as something to revisit when I have the time.

Now, I’m happy to report I’ve created a replacement for the RightClick.js utility, creatively named SWFRightClick. It uses the same approach to handling right-clicks, but does it with a completely new codebase and a few extra goodies. SWFRightClick is compatible with every edition of SWFObject, and is free to use (MIT license).

Check it out on GitHub. I plan to fold it in to my Captivate publishing templates very soon.

HTML5, Flash, Silverlight, and your courseware

What a busy week.

Flash is dead. Sort of, but not really.

In case you haven’t heard, Adobe formally announced the discontinuation of Flash Player for mobile devices (“Flash to Focus on PC Browsing and Mobile Apps; Adobe to More Aggressively Contribute to HTML5“). Adobe employees struggled to come to grips with what has undoubtedly been a tough week for them — aside from the product news, they were also informed of massive layoffs (around Adobe 750 employees). Regardless of your feelings about Flash, your heart must go out to the families affected by a sudden job loss.

Flash critics were quick to declare Flash dead. But in the immortal words of Monty Python’s Eric Idle, “I’m not quite dead yet.”

Yes, Adobe is scaling back on Flash to focus more energy on so-called HTML5 technologies (“so-called” because many of the technologies involved are not actually part of the HTML5 spec.) However, their public announcement was that mobile Flash is dead, and Flash on the desktop will continue to thrive.

Flex is being abandoned

Then the “Official Flex Team Blog” dropped a bombshell: Adobe Flex is going open-source:

Yes. We know Flex provides a unique set of benefits for enterprise application developers. We also know that the technology landscape for application development is rapidly changing and our customers want more direct control over the underlying technologies they use. Given this, we are planning to contribute the Flex SDK to an open source foundation in the same way we contributed PhoneGap to the Apache Foundation when we acquired Nitobi.

Okay. Flex is being open-sourced. To be honest, this isn’t really shocking news to me since Flex has mingled with Eclipse for a long time and has had its toes in open source for ages. The part that made my jaw drop was in the next section:

Does Adobe recommend we use Flex or HTML5 for our enterprise application development?
In the long-term, we believe HTML5 will be the best technology for enterprise application development. We also know that, currently, Flex has clear benefits for large-scale client projects typically associated with desktop application profiles.

Given our experiences innovating on Flex, we are extremely well positioned to positively contribute to the advancement of HTML5 development, starting with mobile applications. In fact, many of the engineers and product managers who worked on Flex SDK will be moving to work on our HTML efforts. We will continue making significant contributions to open web technologies like WebKit & jQuery, advance the development of PhoneGap and create new tools that solve the challenges developers face when building applications with HTML5.

In the long-term, we believe HTML5 will be the best technology for enterprise application development. For those of you unfamiliar with Flex, it’s Adobe’s ‘enterprise-level’ Flash development path. Designers use Flash Professional and the timeline, while programmers use Flex and its pure ActionScript/MXML environment. This line is basically saying “see ya, Flex, was nice knowing you. We’re putting our best guys — the ones who weren’t laid off — on the ‘HTML5’ train.”

By extension, if Adobe is wiping its hands of Flex, it means they’re seriously scaling back support for the entire Flash ecosystem.

Whoa. The anti-Flash crowd really does have reason to celebrate. But not so quick, my friends: Support for the HTML5 environment is nowhere near Flash’s level of ubiquity, it will take a couple of years to get there, and even then, I doubt it will have the full capabilities Flash offers today.

But does it matter? Let me come back to that in a minute.

Silverlight is dead. Sort of, but not really

For now, lets turn our attention to our friends in Redmond, WA. While Adobe created this week’s loudest thunder, Microsoft made a few waves itself. If you recall, about a year ago one of Microsoft’s presidents (yes, they have more than one) caused a stir when he said “HTML is the only true cross platform solution for everything, including (Apple’s) iOS platform”. Microsoft quickly issued a statement backtracking from the implication that Silverlight was on its way out, and stressed their commitment to Silverlight.

A year later, the circus is back in town. According to some industry sources, Microsoft is planning to stop development of future editions of Silverlight; version 5 may be the last hurrah, except for security updates. Stop me if this sounds familiar.

However, this time around, the rumor sounds much more feasible because of Internet Explorer 10.

Microsoft’s Internet Explorer, that grand ol’ browser, is going through a major metamorphosis. In general, Internet Explorer 10 will behave less like its ancestors and more like Firefox and Webkit (Safari/Chrome). For example, IE 10 will not support Microsoft’s long-standing conditional comments. This is a big deal for web developers, as conditional comments have become a major crutch when dealing with Internet Explorer. But the biggest news I’ve heard about IE 10 thus far is that it will not support plugins — Flash, Silverlight, Quicktime, etc. — when running in “Metro” mode. Microsoft’s Windows 8 team believes that removing plugins from IE will result in improvements across-the-board: security, reliability, privacy, battery life in mobile devices, etc.

For the web to move forward and for consumers to get the most out of touch-first browsing, the Metro style browser in Windows 8 is as HTML5-only as possible, and plug-in free. The experience that plug-ins provide today is not a good match with Metro style browsing and the modern HTML5 web.

Running Metro style IE plug-in free improves battery life as well as security, reliability, and privacy for consumers. Plug-ins were important early on in the web’s history. But the web has come a long way since then with HTML5. Providing compatibility with legacy plug-in technologies would detract from, rather than improve, the consumer experience of browsing in the Metro style UI.

Plugins will still be supported in the non-Metro environment, but it appears Microsoft’s way of the future is Metro, and Metro is HTML5-based, with no plugins allowed, just like Apple iOS. Microsoft is clearly serious about leaving plugins behind, including its own Silverlight Player.

The analysis

Flash and Silverlight are not dead, but are being minimized by their owners. If you read the articles closely, you’ll see a pattern: both Adobe and Microsoft hope to port some of the technology from their plugins to the open web, aka the “HTML5 ecosystem”.

As it currently stands, browser technology and HTML5 ubiquity will take a number of years to catch up to the baseline capabilities of Flash and Silverlight. Even then, I doubt browsers will have some of the advanced capabilities Flash offers today.

The thing is, I don’t think most Flash developers take full advantage of Flash’s capabilities; they seem to use the same basics they’ve been using for years. The cutting-edge stuff is nowhere near as common as the mundane: video players, slideshows, and simple games.

Today’s editions of Firefox, Chrome, Safari, and even Internet Explorer 9 are massively capable browsers. JavaScript, canvas, and CSS3 can be used to replicate many of Flash’s features. The real question becomes: what are you trying to do?

Video

The vast majority of Flash usage appears to be for serving streaming video. HTML5’s video support is not quite ready to replace Flash in this regard, but it’s getting pretty close. The codec wars are the biggest stumbling block, followed by support for DRM and perhaps a handful of advanced features. If you have a simple video that doesn’t need DRM and can be served via two codecs (H264 and OGG or WebM), you can proceed directly to Go, collect $200.

Need to add some interactivity to your movie? Canvas and JavaScript can do an awful lot. Projects like WebGL and Mozilla’s popcorn.js are upping the ante. The future looks pretty bright for native online video.

Gaming

Flash-based gaming is another strong user base. My kids have been using NeoPets and similar sites for years, so I understand just how far and deep Flash gaming goes in our community. But some of these games are deceptively simple with their graphics, and I wouldn’t be surprised if Adobe’s HTML5 tools soon enable Flash game developers to port their games to the HTML5 ecosystem.

Apple’s iOS and Facebook have created big-money opportunities for game developers willing to switch gears to HTML5. We’ve already seen JavaScript/HTML ports of Nintendo games, including the best video game ever made: Super Mario Brothers. They work. They’re fun. Yes, it will take a long time to get market saturation, but HTML5 already has its foot in the door and a lot of money behind it — gaming is big business!

E-Learning

If you’re reading this blog, chances are good that you’re an e-learning developer. You probably also know that most major e-learning rapid development software outputs Flash SWFs. I can’t think of any industry aside from online gaming that is so utterly dependent on Flash at the moment. This has worried me for years — I’ve long preached that e-learning developers should be less reliant on Flash.

When Apple’s iPad was released without Flash support, the outcry from the e-learning industry was fierce. I also feel it was misplaced — it clearly showed the industry’s reliance on a plugin, and rather than focus on removing the Flash requirement barrier, many people chose to plant their feet in the ground and buy non-Apple devices as a form of protest. Some vendors, to their credit, modified their products (Raptivity, Lectora, Rapid Intake, etc.) to take steps away from plugin requirements and towards modern HTML5 experiences. (I admit I haven’t tried the HTML5 editions of these products yet, so I can’t speak to their quality.)

Don’t get me wrong, I’m not anti-Flash — anyone who follows this blog knows that I work with ActionScript and am a member of the SWFObject project team — I just think browsers plugins should be used sparingly, with decent fallbacks in place for people who don’t have Flash, such as video transcripts for people who can’t see the video.

My personal development approach has been to develop the course interface and base pages in HTML/JavaScript, and only use Flash when necessary, such as when playing a video or providing a software simulation experience. Now that native video playback is possible in modern browsers, it’s easy to go back and retrofit courses to use HTML5 video. Same goes for audio if you like to have narration in your course. Some situations, such as software simulations, are much trickier, but this is where we rely on tool vendors to improve their tools. For example, Adobe is working on enabling Captivate to output to HTML5 (it currently only outputs a non-interactive video version of the SWF).

Quizzes, fun interactions, and elegant slideshows are all possible with JavaScript today, even in lowly Internet Explorer 6. I have no idea why a simple course system like Articulate Presenter doesn’t provide an HTML5 publishing option yet… interactions can be handled via JavaScript. Narration can be handled via HTML5’s audio element. Quizzes can be handled by JavaScript (though I admit the security is not as strong when you can’t obfuscate answers via a SWF). Even Captivate should be farther along than it is, but at least they’re making progress.

Hopefully this week’s news about Adobe and Microsoft shifting gears away from plugins will help the more hard-headed among us in the e-learning industry to take a more active role in moving away from Flash and towards a true native web experience.

For Your Reading Pleasure: EasyCaptions

Introducing EasyCaptions: A simple system for adding captions and an interactive transcript to online videos. EasyCaptions uses progressive enhancement to provide the best possible experience for all visitors, regardless of their browser’s JavaScript, HTML5 or Flash support.
Demonstration

Background

I don’t produce much video these days, but as a web surfer I often encounter other people’s videos, and was recently impressed by two video implementations: a TED Talks video page, and an HTML5 video demo produced by Bruce Lawson.

The TED Talks page had a great feature I’d never really seen anywhere else: an interactive transcript of the video that you can read and click. For example, you can click the third sentence of the transcript and the video will jump to that point.

Bruce Lawson’s demo illustrated a dead-simple way to add captions to an HTML5 video using just a wee bit of HTML markup and JavaScript.

Both of these pages shared a unique attribute: they stored complete transcripts of the videos in the markup of the HTML page itself, NOT in an external XML file, as is most commonly seen. What’s the big deal about inline transcripts, you ask? Well, today’s most common captioning options require placing your caption text in an external XML file that loads via ActionScript or JavaScript/AJAX. Both the TED page and Bruce’s demo eschew that approach and place the full transcript in the HTML. This means the transcript is always available to the visitor, regardless of that browser’s support for HTML5, JavaScript or Flash Player.

In both cases, the transcript had been marked up as paragraphs, with extra markup denoting phrases that align with the video’s timecode. The TED site used a tags to mark each phrase, with inline JavaScript triggering the playback of the clicked link. This is functional when JavaScript is enabled, but fails when JavaScript is disabled, leaving you with a ton of bloated markup that doesn’t work:


<p>
<a href="#" class="transcriptLink" onclick="seekVideo(0); return false;">One way to change our genes is to make new ones,</a> 
<a href="#" class="transcriptLink" onclick="seekVideo(2000); return false;">as Craig Venter has so elegantly shown.</a>
</p>

Bruce’s span tags, on the other hand, are semantically sound — a span is a neutral, unobtrusive inline element meant to be a child of a block element such as p. Perfect. But wait, there’s more! Since this was an HTML5 demo, Bruce took advantage of the new data attribute that can be used on just about any HTML element. (The short version is: you can create any custom attribute you want, so long as the name begins with data-.) Bruce decided to create two attributes for each span: data-begin, which indicates (in seconds) when the phrase starts in the video, and data-end, which indicates when that phrase has ended in the video. Simple as can be, and extremely efficient:


<p>
<span data-begin=1 data-end=6>Hi, my name's Dr Archimedes Einstein 
and I'm a Dctor of Science at the University of Science</span>
</p>

Enter: EasyCaptions

I was incredibly inspired by these two pages and decided to combine their features, creating a new system I call EasyCaptions. The goal of EasyCaptions is to make it as simple as possible to add useful captions to your online videos. I want to eliminate the headaches of captioning as well as the excuses (too hard, confusing, etc.).

What EasyCaptions does:

  • Dynamically generates a div under your video that will display caption text (the div‘s style and positioning is completely configurable via CSS).
  • Dynamically makes each span in your transcript clickable, jumping to that point in the video. This is done via progressive enhancement and event delegation, leaving your markup clean and avoiding heavy-handed use of onclick.
  • Works out-of-the-box with HTML5.
  • Includes HTML5 video support detection, enabling you to use a Flash-based fallback if you desire.
  • Provides a hook for Flash-based fallback systems, enabling the captions and transcript to behave identically to the HTML5 version.

What you’ll need to do:

  • Type up the transcript of your video, using <span> tags to wrap each phrase, just like Bruce’s example above.
  • Place the transcript in a container element with a unique ID (such as div id="transcript").
  • Add a teeny bit of JavaScript (about 4 lines).

The final product. Successfully tested in Firefox, Safari, Opera, and Internet Explorer (IE and older versions of the other browsers all use a Flash Fallback if provided).

Peruse the test suite to get an idea of how flexible the system is.

The documentation and downloads are located on the EasyCaptions page.

Dear Apple and Adobe

 

Update: Steve Jobs Responds! Well, not to my letter directly, but it hits on the major points and is a well-written explanation of Apple’s position.

Dear Apple and Adobe

I’m a long-time customer and have spent more money on your products than I have on just about any other aspect of my life. I’ve spent more money on your products than I’ve spent on my healthcare, vacations, kitchen appliances, children’s school supplies, or home entertainment system.

In return, you’ve increasingly shown a disregard for my needs and concerns, and have acted in ways that demonstrate all you want from me is my money.

For example, both of you have constantly forced me (or at a minimum pressured me) to buy updates to products I already paid for. For years I went along with it because I bought into the sales hype and assumed these updates would somehow make my life better.  In most cases, they did not.

Adobe, your constant tinkering with the Creative Suite has brought a few nifty tools to the world, but these new tools will not get me to overlook the incredible bloat you’ve unleashed on my computers — almost 6GB of program files on my Windows PC at work, and over 7GB of app files on my Mac at home. Your applications feel more unstable with every release, and your UI feels slow and unresponsive despite the extra RAM and other hardware upgrades on my machines. Some of the biggest security holes on my computers are due to your Acrobat software — the very same Acrobat software I’ve learned to hate because of how bloated, complicated, and unfriendly it has become. It feels like it gets worse with each release.

Apple, your innovation is refreshing. Adobe could learn a thing or two by examining your software: increased productivity through reduced feature sets and cleaner UI. Simple is usually best. However, despite your continued excellence in design, your behavior is repulsive. You’ve consistently screwed your early adoptors via your pricing schemes and forced millions of Americans to use a phone network they detest. (Why? Because AT&T was willing to give you a bigger cut of the revenue?) Worst of all, the totalitarianism displayed in your latest iPhone developer agreement is breathtaking. It appears your goal is to piss off everyone, even your staunchest allies… like Adobe.

Apple and Adobe, you used to play well together. You both benefited from your long-term relationship and grew into very large, very successful companies. I sincerely doubt either of you would have survived the 1990s intact if it weren’t for your partnership. Desktop publishing was the Mac’s forte and the one thing that kept it afloat when the buzzards were circling. And who provided the most popular DTP software? Adobe (and the companies Adobe acquired, like Aldus and Macromedia).

Adobe, I know you’re mad because Apple won’t let you put your Flash technology on the new iPhone platform (iPhone, iPod, iPad). Honestly, if I were controlling a platform, I would have major concerns, too. As I mentioned earlier, your track record for software quality seems to be in a steady decline. Your products have become infamous for security holes, bloat, and crashing. It didn’t used to be that way. Somewhere along the line you dropped the ball, and now it’s coming back to bite you. The good news is that it isn’t too late for you to reign things in and regain control of your software. Stop trying to please everyone by adding every conceivable feature under the sun, and really focus on the most important elements. Drop the cruft. Clean the cupboards. Get that lint out of your bellybutton. Once your software is respectable again, you’ll be in a much stronger position to complain about Apple.

Apple, I don’t know what happened to you. You went from being a popular underdog to being the class bully. You’re in danger of becoming as popular as Microsoft in the European court system. From where I sit, your biggest mistake has been the idea that you can take over the world, one industry at a time. Of course, many companies are aggressive and set big goals for themselves, but they don’t stab their partners in the back as quickly and viciously as you seem to do. Your hubris and eagerness to expand into your partners’ markets is going to be your downfall. People have liked you because of your design sensibilities and because you were the hip underdog. You are no longer the hip underdog, and with time, other companies will create products that will be (almost) as stylish but also cheaper and with equivalent or greater capabilities.

The bottom line is that neither of you are choir boys, and I’m fed up with your bickering.

Adobe, stop playing the sympathy card. It’s a complete turn-off because I know how crappy your software can be. Granted, it’s unfortunate that so many people depend on Flash and Flash doesn’t work on the iPhone platform, but Flash is not a web standard. For all its shortcomings, the iPhone platform has one excellent quality: a top-notch HTML5 browser. Standardistas have been warning people not to go all-in with Flash for years, and now we see why. If it isn’t part of a standard, it will not be incorporated into some products. It’s the vendor’s choice. Simple as that.

Apple, stop trying to take over the world. We’ve seen what happens to other companies who try it, and it never looks pretty. Focus on your core values and let your partners do their thing without stepping on their toes.

Oh, and ditch AT&T already, will ya?

Respectfully,

Philip

HTML5 Video, minus Ogg

As you’ve probably read somewhere on the interwebs, HTML5 is bringing native video support to browsers. This will enable us to embed a video file directly in our HTML, much like a SWF or image.

Background

You may have also heard that there’s currently a big controversy over what kind of video files will be supported. The defacto standard is MP4/H.264, which is used by Adobe in their Flash video format, and by huge media sites like YouTube. Mozilla, the makers of Firefox, refuse to support the MP4/H.264 standard because it isn’t open-source and free from licensing constraints.

Turns out H.264 is not public domain. Although the company that owns the H.264 patent has temporarily agreed to waive royalty fees for the next decade or so, they reserve the right to charge fees later on. Mozilla says no way, we will only support a video format that is free from licensing issues and has no patent holders (because patent holders can decide to sue some day). Mozilla supports the completely free/open-source Ogg format.

Apple and Adobe, already knee-deep in MP4/H.264 with their Quicktime and Flash video products, vow to press on with H.264.  Google also supports H.264 because YouTube relies on it, and because Google’s new Chrome browser is based on the WebKit project, which has Apple as a main code contributor. In case you haven’t noticed, Apple, Adobe and Google have pretty much cornered the internet video market the past few years, so if they’re throwing their support behind H.264, you can count on it being around for a while. Not to mention that many mobile devices, including the iPhone and most Android phones, have hardware that is designed specifically to support H.264 video, enabling smoother playback and less battery drain than non-dedicated hardware.

(For what it’s worth, Opera is in agreement with Mozilla and supports Ogg. However, not many people seem to pay attention to Opera these days, so they don’t appear to have much influence in this race. Microsoft has endorsed H.264 with it’s upcoming IE9 browser, but it won’t be available for some time.)

The problem

Firefox and Opera are essentially forcing websites to offer two versions of each video: an Ogg version and an MP4 version. In my opinion — and the opinion of many others — this simply will not do. Providing two different video files is not realistic, Ogg’s quality is inferior to H.264, and many computers and mobile devices have direct hardware support for H.264 but not Ogg. In reality, without MP4 support, HTML5 video is rendered useless for most site developers in Firefox and Opera.

The most logical workaround is to code <video> elements to work for MP4 and have a Flash Player-based fallback for older browsers and browsers that only support Ogg. Since the <video> element is designed to allow for fallback content, just like the <object> element, we can do this:


<video id="myvideo" width="480" height="360" controls>
    <source src="/video/file.m4v" type="video/mp4"></source>
    <object data="flash-video-player.swf" type="application/x-shockwave-flash" width="480" height="360">  
        <param value="flash-video-player.swf" name="movie"/>  
        <param value="file=/video/file.m4v" name="flashvars"/>
    </object>
</video>

This works fine in Safari, Chrome, Internet Explorer and older versions of Firefox and Opera that don’t support the <video> element. However, Firefox 3.6 and Opera 10.5 do something very irritating: even though they KNOW their <video> doesn’t support “video/MP4”, they load the <video> element anyway. Which is … like … OMG so duh … because the video can’t possibly play!

If the <video> element is loaded, Firefox and Opera will never load the fallback <object> containing the Flash-based fallback.

Because this behavior cannot be fixed with markup, we’re forced find a scripted workaround (notice that we haven’t used a single bit of JavaScript yet). Thankfully, there’s a pretty straightforward solution: Delete the <video> element in Firefox and Opera.

A Solution

Here’s a simple script that will detect whether HTML 5 video is supported, and if it is, will check to see if MP4 is supported. If HTML5 video is supported but MP4 is NOT supported, the script deletes the specified <video> element but leaves the Flash fallback in its place.


var detectVideoSupport = function (){
    var detect = document.createElement('video') || false;
    this.html5 = detect && typeof detect.canPlayType !== "undefined";
    this.mp4 = this.html5 && (detect.canPlayType("video/mp4") === "maybe" || detect.canPlayType("video/mp4") === "probably");
    this.ogg = this.html5 && (detect.canPlayType("video/ogg") === "maybe" || detect.canPlayType("video/ogg") === "probably");
    return this;
};

var replaceVideoWithObject = function (video_id){    
    if(!video_id){ return false; }
    var video = document.getElementById(video_id);
    if(video){
        var obj = video.getElementsByTagName("object")[0];
        if(obj){
            var obj_copy = obj.cloneNode(true);
            video.parentNode.insertBefore(obj_copy, video);
            video.parentNode.removeChild(video);
        }
    }
};

window.onload = function (){
    var video = detectVideoSupport();
    //Both Opera and Firefox support OGG but lack MP4 support
    if(video.ogg && !video.mp4){
        replaceVideoWithObject("myvideo");
    }
};
</script>

Functioning demo.

A few notes

Tested successfully in:

  • Windows XP: Firefox 3.0, Firefox 3.5.8, Internet Explorer 6, Internet Explorer 8, Google Chrome 4.1.2
  • Windows Vista: Internet Explorer 7
  • Mac OS X (10.6): Firefox 3.6, Safari 4.01, Chrome 5 (beta), Opera 10.1, Opera 10.5b

(Note: IE6, IE7 & IE8 give an unexplained “object required” error in the demo page, but everything works fine. I will investigate as time permits.)

This demo uses the JW Media Player as the Flash fallback video player, but you can use any Flash-based video player on your page.

This demo doesn’t do any Flash Player version detection to ensure the visitor has a supported version of Flash Player. If you need to add Flash Player version detection, you can use SWFObject to embed your Flash file.

Update April 3, 2010: This post was updated to add Opera 10.5 to the list of non-behaving browsers and remove Firefox user agent sniffing.

Introducing LearnSWFObject.com

I’m happy to introduce you to my latest project, LearnSWFObject.com

It’s been many, many months in the making, and has rudely been put aside a number of times when my life left me no free time to work on it. Happily, I’ve been able to push through the last remaining barriers and get the site out the door. Hopefully someone will find it useful.

I’ve ported my most popular SWFObject tutorials and examples to the new site — updated, of course — and have added a few new tutorials, too. As part of the housecleaning effort, I’ve removed all SWFObject examples and tutorials from pipwerks.com. 301 redirects will re-route the most popular posts to their LearnSWFObject.com equivalent, but the odds and ends have been sent to the compost heap.

I’m most excited about the brand-new code generator (written from scratch) that will write your SWFObject embed code for you. Some notes about the generator:

  • It can write HTML 5, HTML 4 (transitional/strict), and XHTML 1 (transitional/strict) doctypes
  • It provides code for both types of SWFObject embeds: dynamic (JavaScript) and static (markup)
  • It includes a download link that lets you save your generated markup in an HTML file
  • The static publishing option includes an option for a nice, shorter syntax
  • The generator uses progressive enhancement techniques, meaning it’s nicer with JavaScript enabled but fully functional without it

Give it a try!

LearnSWFObject.com will remain a work in progress, as SWFObject itself continues to evolve and people find news ways to break use it in their sites. I will continue to play with the formatting and layout from time to time.

In other SWFObject news, I think I’m allowed to report that SWFObject 2.3 is in the works, and will contain mostly bug fixes and an enhancement or two. Bug reports and feature requests are always welcome at SWFObject’s Google Code site. If you have questions about how to use SWFObject that aren’t answered by the LearnSWFObject.com tutorials, please post them on the SWFObject Google Group.

What feature is missing from your e-learning development tool?

I have some simple questions I’d love to get feedback on.

I’m curious about what people are looking for in their e-learning authoring tools, specifically:

  1. What feature is your current tool missing that you would love to see implemented? Support for team collaboration? Support for themes or custom CSS styling? Support for language localization? A beer dispenser? Etc.
  2. What feature does someone else’s tool have that you’re jealous of?

For me? I wish the most popular tools (Captivate, Articulate, Lectora, etc.) would output cleaner HTML and JavaScript. I also wish there was less reliance on Flash and PowerPoint. But if you read my blog regularly, you probably already knew that!

Please post your opinion in the comments below, and please ask others to give their 2 cents. (You can also reply via twitter if you prefer)

I’m really looking forward to your comments. Thanks!

Adobe E-Learning Products “Sneak Peeks”

Today’s Adobe Summit had a session named “Sneak Peeks.” It was (unofficially I think) mentioned that the Adobe E-Learning Suite is coming, and will include Captivate 4, Flash, Photoshop, Acrobat Professional, Device Central, and more.

Here’s a quick list of topics covered.

Versions of Dreamweaver and Flash in E-Learning Suite will NOT be same as those in CS4 suites, and will include e-learning specific bits.

No date for E-Learning Suite given; will only say 2009.

Captivate 4 will include:

  • Automatic panning that follow your screen actions.
  • Previewing in Device Central
    • Allows you to preview on an actual mobile device
    • Allows you to preview with fake screen reflections
  • Inline text editing for captions (no more dialog boxes)
  • Basic drawing tools (shapes)
  • Integration with Adobe Bridge
  • Import > Photoshop files (PSD)
    • Can flatten layers or choose to have them import to separate layers
    • Converts each layer to hi-res PNG
    • Allow syou to animate layers individually on a single slide
  • Support for custom variables, such as using a person’s first name throughout the course
    • Uses $$variablename$$ syntax
    • Means you can use custom static variables throughout a course
    • You can customize an external RDL file to templatize variable use; means you can instruct Captivate to automatically insert variables when recording a demo without having to manually edit the recording afterwards
  • Support for Flash widgets
    • Will work much like components in Flash Professional
    • Captivate 4 will ship with many widgets (including source FLA)
    • Widgets can be customized, and users can create own from scratch
    • Can talk to Captivate (including quizzes) and retrieve variables
    • Example: certificate widget can display person’s name, score for course, date, etc.
    • Example: “perpetual buttons” widget that customizes navigation (forward/back) for movie; hides “back” biutton when on first slide, hide “next” button when on last slide
    • Captivate Exchange on Adobe site will be available for users to post and download custom widgets
  • Support for multiple actions on a single slide
  • Support for ActionScript 3 (can publish to AS2 or AS3 depending on user settings)
  • Can create image slideshow
  • Ability to show/hide toolbar during a course without using customizations
  • Captivate 4 will be on Windows, but work will soon begin on a Mac version
  • Can publish directly to PDF (embeds SWF into new PDF file)
  • Single-SWF output (all files are embedded into single SWF, including nav and full-motion recordings)
  • Auto-generation of Table of Contents (embedded into SWF)
  • SWFs are searchable (text caption content is searchable)
  • “Aggregator” feature allows you to add external Captivate SWFs to project (package multiple SWFs together… is a SWF that loads other SWFs)
  • Supports Flash Player 7, 8, 9 & 10
  • “Reviewer” feature allows users without Captivate to comment on a Captivate SWF
    • Uses Adobe AIR
    • Synchronizes comments form multiple reviewers
  • Supports placeholders to demonstrate where content will be, such as an FLV that isn’t available yet.
    • Helpful when using Reviewer feature so others can know what you have planned
    • Placeholders make it easy to insert content when it becomes available
  • Improved Support for PowerPoint imports
    • Support for dynamic link to PowerPoint presentation — if PowerPoint file is updated, changes wil be reflected in Captivate file
    • Makes PPT file a smart object; you can open and edit PPT file via Captivate (just like how smart vector objects in Photoshop open in Illustrator)