JW Player and HTML5 Video

When it comes to embedded media on the web, the JW Player from LongTail Video is widely regarded as one of the best solutions in terms of producing consistent results across a wide range of browsers and devices.

Since release 5.3, JW Player has included a specific embedding method that caters for the creation of a video player with seamless failover between Flash and HTML5 modes (and vice versa). It also introduced the ability to use the parameters defined in HTML5 video and source tag attributes as the default configuration options for setting up the player. This means that once the JW Player JavaScript library has been loaded in the page, a JW Player instance can be generated using a simple video tag and zero extra JavaScript – an excellent enhancement to ensure the widest availability of video regardless of a user’s individual browser configuration.

JW HTML CONFIG

In my role as Web Developer for a large UK organisation, I recently tasked myself with writing an extension to the JW Player solution with the primary aim of simplifying the configuration process (particularly in relation to playlists) whilst additionally producing an output that reacts responsively. The aim was to provide a solution which doesn’t require additional, separate JavaScript configuration – all settings are to be driven from simple inline HTML.

In much the same way the JW Embedder uses the principle of progressive enhancement by interrogating the HTML5 video tag and attributes and then dynamically producing a configuration object, which is used as the basis for the player setup, the solution makes use of additional structured settings blocks (tags with data- attributes) to extend this concept further.

During the development of this extension (which takes the form of a small JavaScript library, and details can be found here) I encountered a number of “setup and playback inconsistencies” which, when checking in the JW Player forums, became clear I was not alone in experiencing. These “unexplained hiccups” appear to be increasingly common and a worrying source of frustration for users who are implementing (or more likely updating) JW Player to make use of the more elegant and accessible HTML5 first approach.

UNEXPLAINED HICCUPS

To understand more about how these frustrations manifest themselves – let’s explore a likely route to JW Player/HTML5 implementation :

Ok. Let’s say for arguments sake I have some experience in implementing JW Player, going back to before 5.3, when I happily managed to get MP4 videos playing on my website using the Flash embed method. Life is sweet! A single video format played through a 3rd party plugin. If you don’t have the plugin, you don’t get the video – that’s just the way it is I’m afraid!

[Time passes by]

Now I’ve been thinking, maybe my website should make more of an effort to cater for a wider audience and start supporting HTML5 video – specifically to serve those Flash intolerant devices! No problem. As I understand it these devices support MP4 video natively, so all I need to do is upgrade to the latest version of JW Player (let’s say 5.7) and let it handle the Flash to HTML5 failover as required. Perfect.

Erm… or maybe not! It seems that my MP4 video files don’t actually play on the latest iDevices. After a mild panic, and a little bit of research, I am able to figure out that while it seems Flash is quite forgiving about the encoding of video, the iDevice is not. Resolving the problem involves re-encoding the videos specifically for the iDevice, which although a bit of a headache, is perfectly manageable and the result is worth it.

[More time passes]

As my understanding of my website audience evolves further, and my desire to reach as many different users – on a broad range of devices – becomes a key objective, I decide to switch my video strategy to serve HTML5 first with a Flash fall-back.

After a bit of research, I decide that the best approach (involving the least effort) involves simply switching my JW Embed method to make use of the native video and source tags and providing a WebM version of the video – to cater for native playback in Firefox & Opera. I also update JW Player to version 5.10 and specifically include the modes block in my setup object – telling it to use html5 mode first. This is as described in the JW Player documentation:

<script src="/jwplayer/jwplayer.js" type="text/javascript"></script>
<video height="270" width="480" id="myVideo">
<source src="/static/bunny.mp4" type="video/mp4">
<source src="/static/bunny.webm" type="video/webm">
</video>
<script type="text/javascript">
// <![CDATA[
jwplayer("myVideo").setup(
{ modes:
[
{ type: 'html5' },
{ type: 'flash', src: '/jwplayer/player.swf' }
]
});
// ]]>
</script>

One change I make to this code is to place the script tags at the bottom of the body – as this is commonly best practice right?! My code now looks like this:

<html>
<head>

</head>
<body>
<!-- Navigation stuff -->

<!-- Content Area -->
<h2>Check this HTML5 video out!</h2>
<video height="270" width="480" id="myVideo">
<source src="/static/bunny.mp4" type="video/mp4">
<source src="/static/bunny.webm" type="video/webm">
</video>
<!-- More content -->

<!-- Right hand column -->

<!-- Footer stuff -->
<script src="/jwplayer/jwplayer.js" type="text/javascript"></script>
<script type="text/javascript">
// <![CDATA[
jwplayer("myVideo").setup(
{ modes:
[
{ type: 'html5' },
{ type: 'flash', src: '/jwplayer/player.swf' }
]
});
// ]]>
</script>
</body>
</html>

This in theory caters for the base position of native playback in all the mainstream browsers (Firefox & Opera – playing WebM; IE9, iOS, Safari & Android – playing MP4; Chrome – playing MP4 or WebM).

It also has the benefit that if JavaScript is enabled, JW Player will progressively enhance the player to ensure a consistent look-and-feel across all these browsers, plus provide a Flash fall-back for earlier versions of IE – playing the MP4.

Well that wasn’t too hard to implement… Except… What the…?!

After testing thoroughly and ensuring the videos play in all the browsers as expected I push my site live and I am greeted with… issues. Not only that, the worse kind of issues – inconsistent issues.

A selection of problems encountered include:

  • Chrome not playing the MP4, but playing the WebM. Safari plays the MP4
  • Firefox not playing WebM. Chrome plays the WebM
  • Safari not playing natively – rendering the Flash player
  • Safari not showing the player at all
  • IE7 & IE8 not showing the player at all
  • IE9 not showing controls
  • IE9 not showing anything

Given the specific nature of these issues, they should be reasonably easy to debug and determine a solution to.

However the following “generic” issues have also been experienced. These are much less obvious in terms of cause and can end up being responsible for hours of debugging frustration:

  • Previously “playable” video suddenly not loading – an error is presented
  • Video appearing to endlessly buffer – pressing pause and then resume presents a black screen
  • Video buffering for a long time before finally playing
  • Video plays on the first visit to a page, then fails on subsequent visits
  • Video doesn’t play on initial page load, but plays on refresh
  • Video playing with no audio
  • Audio playing with no video
  • Video playing and then stalling or playing out of sync with audio
  • Video format being played is not the expected one

Now, there are a number of phrases that might be used to describe this situation – the most likely in a business context being “show-stopper”. This may quickly be followed by the dreaded question “Is this the right solution for us?”.

In my opinion JW Player is indeed a very good solution – the support forums are a great means for getting issues resolved, and there is usually willing assistance with helping you understand the complexities of producing video for the web. However, as with many situations, knowing what question to ask in this context can sometimes make the difference between getting a speedy resolution versus a gradual increase in grey hairs and heightened blood pressure!

TIME FOR SOME FOR ANSWERS

Having experienced a number of these issues first-hand, and worked through them, I thought it would be useful to share my findings. The following pointers should cover off the specific issues detailed above and will also cater for some of the “generic” issues:

ENCODING

One of the most common stumbling blocks (and therefore most likely to resolve issues) is ensuring the encoding used is correct. It has been said many times before and I’m sure it will be said again – encoding is key. This is particularly relevant when it comes to producing an MP4 which is intended for playback on multiple devices – desktop browsers, iDevices, Android phones/tablets and recent Blackberry phones.

According to the JW Player documentation, the following specification should be adhered to:

> Container format: MP4, headers at the beginning of the file (for seeking)
> Video format: H.264, Baseline profile, 480×270 pixels, around 400/600 kbps (kilobits per second)
> Audio format: AAC, Low Complexity profile, 44.1 kHz stereo, 96 kbps

Also, within the support forums you will regularly see advice to “encode video to H.264 using Handbrake” and the JW documentation continues to say:

Handbrake has a built-in present called iPhone & iPod Touch, which has exactly the right settings.

Whilst good advice, this actually falls short of ensuring you get the correct results. One important option that is unchecked by default in all Handbrake profiles is the “Web optimized” option. Selecting this option reorders some of the metadata within the encoded video to the start of the container. This ensures the video can stream and allows the user to jump to specific points while the rest is downloading in the background.

If the metadata appears at the end of the video, the whole file needs to be downloaded before the video will start playing. I therefore highly recommend checking the “Web optimised” option when you encode. If you have files that are already encoded and are concerned that the metadata is not at the start, you can use another tool such as qt-faststart to move the metadata retrospectively.

More recent versions of Handbrake include the “iPhone 4”, “iPad” & “AppleTV” profiles which have the “Large file size” option checked by default. They also include the “Maximum B-Frames” set to “3” along with a host of other default settings. I would advise considering these settings further when deciding which profile to use.

For further detail regarding HTML5 video support, encoding options and much more useful information check out the excellent “Video on the Web” resource by Mark Pilgrim

SERVER SETTINGS – MIME-TYPES & GZIP

MIME-TYPES

For video to be correctly rendered in WebKit based browsers, the MIME-type provided by the server must correctly correspond to the encoding used.

Specifically:

  • Firefox requires “video/webm” to be set for the “.webm” filetype
  • IE9 requires “video/mp4” to be set for the “.mp4” filetype

In terms of what to include in the type attribute of the HTML source tag, I would suggest only including the MIME-type string – don’t bother with the “codecs” element as this is largely ignored anyway. Note: Including the MIME-type here DOES NOT negate the need to have the server configured correctly – they are not related.

If you experience problems with MP4 video playing on earlier version of Android you may wish to explore removing the type attribute for the MP4 source – as suggested in Peter Gasston’s post.

GZIP

In a bid to try and save bandwidth, some web hosts will gzip everything by default – including video files. In Firefox and Opera, seeking will not be possible or the video may not play at all if a video file is gzipped. If this is occurring it would be prudent to check your server / hosts and disable the gzipping of media files.

QUICKTIME & FLASH

It goes without saying that in order to play video via the Flash player, the browser needs to have the Flash plugin installed. Don’t assume this is the case for all browsers and try to provide a positive fall-back option – this also helps to debug issues.

Flash v9+ is required to playback H.264 encoded MP4’s.

A less obvious gotcha is the dependency of Quicktime for Safari when playing native video on Windows. Again support for this comes via a plugin, which is commonly not installed. Failure to have this available, plus the lack of the Flash plugin can actually lead to no player being rendered in Safari.

IE SPECIFICS

IE7 & IE8 don’t understand the native video & source tags and so will fall-back to Flash mode. However there are two factors that need to be in place in order for this to happen:

Firstly the Flash plugin needs to be available.

Secondly the use of a HTML5 Shim in order to “register” the availability of the video tag. JW Player cannot interrogate and build a configuration object if the video tag has not been “modernised”. This can be done by including a 3rd party HTML5 Shim library or more sensibly by loading the JW Player library in the head – JW Player will modernise the video tag if needed.

ADDITIONAL VIDEO PARAMETERS

The following parameters and their impact can often be overlooked when it comes to implementing a native video tag in conjunction with JW Player:

autoplay () : include this attribute (no value) to let the video start as the page gets loaded. Use sparsely, since this may frustrate your viewers. Note that it does not work on iOS.
controls () : include this attribute (no value) to display a simple controlbar, provided by the browser. The design of this controlbar varies per browser (which is a common issue JW Player resolves gracefully).
height (number) : height of the video on the page in pixels – is required.
loop (): Include this attribute to let the browser continuously repeat playback of the video.
poster (string): the url of a poster frame that is shown before the video starts. Can be any PNG, JPG or GIF image. By default, it is not set (i.e. no preview).
preload (‘auto’, ‘metadata’, ‘none’) : This attribute defines whether the entire video is loaded upon webpage load, or whether only the metadata is initially loaded. Or, you can choose to not load the video at all upon webpage load. The default (on most browsers) is metadata.
src (string) : The URL of the video to play. This is not required, since source files can also be listed using tags.
width (number) : width of the video on the page, in pixels. Is required.

The JW Documentation also states:

… Plus there’s the inevitable browser bugs (especially on devices) and different looking controls that cannot be styled.

The JW Player helps to fix all these issues. It sniffs out the contents of the video tag – the sources, as well as the poster, loop, autoplay, etc.). It also detects the browser capabilities and then uses JavaScript to, for example, work around bugs, add new features or embed a Flash version of the player in place of the tag.

Given these statements you might expect the controls, autoplay, loop and preload attributes to be directly controllable via the video tag and for the results to be consistent across browsers/devices.

Sadly this is not the case. Furthermore, these seemingly supplementary attributes are the reason behind most of the unexplainable “generic” issues detailed above.

Firstly it’s important to note that, contrary to what is stated, JW Player does not directly make use of these parameters.

It does have similar acting parameters which can only be included as part of the JavaScript setup configuration (controlbar, autostart, repeat and bufferlength) but these are not derived from the original parameters and are also not directly included as part of the newly generated JW video tag – they are implemented through pseudo JavaScript methods.

The second part of this is understandable as the JW Player needs to override the browsers built in functionality with it’s own implementation in order to provide consistency.

If JW Player determines that it should render in HTML5 mode, then it simply creates an empty video tag stub with none of these attributes included. This is because it also doesn’t set the source of the video (or src in the case of a single playable video or playlist file) until the user clicks the play button (or the file link in the playlist).

Coming back to the first part, why not allow the attributes to act as the configuration settings for the JW Player – even if they ultimately translate to JW’s own implementation? Surely this is a bug… Actually I believe it’s an oversight that LongTail Video are happy to brush under the carpet.

DIGGING A BIT DEEPER

Lets update my original code with these parameters and work through what happens:

<html>
<head>

</head>
<body>
<!-- Navigation stuff -->

<!-- Content Area -->
<h2>Check this HTML5 video out!</h2>
<video height="270" width="480" id="myVideo" preload="auto" controls autoplay>
<source src="/static/bunny.mp4" type="video/mp4">
<source src="/static/bunny.webm" type="video/webm">
</video>
<!-- More content -->

<!-- Right hand column -->

<!-- Footer stuff -->
<script src="/jwplayer/jwplayer.js" type="text/javascript"></script>
<script type="text/javascript">
// <![CDATA[
jwplayer("myVideo").setup(
{
modes:
[
{ type: 'html5' },
{ type: 'flash', src: '/jwplayer/player.swf' }
]
});
// ]]>
</script>
</body>
</html>

The most obvious thing to note given this configuration is that the native video tag will be interpreted and rendered well before the JW Player library is loaded and the setup call is made. This will result in the following:

  1. The native video placeholder being rendered according to the width & height specified
  2. Native video controls being displayed over the video
  3. The video will be buffered in accordance with the preload setting
  4. The video will start to play as soon as enough data is available to do so

Then when the JW Player library is loaded and the setup call is made[*] the following will happen:

  1. JW will “overwrite” the existing (already playing) video tag with it’s own player furniture comprising a number of nested div tags and a new, empty video tag
  2. The poster image will be displayed
  3. The JW controls will be displayed over the poster image
  4. The video will not automatically start to play

[*] Depending on what follows the video tag in the rest of the page – how long it takes to be rendered, and in the case of external scripts, how much they block – this could potentially be a number of seconds later.

Clearly this doesn’t result in a great user experience and depending on your requirements/flexibility there are things that can be done to make it better, however it does raise an interesting question – what *actually* happens to the video that is already playing when JW overwrites it?

You may rightly expect either the browser, or the JW embed process, to handle the clean up of the existing video – removing it from the DOM and wiping clean any future demands the resource may have on the browser, network or memory… You might therefore be surprised to learn that this is not the case!

Through my testing (yes – I ended up building a video testbed) I was able to determine the following:

If the native video is given enough time[1] to start doing “stuff”[2] before it is wiped out by the JW Embed process, the video will continue to exist in memory, can continue to make demands on the network and amazingly may even continue to play.

[1] Typically I found this to be anything over 0.5 sec.
[2] Either buffering data as a result of the preload attribute being set (or more likely not being set – see below) or downloading and playing as a result of the autoplay attribute.

The result of this situation leads to all manner of problems. I have actually seen, and more importantly can reproduce, the “phantom video” causing the following:

  • The newly JW embedded video delays playing until the “phantom” has finished downloading – misleadingly this can make you think the video encoding is wrong as it resembles the experience of having the metadata at the end of the file
  • The newly JW embedded video won’t play until the “phantom” has finished playing – resulting in a black screen and audio only
  • The newly JW embedded video buffers infinitely – this is as a result of the JW video believing it has buffered enough data to be able to play but actually it’s buffer being empty. It gets confused with the “phantom” video’s buffer which is ready to play – but never does
  • In the case of Firefox where it would natively play the WebM version, coupled with the default JW playback mode being “Flash-first” (supporting MP4), both videos are played simultaneously resulting in what appears to be echoey audio

Because the manifestation of these issues is closely related to the time taken between rendering a native video tag on the page and the JW Embedder overwriting it, you can now start to see how a page that one day has a working video, might suddenly start to exhibit inconsistent behaviour – maybe an external script library is blocking just a bit longer than normal (jQuery on a CDN for example) or you’ve upgraded to a new version of JW Player and the files are not cached the first time you revisit the page.

When you consider the negative impact preload and autoplay can have depending on your embedding approach, coupled with the fact these two attributes are not respected on the majority of mobile devices it becomes “understandable” why LongTail Video might prefer you not to use them – it just would be nice if the documentation were to accurately reflect this.

AND THE SOLUTION?

You may deduce that it’s as simple as to not include the preload or autoplay attributes in the native video tag. However when checking the current default values for preload (in the case that you don’t specify the attribute) I found the following:

Chrome Firefox Safari Opera <IE9 IE9 iOS 5 Android
auto "" auto metadata n/a metadata auto auto

Which suggests that most browsers will attempt to load the entire video upon webpage load – or more accurately, when the browser has enough information about the video tag to be able to render it – again this is not in line with what the JW documentation states.

THE RECOMMENDATION

Phew… that’s a lot to take on-board.

Here’s a quick summary of the approach I recommend in order to minimise the likelihood of problems when using JW Player to progressively enhance an HTML5 video tag.

  • Ensure your videos are encoded correctly
  • Ensure the MIME-types are set correctly on your server
  • Ensure GZIP is not enabled for media files
  • Load JW Library in the head of your web page – although the JW Documentation actually recommends this (in places) the supporting examples and suggestions in forums often overlook the importance of this
  • Don’t include the autoplay attribute in the native video tag
  • Include preload="none" in the native video tag
  • Include the setup script as close as possible inline after the video tag – in order to minimise the time it has to do “stuff” before it is overwritten
  • Include the JW equivalent versions of preload and autoplay in the setup configuration if really needed

HOLD ON, WHAT IF I ACTUALLY WANT TO preload OR autoplay…

Indeed… there is a use case scenario where the JW Player is not actually initiated (JavaScript is disabled) and so the native video tag (and it’s attributes) will be the mechanism used to render and play the video.

Following my recommendation unfortunately means that the native video will not preload or autoplay in this scenario.

On the one hand, maybe this is a small compromise to pay in order to overcome the wider issues of inconsistent video playback.

On the other hand, surely there is a way to clean up the “phantom video” issue and have the native video attributes *actually* drive the configuration of the JW player.

Oh, and while we’re at it, wouldn’t it be great if:

  • JW Player was fully configurable through HTML tags – including playlists – with no JavaScript knowledge required
  • Hybrid playlists could be built which were capable of merging YouTube playlists with individual videos – in Flash and HTML5 mode
  • And JW Player acted responsively within the body of the page or viewport of a mobile device whilst maintaining the correct aspect ratio

Well I thought it would be great. Which is why I’ve enhanced my extension to JW Player so that is does exactly that. You can find more details about this here.

WAIT NOW… WHAT’S THIS… JW6 IS COMING…

And unbelievably LongTail Video are removing support for embedding over a video tag!

Taken from the beta documentation:

JW6 supports many functionalities not supported by a video tag (e.g. Flash, RSS, YouTube, RTMP, VAST). Additionally, longer term promises of an inline video element (progressive enhancement, video SEO, accessibility) are not there yet. Therefore, JW6 does not support the enhancing of a video element yet.

Wow… After being leading advocates of the approach to progressively enhance HTML5 video tags and all the logical reasoning that goes with it, I find it amazing that LongTail Video can drop support for it in such a way.

JW6 supports many functionalities not supported by a video tag – yeah right, but a video tag supports more of these in a fall-back position than a div ever will!

A cynical person might think that by dropping support for this embed method, LongTail Video are looking to reduce the number of support issues that potentially fall under their remit. I imagine that what will actually happen when developers come to implement JW6 is they will simply wrap div‘s around their native video tags and target these div‘s instead – potentially creating more issues than solving. Only time will tell.

In the mean time, I’ve just added a new requirement to the JW HTML CONFIG JavaScript extension I’ve been developing – to support JW6!

 

Updated – 18th October 2012

Received an interesting response to this article from Jeroen @ LongTail Video today:

As James says at the bottom, all embed issues will be solved with JW6. It’s a full rewrite of our HTML5 mode. The MP4 encoding issues (MOOV atom, baseline, etc) remain though. Luckily, WebM seems about to leave the stage soon.

We specifically chose to not support the video element, because of all the issues we ran into with JW5. We do evangelize HTML5 video, for its longer-term promises. We do not support decorating the video element in JW6, because those promises have not yet been fulfilled. There’s no search/accessibility/fallback advantages for using a video element today. There’s also many reasons for preferring Flash in HTML5 browsers (codecs, streaming, security, performance, consistency, advertising, etc).

Over time, we WILL support decoration of video tags as an embed method. It’s just not yet feasible. We have two blog posts lined up for this, in which we’ll elaborate.

Firstly I’m guessing Jeroen actually means “…Ogg seems about to leave the stage soon…” not WebM.

Secondly, it’s interesting to note that JW6 has switched the default playback mode from Flash first in JW5 to HTML5 first – which conflicts with the idea that there are “… many reasons for preferring Flash in HTML5 browsers…”. HTML5 mode is all about presenting and decorating the native video tag.

For me, it still makes more sense to start with a native video tag, whilst providing good documentation (and even a code fix?!) to highlight and overcome the “Phantom Video” issue. This will at least ensure some accessibility and SEO benefits over and above an empty div tag – which relies on JavaScript enhancements to function at all.

Leave a reply