Digital Fiction iPad Project: the Good and Bad Stuff

I thought it might be interesting to reflect on how we’re finding the iPad as a development platform regarding our latest digital fiction project ‘Changed‘, bearing in mind that we’re not using the Apple SDK or exporting an App from Flash CS5 to produce this piece.

Changed is an atmospheric story set beneath a roadway tunnel. It’s based on a script by Lynda Williams (see Grace, a short film which Lynda wrote, here). It contains a mix of text, video, audio and some elements of interactivity. It’s self-reflective and strongly visual, the tunnel itself forming the ‘canvas’ onto which all other aspects of the story unfold.

The work is being designed as a ‘Web App’, which means it can be found on Mobile Safari on the iPad, bookmarked, and then added to the iPad’s Home Screen as if it were a native App. Theoretically, once downloaded, it should run offline where no internet connection is available. We’re also looking to try and keep it compatible with desktop computers/browsers.

It’s a bit of an experiment into what’s possible – but here’s what we’ve liked and disliked about the iPad development part thus far:

The iCool Stuff

The iPad’s natively smooth touch/scrolling navigation method bypasses the need to create easing effects when it comes to exploring large-scale websites/environments. As such, it feels nice exploring long/wide site designs and text appears crisp and sharp. Everything generally looks good on the iPad – the brightness and resolution of the screen is impressive and it’s nice to be so ‘close up’ to what you’re experiencing and to be able to experience it without being sat up to a monitor or laptop with a keyboard and mouse.

HTML5 elements are generally rendered accurately and most jquery/javascript routines work in the mobile version of Safari (via jQuery Touch). Good support for CSS3 and most Webkit tips and tricks seem to carry through fine.

Bookmarking to the Homescreen results in ‘Web Apps’ looking like native Apps and being able to use HTML5 Manifest to ‘cache’ data. This means Web Apps should work offline, ie, even when there’s no internet connection.

Web Apps run in full screen, which looks great. They can also have custom App icons and you can even design a ‘loading screen’ whilst they boot up. Using ‘meta tags’ it’s also possible to set the initial scale and resolution of the viewing area, thus customising the user experience.

Through the use of floating div tags and ‘ontouch’ javascript commands, objects can be set to ‘hover’ and be dragged around with your fingers, potentially allowing basic interactive environments/puzzles with very little coding effort. 24-bit PNG support combined with this allows for some impressive overlay effects.

The iShite Stuff

Building for the iPad without an Apple/the Apple SDK leaves you with no testing platform or emulator other than the iPad device itself, so development is notorously slow and painstaking and has to be done via uploading to a web server and continuously refreshing on the iPad to see the results.

There is only very basic web error message reporting available on the iPad (enabled in Settings) and often no error messages are reported when links fail or video/audio media does not connect or is in some way incompatible. It’s a huge matter of trial and error which again slows down any kind of development workflow.

All media is *by default* forced to appear ‘separate’ because of the way HTML5 tags operate. It is extremely difficult to make media look as if it’s blending together in any way; it always apppears totally boxed out. Video elements regardless of format always seem to display the ‘Q’ quicktime icon before they kick in to operation, which looks naff.

Activating one HTML5 media element automatically shuts off all others. So for instance, you cannot keep a continuous background sound active whilst also displaying a video sequence. This is not a limitation of HTML5 on desktop browsers, only on the iPad/Mobile Safari.

Video/audio elements will not autoplay because of restrictions imposed by Apple on the device. They claim that if you’re using 3G and potentially paying for your bandwidth, you won’t want media to start streaming automatically. So they’ve forced the function out.

Jpeg images cannot be larger than 1024 pixels in size. PNG images get around this somewhat, but even they fail to show up if they’re bigger than 2mb.

Even simple jQuery transitions stagger on mobile Safari, making straightforwardly smooth graphical fades/scales either slow or totally redundant.

For some reason Web Apps always start in ‘portrait’ mode even if you were in ‘landscape’ when you launched the App, which means you have to swing the device around needlessly.

HTML5 Manifest (for caching) does not report if it has any problems (other than saying it needs to connect to the internet) and automatically checks for updates at apparently random intervals. Sort of missing the point, then.

However

Despite the amazing number of bizarre limitations we’re not doing too bad and the piece is gaining a sense of identity and atmosphere, although it’s a long way from the look, feel and complexity of our other, Flash-based work. As an experiment into testing the creative possibilities of HTML5 and CSS3 alone, it’s proving worthwhile, and indeed continuously feeding useful techniques across into to our current (commercial) web design jobs.

I thought it might be interesting to reflect on how we’re finding the iPad as a development

platform regarding our latest digital fiction project ‘Changed’, bearing in mind that we’re

not using the Apple SDK or exporting an App from Flash CS5 to produce this piece.

Changed is an atmospheric story set beneath a roadway tunnel based on a script by Lynda Wright

(see Grace, a short film which Lynda wrote, here). It contains a mix of text, video, audio and

some elements of interactivity. It’s self-reflective and strongly visual, the tunnel itself

forming the ‘canvas’ onto which all other aspects of the story unfold.

The work is being designed as a ‘Web App’, which means it can be found on Mobile Safari on the

iPad, bookmarked, and then added to the iPad’s Home Screen as if it were a native App.

Theoretically, once downloaded, it should run offline where no internet connection is

available. We’re also looking to try and keep it compatible with desktop computers/browsers.

It’s a bit of an experiment into what’s possible – but here’s what we’ve liked and disliked

about the iPad part thus far:

The iCool Stuff

- The iPad’s natively smooth touch/scrolling navigation method bypasses the need to create

easing effects when it comes to exploring large-scale websites/environments. As such, it feels

nice exploring long/wide site designs and text appears crisp and sharp. Everything generally

looks good on the iPad – the brightness and resolution of the screen is impressive and it’s

nice to be so ‘close up’ to what you’re experiencing and to be able to experience it without

being sat up to a monitor or laptop with a keyboard and mouse.

- HTML5 elements are generally rendered accurately and most jquery/javascript routines work in

the mobile version of Safari (via jQuery Touch). Good support for CSS3 and most Webkit tips

and tricks seem to carry through fine.

- Bookmarking to the Homescreen results in ‘Web Apps’ looking like native Apps and being able

to use HTML5 Manifest to ‘cache’ data. This means Web Apps should work offline, ie, even when

there’s no internet connection.

- Web Apps run in full screen, which looks great. They can also have custom App icons and you

can even design a ‘loading screen’ whilst they boot up. Using ‘meta tags’ it’s also possible

to set the initial scale and resolution of the viewing area, thus customising the user

experience.

- Through the use of floating div tags and ‘ontouch’ javascript commands, objects can be set

to ‘hover’ and be dragged around with your fingers, potentially allowing basic interactive

environments/puzzles with very little coding effort. 24-bit PNG support combined with this

allows for some impressive overlay effects.

The iShite Stuff

- Building for the iPad without an Apple/the Apple SDK leaves you with no testing platform or

emulator other than the iPad device itself, so development is notorously slow and painstaking

and has to be done via uploading to a web server and continuously refreshing on the iPad to

see the results.

- There is only very basic web error message reporting available on the iPad (enabled in

Settings) and often no error messages are reported when links fail or video/audio media does

not connect or is in some way incompatible. It’s a huge matter of trial and error which again

slows down any kind of development workflow.

- All media is *by default* forced to appear ‘separate’ because of the way HTML5 tags operate.

It is extremely difficult to make media look as if it’s blending together in any way; it always apppears totally boxed out. Video elements regardless of format always seem to display the ‘Q’ quicktime icon before they kick in to operation, which looks naff.

- Activating one HTML5 media element automatically shuts off all others. So for instance, you

cannot keep a continuous background sound active whilst also displaying a video sequence. This

is not a limitation of HTML5 on desktop browsers, only on the iPad/Mobile Safari.

- Video/audio elements will not autoplay because of restrictions imposed by Apple on the

device. They claim that if you’re using 3G and potentially paying for your bandwidth, you

won’t want media to start streaming automatically. So they’ve forced the function out.

- Jpeg images cannot be larger than 1024 pixels in size. PNG images get around this somewhat,

but even they fail to show up if they’re bigger than 2mb.

- Even simple jQuery transitions stagger on mobile Safari, making straightforwardly smooth

graphical fades/scales either slow or totally redundant.

- For some reason Web Apps always start in ‘portrait’ mode even if you were in ‘landscape’

when you launched the App, which means you have to swing the device around needlessly.

- HTML5 Manifest (for caching) does not report if it has any problems (other than saying it

needs to connect to the internet) and automatically checks for updates at apparently random

intervals. Sort of missing the point, then.

However, despite the amazing number of bizarre limitations we’re not doing too bad and the piece is gaining a sense of identity and atmosphere, although it’s a long way from the look, feel and complexity of our other, Flash-based work.

As an experiment into testing the creative possibilities of HTML5 and CSS3 alone, it’s proving

worthwhile, and indeed continuously feeding useful techniques across into to our current

(commercial) web design jobs.

4 Responses to “Digital Fiction iPad Project: the Good and Bad Stuff”

  • Wow, so THAT’S what it’s like on the cutting edge of the APP world. As long as you’re wearing a lifeline, Andy, we’ll be watching and waiting to see what happens next.

    Seriously, it sounds like you have been doing a lot of good work on this, for which I, and I’m sure others, are very thankful. I am envious simply that you have an iPad, and with the added gift of creating works for it–you fashion the dreams, we will read them any way we can…

  • Thanks for these incredibly useful insights, Andy. Can’t wait to see ‘Changed’ on my prize new iPad (thanks to NMWP and pardon me for boasting ;) The iPad really is a marvellous device, despite the lack of Flash, which I seriously miss, but I love curling up with it and reading/viewing/playing media, new and old. It can be a wonderfully immersive experience, so Flashless and iShite notwithstanding, I think it’s definitely worth the effort developing Web Apps for it using web standards.

  • Mark bernstein:

    Why not do preliminary testing on Safari or Firefox? That gives you local connection, fast iteration, a debugger, and you’re still in WebKit.

    Then do final tuning for iPad.

    Seems especially desirable right mow, with iPad about to move to iOS 4!

  • Hi Mark,

    Thanks for this. Yes believe me, I have been testing in Safari and Firefox as much as possible! But things seem to work very differently on the iPad’s mobile browser, so I am finding I’m having to use the iPad itself quite a lot to make sure it’s still a happy bunny. Here are a few examples.

    1. Many HTML5 tags for audio and video do not work – or work differently – on the iPad even if they appear OK on Firefox/Safari desktop. Over-riding with javascript sometimes works, but not always.

    2. Some video/audio encoded file types do not seem to work on the iPad even though they do on Firefox/Safari desktop. There is no real explanation for this other than the iPad is just very fussy.

    3. Javascript touch commands are completely ignored by desktop browsers. (Although obviously this can be worked around with a script to detect the user-agent and switch to onclick if required.)

    4. Javascript/jquery that works great on Firefox/Safari desktop doesn’t always on the iPad. It either fails or works sluggishly, so it has to be checked out on the device. This is the case for even the most basic javascript scrolls and transitions.

    5. The iPad has all sorts of file size restrictions that are not imposed by desktop computers/browsers. I keep finding new limitations on this, although so far I’ve worked around them all.

    Perhaps some of these things will change, as you say, with iOS4.

    Best regards,

    Andy

Leave a Reply