Surprise videos

I’m updating my post of 14 December 2013 #174 How to slow down time, which started with a reference to slow motion video capture. Slo-mo recordings via smartphone struck me as a way of revealing what is otherwise hidden from view by extending our ability to attend to temporal detail. I created a slow-mo video of a ball bearing dropped in a glass of water.

Since then there have been major changes in the production and consumption of short videos:

• easy, speedy and competent short-form video creation
• rapid video publication on social media
• ready availability of content designed, generated or edited to grab attention
• social media’s automation of content recommendations based on viewing habits
• seamless streaming of recommended content
• monetisation of short video publication motivating cycles of further video production.

The immediate temporal effect for the consumer is a seemingly limitless source of time-filling speeded-up hyper-engaging content.

Shock events

Surprise and calamity play important roles in short on-line video. I’m reminded of C.S. Peirce’s identification of the semiotic event that is immediately arresting (a “dicent indexical sinsign”). See post Semiotic shock.

As I have explored in an earlier post (What a calamity!), a shock event presents as a class of sign that is raw, immediate, sudden, with an obvious cause, and with immediately accessible qualities: a flash of light from a faulty circuit, an explosion from a firecracker, a tray of dishes crashing to the floor, or something fatal. 

Certain short form videos are designed to deliver this immediacy, either vicariously in the response of the video subjects (a startled gorilla watching a magic trick) or in the viewer (“I didn’t expect that!”), or the pleasure derived from repeat viewings of the surprise moment. The immediate consumption of such content requires very little from the viewer in terms of reflection or analysis for its effect. See blog post Experts are better than algorithms.

AI generation

Now we have tools for creating videos, including AI video generation from text prompts, automated editing and scene detection, frame interpolation and motion smoothing, resolution enhancement and restoration, and real-time filters and effects.

Thanks to convincing automated video production and enhancement we are also unsure about the provenance and authenticity of such imagery: animals in improbable circumstances, e.g. a domesticated cat terrorising a streak of tigers; extreme geo-weather events, e.g. a tsunami enveloping a city; before-and-after transformations. e.g. building components fly into place; danger narrowly avoided. e.g. diver plunges from a great height into a rock pool.

Social media recommendation systems now stream video not simply by topic but by monitoring viewer behaviour: how long you watch, when you pause, what you replay or skip. Content is sequenced to maintain continuous engagement calibrated to individual attention rhythms. Your feed calculates how quickly you tire, what reactivates you, and how to prevent exit.

My first AI video

RunwayML belongs to a class of cloud-based platform for creating and editing video using AI. From a spatial point of view it sidesteps the need for 3D modelling and rendering to deliver plausible movement, parallax and other geometrical aspects of spaces and objects. So you can effectively move into static 2D images as if 3D.

I prompted RunwayML with one of my photographs and some explanation: “This is a 180 of a straight row of Buddha statues in a sacred cave in Sri Lanka. Generate a video with the camera moving along the row and with the row, number of statues and the cave extending to infinity. Turn the camera to the right so we can see the row of Buddhas disappearing to infinity.”

The AI generated a 10 second video. I think I would need to introduce further prompting to realise the “infinite Buddha” scenario, though the app managed to extend six statues to ten and extend the cave space beyond the confines of the photograph.

The surprise element here is that the AI was able to infer highly plausible movement from a static image, rather than the impossibility of an infinite array of Buddhas, but that’s for another time.