Lecture given on Sept. 5, 2012, to the RTF 319 class at the University of Texas at Austin.

1. Introduction to
Digital Media & New
Media
William J. Moner
September 5, 2012
RTF 319 – Intro to Digital Media

2. Reading Due Today
• Manovich, L. (2002). The Language of New
Media. pp. 43 – 74
• Sefton-Green, J. (2005). ―Timelines,
Timeframes and Special Effects: software
and creative media production.‖

3. HOUSEKEEPING
• Moner’s office hours are now
MONDAY 9 – 12, CMA A6.117
• Labs begin next week
(Tuesday/Wednesday)
• Introduction to Classes shared drive

4. TODAY
• Introduction to NEW MEDIA
• Discussion of SOFTWARE STUDIES as part
of MEDIA STUDIES
• Characteristics of DIGITAL MEDIA

5. Lev Manovich
Professor at the Visual Arts
Department, University of
California - San Diego
(UCSD)
Director of the Software
Specializations:
Studies Initiative at California
Institute for digital art and history
Telecommunications and theory of digital culture
Information Technology. digital humanities
(CALIT2).
http://www.manovich.net

6. Julian Sefton-Green
Principal Research Fellow, London School of
Economics, University of London, Department of
Media and Communications
Specializations in youth media and technology
Community and education policy
Creativity, learning, and arts research
http://www.julianseftongreen.net/

7. Lev Manovich (2002)
THE LANGUAGE OF NEW MEDIA

8. NEW MEDIA
• Convergence of two distinct historical
trajectories
• COMPUTATIONAL MEDIA
• Babbage’s early computer
• Information systems, computers
• AUDIOVISUAL MEDIA
• Daguerreotypes
• Photographs, movies
• Recording technologies
• Transmission: television, radio

9. The Language of New Media
• The arts and media have distinct visual languages
• strategies to organize information and viewers’
experiences
• Material properties of the computer
• its use in modern society, the structure of its interface
and key software applications
• Contemporary visual culture
• The internal organization, iconography, iconology and
viewer experience of various visual sites in our culture
• Contemporary information culture
• maps, wayfinding, navigation, visual cues

10. Representation in New Media
• Any new media object represents, as well as
helps to construct, some outside referent
• a physically existing object; historical information
presented in other documents; a system of categories
currently employed by culture as a whole or by some
social groups or interests.
• New media representations are also always
biased.
• They represent / construct some features of physical
reality at the expenses of others, one world view
among many, one possible system of categories
among numerous others possible.

11. Principles of New Media: Numeric
All new media objects are composed of digital
code (bits: 0s and 1s)
• Can be described formally
• Subject to algorithmic manipulation

12. Principles of New Media: Modularity
• Founded on various ―discrete samples‖
• Image = ―pixels‖ (picture elements, 1x1 square)
• 3D image = ―polygons‖ (triangles + for meshes)
• 3D object = ―voxels‖ (3D elements, 1x1x1 cube)
• Objects can be combined into new media
compositions without losing their individual
characteristics

13. Principles of New Media: Automation
• Role of programming, scripting, and
sequencing
• Algorithmic in nature
• ―Automatic‖ rendering of scenes,
backgrounds, etc.
• In computer games, enemies can only
perform certain actions in reaction to the
avatar’s movements (avatar = on-screen
representation of user)

14. Automation…
• Web and distributed systems rely on
automation
• Consider how Google *exists*
• What does Google *do*?

15. From Manovich (2002)
―By the end of the twentieth century, the problem became
no longer how to create a new media object such as an
image; the new problem was how to find the object which
already exists somewhere.‖
―This led to the next stage in media evolution: the need for
new technologies to store, organize and efficiently
access these media materials. These new technologies
are all computer-based.‖
―The emergence of new media coincides with this second
stage of a media society, now concerned as much with
accessing and re-using existing media as with creating
new one.‖ (p. 55)

16. Principles of New Media: Variability
• Closely connected to automation and
modularity
• Automation: ―instead of identical copies a
new media object typically gives rise to
many different versions … in part
automatically assembled by a computer.‖
• Modularity: ―media elements maintain their
separate identity and can be assembled into
numerous sequences under program
control.‖

17. Variability
• Media elements are stored in a database
• Can separate ―content‖ from interface (think
of the Yelp website versus the Yelp app)
• User information can lead to automatic
customization (think Yelp’s geographic
awareness)

18. Variability
• Branching interactivity
• pathways emerge from user interactive choices
• Hypermedia experiences assembled through
different pathways
• think of how your experience differs from that of
someone else’s when playing Sims Online or
browsing YouTube

19. Variability
• Periodic updates ensure the user experience
is never quite the same from version to
version or moment to moment
• Scalability in the visual and informational
sense
• Think of all of the information contained in Google
Maps versus the way we interface with a
traditional paper map
• Except in virtual worlds, we could zoom infinitely!

20. From Manovich (2002)
In a post-industrial society, every citizen can
construct her own custom lifestyle and "select" her
ideology from a large (but not infinite) number of
choices. Rather than pushing the same
objects/information to a mass audience, marketing
now tries to target each individual separately.‖
New social logic: ―Every visitor to a Web site
automatically gets her own custom version of the
site created on the fly from a database. The
language of the text, the contents, the ads
displayed — all these can be customized‖ by
interpreting data. (p. 60)

21. Variability
• In new media (particularly interactive media),
the user rests control from the storyteller
• ―By passing these choices to the user, the
author also passes the responsibility to
represent the world and the human condition
in it.‖ (p. 62)

22. Principles of New Media: Transcoding
New media’s structures now follow the
established conventions of computer's
organization of data.
The examples of these conventions:
• data structures such as lists, records and arrays
• substitution of all constants by variables
• separation between algorithms and data
structures
• modularity

23. Implications for storytelling
Two distinct layers: the “cultural layer” and
the “computer layer.”
Cultural layer: Computer layer:
• Encyclopedia and a • process and packet
short story transmission
• Story and plot • sorting and matching
• Composition and • function and variable
point of view • computer language and
• Mimesis and a data structure
catharsis
• Comedy and
tragedy

24. Manovich’s Project
• Distinguish ―digital‖ from ―new media‖
• Establish a domain of study called
software studies
• Moves beyond traditional media studies domains
(radio, television, film, media industries, media
technology) to introduce the logic of
computerization, digital communication, and
human-computer interface (HCI)
• Finds a middle ground between HCI and
media/cultural studies

25. PRODUCTION SOFTWARE

26. Software Functions
• Cut and paste
• Creating new / opening / saving documents
• Image manipulation includes:
• Cropping
• Rotating
• Resizing
• Framing

27. Creative Media Production
• Sefton-Green asserts that mastery of
Photoshop is a core or key digital production
skill
• Competency in digital imaging becomes an
important concern in media literacy
• The image (creation, manipulation)
underpins nearly all digital media production
endeavors

28. Filtering
• Addition of special effects to digital images
• Blurs, warps, distortions
• Preset options and customizations available

29. Filters in other software
• Audio effects (envelopes, distortion, reverb,
autotune)
• Video filters (lighting and gels, diffusion)
• Motion graphics (motion blurs, tweening)

30. QUESTION:
• Do you believe that once you’re familiar with
one production suite, you can easily map the
software skills to other production suites?
• (e.g. once you know Final Cut, do you think
you could master Maya more easily?)

31. Photoshop and Layers
• Key concept
• Layers can be used to ―stack‖ elements of
an image
• Also, you can:
• Blend layers using various algorithms
• Mix light sources and colors together
• Combine and merge layers into a single layer

32. From Layers to Timelines
• Timelines are suited for image sequences
played back in rapid fashion frame by frame
• VIDEO, ANIMATION
• Timelines are also suited for audio editing
and sequencing

33. Channels and Tracks
• In audio, layers may be referred to as
channels or tracks in an homage to
traditional audio editing practices
• In video, layers may be referred to as
channels

34. The Takeaway
• Software for digital production bear the burden
of using traditional metaphors for their given
spaces
PHOTOSHOP = DARKROOM EDITING
AUDACITY = TAPE-BASED AUDIO STUDIO
ENGINEERING
AVID = REEL-TO-REEL VIDEOTAPE EDITING

35. Moving beyond metaphors
PROGRAMMING AND SCRIPTING
• Adobe Flash permits (and, quite frankly,
requires) a level of aptitude in programming
to unlock the value of the IDE (integrated
development environment) using
Actionscript
• After Effects (a timeline-based visual
effects editing app) permits use of
Javascript for common scripting behaviors
(looping, randomization)

36. Programming
• Flash makes use of objects which emerge
from a class of items (think Object =
baseball; Class = ball)
• Objects can possess properties and have
methods (or actions) associated with them
• Object-oriented programming becomes a
fundamental competency when creating
modern games, interactive environments,
and sequences

37. Interactivity
With the advent of software and gaming, the
communicative capabilities of the internet
(communication protocols), the visual nature of
the web (HTML/CSS) and now with the
abundance of multi-touch screens (HCI,
UX/UI), interactivity becomes a new mode of
thinking about visual experiences and
storytelling

38. Databases
How do we organize digital materials?
• Libraries of common sound effects
• Archival footage
• Attaching tags, labels, information to media
Digital technologies work via the logic of
data

39. Synaesthesis / Translation /
Comparative Effects
• Be aware / ―beware‖ of metaphors
• iTunes does not just sell ―tunes‖
• Channels in Photoshop refer to channels of
light (Red/Green/Blue)
• Metaphors may be a ―head fake‖
• Translation of experiences between one discipline to
another
• Be cognizant of the differences

40. For Monday
• Read Manovich (2002) (pp. 43-74; 115 –
160)
• [the entire text is recommended reading]
• Read Kleinrock (2010)
[RECOMMENDED]
• Read Okin (2005) & Jordan (1999)