The document discusses doing digital multimodal composing in English for specific purposes courses. It provides an overview of emerging needs of ESP learners, digital multimodal composing, and research findings and evaluation. It defines digital multimodal composing as going beyond traditional writing forms to include other modes made available by digital media like videos, infographics, posters and brochures. Benefits of digital multimodal composing include meeting real-world learner needs, enhancing language teaching strategies, engaging students with authentic audiences, and increasing learner autonomy.

1. Doing digital multimodal composing on
English for specific purposes courses
Christoph A. Hafner
Department of English, City University of Hong Kong
Workshop presented at English Department, Wenzao
Ursuline University of Languages

5. www.lsppc.org

6. Overview
1. Emerging needs of ESP learners
2. Digital multimodal composing
3. Research findings and evaluation

7. What do you mean by English for Specific Purposes?

8. Emerging needs of ESP learners
The role of digital media in contemporary
communication

9. Affordances
of digital
tools
Ways of
doing
Ways of
relating
Ways of
meaning
Ways of
being
Ways of
thinking

10. Scientists urgently need to be able to
speak with clarity to funders, policy-
makers, students, the general public, and
even other scientists.
I’d like to try a playful experiment. Would
you be willing to have a go at writing your
own explanation of what a flame is – one
that an 11-year-old would find intelligible,
maybe even fun? The Centre for
Communicating Science is looking for new
ways to light up people’s minds with
science, and you might point the way.
We’ll try out the entries on real 11-year-
olds and see which work best. (Alda, 2012,
p. 1019)

12. Some other examples
• Crowdfunding proposals (Mehlenbacher, 2017)
– Hybrid genre, aimed at multiple audiences
• Academic blogs (Blanchard, 2011; Luzón, 2012, 2013)
– Writing for a diversified audience
– Interacting with and responding to that audience

13. Additional forms of expression
JoVE began in response to a
universally experienced problem
within the world of biomedical
science: today only 10-30% of
published scientific articles can be
successfully reproduced. As a result,
scientists and researchers around
the world end up spending months
and years of time, effort, and
funding simply trying to replicate
the findings of other labs instead of
advancing toward new discoveries.
(www.jove.com, emphasis in
original) Journal of visualized experiments

14. How a protocol is being executed is being captured visually.
So there is like no mistake on how it is being done,
especially if, say, if we’re doing micro-injection of DNA into
eggs, okay, how do you handle the pipette? How deep do
you penetrate it? So there’s lots of finer details which are
being sort of captured visually (…)
(Interview with biologist at City University of Hong Kong)

15. Another example
• The Article of the Future (Pérez-Llantada, 2013)
– ‘More dynamic’ and ‘user-friendly’ with ‘enhanced
elements’
– Did not affect ‘actual text-composing and reception
practices’

16. ‘The ability to produce and
understand text-visual
interrelations is now an
essential component of an
academic literacy.’ (Hyland
& Hamp-Lyons, 2002, p. 8)
‘In fact, it is now no longer
possible to understand
language and its uses
without understanding the
effect of all modes of
communication that are
copresent in any text.’
(Kress 2000, p. 337)

17. Digital multimodal composing

18. A definition
• Digital multimodal composing in ESP classes
means going beyond traditional writing forms
to include other modes made available by
digital media.
Digital videos Infographics Posters Brochures
Academic
text/image
interaction

19. Benefits of DMC
• Meets the real-world needs of learners in a digitally mediated world;
• Enhances state-of-the-art language teaching strategies like task-
based language teaching and learning, by allowing for the easy
integration of multiple skills;
• Engages students with authentic audiences, providing real-world
motivation;
• Is ‘voice-enhancing’ and can ‘embolden struggling writers to express
themselves’ (p. 82);
• Heightens genre awareness as a result of transformation processes
that occur in the composition process;
• Increases learner autonomy and encourages independent language
practice.
(Belcher, 2017)

20. The controversy
• Does DMC ‘distract’ learners from the ‘real’
work of language learning?

21. Distracting or motivating?
Candy: At the beginning of the video, we want to attract
concentration on the video. So introduction is –
we make a lot of effort on it.
Lindsay: Would you make the same effort if you were just
doing a stand-up presentation with PowerPoint?
Jane: Probably not.
Candy: No. Introduction is not that important maybe in
PowerPoint, maybe the information, maybe the –
in the middle of your presentation, is the main
point, so introduction is not that important. (Focus
group: Phase 2)

22. Implementing DMC
A project-based course in English for science

23. Background and context
• University course in English for science students
– Applied Biology
– Applied Chemistry
– Applied Physics
– Architectural Studies
– Computing Mathematics
– Environmental Science and Management
– Surveying

24. English for science project
A. Digital video project
B. Written scientific
report
English for
science
project
A1. Reading/data
collection
A2. Scripting/
storyboarding
A3. Performing/
recording
A4. Editing
A5. Sharing
B2. Writing
B1. Reading/
outlining
B3. Editing/
proofreading

25. Sample topics
Dim the lights
(2013)
Taste me if you can
(2009)
Vitamin C
(2012)

26. Technology and support

27. Addressing
multimodality by
analysing ‘models’
• BBC documentary
• Al-Jazeera documentary
• Student documentary
Rhetorical
strategies
Linguistic
analysis
Narrative
images
Conceptual
representations
Offer and
demand
Camera angle
Visual
coherence

28. Scaffolding the project
• Week 2, students to form groups for scientific
documentary;
• Week 4, data collection for scientific documentary
complete;
• Week 5, script complete;
• Week 7, final video uploaded and shared through
YouTube.
• Week 11, students to submit a draft scientific
report for individual feedback

29. Negotiating consent
In order to protect privacy in these online
environments, we ask that you follow these
terms of use for online learning on GE2401. In
particular, you agree that you will:
• Only use your first name in our writing;
• Only post pictures of people if you have
asked for their permission;
• Be responsible and sensible in our writing;
• Be positive and constructive in our
comments;
• Respect other users’ opinions and advice;
• Post only original works (for all text types
and files);
• Respect intellectual property; and
• Be responsive and open to opinions and
advice.

30. Blind as a bat

31. Some sample activities

32. Discoursal identities
Hafner, 2014

33. Student perspectives on the task

34. The challenge of attention
The first impression of audience is the critical point
to determine the success of a good documentary
since if the audience’s attention cannot be attracted
at the beginning, they will have no interest to
continue to watch the video even the information is
rich and constructive. I agree with t01_john. He said
that visual stimulation would be the easiest way to
make them remember the video. [Student blog post,
October 7th, 2009 at 11:22 pm]

35. The challenge of multimodality
I think the most challenging thing is how to give
an attractive and interesting present[ation]
because we use lots of method involved in our
video. For example, use pictures, use a narrator,
stand in front of the camera for speaking and use
music or many, many elements we involved in
order to give a whole product to make it more
interesting. [Student focus group interview]

36. Thinking about multimodality
• Moving images and animation
• Charts and tables for scientific data
• Subtitles
• Different camera angles and lighting,
• Background music
• Sound effects
• Interesting locations
• Interesting participants
• Facial expression

37. 3 Cases and rhetorical devices
• Investigation of a startling fact
– Did you realize there is a blind spot in your eye?
• Investigation of a social issue
– Why does the cafeteria food taste so bad? Is it only
the taste, or is it the smell as well?
• Investigation of a personal issue
– Why can’t I taste this orange juice? Is there
something wrong with me?

38. Case #1
Investigation of a startling fact

39. Student identity
• Student as ‘scientist’
• Narrator role is backgrounded, increased social
distance between the producers and their
audience

40. Opening: Case 1, shot 3

42. Background and theory: Case 1, shot 10

44. Discussion: Case 1, shots 51-52

46. Case #2
Investigation of a social issue

47. Student identity
• Student as ‘reporter’
• Narrator role is more prominent, with an on-
screen narrator (the reporter), who appears at
various points throughout the documentary
• Watch for intertextual references and use of
sound

48. Opening: Case 2, shots 1-8

50. Case #3
Investigation of a personal issue

51. Student identity
• Student as ‘traveller’ on a ‘journey of
experiment’
• Narrator role is prominent, the narrator is a
part of the action as in a first person
documentary
• Watch for the range of visual information and
effective use of sound

52. Opening: Case 3, shots 55-65

54. Semiotic remediation
Hafner & Miller, 2019

55. TRANSFORM
Video documentary
Scientific report
For a public audience
To educate and entertain
Student as...
• Scientist
• Reporter
• Traveller
• Secret Agent
For a specialist audience
To inform and persuade
Student as scientist

56. Key questions
• For each genre:
– What content is appropriate?
– How should it be organized?
– How should it be expressed through lexical,
grammatical and multimodal resources?
– How actively can I engage with the audience?
– How forcefully can I present my stance?

57. Establishing novelty

58. Establishing novelty
Do you know what the
problem of light pollution
is?
Pun and So (2011) present
the night-sky brightness
taken at 199 distinct
locations in Hong Kong
and state that the light
pollution in Hong Kong is
severe. However, they do
not further investigate the
problems of light pollution
and the relationship
between light pollution and
other parameters.

59. Methods

60. Methods
We have organized a group
of special agents to search
for the answer. … We have
selected four locations for
our study, including Sai
Kung, a countryside, Mong
Kok and Causeway Bay,
urban areas, and Sha Tin, a
new town.
Four locations
including Causeway
Bay, Mong Kok, Sha
Tin and Sai Kung,
were selected for this
study because they
show different degrees
of urbanization.

61. Presenting results

62. Presenting results
Narrator: Causeway Bay is also one
of the Hong Kong’s major shopping
districts. As there are many indoor
shopping campuses such as Times
Square and Sogo, many tourists will
live there…
Do tourists living in Causeway Bay
think that light pollution is a great
problem?
The light intensity in Causeway Bay
is 550 LUX.
Fig. 3 shows the average
scores given by residents to
assess the negative impact on
their daily life by light pollution
in the daytime.

63. Interpretations and claims

64. Interpretations and claims
Host1: Attractions should
be lightened up. Tourists
love light. They enjoy to
discover more about the
city.
Fig. 3 and 4 suggest
that in reality, greater
light pollution may not
be perceived as
leading to more
discomfort.

66. Collaboration

67. Digital tools used
• Communication tools
– Facebook groups
– WhatsApp messenger, Facebook messenger
– Email
• Collaboration tools
– Google Drive
– Dropbox

68. Collaborative practices
• supporting/motivating
• managing the group/task
• negotiating the text
• sharing files and links
• requesting and giving feedback
• peer teaching and learning from peers

69. Conclusions