SLIDES & NOTES - VWBPE - Composing in the Hybrid Metaverse 23,24,25-Mar-2023.pdf

Composing in the Hybrid Metaverse
This presentation provides definitions and
examples of cross-dimensional thinking,
hybrid tech, and metaverse with respect to
creative composition in ideas, vision, and
sound. Examples are given for the use of
hybrid ideas and technologies in music and
visual composition. Included are individual
and collaborative creative compositions
taken from a weekly live stream on Twitch.
Participants will see new ways to create,
communicate, and collaborate using
emergent technologies.
Participants will be inspired to express their
own creative ideas in new media.
This presentation shares results from
creative problem solving while live and
online, with others present and observing.
Included are times when compositions fail
and when they succeed - this relates to
finding meaning and interpreting results.
Also, the iterative use of visual and aural
expressions for the same composition -
new ways to approach a solution. Also, the
use of unconventional and conventional
ideas of music theory and metaverse to
make viable new compositions breaking
new ground.
This presentation shares examples of
composing using cross-dimensional
thinking and online presence.
1
Composing in the Hybrid Metaverse - From My World to Yours
jn: 23,24,25-mar-2023
Development One LLC
01 Composing in the Hybrid Metaverse
Ladies and gentlemen and those who identify otherwise, thank you for your presence
at this presentation on Composing in the Hybrid Metaverse.
While I do not necessarily advocate the methods and apps I present today, I do enjoy
sharing them with you and hope that you will enjoy the experience.

Composition Elements
● Intelligible
○ Ideas
○ Feelings
● Audible
○ Sound
○ Speech
○ Music
● Visible
○ Color
○ Shape
○ Image
Do It Yourself
● Word to sound
● Sound to composition
INTRO - Composing in the Hybrid Metaverse
Objectives
● See examples of composing in the
hybrid metaverse
● See examples of using
cross-dimensional thinking
● Experience ways to express new ideas
in senses and media
Agenda
● Tell it as a story
● Recreate it as a shared exercise
2
jn: 23,24,25-mar-2023
Development One LLC
● Tasteable
○ Flavors
● Smellable
○ Odors
● Touchable
○ Textures
Our objectives today are to see examples of composing and cross-dimensional
thinking and experiencing new ways to express ideas.
When I think of composition I think of arranging, in a pleasing manner, elements
perceivable by our senses. These elements include ideas, sounds, colors, flavors,
odors, and textures.
When I think of metaverse I think of an evolving ecosystem of physically and digitally
connected apps, and in particular I think of apps allowing instant, online, two-way,
co-creative collaboration.
In this presentation I will share my experiences composing online in Twitch over the
last 3 years, and conclude with inviting you to participate in some group exercises
where we collaboratively compose together.
So we will “Tell it as a story” and “Recreate it as a shared exercise.”

Patterns give inspiration
3
jn: 23,24,25-mar-2023
Development One LLC
Image courtesy Passion Astro
APP: Book of Changes <300 BCE Chinese text
for divination
https://www.youtube.com/watch?v=_OWaw7IWFWM
03 Patterns give inspiration
So a long time ago I was writing poetry in the form of 6-line sestets and each line had
an alternating pattern corresponding to the 64 I Ching hexagrams, with some lines
representing dynamic “yang” and some magnetic “yin”.
I went through 2 cycles of these and eventually had a 128 poem collection, some very
good I felt and some OK. But all fitting the pattern of 6 lines with each line having a
consistent length and meter
This slide shows the pattern, “Confining” which has a shorter length in the first, fourth,
and sixth positions. Correspondingly in my poem, if you were to count syllables, my
short lines have eight and my long lines have nine.
So then I thought, how might I put these to music?
I improvised a handful of themes that fit the words and meter and enjoyed them very
much and, again, some I felt were good and some OK.
For example, for the poem shown on the slide, here are the last two lines, long and
short, expressed musically:
~SOUND: 01
WHAT DOES THIS SOUND MAKE YOU THINK OF?

TYPE “_your thought here_” or “not sure”
So in this way I returned to something I had done in earlier days but ceased, namely,
attempting to compose music
I decided to use custom scales patterned on 6 because the poems were six lines
long.
And I ended up using pentatonic scales of 5 notes each because well… The best
explanation I can come up with in hindsight is that six lines have five spaces
separating them and I wanted the end line to return to the first line musically - at least
that's what I was trying for back then.
Along the way I ended up using a spreadsheet to lay out possible 5 note scales
because after all there are 12 tones possible

Pitch classes give alphabets
4
jn: 23,24,25-mar-2023
Development One LLC
APP: MuseScore software for music composition
and notation
04 Pitch classes give alphabets
This slide shows the twelve tones which sound like this:
~SOUND: 02
WHAT DIRECTION ARE THESE SOUNDS?
TYPE “_your thought here_” or “not sure”
I discovered that “notes” are “pitch classes” because our brains perceive notes as the
same if doubled or halved in frequency, So a low “C” on a tuba sounds like a high “C”
on a flute and so on.
Here let me digress: all the white notes on the piano make what is called a major
scale and if you use 3 black notes instead of 3 white notes you get what is called a
minor scale and these scales are related to each other because they all start on the
same note C. More on that later.

Music channels give inspiration
5
jn: 23,24,25-mar-2023
Development One LLC
APP: YouTube online video sharing service for
social media
05 Music channels give inspiration
By this time out of curiosity I began looking for YouTube channels related to music
and I found many of them. Some of my favorites are _12tone_ and
_MusicTheoryForGuitar_. This slide shows their channel thumbnails.

Notations give perspectives
6
jn: 23,24,25-mar-2023
Development One LLC
APP: Google Sheets online document editor for
spreadsheets
https://docs.google.com/spreadsheets/d/1OA8PqwQsixBK2a0eJQ-OxqvMA5n7lpBL1A1BT2dPqnY/edit#gid=1712634005
06 Notations give perspectives
I found that traditional notation for chords and music theory started simple and
became, I felt, confusing. So I began to make my own simplified and extended
interpretations
This slide shows C34 and C43 chords which are traditionally called C minor and C
major. But there are a lot of other possible chords, 120 in all, and believe me, I have
enumerated and used them all! *humor*
_12tone_ (the channel) also introduced the idea of one of the notes in a traditional
scale not having a traditional function so _12tone_ made up a suggested name: they
called it “ambivalent.” So I added “ambivalent” to my extended vocabulary.

Perspectives give options
7
jn: 23,24,25-mar-2023
Development One LLC
SCALE: hp01 C3443 7710 (heptatonic parallel
minor interval 3, major interval 4)
07 Perspectives give options
By this time I had run across more YouTube music videos and was hearing about
something called “negative harmony” which caught my interest.
In looking I discovered _MusicTheoryForGuitar_ who gave a simple elegant
explanation and demonstration of negative harmony which is that all minor scales are
reflections of major scales and vice versa.
If you draw the major scale on the upper arc of a circle and the minor scale on a lower
arc it looks like positive on the top and negative on the bottom and that's why it's
called negative harmony.
This slide shows the reflected minor and major scales and here is what they sound
like:
~SOUND 03
IF YOU HEAR A DIFFERENCE TYPE “diff”
IF THEY SOUND THE SAME TYPE “same”
IF YOU ARE NOT SURE TYPE “not sure”

Hybridizing gives options
8
Eight five-note parallel minor/major scales
SCALE: pp01 C3663 557 (pentatonic parallel
minor interval 3, major interval 6)
jn: 23,24,25-mar-2023
Development One LLC
https://docs.google.com/spreadsheets/d/1jZ04dSSSmYkioE6Z0opwX1kdcR--Kx_UNzMghBnl5ZQ/edit#gid=0
08 Hybridizing gives options
So I had everything I needed: I was going to make 5 note scales and I was going to
make them with a matching major and minor version each and they were going to be
perfect reflections of each other.
Using my trusty spreadsheet I figured out all kinds of possible scales and ended up
picking a core set each with its own major and minor interval.
Here is what one of the new scales sounds like:
~SOUND 04
IF YOU HEAR A DIFFERENCE TYPE “diff”
IF THEY SOUND THE SAME TYPE “same”
Let me rewind a little in the timeline. As I said, I was improvising on my home piano
and in order to write down my ideas I was using Post-it notes and a very awkward
notation writing down each note by hand using letters of the alphabet and numbers
from the number line
So I looked online for a music notation tool that did not cost an arm and a leg - I

already knew of one that did cost an arm and a leg - and I found an open source free
version called MuseScore which I use to this day.
With much trepidation I re learned how to enter notes on a traditional music staff.

Hybridizing hybridizing gives options
11
jn: 23,24,25-mar-2023
Development One LLC
APP: MuseScore software for music composition
and notation - SOUND LIBRARY
https://www.youtube.com/watch?v=t-o3nneUANw
09 Hybridizing hybridizing gives options
To my joy I found that MuseScore could play the notes back at me at any tempo and
volume I wanted and that I could have more than one line of music playing at the
same time which was not possible for me on the piano.
Furthermore I could change it from the sound of a piano to a clarinet or an oboe and I
could even add drum.
This slide shows a MuseScore composition with four parts and sounds. Here is what it
sounds like:
~SOUND 05
WHAT DOES THIS SOUND FEEL LIKE?
TYPE “_your feeling here_” or “not sure”
So I ended up using my spreadsheet to generate all possible chords for each scale.
But you can't play a spreadsheet so then I made a matching score in MuseScore
which listed all the possible chords for each scale and I could play those scores just
by clicking on them.

So now I had a do-it-yourself pick-and-copy chords from the reference score and
paste them into a blank new composition score… and I was off and running.

Sharing gives options
10
jn: 23,24,25-mar-2023
Development One LLC
APP: SoundCloud livestreaming service for
sharing prerecorded content
https://soundcloud.com/sitearm-1/shake-trigram-etude-24-24-42
10 Sharing gives options
One other rewind point - I knew that I wanted to share the music I was creating and
the only thing I knew how to do was use YouTube to share videos so I checked out
options like SoundCloud to share audio files.
My first-ever, what I called “Shake Trigram 24 24 42” was posted online. This slide
shows that post and here is part of what it sounds like:
~SOUND 06

Hybridizing sharing gives options
14
APP: Twitch livestreaming service for sharing live
content - YOUR COLLECTIONS
jn: 23,24,25-mar-2023
Development One LLC
social media - YOUR PLAYLISTS
https://www.youtube.com/@sitearm/playlists?view=50&sort=dd&shelf_id=7 https://www.twitch.tv/sitearm/videos
11 Hybridizing sharing gives options
After sharing some of this story with a colleague and also telling them about how I
had been following Twitch streamers and admiring their skill and their presence
online, my colleague suggested, why don't you stream composing on Twitch.
And so I did.
This slide shows Twitch composing collections I’ve completed so far on the right side,
and corresponding YouTube playlists on the left.
I actually began with a traditional scale and song which I called “Ballad” but I
deliberately used non-traditional chords to tell a story of going from weird to familiar.
The metaphor was being in a battle and recovering and growing from the experience.
Then I launched full bore into the pentatonic parallel scales.
At this time I had given up trying to fit the five note scales to my 6 line poems. They
just didn't match well except for my original tunes which had been composed, it turned
out, in traditional Western methods.
After several series working my way through my 8 different custom scales I began to

experiment with combining them.
I used the metaphor of sunlight and shadow where sun casts cloud shadows on the
ground and if you look at the ground there are bright areas and shadowed areas
So I ended up making new improvised-on-the spot parallel scales. Some of them had
4 notes instead of 5 and some had 6 or 3.
I also looked at a traditional song called “To a Wild Rose” and analyzed it and
discovered it was using a variation on the traditional 7 note Western scale where it
had changed 2 of the notes and I loved it so I tried composing with it as well.

Visualizing gives options
12
jn: 23,24,25-mar-2023
Development One LLC
APP: Music Animation Machine MIDI (Musical
Instrument DIgital Interface) Player
https://www.youtube.com/watch?v=jmXbUp-xEHM
12 Visualizing gives options
Now let me go back to the issue of sharing my music. I ended up deciding I was going
to have to use YouTube to share the music but just showing a blank screen as music
plays was as bad as SoundCloud just showing a waveform, I felt.
I had a freeware animator that would display the notes in piano roll form so I made
some videos like that.
This slide shows one of those piano roll animated compositions and here is what it
sounds like:
~SOUND 07
WHAT COLOR ARE THE TINKLY SOUNDS?
TYPE “_your color here_” or “not sure”

Hybridizing visualizing gives options
13
jn: 23,24,25-mar-2023
Development One LLC
https://sitearm.wordpress.com/2019/03/03/custom-12-tone-rgb-color-palette-using-munsell-color-scale-maximum-chroma/
13 Hybridizing visualizing gives options
The piano roll animator had an option to show notes by color and this got me looking
into creating my own color palette to match pitch classes. This slide shows what I
came up with.
ARE THE COLORS ON THE LEFT THE SAME AS THE COLORS ON THE RIGHT?
IF YOU SEE A DIFFERENCE TYPE “diff”
IF THEY LOOK THE SAME TYPE “same”

Synchronizing gives options
18
jn: 23,24,25-mar-2023
Development One LLC
APP: Magic Music Visualizer
https://www.youtube.com/watch?v=LExWDznoMYo
14 Synchronizing gives options
And then I looked for and found an animator that would let you use and create custom
shapes that were driven by and responded to the sound.
This was a joy because in MuseScore I could separate the parts of my music by
different lines and export them each as its own musical actor and then in the
visualizer give each actor its own colored shape.
And they would dance together or act together on the stage, so to speak. And I could
even change the scenes on the stage to transition with the music timeline.
This slide shows one of those custom animations and here is part of what it sounds
like:
~SOUND 08
So that brings us up to the relative present day.
Where I am today is that I feel that I'm reasonably comfortable and competent with

using 12 tones in different custom and traditional scales.
And taking a set of chords and picking out a pleasant sequence, extracting a melody,
adding ornamentation, adding color and shimmer lines, and making a visual
animation of them.
But as usual we all want more. I wanted to add some kind of 3-D expression and I
was able to download some 3-D elements from Kitely and add them to my music
visualizer where they can rotate and expand and contract in response to the music.
And I've been looking back at Second Life and been amazed by the new custom
synchronized choreographed dances and particle light effect shows now scheduled
regularly.
I've also looked into virtual streaming avatars called “vtubers” and have one that I use
occasionally on my stream for fun.
Her head bobs and turns in response to mine from the webcam and her lips are
synchronized to mine, just as Second Life avatar lips are synced to our own voices.
So there's further to go.

Sharing online gives options
20
jn: 23,24,25-mar-2023
Development One LLC
content - CHAT
15 Sharing online gives options
While streaming live on Twitch I noticed it was a different experience from composing
offline. At any moment I felt somebody could be watching my stream, whether they
typed something in chat or not.
I ended up calling them Silent Lurker because I did find from my after-action stream
reports that there were people on my stream from time to time.
At times some of the people in chat actually said hello and at times I had fun dialogs
with them. For example the Austrian music teacher who used MuseScore to
customize and print scores for his junior high band including his star young girl lead
trumpet.
This slide shows another interactive chat where I got a whole new “aha!”. And I quote,
“this is a studio experience… but over the internet.”
WHAT DOES IT FEEL LIKE CHATTING ONLINE?
TYPE “_your experience here_”
I did some collaborative compositions with members of chat. One provided a

soundtrack by email and I animated it live with them looking on and sent the recording
back to them.
I also did collaborative compositions with members of the Renderosity digital art
community who gave me permission to use screenshots of their work in what were
basically animated slideshows using some of my music.
I finally realized I could also combine my music with my video machinima work. For
example recordings of presentations given by myself and others in Second Life for
university students.
This especially highlighted for me the idea of “lead actor” and “accompaniment”
design in a combined sound and sight recording.
Sharing the process of composing has been an enriching experience.
For example, being able to think out loud and work ideas out into a medium while live
and having people with you in the process.

When I see live video of myself…
When I listen to what I am saying, as I am
saying it, as if someone else is speaking…
When I set something aside, and look at it /
listen to it again the next day…
When I am streaming live, aware that
someone may be listening…
Now… ow… w…
Feedback gives options
22
jn: 23,24,25-mar-2023
Development One LLC
Image courtesy Marc Evanstein
social media
content
16 Feedback gives options
Composing music became inextricably entangled with composing presentations. I
found both are creative endeavors and share a similar process of taking an idea from
pure thought to trial drafts to more or less completed work which is then presented.
In the earlier stages I was anxious to not only share what I was doing but to get
feedback. I tried different ways to cajole, evoke, prod, or simply ask for feedback
Some feedback I got was amazing like the piano tuner who said one piece made
them think of a huge automated factory floor with hundreds working in synchrony
Some feedback I got was disconcerting for example what do you want to know why
are you asking me, they would say
Some feedback was unsatisfying like, it sounds somber or it sounds like science
fiction
Finally I realized all the feedback was great because didn't I have a wide array of my
own response to the creative work of others?
This slide shows a composite of my own and another writer’s experiences of

composing and feedback. My part of the experience sounds like this:
~SOUND 09
IS IT SAME OR DIFFERENT WORKING WITH SOMEONE ELSE PRESENT?
IF YOU EXPERIENCE DIFFERENT TYPE “diff”
IF YOU EXPERIENCE THE SAME TYPE “same”
I came to realize that feedback is simply information that helps feed forward to
whatever I work on next
Today I feel that each of us are creators who basically work from some inner impetus
that is based, half on memory of the past, and half on some kind of immediate flash
forward insight of what might be possible next
When I said that sharing composing online has been enriching I include pretty much
all areas of my life.
For example seeing all human activity and realms as forms of composition such as
cooking a meal, sewing a textured fabric, compositing a new perfume
Writing a school assignment or preparing a business report
Most recently my attention has been caught adding the dynamic of performing music
live to the dynamic of writing it.
Writing a script and presenting it are 2 sides of a creative process, I feel
Even presenting extemporaneously creates a memory from which we remember a
few good lines that we will repeat the next time we speak on that topic
And I remember with dread the few times I spoke extemporaneously that frankly
sucked because I had not taken the time to think ahead and write down a few good
lines ahead of time
So currently I speculate that having written a few good lines and then repeatedly
presenting them with some spontaneity added generates new energy.
And creates more of that instantaneous flash forward insight of seeing what else
could be possible next

Doing something live gives options
24
jn: 23,24,25-mar-2023
Development One LLC
17 Doing something live gives options
[Match the sound to a word…]
For this part I am going to rely on selected volunteers from the audience.
[volunteers step forward]
Everyone please acknowledge our volunteers!
[first volunteer]
Choose one of the buttons on the left side: A, B, C, D, or E.
What button did you choose?
Now click the button you chose to hear its sound.
Can you hear it OK?
Now choose one of the words on the top: Fire, Earth, Metal, Water, Wood.
What word do you choose?
Good! Now I will slide the button you chose under the word you chose. Please stand
under the button and word you chose.

[repeat until all volunteers are matched to their buttons and words]
~
[Composition]
Now for our live performance!
The name of our composition is “Feng Shui Elements.”
[“constructive” cycle - shown]
FIRE EARTH METAL
METAL WATER WOOD
[“deconstructive cycle - improvised]
WOOD EARTH WATER
WATER FIRE METAL
[“harmony of yin and yang” - improvised]
FIRE EARTH METAL WATER WOOD
~
Please acknowledge our performers!

EXTRO - Composing in the Hybrid Metaverse
In this presentation we shared examples of
composing in the hybrid metaverse and
cross-dimensional thinking.
● What caught your attention?
Methods
● Patterns
● Pitch Classes
● Music Channels
● Notations
● Perspectives
● Hybridizing
● Sharing
● Visualizing
● Synchronizing
Apps
● MuseScore
● YouTube
● Google Sheets
● SoundCloud
● Twitch
● Music Animation Machine
● Magic Music Visualizer
18
jn: 23,24,25-mar-2023
Development One LLC
https://docs.google.com/presentation/d/1zhXsvvpQiDJtxs6_juasrS13xL-Bimhg7zxpP9_C16k/edit?usp=share_link
18 EXTRO - Composing in the Hybrid Metaverse
So now we've reached the end of our brief journey into the adventure of Composing in
the Hybrid Metaverse. I hope you've enjoyed these shared examples of composing
and cross dimensional thinking.
But I hope this is an interim, not a final end! I invite you to use ideas we’ve gleaned
today in further “flash forward” collaborative compositions.
In particular I ask you, what caught your attention?
~~
This concludes our presentation on Composing in the Hybrid Metaverse. Many many
thanks!

SLIDES & NOTES - VWBPE - Composing in the Hybrid Metaverse 23,24,25-Mar-2023.pdf

Recommended

Recommended

More Related Content

Similar to SLIDES & NOTES - VWBPE - Composing in the Hybrid Metaverse 23,24,25-Mar-2023.pdf

Similar to SLIDES & NOTES - VWBPE - Composing in the Hybrid Metaverse 23,24,25-Mar-2023.pdf (20)

More from Siterma The World In 4D

More from Siterma The World In 4D (20)

Recently uploaded

Recently uploaded (20)

SLIDES & NOTES - VWBPE - Composing in the Hybrid Metaverse 23,24,25-Mar-2023.pdf