132nd AES Convention, 26th-29th of April, Budapest, Hungary
The emerging Semantic Web provides a powerful framework for the expression and reuse of structured data. Recent efforts have brought this framework to bear on the field of Semantic Audio, as well as information management in audio applications. This tutorial will provide an introduction to Semantic Web concepts and how they can be used in the context of music-related studies. We will outline the use of the Resource Description Framework (RDF) and related ontology and query languages. Using practical examples, we will demonstrate the use of the Music and Studio Ontologies, and show how they facilitate interoperability between audio applications and linked data sets on the Web. We will explore how signal processing tools and results can be described as structured data and utilised in audio production.
This tutorial focuses on the intersection of the fields of Semantic Audio and the Semantic Web.
1. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Semantic Web and
Semantic Audio
technologies
Tutorial by
György Fazekas and Thomas Wilmering
Centre for Digital Music
Queen Mary University of London
School of Electronic Engineering and Computer Science
132nd Convention
April 26th-29th,
Budapest, Hungary
Saturday, 28 April 12
2. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
We are on the Web
• Slides, examples and other resources are
available at:
• www.isophonics.net/content/aes132-
tutorial
Saturday, 28 April 12
3. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Outline
• Motivations
• Semantic Web Technologies
• Semantic Web Applications
• Short Hands on Session (1)
• Music Ontology
• Studio Ontology
• Semantic Audio Tools
• Short Hands on Session (2)
Saturday, 28 April 12
4. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• The focus of this tutorial is the
intersection of the two fields
Semantic
Audio
Semantic
Web
Saturday, 28 April 12
5. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is Semantic Audio ?
• What is the Semantic Web ?
• How are they related,
• and why should we care?
Saturday, 28 April 12
6. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is Semantic Audio ?
• a confluence of technologies for
• interacting with audio in human terms
• Semantic Audio technologies include:
• Audio content analysis
• e.g. Digital Signal Processing and Machine Learning
• Information Management
• Knowledge Representation
• e.g. Logic, Ontologies, and database technologies
Saturday, 28 April 12
7. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is Semantic Audio ?
• a confluence of technologies for
• interacting with audio in human terms
• Semantic Audio technologies include:
• Audio content analysis
• e.g. Digital Signal Processing and Machine Learning
• Information Management
• Knowledge Representation
• e.g. Logic, Ontologies, and database technologies
Saturday, 28 April 12
8. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is Semantic Audio ?
• a confluence of technologies for
• interacting with audio in human terms
• Semantic Audio technologies include:
• Audio content analysis
• e.g. Digital Signal Processing and Machine Learning
• Information Management
• Knowledge Representation
• e.g. Logic, Ontologies, and database technologies
Saturday, 28 April 12
9. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is the Semantic Web ?
• (1) a diverse network of interconnected
data and services
• in principle, it is similar to how
documents are linked using hypertext
• (2) a machine-interpretable
representation of the World Wide Web
Saturday, 28 April 12
10. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is the Semantic Web ?
• (1) a diverse network of interconnected
data and services
• in principle, it is similar to how
documents are linked using hypertext
• (2) a machine-interpretable
representation of the World Wide Web
Saturday, 28 April 12
11. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is the Semantic Web ?
• (1) a diverse network of interconnected
data and services
• in principle, it is similar to how
documents are linked using hypertext
• (2) a machine-interpretable
representation of the World Wide Web
Saturday, 28 April 12
12. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Introduction
• What is the Semantic Web ?
• The objective is:
• Enable machines to complete complex
(search) tasks currently requiring
human-level intelligence
Saturday, 28 April 12
13. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Motivations
• How Semantic Audio and the Semantic
Web are they related?
• A proliferation of music content on the
Web requires Semantic Audio
technologies for better access to this
content.
Saturday, 28 April 12
14. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Motivations
• How Semantic Audio and the Semantic
Web are they related?
• Semantic Web technologies enable better
representation and access to music
related information.
Saturday, 28 April 12
15. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Motivations
• Why should we care?
• Music Information Retrieval:
• Find me upbeat and catchy songs
between 130-140 bpm, performed by
artists collaborating in the London-
Shoreditch area, and sort them by
musical key.
Saturday, 28 April 12
16. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Motivations
• Why should we care?
• Music production:
• Find me guitar riffs in all my recording
projects where an echo and compressor
were applied with the given parameters.
Saturday, 28 April 12
17. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Motivations
• Why should we care?
• These queries/applications require clever
• content analysis
• knowledge representation
• information management
Saturday, 28 April 12
18. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Tutorial Focus
• The focus of this tutorial is the
intersection of the two fields
Semantic
Audio
Signal
Processing
Machine
Learning
URI
HTTP Web
Technologies
Audio Content
Analysis and MIR
Semantic
Web
Logic and
Reasoning
Knowledge
Repr.
Linked
Data
Web
Ontologies
RDF
Semantic
Audio
Signal
Processing
Machine
Learning
URI
HTTP Web
Technologies
Audio Content
Analysis and MIR
Semantic
Web
Logic and
Reasoning
Knowledge
Repr.
Linked
Data
Web
Ontologies
RDF
OWL &
RDFS
Saturday, 28 April 12
19. Semantic
Audio
Signal
Processing
Machine
Learning
URI
HTTP Web
Technologies
Audio Content
Analysis and MIR
Semantic
Web
Logic and
Reasoning
Knowledge
Repr.
Linked
Data
Web
Ontologies
RDF
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Tutorial Focus
• The focus of this tutorial is the
intersection of the two fields
Semantic
Audio
Signal
Processing
Machine
Learning
URI
HTTP Web
Technologies
Audio Content
Analysis and MIR
Semantic
Web
Logic and
Reasoning
Knowledge
Repr.
Linked
Data
Web
Ontologies
RDF
OWL &
RDFS
Saturday, 28 April 12
20. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Semantic Web Technologies
Saturday, 28 April 12
21. Linked Data Semantic Web Web of Data
• These concepts are often used
interchangeably
• Linked Data is a recent movement that
focusses on creating a web of data
• Just like the Web is a web of documents
• Broader premises of the Semantic Web will be
realised in the future
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Linked Data
=
?
=
?
Saturday, 28 April 12
22. Linked Data Semantic Web Web of Data
• These concepts are often used
interchangeably
• Linked Data is a recent movement that
focusses on creating a web of data
• Just like the Web is a web of documents
• Broader premises of the Semantic Web will be
realised in the future
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Linked Data
=
?
=
?
Saturday, 28 April 12
23. Linked Data Semantic Web Web of Data
• These concepts are often used
interchangeably
• Linked Data is a recent movement that
focusses on creating a web of data
• Just like the Web is a web of documents
• Broader premises of the Semantic Web will be
realised in the future
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Linked Data
=
?
=
?
Saturday, 28 April 12
24. Linked Data Semantic Web Web of Data
• These concepts are often used
interchangeably
• Linked Data is a recent movement that
focusses on creating a web of data
• Just like the Web is a web of documents
• Broader premises of the Semantic Web will be
realised in the near future
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Linked Data
=
?
=
?
Saturday, 28 April 12
25. As of September 2011
Music
Brainz
(zitgist)
P20
Turismo
de
Zaragoza
yovisto
Yahoo!
Geo
Planet
YAGO
World
Fact-
book
El
Viajero
Tourism
WordNet
(W3C)
WordNet
(VUA)
VIVO UF
VIVO
Indiana
VIVO
Cornell
VIAF
URI
Burner
Sussex
Reading
Lists
Plymouth
Reading
Lists
UniRef
UniProt
UMBEL
UK Post-
codes
legislation
data.gov.uk
Uberblic
UB
Mann-
heim
TWC LOGD
Twarql
transport
data.gov.
uk
Traffic
Scotland
theses.
fr
Thesau-
rus W
totl.net
Tele-
graphis
TCM
Gene
DIT
Taxon
Concept
Open
Library
(Talis)
tags2con
delicious
t4gm
info
Swedish
Open
Cultural
Heritage
Surge
Radio
Sudoc
STW
RAMEAU
SH
statistics
data.gov.
uk
St.
Andrews
Resource
Lists
ECS
South-
ampton
EPrints
SSW
Thesaur
us
Smart
Link
Slideshare
2RDF
semantic
web.org
Semantic
Tweet
Semantic
XBRL
SW
Dog
Food
Source Code
Ecosystem
Linked Data
US SEC
(rdfabout)
Sears
Scotland
Geo-
graphy
Scotland
Pupils &
Exams
Scholaro-
meter
WordNet
(RKB
Explorer)
Wiki
UN/
LOCODE
Ulm
ECS
(RKB
Explorer)
Roma
RISKS
RESEX
RAE2001
Pisa
OS
OAI
NSF
New-
castle
LAAS
KISTI
JISC
IRIT
IEEE
IBM
Eurécom
ERA
ePrints dotAC
DEPLOY
DBLP
(RKB
Explorer)
Crime
Reports
UK
Course-
ware
CORDIS
(RKB
Explorer)
CiteSeer
Budapest
ACM
riese
Revyu
research
data.gov.
ukRen.
Energy
Genera-
tors
reference
data.gov.
uk
Recht-
spraak.
nl
RDF
ohloh
Last.FM
(rdfize)
RDF
Book
Mashup
Rådata
nå!
PSH
Product
Types
Ontology
Product
DB
PBAC
Poké-
pédia
patents
data.go
v.uk
Ox
Points
Ord-
nance
Survey
Openly
Local
Open
Library
Open
Cyc
Open
Corpo-
rates
Open
Calais
OpenEI
Open
Election
Data
Project
Open
Data
Thesau-
rus
Ontos
News
Portal
OGOLOD
Janus
AMP
Ocean
Drilling
Codices
New
York
Times
NVD
ntnusc
NTU
Resource
Lists
Norwe-
gian
MeSH
NDL
subjects
ndlna
my
Experi-
ment
Italian
Museums
medu-
cator
MARC
Codes
List
Man-
chester
Reading
Lists
Lotico
Weather
Stations
London
Gazette
LOIUS
Linked
Open
Colors
lobid
Resources
lobid
Organi-
sations
LEM
Linked
MDB
LinkedL
CCN
Linked
GeoData
LinkedCT
Linked
User
Feedback
LOV
Linked
Open
Numbers
LODE
Eurostat
(Ontology
Central)
Linked
EDGAR
(Ontology
Central)
Linked
Crunch-
base
lingvoj
Lichfield
Spen-
ding
LIBRIS
Lexvo
LCSH
DBLP
(L3S)
Linked
Sensor Data
(Kno.e.sis)
Klapp-
stuhl-
club
Good-
win
Family
National
Radio-
activity
JP
Jamendo
(DBtune)
Italian
public
schools
ISTAT
Immi-
gration
iServe
IdRef
Sudoc
NSZL
Catalog
Hellenic
PD
Hellenic
FBD
Piedmont
Accomo-
dations
GovTrack
GovWILD
Google
Art
wrapper
gnoss
GESIS
GeoWord
Net
Geo
Species
Geo
Names
Geo
Linked
Data
GEMET
GTAA
STITCH
SIDER
Project
Guten-
berg
Medi
Care
Euro-
stat
(FUB)
EURES
Drug
Bank
Disea-
some
DBLP
(FU
Berlin)
Daily
Med
CORDIS
(FUB)
Freebase
flickr
wrappr
Fishes
of Texas
Finnish
Munici-
palities
ChEMBL
FanHubz
Event
Media
EUTC
Produc-
tions
Eurostat
Europeana
EUNIS
EU
Insti-
tutions
ESD
stan-
dards
EARTh
Enipedia
Popula-
tion (En-
AKTing)
NHS
(En-
AKTing) Mortality
(En-
AKTing)
Energy
(En-
AKTing)
Crime
(En-
AKTing)
CO2
Emission
(En-
AKTing)
EEA
SISVU
educatio
n.data.g
ov.uk
ECS
South-
ampton
ECCO-
TCP
GND
Didactal
ia
DDC Deutsche
Bio-
graphie
data
dcs
Music
Brainz
(DBTune)
Magna-
tune
John
Peel
(DBTune)
Classical
(DB
Tune)
Audio
Scrobbler
(DBTune)
Last.FM
artists
(DBTune)
DB
Tropes
Portu-
guese
DBpedia
dbpedia
lite
Greek
DBpedia
DBpedia
data-
open-
ac-uk
SMC
Journals
Pokedex
Airports
NASA
(Data
Incu-
bator)
Music
Brainz
(Data
Incubator)
Moseley
Folk
Metoffice
Weather
Forecasts
Discogs
(Data
Incubator)
Climbing
data.gov.uk
intervals
Data
Gov.ie
data
bnf.fr
Cornetto
reegle
Chronic-
ling
America
Chem2
Bio2RDF
Calames
business
data.gov.
uk
Bricklink
Brazilian
Poli-
ticians
BNB
UniSTS
UniPath
way
UniParc
Taxono
my
UniProt
(Bio2RDF)
SGD
Reactome
PubMed
Pub
Chem
PRO-
SITE
ProDom
Pfam
PDB
OMIM
MGI
KEGG
Reaction
KEGG
Pathway
KEGG
Glycan
KEGG
Enzyme
KEGG
Drug
KEGG
Com-
pound
InterPro
Homolo
Gene
HGNC
Gene
Ontology
GeneID
Affy-
metrix
bible
ontology
BibBase
FTS
BBC
Wildlife
Finder
BBC
Program
mes BBC
Music
Alpine
Ski
Austria
LOCAH
Amster-
dam
Museum
AGROV
OC
AEMET
US Census
(rdfabout)
Media
Geographic
Publications
Government
Cross-domain
Life sciences
User-generated content
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Source: http://richard.cyganiak.de/2007/10/lod/
Linked Data
Saturday, 28 April 12
26. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Demo Videos
• What is possible now?
• The following demos show:
• (1) audio applications that collect and use data
from the Semantic Web
• (2) audio applications that utilise Semantic Web
technologies (but not necessarily external data)
Saturday, 28 April 12
27. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Demo Video 1
Saturday, 28 April 12
28. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Demo Video 2
Saturday, 28 April 12
29. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Demo Videos
• How do these applications really work?
• They combine information from different
sources
• To achieve this we need:
• interoperability
• queryability
• and also:
• extensibility
• modularity
Saturday, 28 April 12
30. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Demo Videos
• How do these applications really work?
• They combine information from different
sources
• To achieve this we need:
• interoperability
• queryability
• and also:
• extensibility
• modularity
Saturday, 28 April 12
31. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Basic Requirements
• How do these applications really work?
• They combine information from different
sources
• To achieve this we need:
• interoperability between different data sources
• queryability
• and also:
• extensibility
• modularity
Saturday, 28 April 12
32. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Basic Requirements
• How do these applications really work?
• They combine information from different
sources
• To achieve this we need:
• interoperability
• queryability
• and also:
• extensibility
• modularity
Saturday, 28 April 12
33. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Metadata Structural Diversity
Artist
Instrument
Tempo
Genre
• But, the heterogeneity of musical metadata
presents a problem
Saturday, 28 April 12
34. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Metadata Structural Diversity
Artist
Instrument
Tempo
Genre
Artist
Instrument
Tempo
Sub-
Genre
Gender
Date of
Birth
Frequency
range
Registers
Unit of
measure
Genre
• But, the heterogeneity of musical metadata
presents a problem
Saturday, 28 April 12
35. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Metadata Structural Diversity
Artist
Instrument
Gender
Date of
Birth
Frequency
range
Registers
• But, the heterogeneity of musical metadata
presents a problem
Saturday, 28 April 12
36. • The XML Factor:
132nd AES Convention, 26th-29th of April, Budapest, Hungary
XML and Metadata Standards
Image Credit: Dan Zambonini (O’Reilly XML.com blog) http://www.oreillynet.com/xml/blog/
Saturday, 28 April 12
37. • XML and XML-based metadata standards
• only specify the syntax of documents
• meaning (a.k.a. semantics) is implicit,
• and hard coded in procedural software
132nd AES Convention, 26th-29th of April, Budapest, Hungary
XML and Metadata Standards
Saturday, 28 April 12
38. • The XML Factor:
132nd AES Convention, 26th-29th of April, Budapest, Hungary
XML and Metadata Standards
Image Credit: Dan Zambonini (O’Reilly XML.com blog) http://www.oreillynet.com/xml/blog/
Saturday, 28 April 12
39. • The XML Factor:
There is no shared model of information
and knowledge
132nd AES Convention, 26th-29th of April, Budapest, Hungary
XML and Metadata Standards
Image Credit: Dan Zambonini (O’Reilly XML.com blog) http://www.oreillynet.com/xml/blog/
Saturday, 28 April 12
40. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
• RDF provides a simple
model of information
Image Credit: Dan Zambonini (O’Reilly XML.com blog) http://www.oreillynet.com/xml/blog/
• How does it work?
Saturday, 28 April 12
41. • In RDF information is decomposed into simple
statements.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Subject Object
predicate
Saturday, 28 April 12
42. • In RDF information is decomposed into simple
statements.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Dave Brubeck Music Artist
is a
Saturday, 28 April 12
43. • In RDF information is decomposed into simple
statements.
Paul Desmond
Quartet
Music Group
is a
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Saturday, 28 April 12
44. • In RDF information is decomposed into simple
statements.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Dave Brubeck
Paul Desmond
Quartet
member
Saturday, 28 April 12
45. • When combined, statements form a Graph
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Dave Brubeck Music Artist
is a
Paul Desmond
Quartet
Music Group
is a
member
Saturday, 28 April 12
46. • When combined, statements form a Graph
• more precisely a Directed Graph
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Dave Brubeck Music Artist
is a
Paul Desmond
Quartet
Music Group
is a
member
Saturday, 28 April 12
47. • In RDF information is decomposed into simple
statements.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Dave Brubeck Music Artist
is a
Saturday, 28 April 12
48. • These statements are also called triples
• of terms or resources:
• (subject, predicate, object).
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Resource Description Framework
Subject Object
predicate
Saturday, 28 April 12
49. • Every term gets a Unified Resource Identifier
(URI)
• When RDF is combined with URIs we can create
a globally distributed Web of Data that
• scales just like the World Wide Web
132nd AES Convention, 26th-29th of April, Budapest, Hungary
<http://Subject> <http://Object>
<http://predicate>
RDF & Unified Resource Identifiers
Saturday, 28 April 12
50. • Every term gets a Unified Resource Identifier
(URI)
• When RDF is combined with URIs we can create
a globally distributed Web of Data that
• scales just like the World Wide Web
132nd AES Convention, 26th-29th of April, Budapest, Hungary
<http://Subject> <http://Object>
<http://predicate>
RDF & Unified Resource Identifiers
Saturday, 28 April 12
51. • Using Web URIs ensures that elements of RDF
statements are uniquely identified
• We can also:
• retrieve additional information
• for instance, about the meaning of terms
• store different parts of the graph at different
databases / locations
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF & Unified Resource Identifiers
Saturday, 28 April 12
52. • Using Web URIs ensures that elements of RDF
statements are uniquely identified
• We can also:
• retrieve additional information
• for instance, about the meaning of terms
• store different parts of the graph at different
databases / locations
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF & Unified Resource Identifiers
Saturday, 28 April 12
53. • Using Web URIs ensures that elements of RDF
statements are uniquely identified
• We can also:
• retrieve additional information
• for instance, about the meaning of terms
• store different parts of the graph at different
databases / locations
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF & Unified Resource Identifiers
Saturday, 28 April 12
54. • How do we express / store information
described by an RDG graph?
• just write down triples of URIs as sentences.
• this is called N-Triples.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Syntax and Serialisation
<http://SUBJECT>
<http://PREDICATE>
<http://OBJECT> .
Saturday, 28 April 12
55. • How do we express / store information
described by an RDG graph?
• just write down triples of URIs as sentences.
• this is called N-Triples.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Syntax and Serialisation
<http://SUBJECT>
<http://PREDICATE>
<http://OBJECT> .
Saturday, 28 April 12
56. • How do we express / store information
described by an RDG graph?
• just write down triples of URIs as sentences.
• this is called N-Triples.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Syntax and Serialisation
<http://SUBJECT>
<http://PREDICATE>
<http://OBJECT> .
Saturday, 28 April 12
57. • RDF is not RDF/XML !
• XML was the first standardised syntax for RDF,
but
• there are many others available that are:
• easier to use
• easier to read (by a human)
• easier to parse (by a machine)
• more concise
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Syntax and Serialisation
Saturday, 28 April 12
58. • RDF is not RDF/XML !
• XML was the first standardised syntax for RDF,
but
• there are many others available that are:
• easier to use
• easier to read (by a human)
• easier to parse (by a machine)
• more concise
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Syntax and Serialisation
Saturday, 28 April 12
59. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Syntax
• Some common
syntaxes:
• N Triples
• Turtle
• RDF/XML
• RDFa
• JSON-LD
• N3 (this goes beyond the
RDF model and the scope
of this tutorial)
Saturday, 28 April 12
60. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Syntax
• Some common
syntaxes:
• N Triples
• Turtle
• RDF/XML
• RDFa
• JSON-LD
• N3 (this goes beyond the
RDF model and the scope
of this tutorial)
Saturday, 28 April 12
61. • Here is a statement in N Triples.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
<http://dbpedia.org/resource/Dave_Brubeck>
<http://dbpedia.org/ontology/genre>
<http://dbpedia.org/resource/Cool_Jazz> .
RDF N Triples Syntax
Saturday, 28 April 12
62. • Using CURIEs and the prefix notation.
• Still 3 lines of RDF but given a large set of
statements this is a significant reduction.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
@prefix dbpr: <http://dbpedia.org/resource/> .
@prefix dbpo: <http://dbpedia.org/ontology/genre> .
dbpr:Dave_Brubeck dbpo:genre dbpr:Cool_Jazz .
RDF Turtle Syntax
Saturday, 28 April 12
63. • Using CURIEs and the prefix notation.
• Still 3 lines of RDF but given a large set of
statements this is a significant reduction.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
@prefix dbpr: <http://dbpedia.org/resource/> .
@prefix dbpo: <http://dbpedia.org/ontology/genre> .
dbpr:Dave_Brubeck dbpo:genre dbpr:Cool_Jazz .
RDF Turtle Syntax
Saturday, 28 April 12
64. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Turtle Syntax
:Dave_Brubeck rtf:type mo:MusicArtist ;
:member :Paul_Desmond_Quartet.
Dave Brubeck Music Artist
is a
Paul Desmond
Quartet
Music Group
is a
member
• The prefix can remain empty (:resource) to
represent the local scope.
Saturday, 28 April 12
65. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Turtle Syntax
:Dave_Brubeck rtf:type mo:MusicArtist ;
:member :Paul_Desmond_Quartet.
Dave Brubeck Music Artist
is a
Paul Desmond
Quartet
Music Group
is a
member
• The semicolon can be used to group
statements about the same resource.
Saturday, 28 April 12
66. • Blank nodes represent unnamed resources
• They are very useful when representing
complex data
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Turtle Syntax: Blank nodes
:resource [
:name “parameter name” ;
:value “20”
] .
Some resource
Blank Node
parameter
"Name" "Value"
name value
Saturday, 28 April 12
67. • owl:sameAs predicate can be used to link
resources in different datasets that hold
information about the same resource.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix mo: <http://purl.org/ontology/mo/> .
<http://www.bbc.co.uk/music/artists/1545000730-525f-4ed5-
aaa8-92888-f060f5f#artist>
rdf:type mo:MusicArtist ;
owl:sameAs <http://dbpedia.org/resource/Dave_Brubeck> .
Linking different datasets
Saturday, 28 April 12
68. • URIs : <http://some_resource.org>
• CURIEs: mo:MusicArtist
• @prefix: declare namespaces
• Blank nodes: [ ... ] or _:bnode
• Literal values: “some string”
• Typed literals: “20”^^xsd:int
• Group statements: semicolon ( ; )
• Group objects: colon ( , )
• Close statements: dot ( . )
• Shorthand for rdf:type: a
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Turtle Syntax: Summary
Saturday, 28 April 12
69. • Linked data repositories
• use eg. HTTP GET
• this is usually done through content
negotiation
• Triple Stores
• Garlic’s 4Store
• Openlink Virtuoso
• Lots of programming libraries
• redland (C), rdflib (Python), Jena (Java)
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Storage and Databases
Saturday, 28 April 12
70. • Linked data repositories
• use eg. HTTP GET
• this is usually done through content
negotiation
• Triple Stores
• Garlic’s 4Store
• Openlink Virtuoso
• Lots of programming libraries
• redland (C), rdflib (Python), Jena (Java)
132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF Storage and Databases
Saturday, 28 April 12
71. • SPARQL RDF protocol and Query Language
• Similar to Turtle
• It has several query types, e.g.
• SELECT
• CONSTRUCT
• variables: ?x
• These allow to form query patterns to be
matched
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Querying RDF with SPARQL
Saturday, 28 April 12
72. • SPARQL RDF protocol and Query Language
• Similar to Turtle
• It has several query types, e.g.
• SELECT
• CONSTRUCT
• variables: ?x
• These allow to form query patterns to be
matched
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Querying RDF with SPARQL
Saturday, 28 April 12
73. • SPARQL RDF protocol and Query Language
• Similar to Turtle
• It has several query types, e.g.
• SELECT
• CONSTRUCT
• variables: ?x
• These allow to form query patterns to be
matched
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Querying RDF with SPARQL
Saturday, 28 April 12
74. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Querying RDF with SPARQL
PREFIX dbpr: <http://dbpedia.org/resource/>
PREFIX dbpo: <http://dbpedia.org/ontology/>
SELECT ?genre
WHERE {
dbpr:Dave_Brubeck dbpo:genre ?genre .
}
• Find a genre classification according to
DBPedia
Saturday, 28 April 12
75. • Find other artists (?x) having the same genre
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Querying RDF with SPARQL
PREFIX dbpr: <http://dbpedia.org/resource/>
PREFIX dbpo: <http://dbpedia.org/ontology/>
SELECT ?x
WHERE {
dbpr:Dave_Brubeck dbpo:genre ?genre .
?x dbpo:genre ?genre .
}
Saturday, 28 April 12
76. • Let’s try this in practice
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Querying RDF with SPARQL
PREFIX dbpr: <http://dbpedia.org/resource/>
PREFIX dbpo: <http://dbpedia.org/ontology/>
SELECT ?x
WHERE {
dbpr:Dave_Brubeck dbpo:genre ?genre .
?x dbpo:genre ?genre .
}
Saturday, 28 April 12
77. • There are many music related linked data
services and applications available
• DBTube.org
• http://dbtune.org/
• Linked Brainz (MusicBrainz database)
• http://linkedbrainz.c4dmpresents.org/
• Musicnet
• http://musicnet.mspace.fm/
• BBC Music website
• http://www.bbc.co.uk/music
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Linked Data Services
Saturday, 28 April 12
78. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontologies
Saturday, 28 April 12
79. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF and Ontologies
• Elements of RDF statements can be selected in
an ad-hoc manner.
• We need a way to give “meaning” to each
subject, predicate and object.
• Represent knowledge in a formal way.
Saturday, 28 April 12
80. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
RDF and Ontologies
• Elements of RDF statements can be selected in
an ad-hoc manner.
• We need a way to give “meaning” to each
subject, predicate and object.
• Represent knowledge in a formal way.
Saturday, 28 April 12
81. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Knowledge Representation
• This can be done using First Order Logic
• But this is too hard for practical reasoning
• Description Logics are subsets of this logic
that provide the logical foundations for Web
Ontologies and Ontology languages
Saturday, 28 April 12
82. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Knowledge Representation
• This can be done using First Order Logic
• But this is too hard for practical reasoning
• Description Logics are subsets of this logic
that provide the logical foundations for Web
Ontologies and Ontology languages
Saturday, 28 April 12
83. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontologies
• An Ontology is:
• a shared conceptualisation of a world or
domain
• it includes:
• 1) individuals,
• 2) classes, groups of individuals that have
something in common,
• 3) possible relationships between them
Saturday, 28 April 12
84. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontologies
• An Ontology is:
• a shared conceptualisation of a world or
domain
• it includes:
• 1) individuals,
• 2) classes, groups of individuals that have
something in common,
• 3) relationships possible between them
Saturday, 28 April 12
85. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontologies
• There might be many ontologies for the same
domain
• They all may be valid (and useful),
• but it is unlikely they cover everything,
• or equally useful in all applications.
Saturday, 28 April 12
86. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontologies
• There might be many ontologies for the same
domain
• They all may be valid (and useful),
• but it is unlikely they cover everything,
• or equally useful in all applications.
Saturday, 28 April 12
87. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontology Languages
• There is a stack of languages
(W3C recommendations)
• OWL2: extended data model
• OWL: allows for equivalence,
cardinality constraints, etc...
- OWL-Full
- OWL-DL
- OWL-Lite
• RDFS: allows for describing class
and property hierarchies
More
complex
reasoning
support
Expressiveness
Saturday, 28 April 12
88. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontology Languages
• There is a stack of languages
(W3C recommendations)
• OWL2: extended data model
• OWL: allows for equivalence,
cardinality constraints, etc...
- OWL-Full
- OWL-DL
- OWL-Lite
• RDFS: allows for describing class
and property hierarchies
More
complex
reasoning
support
Expressiveness
Saturday, 28 April 12
89. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Domain
• To describe music we need to communicate:
• editorial (bibliographic) information
• information about intellectual works and
workflows
• people and their works
• cultural and social information
• content-based information
• provenance and trust
• who says what and can we trust it?
Saturday, 28 April 12
90. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Domain
• To describe music we need to communicate:
• editorial (bibliographic) information
• information about intellectual works and
workflows
• people and their works
• cultural and social information
• content-based information
• provenance and trust
• who says what and can we trust it?
Saturday, 28 April 12
91. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Some useful ontologies
• Dublin Core
• Friend of a Friend (FOAF) vocabulary
• to talk about people, groups, and
• OWL-Time:
• basic temporal concepts
• Timeline Ontology:
• relate temporal concepts with regards to different
timelines
• Event Ontology:
• describe time based events
Saturday, 28 April 12
92. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Ontology
Saturday, 28 April 12
93. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Ontology
mo:MusicArtist
rdf:type owl:Class ;
rdfs:comment """A person or a group of people (or
a computer, whose musical creative work shows
sensitivity and imagination """ ;
rdfs:isDefinedBy <http://purl.org/ontology/mo/>;
rdfs:label "music artist" ;
rdfs:subClassOf foaf:Agent .
Credit: Yves Raimond et al, http://musicontology.com/
• Combines several ontologies to describe music
related information
Saturday, 28 April 12
94. • Combines several ontologies to describe music
related information
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Ontology
mo:MusicArtist
rdf:type owl:Class ;
rdfs:comment """A person or a group of people (or
a computer, whose musical creative work shows
sensitivity and imagination """ ;
rdfs:isDefinedBy <http://purl.org/ontology/mo/>;
rdfs:label "music artist" ;
rdfs:subClassOf foaf:Agent .
Credit: Yves Raimond et al, http://musicontology.com/
Saturday, 28 April 12
95. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Timeline and Event Ontologies
• The Timeline Ontology extends OWL-Time and
defines the TimeLine concept.
• Temporal objects (signal, video, performance,
work, etc.) can be associated with a timeline.
• The Event ontology relates arbitrary events to:
• temporal entities
• geographical coordinates
• participating agents
• passive factors (such as tools)
• and products (results of an event)
• allows to decompose complex events into sub-events
Saturday, 28 April 12
96. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Timeline and Event Ontologies
• The Timeline Ontology extends OWL-Time and
defines the TimeLine concept.
• Temporal objects (signal, video, performance,
work, etc.) can be associated with a timeline.
• The Event ontology relates arbitrary events to:
• temporal entities
• geographical coordinates
• participating agents
• passive factors (such as tools)
• and products (results of an event)
• allows to decompose complex events into sub-events
Saturday, 28 April 12
97. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Timeline and Event Ontologies
http://purl.org/NET/c4dm/event.owl#
• An event may be
(for instance):
• a concert,
• a performance
or
• a note onset
Saturday, 28 April 12
98. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Ontology
• Defines a Music Production Workflow Model
mo:
Composition
mo:Musical
Work
mo:
Recording
mo:Release
Event
mo:
Performance
mo:Sound mo:Signal mo:Release
mo:Record
mo:produced_work mo:produced_sound mo:produced_signal mo:release
mo:record
mo:Track
mo:published_as
mo:performed_in mo:recorded_in
mo:
AudioFile
mo:available_as mo:track
event:factor_of
Event
Musical Work
Musical Expression
Musical Manifestation
Musical Item
Saturday, 28 April 12
99. • A large set of extensions are available,
including:
• The Audio Features Ontology
• The Chord ontology
• The Studio Ontology
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Ontology
Saturday, 28 April 12
100. • A large set of extensions are available,
including:
• The Audio Features Ontology
• The Chord ontology
• The Studio Ontology
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Music Ontology
Saturday, 28 April 12
101. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontology Frameworks
Music Ontology
Event
Timeline FRBR
FOAF
Chords
Audio
Features
Symbolic
Notation
Vamp
plugins
Instrument
Temperament
Studio Ontology
Device
Multitrack
Edit
Audio
Mixer
Microphone
Audio
E!ects
Sig.Proc. extension
base
Saturday, 28 April 12
102. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
Saturday, 28 April 12
103. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
• Enables collecting information about audio
production.
• Motivations
• Notation for capturing the contribution of the
engineer to creative work
• Improved Information and workflow
management in the studio
• Exploit music production data in MIR systems
• Enable building intelligent music production
systems
Saturday, 28 April 12
104. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
• Enables collecting information about audio
production.
• Motivations
• Notation for capturing the contribution of the
engineer to creative work
• Improved Information and workflow
management in the studio
• Exploit music production data in MIR systems
• Enable building intelligent music production
systems
Saturday, 28 April 12
105. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
• Enables collecting information about audio
production.
• Motivations
• Notation for capturing the contribution of the
engineer to creative work
• Improved Information and workflow
management in the studio
• Exploit music production data in MIR systems
• Enable building intelligent music production
systems
Saturday, 28 April 12
106. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
• Defines a Studio
Production
Workflow Model
• Two parts:
• Technical (domain
independent)
• Musical (domain
specific)
Saturday, 28 April 12
107. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
• Defines a Studio
Production
Workflow Model
• Two parts:
• Technical (domain
independent)
• Musical (domain
specific)
Saturday, 28 April 12
108. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
• Domain independent components:
• Technological artefacts (devices) and their
connections
rdfs:subClassOf rdfs:subClassOf
device:service device:state
device:Device
device:component
device:
AbstractDevice
device:
PhysicalDevice
device:Statedevice:Service
Saturday, 28 April 12
109. • Domain independent components:
• Technological artefacts (devices) and their
connections
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
rdfs:subClassOf
rdfs:subClassOf
con:connector con:protocol
con:Terminal
con:Optical
Terminal
con:Electrical
Terminal
con:Protocolcon:Connector
rdfs:subClassOf
rdfs:subClassOf
con:Analog
Terminal
con:Digital
Terminal
con:XLR_3M
rdf:type
con:AES42
rdf:type
Saturday, 28 April 12
110. • A model of audio processing devices
• Phenomenon: a physical process that
produces for instance an audio effect
• Model: a computational model of the process
• Implementation: a particular implementation
of the model, e.g. in C++
• Device: a concrete device that someone can
own
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
Saturday, 28 April 12
111. • A model of audio processing devices
• Phenomenon: a physical process that
produces for instance an audio effect
• Model: a computational model of the process
• Implementation: a particular implementation
of the model, e.g. in C++
• Device: a concrete device that someone can
own
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
Saturday, 28 April 12
112. • A model of audio processing devices
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
Phenomenon
representation
Model Implementation Device
Algorithm
Circuit
Design
actualisation instantiation
Computer
Code
Hardware
Design
Software
Plugin
Hardware
Unit
Audio
Effect
Visual
Effect
abstract concrete
Work
realisation
Expression Manifestation Item
embodiment exemplar
FRBR model
Signal Processing Device model
possible subclass
property relation
Saturday, 28 April 12
113. • Signal processing workflow model
• with separate
• Event flow
• Signal flow
• This supports the requirements of real-time
recording and audio processing scenarios
• as well as post-production.
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
Saturday, 28 April 12
114. • Signal processing workflow model
• with separate
• Event flow
• Signal flow
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
signal flow
RecordingSession PostProductionSession
event flow
mo:Music Ontology
studio:Studio Ontology
con:Connectivity Ontology
mo:Recording
con:Output
Terminal
studio:
Microphone
studio:
microphone
device:output
studio:signal
mo:produced_signal
con:Output
Terminal
studio:Mixing
Console
device:output
con:Input
Terminal
studio:signal
studio:produced_signal
studio:
console
device:input
con:Output
Terminal
studio:Effect
Unit
device:output
studio:signal
con:Input
Terminal
consumed_signal
studio:signal
studio:produced_signal
studio:
effect
device:input
studio:
Transform
studio:
Mixing
studio:signal
mo:Signalmo:Signalmo:Signal
consumed_signal
studio: studio:
Saturday, 28 April 12
115. • Extensions in 4 areas (more modules are in
preparation)
• Audio Recording
• Audio Mixing
• Audio Effects
• Audio Editing
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Studio Ontology
Saturday, 28 April 12
116. • A model of audio effects from physical
phenomena to concrete devices
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio Effects Ontology
device:
Phenomenon
device:
representation
device:
Model
device:
Implementation
device:
Device
device:
actualisation
device:
instantiation
abstract concrete
subclass or
subproperty
property relation
fx:AudioEffect
fx:model
fx:
Model
fx:
Implementation
fx:
EffectDevice
fx:implementation fx:device
fx:
Chorus
fx:Reverb
Model
fx:
VST
fx:
LADSPA
fx:Effect
Unit
fx:
Reverb
fx:Effect
Plugin
fx:
Schroeder
fx:
FDN
fx:Reverb
Impl.
fx:Reverb
Impl1
fx:Reverb
Impl2
Saturday, 28 April 12
117. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontologies and tools for
Semantic Audio
Saturday, 28 April 12
118. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Audio Features Ontology
• Key points:
• Features represented by Events or Signals
• Timelines link things together
• Basic feature types:
• Instants: Time point like features,
• e.g. a note onset
• Intervals: Temporal segments,
• e.g. the duration of the intro of a song
• Dense features: signal like features,
• e.g. a spectrogram
Saturday, 28 April 12
119. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Audio Features Ontology
• Key points:
• Features represented by Events or Signals
• Timelines link things together
• Basic feature types:
• Instants: Time point like features,
• e.g. a note onset
• Intervals: Temporal segments,
• e.g. the duration of the intro of a song
• Dense features: signal like features,
• e.g. a spectrogram
Saturday, 28 April 12
120. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Audio Features Ontology
Saturday, 28 April 12
121. • (1) A note onset on the signal timeline:
• An instant on a timeline
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Audio Features Ontology
@prefix tl: <http://purl.org/NET/c4dm/timeline.owl#>.
@prefix af: <http://purl.org/ontology/af/>.
!
:signal_timeline a tl:Timeline .
:onset_23 a af:Onset;
! event:time [
! a tl:Instant ;
! tl:timeline :signal_timeline ;
! tl:at "PT1.710S"^^xsd:duration ;
! ] .
Saturday, 28 April 12
122. • (2) A key segment:
• An interval on a timeline
132nd AES Convention, 26th-29th of April, Budapest, Hungary
The Audio Features Ontology
:signal_timeline a tl:Timeline .
:key_segment_1 a af:Segment;
! ! rdfs:label """Bb major""" ;
! ! af:feature "11" ;
! ! event:time [
! ! ! a tl:Interval ;
! ! ! tl:timeline :signal_timeline ;
! ! ! tl:start "PT30.1S";
! ! ! tl:duration "PT200S";
! ! ] .
Saturday, 28 April 12
123. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Semantic Audio Tools
• Tools that produce and read RDF according to
these ontologies include:
• Sonic Annotator
• Sonic Visualiser
• SAWA
Saturday, 28 April 12
124. • An Application Programming Interface for
feature extraction
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Vamp Plugins
http://www.vamp-plugins.org/download.html
Saturday, 28 April 12
125. • Vamp plugins take audio input and return
structured data (but not RDF!)
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Vamp Plugins
http://www.vamp-plugins.org/download.html
Saturday, 28 April 12
126. • Vamp Plugin Ontology:
Links the results with a plugin and the
enclosed algorithm that computed them.
• Vamp Transform Ontology:
Allows to express the parameters (e.g. window
size) that were used to obtain a particular set
of results.
• Plugins, parameters and results are linked,
and described using the same format!
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Vamp Plugins
Saturday, 28 April 12
127. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Sonic Annotator
• A command line Vamp plugin host that
outputs RDF
• Key features:
• A program for analysing large collections
available locally, or on the Web.
• It can read a very wide range of audio file
formats.
• Reads Vamp plugin configuration in RDF
• Returns the features in RDF linked with the
configuration and editorial data (if available)
Saturday, 28 April 12
128. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Sonic Annotator
• A command line Vamp plugin host that
outputs RDF
• Key features:
• A program for analysing large collections
available locally, or on the Web.
• It can read a very wide range of audio file
formats.
• Reads Vamp plugin configuration in RDF
• Returns the features in RDF linked with the
configuration and editorial data (if available)
Saturday, 28 April 12
129. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Sonic Annotator
• A command line Vamp plugin host that
outputs RDF
• Key features:
• A program for analysing large collections
available locally, or on the Web.
• It can read a very wide range of audio file
formats.
• Reads Vamp plugin configuration in RDF
• Returns the features in RDF linked with the
configuration and editorial data (if available)
Saturday, 28 April 12
130. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Sonic Annotator
• (1) Create an RDF transform skeleton:
• (2) Edit the file if necessary and run the
feature extractor:
• This will dump the results on the standard
output.
• A detailed tutorial is available at
• http://www.omras2.org/SonicAnnotator
$ sonic-annotator -s
vamp:vamp-example-plugins:fixedtempo:tempo > transform.n3
$ sonic-annotator -t transform.n3
vamp:vamp-example-plugins:fixedtempo:tempo
-w rdf --rdf-stdout audio_file.wav
Saturday, 28 April 12
131. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Sonic Annotator
• (1) Create an RDF transform skeleton:
• (2) Edit the file if necessary and run the
feature extractor:
• This will dump the results on the standard
output.
• A detailed tutorial is available at
• http://www.omras2.org/SonicAnnotator
$ sonic-annotator -s
vamp:vamp-example-plugins:fixedtempo:tempo > transform.n3
$ sonic-annotator -t transform.n3
vamp:vamp-example-plugins:fixedtempo:tempo
-w rdf --rdf-stdout audio_file.wav
Saturday, 28 April 12
132. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SAWA
• Sonic Annotator Web Application
• A tool for Web-based audio analysis
• Runs Vamp feature extractor plugins on a
small uploaded audio collection
• Configured using RDF and return RDF data
according to the Audio Features Ontology.
Saturday, 28 April 12
133. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SAWA
• Sonic Annotator Web Application
Saturday, 28 April 12
134. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Ontologies and Tools for Music
Production
Saturday, 28 April 12
135. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Ontology
Music Ontology
Event
Timeline FRBR
FOAF
Chords
Audio
Features
Symbolic
Notation
Vamp
plugins
Instrument
Temperament
Studio Ontology
Device
Multitrack
Edit
Audio
Mixer
Microphone
Audio
E!ects
Sig.Proc. extension
base
Saturday, 28 April 12
136. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Ontology
Music Ontology
Event
Timeline FRBR
FOAF
Chords
Audio
Features
Symbolic
Notation
Vamp
plugins
Instrument
Temperament
Studio Ontology
Device
Multitrack
Edit
Audio
Mixer
Microphone
Audio
E!ects
Sig.Proc. extension
base
Saturday, 28 April 12
137. •enable communication between musicians,
developers and engineers
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Ontology
Saturday, 28 April 12
139. •enable communication between musicians,
developers and engineers
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Ontology
Composer/
Performer
Developer
Engineer
Auditor
(aesthetic limits)
Instrument
(technical limits)
Sound
Saturday, 28 April 12
140. •enable communication between musicians,
developers and engineers
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Ontology
•interdisciplinary classification of audio effects:
• perceptual attributes
• implementation techniques
• application
Composer/
Performer
Developer
Engineer
Auditor
(aesthetic limits)
Instrument
(technical limits)
Sound
Saturday, 28 April 12
141. •enable communication between musicians,
developers and engineers
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Ontology
•Modularised
• Vocabulary
• List of FX
• Descriptors
• Application of FX
• Classifications
•interdisciplinary classification of audio effects:
• perceptual attributes
• implementation techniques
• application
Composer/
Performer
Developer
Engineer
Auditor
(aesthetic limits)
Instrument
(technical limits)
Sound
Saturday, 28 April 12
142. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Perceptual Classification
•Loudness
•Pitch/Harmony
•Space
•Timbre
•Time/Duration
Saturday, 28 April 12
143. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Description
:fx_0
fx:Implementation
fx:HiPassFilter
"AUHipass"
"Apple"
fx:Au
fx:Osx "aufx hpas appl"
fx:NumParameter
"0"
"cutoff
frequency"
"10.0"
"6900.0"
"22050.0"
implementation_of
dc:title
dc:creator
available_as
api
au_id
platform
parameter_name
parameter_type
min_value
max_value
default_value
rdf:type
implementation_version
has_parameter
rdf:type
"Hertz" parameter_id
• general descriptors
Saturday, 28 April 12
144. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Description
:fx_0
fx:Implementation
fx:HiPassFilter
"AUHipass"
"Apple"
fx:Au
fx:Osx "aufx hpas appl"
fx:NumParameter
"0"
"cutoff
frequency"
"10.0"
"6900.0"
"22050.0"
implementation_of
dc:title
dc:creator
available_as
api
au_id
platform
parameter_name
parameter_type
min_value
max_value
default_value
rdf:type
implementation_version
has_parameter
rdf:type
"Hertz" parameter_id
• specific version
Saturday, 28 April 12
145. :fx_0
fx:Implementation
fx:HiPassFilter
"AUHipass"
"Apple"
fx:Au
fx:Osx "aufx hpas appl"
fx:NumParameter
"0"
"cutoff
frequency"
"10.0"
"6900.0"
"22050.0"
implementation_of
dc:title
dc:creator
available_as
api
au_id
platform
parameter_name
parameter_type
min_value
max_value
default_value
rdf:type
implementation_version
has_parameter
rdf:type
"Hertz" parameter_id
132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Description
• parameters
Saturday, 28 April 12
146. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Description
• parameters
Saturday, 28 April 12
147. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Description
:fx_0
fx:NumParameter
"0"
"cutoff
frequency"
"10.0"
"6900.0"
"22050.0" parameter_name
parameter_type
min_value
max_value
default_value
implementation_version
has_parameter
rdf:type
"Hertz" parameter_id
fx:HiPassFilterCutoff
fx:FilterCutoff
rdfs:subClassOf
standard_parameter
• parameters
• definition of standard parameter
classes
Saturday, 28 April 12
148. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Audio FX Description
• link to classification
system
• e.g. perceptual (fxp)
Saturday, 28 April 12
153. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SPARQL Query Example
Which events have been produced by an audio effect
affecting loudness?
What is their track name in the original multitrack
project?
Saturday, 28 April 12
154. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SPARQL Query Example
Which events have been produced by an audio effect
affecting loudness?
What is their track name in the original multitrack
project?
Saturday, 28 April 12
155. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SPARQL Query Example
SELECT ?time ?track WHERE {
?a event:time ?b ;
fx:created_by_fx ?c ;
fx:track_origin ?track.
?b tl:at ?time .
?c fx:transform ?d .
?d fx:implementation_of ?e .
?e fxp:main_attribute fxp:Loudness .
}
Which events have been produced by an audio effect
affecting loudness?
What is their track name in the original multitrack
project?
Saturday, 28 April 12
156. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SPARQL Query Example
SELECT ?time ?track WHERE {
?a event:time ?b ;
fx:created_by_fx ?c ;
fx:track_origin ?track.
?b tl:at ?time .
?c fx:transform ?d .
?d fx:implementation_of ?e .
?e fxp:main_attribute fxp:Loudness .
}
Which events have been produced by an audio effect
affecting loudness?
What is their track name in the original multitrack
project?
Saturday, 28 April 12
157. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SPARQL Query Example
SELECT ?time ?track WHERE {
?a event:time ?b ;
fx:created_by_fx ?c ;
fx:track_origin ?track.
?b tl:at ?time .
?c fx:transform ?d .
?d fx:implementation_of ?e .
?e fxp:main_attribute fxp:Loudness .
}
Which events have been produced by an audio effect
affecting loudness?
What is their track name in the original multitrack
project?
Saturday, 28 April 12
158. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SPARQL Query Example
SELECT ?time ?track WHERE {
?a event:time ?b ;
fx:created_by_fx ?c ;
fx:track_origin ?track.
?b tl:at ?time .
?c fx:transform ?d .
?d fx:implementation_of ?e .
?e fxp:main_attribute fxp:Loudness .
}
Which events have been produced by an audio effect
affecting loudness?
What is their track name in the original multitrack
project?
Saturday, 28 April 12
159. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
SPARQL Query Example
SELECT ?time ?track WHERE {
?a event:time ?b ;
fx:created_by_fx ?c ;
fx:track_origin ?track.
?b tl:at ?time .
?c fx:transform ?d .
?d fx:implementation_of ?e .
?e fxp:main_attribute fxp:Loudness .
}
Which events have been produced by an audio effect
affecting loudness?
What is their track name in the original multitrack
project?
Saturday, 28 April 12
160. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Predicting New Metadata
•feature extraction from effected files is inefficient
•instead: predict and accumulate metadata (where
possible)
•use RDF and the Audio Effects Ontology
Saturday, 28 April 12
161. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Predicting New Metadata
•feature extraction from effected files is inefficient
•instead: predict and accumulate metadata (where
possible)
•use RDF and the Audio Effects Ontology
updated
metadata
feature
extraction
query
metadata FX
audio data
metadata
transformed
audio
audio FX
effect
parameters
Saturday, 28 April 12
162. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
FX-Based Information Retrieval
:event_1 a af:Onset;
event:time [ a tl:Instant;
tl:at "PT2.194007S"^^xsd:duration;
tl:onTimeLine :signal_timeline_0];
fx:created_by_fx :transform_0;
fx:event_used :event_0;
fx:track_origin :drums.
Saturday, 28 April 12
163. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
FX-Database on the Semantic Web
Saturday, 28 April 12
164. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
• Large database on the Web: KVR Audio
FX-Database on the Semantic Web
Saturday, 28 April 12
165. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
• Large database on the Web: KVR Audio
• HTML: Data is not easily reusable
FX-Database on the Semantic Web
Saturday, 28 April 12
166. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
• Large database on the Web: KVR Audio
• HTML: Data is not easily reusable
• Website format may change
FX-Database on the Semantic Web
Saturday, 28 April 12
167. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
• Large database on the Web: KVR Audio
• HTML: Data is not easily reusable
• Website format may change
• No clear Semantics
FX-Database on the Semantic Web
Saturday, 28 April 12
168. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
• Large database on the Web: KVR Audio
• HTML: Data is not easily reusable
• Website format may change
• No clear Semantics
• KVR module for the FX Ontology
FX-Database on the Semantic Web
Saturday, 28 April 12
169. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
FX-Database on the Semantic Web
:fx_0 a owl:Class, fx:PlugIn ;
fx:implementation_of fx:Reverberation, kvr:Reverb ;
dc:title "VariVerb Pro"^^xsd:string ;
dc:creator "Magix"^^xsd:string ;
rdfs:seeAlso "http://www.samplitude.com/eng/vst/variverb.html";
fx:available_as fx:Vst ;
gr:hasPriceSpecification
[ a gr:UnitPriceSpecification ;
gr:hasCurrency "USD"^^xsd:string ;
gr:hasCurrencyValue "199"^^xsd:float ;
gr:validThrough "2012-02-13T20:16:40"^^xsd:dateTime ] .
• KVR module for the FX Ontology
Saturday, 28 April 12
170. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Applications of the FX Ontology
• Music production
• detailed metadata creation
Saturday, 28 April 12
171. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Applications of the FX Ontology
• Music production
• detailed metadata creation
• reproducibility of sound
transformations
Saturday, 28 April 12
172. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Applications of the FX Ontology
• Music production
• detailed metadata creation
• reproducibility of sound
transformations
• recommendation of similar audio
effects and settings
Saturday, 28 April 12
173. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Applications of the FX Ontology
• Music production
• effect search by high level semantic
descriptors
• perceptual/technical descriptors
• link to data on the Semantic Web
Saturday, 28 April 12
174. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Applications of the FX Ontology
• Music production
• effect search by high level semantic
descriptors
• perceptual/technical descriptors
• link to data on the Semantic Web
• semantic metadata as control input for
adaptive audio effects
Saturday, 28 April 12
175. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Applications of the FX Ontology
• Musicological research
• production tendencies of genres/eras
• more detailed descriptors due to
retention of multitrack and transform-
specific metadata
Saturday, 28 April 12
176. 132nd AES Convention, 26th-29th of April, Budapest, Hungary
Summary
• The use of Semantic Web technologies enable
Semantic Audio applications that link and
scale like the Web itself.
• New applications using a mashup of data
sources
• Provide interoperability between tools in
music information sciences and music
production
Saturday, 28 April 12