1.
New Insights from Trademarks
with Natural Language
Processing
Alex Lehmann
CanadianIntellectual Property Office

2.
Acknowledgement
CIPO Economic Research & Strategic Analysis team
• Gray Barski
• Mohammad Dehghan
• Dakshdeep Singh
• Xiang Zhao
2

3.
3
Outline
1. Trademarks as economic indicators
2. COVID-19 and trademark activity
3. Detecting emerging trends
4. Classifying trademarks by industry

4.
Trademarksas Economic
Indicators
4

5.
What is a trademark?
• Form of intellectual
property (IP)
• Means of identifying a
firm’s products/services
• Name, logo, symbol, etc.
5

6.
Economic Importance
• Trademarks help define a firm’s brand
• Leading indicators for market activity
• Registration takes time and resources
CONCLUSION:
Trademarksurveillance shows where firms are interestedin establishing
and maintaining a presence in the market.
6

7.
Text Mining in Trademarks
• Trademark
applicants must
designate goods and
services for their
mark
• Lines of business for
the brand
7

8.
Study Data
• 276,271 registered
trademarks
• Filed 2012-2022
• Available via
IP Horizons
• Formatted
• Lemmatized
• Stop words removed
8
“Providinga web-based portal
for an array of benefits”
“provide web based portal
array benefit”

9.
9
COVID-19and Trademark
Activity

10.
Research Question
• GDP and trademarks are usually correlated
• Pandemic GDP went down, but trademarks
went up
• Mostly first-time trademark filers
QUESTION:Which lines of business were driving the trademarkboom?
10

11.
ModelingApproach
11
Separable
Nonnegative
Matrix
Factorization
Document
topics
Within-topic
word importance
Collection of
documents
(“corpus”)

12.
ModelingApproach
12
Document
topics
Resampling-
Based
Inference
(Bootstrap)
A priori
document
groups
Distributions
for topics
by group

13.
Regional Variation
• How did different
regions’ medical
suppliers respond to
the pandemic?
• Estimate distributions
of “mask” emphasis
in trademarks
13
Regional Prevalence for
“mask” topic

14.
First-Time Filers’ Lines of Business
14
First-time filers
Establishedfilers

15.
15
DetectingEmergingTrends

16.
Motivation
• Emerging markets offer great opportunity for Canadian
business, especially small and medium enterprises
(SMEs)
• Early identification of emerging markets for innovation is
a crucial component of building IP awareness and
advancing innovation
16

17.
Emerging Topic Model
Text Mining (per-period)
1. Embed trademark text using
machine learning.
2. Find clusters of similar
trademarks.
3. Represent clusters with most
relevant words.
Modeling Evolution (overall)
1. Calculate similaritybetween
clusters in each time period.
2. Consider similarclusters
“linked”.
3. Tracking links tells us how
lines of business appear,
change, and terminate.
17

18.
Topic Evolution
1. Continuation
2. Emergence
3. Splitting
4. Merging
5. Termination
18
Initial topic
Continuation
Emergence
Splitting
Merging Termination

19.
Evolution of a Line of Business
19

20.
20
Classifying Trademarksby
Industry

21.
Motivation
• North American firms have a classification under the
North American Industry Classification System (NAICS)
• We want to know when firms move/expand into a new
industry
• Trademarks could be convenient leading indicators for
this activity
21

22.
Classification Methodology
22
Collection of
documents
(“corpus”)
Embedding
Model
(BERT)
Deep
learning
classifier
NAICS code for
each document
“221113”

23.
Preliminary Performance
• Subsector identification task:
• F1 score: ~0.77
• Accuracy > 97% in certain sectors
23
221113
Sector
(Utilities)
Subsector
(Utilities)
Industry group
(Electric power
generation)
Industry
(Electric power
generation)
National sector
(Nuclear electric power)

24.
24
Conclusion

25.
Conclusion
• Trademarks are useful indicators for innovation and
drivers of economic change
• Text mining and natural language processing are
powerful tools for trademark surveillance
• There are myriad opportunities to apply these techniques
to answer important questions about innovation
25

26.
Thankyou!
Alex Lehmann
BusinessImprovement Services
CanadianIntellectual Property Office
alexander.lehmann@ised-isde.gc.ca
26