Impactful SE Research: Some Do's and More Don'ts

Impactful SE Research: 
Some Do’s 
and More Don’ts
Gail C. Murphy 
University of British Columbia 
Tasktop Technologies Inc.
@gail_murphy

“software is eating
the world”
Marc Andreesen

Multiple Paths to Impact
€ Economic
Ý Social
Ħ Policy
á People
! Academic

Do
•understand and use the
real problem as a
beacon
•be open to
opportunities
•continually listen
•collaborate
•adapt
•be generous
•approach problems
with integrity
Don’t
•focus only on home runs
•put the research ahead
of the people
•think innovation only
happens in academia
•think scale doesn’t
matter
•think open source is
enough
•think the message
doesn’t matter

Who Should 
Fix This
Bug?
7
Mylar
Story
Hipikat
Four Stories
PhD:  
Reflexion 
Models

Who Should 
Fix This
Bug?
8
Mylar
Story
Hipikat
Reflexion
Models
A story about
academic and
social impact 
and openness
to opportunity
PhD:  
Reflexion 
Models

A bit of background
B.Sc.
Computing
Science
"♀
Software
Developer
(5 years)
!
Grad
School
(U. Washington)

Ph.D.: Reflexion Models
Murphy, Notkin & Sullivan, FSE 1995. SIGSOFT Retrospective Impact Award.

Goals
•Lightweight
• Apply to a large system in < 30 minutes 
•Flexible
• Multiple kinds of source, relationships between ‘modules’ 
•Human-centric
• Provide information to human in human’s terms

Reflexion Models for Excel
File
Sheet
Graph

Reflexion Models for Excel
Graph
Sheet
File
0
36734
912
1210
...
...

Reflexion Models: Take-aways
•Focus on the problem
• Understanding the implementation 
•Notkin guidance
• “Your Ph.D. doesn’t have to be a home run”
• “It is the start of your career and life’s work not the end” 
•Openness to opportunity
• Be ready to pitch
• Not the perfect case study but lots of good aspects
Ý Social! Academic

Who Should 
Fix This
Bug?
16
Mylar
Story
PhD: 
Reflexion 
Models
Hipikat
Hipikat
A story about 
academic impact
and the need
for scale

Hipikat
• Context:
• The best information about a development is in the heads of the
developers’ 
• Hypothesis:
• Pertinent artifacts to a task can be recommended from a latent group
memory formed from digital assets recorded as part of the development 
• Prototype:
• Built a latent group memory automatically from repository information,
mailing lists, etc.
• Evaluation:
• Wizard-of-oz, controlled lab studies, simulation studies
Çubranić and Murphy, ICSE 2003. Ten year ICSE impact award.

Building Hipikat
• Collaboration with IBM when Eclipse was in development 
• Likely first academic plug-in built for Eclipse 
• IBM personnel participated in lab studies, increasing validity 
• Lots of hiccups to overcome
• New development environment and architecture
• Constant upgrading at IBM to be on the latest

Hipikat: Take-aways
• Collaboration as an enabler
• Joint collaboration with industry partner 
• Scale matters
• Eclipse presented itself as an opportunity we acted upon 
• Messages matter
• Hipikat is still first in Google search
• The water cooler description always did and still resonates
• Research can push policy boundaries
• What was ethical to use from open source development?
! Academic á PeopleĦ Policy

Who Should 
Fix This
Bug?
22
Mylar
Story
PhD: 
Reflexion 
Models
Hipikat
Who Should
Fix This Bug?
A story about 
academic impact
and first base
can be enough

Who Should Fix This Bug?
• Problem:
• Lots of issues filed in open source
• How to reduce the effort and costs of triaging the bugs 
• Proposed solution:
• Use supervised machine learning to learn triage results, like who
should fix a particular bug, which component the bug should be filed
against, etc. 
• Evaluation:
• Applied against a few open source systems
• My favourite review comment: “academically honest” because we
explained where it didn’t work
Anvik, Murphy and Hiew, ICSE 2006. Ten-year ICSE impact award.

A problem before its time
• We cared about how hard it would be to apply the machine
learning approach to a new system 
• Semi-automated configuration of machine learning approach
was important to us for approach to be usable 
• At least 1/2 of John Anvik’s thesis was this semi-automated
configuration 
• Hard to publish because the problem wasn’t understood (and we
didn’t explain well enough) 
• We persevered

Who Should Fix This Bug: Take-aways
• Messages matter
• The title captured attention 
• Integrity is important
• Outlining context where something works and where it does not is
valuable 
• “First base” can be enough
• We opened a new area more than solving a problem completely 
• Sometimes impact is not where you expect
• Would I have predicted this as the most-cited paper from the group?
! Academic á People

Who Should 
Fix This
Bug?
26
Mylar
Story
PhD: 
Reflexion 
Models
Hipikat
The Mylar/
Tasktop
A story about 
academic and 
industrial
impact

TIMELINE
2004 Mylar
Information Overload
Mylar 
 
See only what you’re working
27

TIMELINE
28
2007 Tasktop
Mik Kersten
Rob Elves
3 Co-founders
Gail Murphy
Shawn  
Minto
Leo Dos
Santos
2 Staff
2004 Mylar

TIMELINE
29
2017 Tasktop
2004 Mylar
43 of the Fortune 100

Mylar Highlights: 2004-2007
2004
2004
2005
2005
2005-06
2007
Mylar invention & innovation
1st industrial trial at IBM Toronto
1st academic publication
EclipseCon/Field Study/Eclipse project
Company discussions; biz competitions
Mik’s Ph.D. Friday; incorporate Monday
IBM 
Trial
Publication
Murphy and Kersten, AOSD 2006. Ten-year Modularity impact award.

2004
2004
2005
2005
2005-06
2007
Mik’s Ph.D. Friday; incorporate Monday
IBM 
Trial
Publication
Robustness
&
community
building

2004
2004
2005
2005
2005-06
2007
Mik’s Ph.D. Friday; incorporated Monday
IBM 
Trial
Publication

2007
2008
2009
2011
2014
2015-17
Mylar becomes Eclipse Mylyn
Tasktop Dev
Tasktop Dev - Enterprise sales
Tasktop Sync
$11M Series A funding
Tasktop Data & Gateway capabilities
Eclipse Mylyn
Various innovations
Mylyn/Tasktop Highlights: 2007-2016

2007
2008
2009
2011
2014
2015-16
Tasktop Dev
Tasktop Sync
Eclipse Mylyn
Various innovations

2007
2008
2009
2011
2014
2015-16
Tasktop Dev
Tasktop Sync
Eclipse Mylyn
Various innovationsVarious
inventions

2007
2008
2009
2011
2014
2015-16
Tasktop Dev
Tasktop Sync
Eclipse Mylyn
Various innovationsVarious
inventions
Continual
innovation &
incremental
validation
2017: $11M Series B Funding 
2017: Tasktop Integration Hub

Robustness
&
community
building
Continual
innovation &
incremental
validation

Mylar/Tasktop: Take-aways
•Collaboration
• With Mik Kersten, with open source community, with
Tasktopians, with industry partners, with customers … 
•Adaptability
• Pivot to Tasktop Sync 
•Open source takes work
• Tasktop supports Mylyn community
!Academic € Economic á People

Who Should 
Fix This
Bug?
41
Mylar
Story
Hipikat
PhD:  
Reflexion 
Models

42
Thanks to…
Mylar + Early Tasktop:
Tasktopians circa 2013
Mik Kersten, Rob Elves, Shawn Minto, Davor Cubranic, Taivo Evard, Nathan Hapke,  
Wesley Coelho, Meghan Allen, Leo Dos Santos, Steffen Pingel
Collaborators: 
David Notkin, Kevin Sullivan, 
Davor Çubranić, John 
Anvik, Lyndon Hiew

Who Should 
Fix This
Bug?
43
Mylar
Story
Hipikat
PhD:  
Reflexion 
Models
Academic
Social
People
Academic 
People
 
Academic
{Policy}
People
Academic 
Social 
Economic
People
Impacts

Do
•understand and use
the real problem as a
beacon
•be open to
opportunities
•collaborate
•adapt
•be generous
•be a great academic
Don’t
•focus only on home runs
•put the research ahead
of the people
•think innovation only
happens in academia
•think scale doesn’t
matter
•think open source is
enough
•think the message
doesn’t matter
At the start of the talk…

All in terms of “Do”s
•understand and use
the real problem as a
beacon
•be open to
opportunities
•collaborate
•adapt
•be generous
•be a great academic
•focus on the problem at
hand
•see the value in people in
your network
•follow innovations in
industry
•consider whether scale
matters
•realize open source
requires effort
•spend time on the
message

Impact can take many forms 
 
Gail C. Murphy 
University of British Columbia 
Tasktop Technologies Inc. @gail_murphy
Focus on the problem
Be open to possibilities
Think outside the box
Do research with integrity

Impactful SE Research: Some Do's and More Don'ts

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Impactful SE Research: Some Do's and More Don'ts

Similaire à Impactful SE Research: Some Do's and More Don'ts (20)

Plus de Gail Murphy

Plus de Gail Murphy (14)

Dernier

Dernier (20)

Impactful SE Research: Some Do's and More Don'ts