This document summarizes a talk given by Prof. Abhik Roychoudhury about skills needed for a PhD. He discusses obvious skills like analyzing papers and identifying research trends. Less obvious skills include choosing impactful problems and determining if one has the right background. The least obvious skill is determining what constitutes a research contribution, which is qualitative rather than quantitative. The talk provides examples of different types of contributions and emphasizes choosing an interesting research area and topic, considering its relevance over time, potential for translation, and avoiding negative perceptions from the community.

1. ISSTA19
Doctoral
Symposium
Art of
Research
Planning
Prof. Abhik Roychoudhury
National University of Singapore
ISSTA 2019 Doctoral Symposium
All comments in this talk are my own, and for discussion only.

2. Skills needed in PhD
Obvious ones
• Reading a paper and quickly judging a
contribution.
• Compare and contrast the contributions of each
paper, and chart the growth of a research area
over time.
• Identify certain emerging research trends alluded
to in recent publications, and check their
feasibility
ISSTA 2019 Doctoral Symposium

3. Obvious Skills needed
Obvious ones
• Why are they obvious?
• They involve effort and discipline, of course!
• They involve analytical skills, given the papers you
need to analyze them, and distill out the positioning
of the papers.
• Involves looking outside, and doing so analytically
• In research, you are expected to be analytical
• It is no big deal
ISSTA 2019 Doctoral Symposium

4. Less Obvious Skills
needed
Less Obvious ones
• Choosing the right problems at the right time
• Why will it be impactful if you study it now?
• What technical skills and background are
needed, AND
• Why you are the right person right now to
tackle this problem?
• Looking inside yourself
ISSTA 2019 Doctoral Symposium

5. Least Obvious Skills
needed
Least Obvious ones
• Learning to decide what is a research contribution?
• An (ISSTA) paper, A degree, x number of citations ??
•NO!!
• Contribution is how you progress the field
• It is argumentative and Qualitative in nature, as
opposed to quantitative
• Paper in a top conference is only a quantitative measure
ISSTA 2019 Doctoral Symposium

6. Nature of contribution
in CS
• Conceptual Contribution
• Focus is on the imagination such as invention of
program representations by Fran Allen, who was
a school-teacher in Peru (NY)
• She revolutionized compilation, so that you can
compile your programs today for your assignment
ISSTA 2019 Doctoral Symposium

7. Nature of contribution
in CS
• Technical Contribution
• [Transplantation / Adaptation style] Amir Pnueli
brought ideas from philosophy to build temporal
logics for specifying properties of programs.
• [Cross-fertilization style] Clarke, Emerson and Sifakis
developed model checking as an automated
procedure for checking programs
• Today, these are being used to check critical
properties of SW in the car you are driving.
• Also, used for checking Intel processors we are using
ISSTA 2019 Doctoral Symposium

8. Nature of contribution
in CS
• Utilitarian Contribution
• e.g. build a novel computer system which acts a great
enabler.
• Need to be careful that it is not implementation of
known ideas.
• Conveys important design goals and principles.
• Invention of Unix operating system by Ken
Thompson, which has influenced Apple’s MacOS
which you and I are running right now.
• Technical and utilitarian contributions often go hand-
in-hand.
ISSTA 2019 Doctoral Symposium

9. Make your own
Contribution
ISSTA 2019 Doctoral Symposium
You have several ideas, and you can even evaluate their
Technical, Conceptual Contributions
Which one to choose?

10. Research area
ISSTA 2019 Doctoral Symposium
Is it interesting to YOU?
Is it topical? Are you the only one interested?
Is it useful even in indirect fashion?
….
Not all the questions need to be answered as YES
Ask yourself

11. Growth
ISSTA 2019 Doctoral Symposium
Will my research area remain relevant ?
Will it contribute to grow?
What are the disruptive technologies which may aid
or prevent its growth.
Ask yourself
Super-hot areas can become super-cold quickly, Why (Question for the group)
Certain areas are broad and ever-green e.g. PL, SE. Comes with choices

12. Research Topic
• Who would be interested?
• Who would fund it, if they had 1B?
• Only a thought experiment
• Does not preclude theoretical research
• Level of interest in a topic in research
community is not necessarily linked translation.
ISSTA 2019 Doctoral Symposium

13. Translation
• Translation is not an immediate need
• Yet it remains a key goal.
• Tough to find a balance
• Never translate anything
• Focus only on translation?
• Try to see a pathway …
ISSTA 2019 Doctoral Symposium

14. Examples from real-life
• I focus on my own community
• Software testing via symbolic execution
• Proposed in 1976 and then again in 2005.
• Aided by growth of Constraint Solver capability
• Uptake from some companies in 10 years
• Some of the research is quite theoretical
• Heavy investment for a company initially
• Pays off in long term, used in multi-national
corporations today.
ISSTA 2019 Doctoral Symposium

15. Example area
ISSTA 2019 Doctoral Symposium
Software
Seed Tests
Engine (calculate path
for seed tests, convert it
to logic, mutate logical
formula to find inputs
which go through other
paths, solve mutated
logic formula and get
more inputs … )
Test
suite
covering
many
paths
Aided by growth in
solver technology

16. Another example
direction
ISSTA 2019 Doctoral Symposium
Debugging
Buggy
Program
Failing
Test
Explanation
Less theoretical in nature
Often involves huge search
Searching in time
Searching in space
Lot of heuristics
Very active 1997 – 2016
….
Less papers now?
Lot of sctivity in. related field
of Automated Program Repair.
Keep an eye. You could be the
leader of the next wave.

17. Translation of research
• Translation gives your research more
credibility
• Each research community goes in waves
• Interest in a technology often reduces.
• Translation over a period of time can sustain
that interest.
• Translation can also move the area into other
directions over a longer time e.g. decades
• Why a certain topic is not translated?
ISSTA 2019 Doctoral Symposium

18. Lack of Translation
• Little bit of a story of start-up success
• Why is a start-up not successful
• Not enough investment
• Steady marketing blitz
• …
• Alternate ways of doing it
• Inferior technology
• …
• Falls through the cracks.
• Need investment for translation, may not be in funds.
This is the nuanced aspect.
ISSTA 2019 Doctoral Symposium

19. Research Topic
• Who would be interested?
• Who would fund it, if they had 1B?
• Only a thought experiment
• Does not preclude theoretical research
• Level of interest in a topic in research community is
not necessarily linked translation.
• Time-line for deployment
• 5-50 years
• Thought experiment only
ISSTA 2019 Doctoral Symposium

20. Imagination
• Satisfying the intellectual curiosity
• Beyond utility
• Does it touch their mind
• [Contrast]
• Does this attempt a useful problem
• [Testing – our example]
• Is this a practical technology
• Not strictly enforced when an area starts
• [Symbolic Execution was not practical when it
started]
ISSTA 2019 Doctoral Symposium

21. Contribution
• Technical contribution
• Smart-ness of solution
• Narrowly focused on Problem solving
• Conceptual contribution
• Charting out an area
• More focused on Problem Formulation
• Utilitarian Contribution
• Translation and Deployment
ISSTA 2019 Doctoral Symposium

22. Timing of a topic
• Not too ahead
• Ultra-futuristic
• Not today’s issue
• Implementation only
• Tomorrow’s issue
• Symbolic execution took off after SMT solvers
became mature 1995- 2005 roughly.
ISSTA 2019 Doctoral Symposium

23. Timing of an area
• PhD takes 5 years
• A research area in CS which is hot in 2019 vey
unlikely to be hot in 2024
• Your job will be affected by what is hot in 2023-24
• Example period 1995-2000
• Computer Networking was hot in 1995-96
• There was a huge supply of Networking graduates
in 2000-01.
ISSTA 2019 Doctoral Symposium

24. Timing of CS itself
• Boom and bust cycles in CS
• We are seeing a boom right now
• Hard to know how long it lasts
• Example: 1997 vs 2002.
• Dot com internet startup boom 1996 – 2001.
• Different situation 2003 – 2010 or so.
• These are global factors which affects ALL CS
graduates!
ISSTA 2019 Doctoral Symposium

25. Perception
• Avoid negative perception of community
• Fitting a solution to a problem
• You have a hammer everything looks like a nail
• Example: Testing via Symbolic Execution
• Problem: Testing of XYZ software
• Technique: Symbolic Execution
• Question: Is symbolic execution an overkill for
XYZ software could simpler techniques be used?
ISSTA 2019 Doctoral Symposium

26. Perception
• Avoid negative perceptions such as:
• Been there done that.
• [Example] Use formal methods to verify critical software
• [Perception] Tried it (20 years ago in 1998) too slow !
• [Reality] More practical in 2018-17 due to powerful
solvers being developed?
• You need to write papers with clarity to avoid such
perceptions.
ISSTA 2019 Doctoral Symposium

27. Perception
• Avoid negative perception of community
• This is easily achievable
• Claiming away your technical contributions.
• Mentioning your contributions as more of an
implementation than ideas.
• Need to write the paper to cater to, and
prevent such reviews.
ISSTA 2019 Doctoral Symposium

28. Perception
• Avoid negative perception of community
• This must have been done before.
• Usually papers not turned down because of not
mentioning related work.
• One may compare your paper at a high level with
another, and claim similarity.
• This is where you need to be very clear yourself
about the new concepts and ideas in your paper.
ISSTA 2019 Doctoral Symposium

29. Takeaways
• Choose an area
• Choose a topic of interest
• Why would it interest others
• Possible translational value you can claim
• Extrapolate to the time you finish PhD
• Evaluate the contributions – be clear yourself
• Evaluate the timing of your work – is the time right?
• Structure the paper to address common negative
perceptions, AND
ISSTA 2019 Doctoral Symposium

30. Takeaways
• Contribution and advance of a paper is dependent
on so many factors.
• Implementation, experiments and experimental
results are important
• But they are far from being the only factor.
• Footnote: What I tell all my students
• THINK hard before we start working hard
IMPLEMENTING!
ISSTA 2019 Doctoral Symposium