The document discusses evaluating web-based bug tracking tools using conceptual graph models of software development processes. It proposes modeling roles, intentions, and statuses to better characterize processes. Current tools often don't address why roles are authorized for changes. Conceptual graphs can formally represent processes for analysis and comparison. Next steps include validating the approach through empirical studies and incorporating the models into organizations to guide distributed collaboration.

1. An evaluation of the pragmatics of
web-based bug tracking tools
Proc. 2nd Intl. Conf. on the
Pragmatic Web
Tilburg, Netherlands
23 Oct 2007
Harry S. Delugach
Intelligent Systems Laboratory
Computer Science Department
Univ. of Alabama in Huntsville, USA
Intelligent Systems Laboratory

2. Context of inquiry
● Why web-based tools to support software
development?
– Distributed environments, or at least where the
participants might have difficulty meeting
regularly face-to-face
– Web-based tools allow them to collaborate in a
generally cost-effective way
– Large number of activities and artifacts to be
created and maintained
Intelligent Systems Laboratory

3. Difficulty with tools
● Effectiveness is determined by how well
participants understand how to use them.
● Mere use of tools is not sufficient to support an
effective process.
● Much current work is focused on characteristics of
artifacts or products
● This work focuses on the roles and purposes of the
participants
Intelligent Systems Laboratory

4. Purpose in developing models
● To better characterize and describe the
processes themselves
● To formally analyze and evaluate tools with
respect to generally accepted process
models
● To formally compare and contrast the
models with each other
● To arrive at a shared process model among
developers
Intelligent Systems Laboratory

5. Techniques
● Workflow modeling
● Software problem tracking
– sometimes called “bug tracking”
● Conceptual graphs:
– a convenient formalism
– easily understood visualization aid to represent
the models
Intelligent Systems Laboratory

6. Workflow modeling
● Aim is to provide framework for describing and
evaluating software development processes
● Neutral with respect to being either normative or
descriptive.
● Being “informal” does not render a model incapable
of being modeled
● We assume that developing a model of a process
is generally more useful than not modeling it at all.
Intelligent Systems Laboratory

7. Workflow step
● Starting with RENISYS
– An organizational role has particular obligations
to an activity, and an intention to accomplish
Intelligent Systems Laboratory

8. Beyond RENISYS
● Include concept “Intention” with respect to a role in
the process.
– Previous models show only the obligations (required,
allowed, prohibited) of a particular role.
– No indication as to why a particular role would be given a
particular assignment.
– For example, why would a program manager be required
to review a change, or why would a developer be
allowed (but not required) to make a change?
● Include status with respect to that intention
Intelligent Systems Laboratory

9. Software problem tracking process
● Part of software configuration management (SCM)
● Many people involved
● Issues to consider:
– Who is involved?
– What are stakeholders’ roles in the process’s success?
– What responsibilities do they hold with respect to the
system?
– What are their communication needs with other
stakeholders?
Intelligent Systems Laboratory

10. Modeling problem resolution processes
● Bug-tracking is one kind of problem
resolution process.
● Standard ISO/IEC 12207.0 clause 6.8
● Processes supported/implied by tools
– Bugzilla
– Trac
– sourceforge
Intelligent Systems Laboratory

11. Existing tools and processes
● No explicit set of roles which are defined in the
process
● Most tools seem implicitly geared toward a
particular change control process
● Some of them appear to imply certain role
● Others appear role-neutral.
● Generic models for software engineering
processes
Intelligent Systems Laboratory

12. Ontology for bug tracking
Intelligent Systems Laboratory

13. Model for reporting a bug
Intelligent Systems Laboratory

14. Model for fixing a bug
Intelligent Systems Laboratory

15. Validation of model graphs
● Data mining
– Use conceptual graph tools to scan the wealth of
existing data (e.g., Ripoche and Sansonnet)
– Emails, forum posts, program source code
comments identifier names in programs
● Human subjects
– Paraphrasing graphs in natural language
– Analyze comparison graphs
Intelligent Systems Laboratory

16. ISO/IEC 12207 process
“… all detected problems are promptly reported and
entered into the Problem Resolution Process; action is
initiated on them; relevant parties are advised of the
existence of the problem as appropriate; causes are
identified, analyzed, and, where possible, eliminated;
resolution and disposition are achieved; status is tracked
and reported…”
● Described by text in passive voice
– E.g., reported by whom, to whom, who initiates
action, etc.
Intelligent Systems Laboratory

17. ISO/IEC 12207 process model
Intelligent Systems Laboratory

18. ISO/IEC 12207 process model
● What’s missing?
Intelligent Systems Laboratory

19. Bugzilla bug tracking
Intelligent Systems Laboratory

20. Bugzilla’s model
● Nothing missing!
Intelligent Systems Laboratory

21. Sourceforge bug tracking
● Assignee - The project administrator to
which a tracker item is assigned. Can be
chosen from one of the administrators
registered in this project.
● Status - This is the (potentially changing)
current status of a bug.
● Priority – a nine-level scale.
● Category – project-specific.
● Group – project-specific.
Intelligent Systems Laboratory

22. Status?
●The online help says:
You can set the status to 'Pending' if you are
waiting for a response from the tracker item
author. When the author responds the status is
automatically reset to that of 'Open'. Otherwise, if
the author doesn't respond within an admin-
defined amount of time (default is 14 days) then
the item is given a status of 'Deleted'.
●Suggests definitions and process
Intelligent Systems Laboratory

23. What does “support” a process mean?
● Consider the attributes of “category” and
“group”
– Each project administrator can choose them
based on their own preferences.
– Automated bug tracker has no capability to:
• Relate categories or groups to each other, or
• To accommodate constraints between particular
categories, groups or values of the other attributes
(except for the ability to search the bug list by value).
 For example, are “category” and “group” orthogonal to each
other, or is a group a sub-category, etc.?
Intelligent Systems Laboratory

24. Modeling “status”
● One educator reports:
– “if the phrases describing subtask status are not defined,
different student teams often give different meanings to
the same phrase. Even worse, sometimes, different
members in the same team would interpret the same
phrase differently.”
● Need to define status phrases indicating which role
and process are involved
– e.g., the status “Ready for Review” meant ready to be
reviewed by the quality assurance (QA) role on the team.
A better way to name this would be an explicit “Ready for
Review by QA” status.
Intelligent Systems Laboratory

25. Status as a derived concept
● Process perspective: item’s status reflects
the process that produced it
Intelligent Systems Laboratory

26. Summary
● Many current tools are incomplete with respect to
organizational needs
– Because tools do not address the issues of why
someone is authorized (or not) to make a change to an
item’s status, it is likely for a bug’s status to be
inconsistent with a process
● The advantages of using conceptual graphs to
represent the processes
– Have the potential to be formally manipulated and
compared
– Are well-defined as in ISO/IEC 24707 (Common Logic)
– Provide an easily understood visual description of the
process for developers and analysts.
Intelligent Systems Laboratory

27. Next steps
● Empirical validation of the approach.
– compare the performance and experience of software
engineering project teams
• some teams using conventional approaches and others using
the pragmatically supported approach.
– incorporate the ideas into an existing production-level
software organization (assuming they already perform
sufficient metrics on their process)
– perform some linguistic analyses of various natural
language used or produced in software development,
especially in distributed communities where a large
portion of the interaction takes place through electronic
means.
Intelligent Systems Laboratory

28. Further steps
● Study the subsequent process of actually
correcting the bugs identified during the process,
with duties and responsibilities assigned to
appropriate roles
– Involves a superset of the same roles involved in
problem tracking.
● Compare different organizations’ processes to find
common elements, and (likely) missing elements;
this is a natural next step.
● Identify where processes actually conflict with each
other.
Intelligent Systems Laboratory