Recommandé
Recommandé
Contenu connexe
Dernier
Dernier (20)
En vedette
En vedette (20)
Empowering End Users in Debugging Trigger-Action Rules
- 1. EMPOWERING END USERS IN DEBUGGING TRIGGER-ACTION RULES FULVIO CORNO, LUIGI DE RUSSIS, ALBERTO MONGE ROFFARELLO e-Lite https://elite.polito.it
- 2. The Internet of Things is a recognized paradigm that already helps society in many different ways, through applications ranging in scope from the individual to the planetary Vint Cerf and Max Senges, Google Research
- 3. A COMPLEX NETWORK OF PHYSICAL AND VIRTUAL OBJECTS
- 4. END USERS CAN PROGRAM THE IOT THROUGH TRIGGER-ACTION RULES
- 5. IF I PUBLISH A POST ON FACEBOOK THEN SHARE IT ON TWITTER IF THE SURVEILLANCE CAMERA RECOGNIZE ME THEN TURN ON THE SMART THERMOSTAT
- 6. TRIGGER-ACTION PROGRAMMING IS A DIFFICULT TASK, BUT CURRENT TOOLS DO NOT PROVIDE ANY HELP FOR DEBUGGING RULES
- 7. Enabling end users to debug their trigger-action rules by properly warning them when they are defining any troublesome or potentially dangerous behavior. Goal
- 9. IF I post a photo on Facebook THEN save it on my iOS library IF I add a new photo on my iOS THEN post it on Instagram IF I post a photo on Instagram THEN post it on Facebook Loops Redundancies Inconsistencies
- 10. IF I play a new song on my Alexa THEN post a tweet on Twitter IF I play a new song on my Alexa THEN save the track on Spotify IF I save a track on Spotify THEN post a tweet on Twitter Loops Redundancies Inconsistencies
- 11. IF I exit home THEN turn off the Philips Hue lamp IF I exit home THEN set the Nest to Away mode IF the Nest is set to Away Mode THEN turn on the Philips Hue lamp Loops Redundancies Inconsistencies
- 12. STRATEGY 1: Highlighting rule conflicts STRATEGY 2: Simulating rule conflicts EUDebug
- 13. STRATEGY 2: Simulating rule conflicts STRATEGY 1: Identifying rule conflicts EUDebug
- 14. STRATEGY 1: Identifying rule conflicts STRATEGY 2: Simulating rule conflicts EUDebug
- 17. IF I exit home (T1) THEN turn off the kitchen Philips Hue lamp (A1) IF I exit home (T2) THEN set the Nest to Away mode (A2) IF the Nest is set to Away Mode (T3) THEN turn on the kitchen Philips Hue lamp (A3) R1 R3 R2 Petri Nets EUPont
- 18. IF I exit home (T1) THEN turn off the kitchen Philips Hue lamp (A1) IF I exit home (T2) THEN set the Nest to Away mode (A2) IF the Nest is set to Away Mode (T3) THEN turn on the kitchen Philips Hue lamp (A3) R1 R3 R2 Petri Nets EUPont
- 19. T1 = I exit home A1 = turn off the kitchen Hue lamp T1 A1 R1 Petri Nets EUPont
- 20. IF I exit home (T1) THEN turn off the kitchen Philips Hue lamp (A1) IF I exit home (T2) THEN set the Nest to Away mode (A2) IF the Nest is set to Away Mode (T3) THEN turn on the kitchen Philips Hue lamp (A3) R1 R3 R2 Petri Nets EUPont
- 21. T2 = I exit home A2 = set the Nest to Away mode T1 A1 R1 T2 A2 R2 Petri Nets EUPont
- 22. T1 = I exit home T2 = I exit home T1 A1 R1 T2 A2 R2 Petri Nets EUPont
- 23. T1|2 = I exit home T1 A1 R1 T2 A2 R2 T1|2 copy Petri Nets EUPont
- 24. IF I exit home (T1) THEN turn off the kitchen Philips Hue lamp (A1) IF I exit home (T2) THEN set the Nest to Away mode (A2) IF the Nest is set to Away Mode (T3) THEN turn on the kitchen Philips Hue lamp (A3) R1 R3 R2 Petri Nets EUPont
- 25. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 T3 = the Nest is set to Away mode A3 = turn on the kitchen Hue lamp Petri Nets EUPont
- 26. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 A2 = set the Nest to Away T3 = the Nest is set to Away Petri Nets EUPont
- 27. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate A2 = set the Nest to Away T3 = the Nest is set to Away Petri Nets EUPont
- 29. EUPont: End User Programming ontology http://elite.polito.it/ontologies/eupont.owl [Corno, De Russis, Monge Roffarello, A High-Level Semantic Approach to End-User Development in the Internet of Things, IJHCS, 2018] Petri Nets EUPont
- 30. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off A1 = turn off the kitchen Hue lamp A3 = turn on the kitchen Hue lamp Petri Nets EUPont
- 31. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off T1|2 = I exit home Petri Nets EUPont
- 32. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off T1 = I exit home T2 = I exit home Petri Nets EUPont
- 33. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off T2 = I exit home A1 = turn off the kitchen Hue lamp Petri Nets EUPont
- 34. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off A1 = turn off the kitchen Hue lamp A2 = set the Nest to Away mode Petri Nets EUPont
- 35. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off A1 = turn off the kitchen Hue lamp T3 = the Nest is set to Away Mode Petri Nets EUPont
- 36. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off A1 = turn off the kitchen Hue lamp A3 = turn on the kitchen Hue lamp Petri Nets EUPont
- 37. T1|2 copy T1 A1 R1 T2 A2 R2 T3 A3 R3 activate EUPont: Lights on EUPont: Lights off A1 = turn off the kitchen Hue lamp A3 = turn on the kitchen Hue lamp Petri Nets EUPont
- 38. Exploratory Study Participants MethodologyResearch Questions
- 39. 9 males 6 females University students 20.34 mean age No experience in computer science and programming Participants Research Questions Methodology
- 40. UNDERSTANDABILITY Do end users understand problems and why the defined rules generate them? Participants Research Questions Methodology
- 41. Is highlighting the detected problems sufficient? Do users need a step-by-step simulation of the involved rules? IDENTIFICATION Participants Research Questions Methodology
- 42. 12 TRIGGER-ACTION RULES Participants MethodologyResearch Questions
- 43. 2 Inconsistencies 2 Redundancies 1 Loop IC1 12 TRIGGER-ACTION RULES IC2 RD1 RD2 LP Participants MethodologyResearch Questions
- 44. 2 Inconsistencies 2 Redundancies 1 Loop IC1 12 TRIGGER-ACTION RULES IC2 RD1 RD2 LP DIRECT PROBLEMS Participants MethodologyResearch Questions
- 45. 2 Inconsistencies 2 Redundancies 1 Loop IC1 12 TRIGGER-ACTION RULES IC2 RD1 RD2 LP INDIRECT PROBLEMS Participants MethodologyResearch Questions
- 46. 12 TRIGGER-ACTION RULES Participants MethodologyResearch Questions 2 Inconsistencies 2 Redundancies 1 Loop IC1 IC2 RD1 RD2 LP
- 48. End users perceive problems in trigger-action rules differently: loops and inconsistencies are dangerous, redundancies can be even acceptable Perceiving Problems Understanding Problems Highlighting or Explaining?
- 50. 97.67 % discarded inconsistencies 80 % discarded loops Perceiving Problems Understanding Problems Highlighting or Explaining?
- 51. 46.67 % saved redundancies 97.67 % discarded inconsistencies 80 % discarded loops Perceiving Problems Understanding Problems Highlighting or Explaining?
- 52. The important thing is that the lamp is turned on, I do not care its color! RD1 - P13 Perceiving Problems Understanding Problems Highlighting or Explaining?
- 53. Some problems are difficult to understand: loops and indirect problems are often misinterpreted Perceiving Problems Understanding Problems Highlighting or Explaining?
- 54. 16,67 % misinterpretations when facing an inconsistency 13,34 % misinterpretations when facing a redundancy 40 % misinterpretations when facing a loop Perceiving Problems Understanding Problems Highlighting or Explaining?
- 55. 26.67 % misinterpretations when facing an indirect problem 10 % misinterpretations when facing a direct problem Perceiving Problems Understanding Problems Highlighting or Explaining?
- 56. I am sure that this problem will never occur with the rules I have defined! Moreover, such rules are useful, because the photo will be saved in 3 places at the same time. LP - P13 Perceiving Problems Understanding Problems Highlighting or Explaining?
- 57. Highlighting the detected problem is often not sufficient: a step-by-step simulation of the involved rules helps users understand difficult problems Perceiving Problems Understanding Problems Highlighting or Explaining?
- 58. 77,81 % of correct interpretations when a problem is highlighted 83,78 % correct interpretations when a problem is simulated Perceiving Problems Understanding Problems Highlighting or Explaining?
- 59. 77,81 % of correct interpretations when a problem is highlighted 83,78 % correct interpretations when a problem is simulated Perceiving Problems Understanding Problems Highlighting or Explaining? 50 % highlighted 71,43 % simulated FOR LOOPS
- 60. The step-by-step simulation helped me in understanding the loop because I could better see the evolution of the rules. It visually told me what happened! LP - P6 Perceiving Problems Understanding Problems Highlighting or Explaining?
- 61. SUMMARY We designed and implemented EUDdebug, a tool to enable end users to debug trigger-action rules for IoT personalization. We explored how users without programming skills would debug their rules with the help of EUDdebug, and we showed that: 1. Users perceive problems in trigger-action rules differently 2. Some problems are more complex than others 3. Step-by-step simulations help understanding problems
- 62. e–Lite https://elite.polito.it EMPOWERING END USERS IN DEBUGGING TRIGGER-ACTION RULES FULVIO CORNO, LUIGI DE RUSSIS, ALBERTO MONGE ROFFARELLO
Notes de l'éditeur
- Thank you for the introduction. Today, on behalf of my co authors I will present you our work on end-user debugging of trigger-action rules.
- Our work stems from the rapid and continous growth of the Internet of Things. The IoT, nowadays, a well recognized paradigm that already helps us in many different contexts
- Even today people interact daily with a growing number of Internet-enabled devices and web applications in many different contexts, ranging from smart homes to smart cities. The result is a complex network of smart objects, either physical or virtual, that are able to interact and communicate with each other, with humans, and with the environment
- In this context, end users can nowadays exploit many different tools to directly program the joint behavior of their devices and web applications, on the basis of their personal needs. Perhaps the most common tool in this filed is IF THIS THEN THAT.
- Typically, the adopted programming paradigm is the trigger-action programming. So for example, users can define trigger-action rules such as “if the surveillance camera recognize me, then turn on the smart thermostat”, or “if I publish a post on Facebook, then share it on Twitter”.
- Despite apparent simplicity, however, the definition of trigger-action rules is not an easy task, especially for users who do not have any programming experience, and so they are likely to make mistakes. Errors in this context, however, can lead to unpredictable and even dangerous behaviors, such as a door that is unexpectedly unlocked. Therefore, one of the most important and urgent challenge in this field is the need to avoid possible conflicts and to assess the rule correctness. Unfortunately, contemporary tools for trigger-action programming do not provide users with any help for debugging their rules.
- This motivated us in exploring novel methodologies for enablig end users to debug their trigger action rules, by properly warning them when they are defining any troublesome or potentially dangerous behavior. 2:07
- We started by trying to understand which problems are more common in our domain, and we settled on 3 main conflict categories: loops, redundancies, and inconsistencies.
- So let’s start with infinite loops. Loops arise when a set of trigger-action rules are continuously activated without reaching a stable state. The 3 rules reported in the slide are an example: they are repeated indefinitely, and they continuously post a photo in 3 different places.
- Then there are redundancies, that arise when two or more rules are activated at the same time, and they reproduce similar functionality. Here, for example, a very similar tweet is posted at the same time, because the first 2 rules share the same trigger, while the second rule implicitly activate the third rule.
- And finally there are inconsistencies. They arise when rules that are activated at the same time try to execute contradictory actions. In the example, when I’m leaving home, the lamp is turned off, and the thermostat is put to away mode. Unfortunately, the third rule is also activated, and the lamp is also turned on. So, we are sending 2 contradictory commands to our lamp at the same time.
- Afert idetifying the problems to be detected, we implemented a tool, named EUDebug, that enables end users to debug their trigger-action rules at composition time according to 2 strategies. With the first one, the tool analyzes all the rules of the user, and it highlights problems that the rule that is being defined may generate.
- If needed, the user can further inspect the problem with a step-by-step simulation of the problematic rules. And this is the second debugging strategy.
- The user interface of our tool is modeled after IF THIS THEN THAT, and now I will show you a brief demo of the tool, by reproducing the inconsistency seen before. 4:25
- So, how the presented tool works? These are the 2 main ingredients that we exploited for implementing our approach: Petri Networks and Semantic Web.
- Roughly speaking, Petri nets are directed graphs, in which directed arcs connect places and transitions. Places may hold tokens, which are used to study the dynamic behavior of the net. We chose petri nets since they can naturally describe the rules as well as their non-deterministic and concurrent environment, and they are also useful to simulate rules step-by-step: by firing a transition at a time, tokens move in the net by giving the idea of a possible execution flow. 6:13
- In particular, we defined a novel petri net formalism to work with trigger-action rules, by defining different types of places and transitions. To exemplify it, I will show you now how the inconsistency example of the demo is modeled with a Petri net. These are the 3 rules that are involved R1, R2, and R3, where R3 is the rule that I was trying to define in the demo
- Let’s start with the first one
- The trigger and the action are modeled as places, while the trasition that connect T1 to A1 is the rule R1.
- Now, let’s add the second rule, R2
- Again, T2 and A2 become 2 places, connected through the rule R2
- Now, since the two triggers, T1 and T2 are equals
- The associated places are connected to a parent place T12 through a copy transition.
- Let’s add now the last rule R3
- Same formalism of before
- However, we must notice that the action of the second rule (set the nest to away mode) implicitly activate the trigger of the third rule (the nest is set to away mode).
- So the formalism provide an activate transition to model such a behavior
- And this is the final net.
- And here comes the second ingredient of our approach, the semantic web, and in particular, the EUPont ontology. EUPont is an ontological high-level representation of trigger-action programming that describes smart devices and online services on the basis of their final capabilities. We used such a model to enrich petri nets, and in particular places, with information about the functionality of triggers and actions, with the aim of detecting conflicts among rules. 8:08
- So, we add to each places of the net «a color», that is simply a label. Therefore, colors represent the functionality of the related trigger or action. Here in the figure, for example, the color of A1 is «Lights off», while the color of A3 is «Lights on»
- The tool uses the generated model to analyze the rules and to find conflicts, in this case the inconsistency seen in the demo. A token is put in the trigger «I exit home», that is the root node of the net, and we can see what happens when the net is executed 9:00
- Trasitions fire, and tokes move in the net
- In the end, when the net execution ends, I can see that there are 2 tokens, one in A1 and one in A3, that, on the basis of the color, represent the execution of 2 contradictory actions in terms of final functionality on the same lamp.
- 9:40
- We ran an exploratory study with 15 participants to evaluate whether EUDebug helps them to debug their trigger-action rules.
- We were guided by 2 main research questions, one related to the understandability of the detected problems
- And the other one related to the strategies we adopted for assisting users. So we investigated whether highlighting a problem is sufficient, or whether a user need a step-by-step simulation to better understand the problem.
- Each participant used our tool to compose 12 predefined trigger-action rules.
- That generated 5 different problems, 2 inconsistencies, 2 redundancies, and 1 loop.
- Problems were of different nature. 2 problems were direct, in the sense that they involved 2 rules, only, with the same trigger and 2 problematic actions.
- The other 3 problems were instead indirect. because they were caused by the implicit activation of a chain of rules.
- When the definition of a rule generated a problem, participants could use the step-by-step simulation, and, at the end, they had to decide whether to save or not the defined rule. Moreover, they had also to provide an interpretation of the problem and their choices.
- We summarized the results into 3 main findings 11:03
- The first one is that different problems are perceived differently by end users. Participants, for example, considered loops and inconsistencies as dangerous, while they accepted redundancies, at least in some cases.
- In particular, participants discarded the rule that generated a loop in 80% of cases
- This percentage was even higher when the rule generated an inconsistency
- However, when faced with a rule that generated a redundancy, participants behaved differently: more than 40% of rules that generated a redudancy were consciously saved.
- In many cases, in fact, participants were interested in the final result, and they didn’t consider multiple similar actions as a problem
- The second finding of our work is related to the uderstanding of the detected problems. We found that some problems are more difficult to understand than others. Loops and in general indirect problems, for example, are often misinterpreted 12:05
- Indepentently of the choice of the participant to save or not the rule that generated the problem, we found that the loop was misinterpreted in 40% of cases. Misinterpretations were instead significantly lower for the other classes of problems.
- A consistent difference also emerged between direct and indirect problems: indirect problems were misinterpreted more often than direct problems
- This is, for example, an interpretation of a participant when facing a loop that continuosly posted the same photo in 3 different plaes. And this interpretation was actually very common among participants. They often understood that the rules involved in the loop implicitly activeted each other, but they did’t recognize that the last rule of the loop re-activated the first one. In other words, they didn’t understand the infinite nature of the loop. Without understanding this, the participant of the slide was even happy, because his photos were saved in 3 places at the same time, automatically. 12:45
- The last finding of our work is that highlighting a problem is often not sufficient for end users to understand problems, in particular the more complex ones. Instead, a step-by-step simulation of the involved rules is helpful for understanding problems.
- Indeed, partcipants provided better interpretations when they used the step-by-step simulation.
- Especially for complex problems like loops.
- The step-by-step simulation, as acknowledged by some participants, helps user understand the problem, because it provides a visual representation of a possible run-time evoultion of the involved rules.
- To conclude, we designed and implemented EUDebug, with the aim of empowering end users in debugging trigger-action rules for IoT personalziation. In an exploratory study we then explored how users without programming skills would debug their rules with the help of our tool, and we found 1 that users perceive problems differently 2 that some problems are more complex than others 3 and that step-by-step simulations help users understand problems
- This concludes my presentation, thank you for your attention, and I’m open to questions