1. disaggregated hot and cold water sensing with minimal calibration
semi-supervised training for infrastructure mediated sensing
Eric C. Larson
UbiComp Lab
electrical
engineering
computer
science and engineering
University of Washington

2. how can indirect sensing and machine
learning be used to reduce our
environmental footprint?

3. 3

4. lake mead 1983
4

5. lake mead 2011
5

6. we are using water faster
than it is being replenished
Pacific Institute for Studies in Development, Environment, and Security, 2011 6

7. we are using water faster
than it is being replenished
Pacific Institute for Studies in Development, Environment, and Security, 2011 6
image: weiku.com

8. $2,994.83
7

9. 8

10. water usage is vastly
misunderstood

11. eco-feedback
10

12. eco-feedback
11 image: gardena, inc

13. eco-feedback
11 image: gardena, inc
image: showersmart
image: iSave

14. eco-feedback
Geographic Comparisons
Dashboards
Metaphorical Unit Designs
Recommendations
12

15. eco-feedback
13

16. eco-feedback
14

17. eco-feedback
15

18. eco-feedback
16

19. eco-feedback
17 video: courtesy Jon Froehlich

20. what are the potential
water savings?

21. eco-feedback in electricity
19

22. 0%
5%
10%
15%
20%
1 2 3 4 5 Untitled 1
20%
12%
9.2%8.4%
6.8%
3.8%
Enhanced
Billing
Web
Based
Daily
Feedback
Realtime
Feedback
Appliance Level
+ Personalized
Feedback
Annual%Savings
Based on 36 studies between 1995-2010. Summarized by Ehrhardt-Matinez et al.
>20% reduction: Gardner et al. (2008) and Laitner et al. (2009)
Appliance
Level
eco-feedback in electricity
aggregate
disaggregated
Courtesy: Sidhant Gupta
20

23. how can we sense
water usage?

24. 22
15
Turbine
Insert
Thermistor
Flow
image: LBNL

25. 22
15
Turbine
Insert
Thermistor
Flow
image: LBNL

26. meters
flow rate fixture flow
inline water
23

27. meters
flow rate fixture flow
inline water
water
pressure
pressure
sensor
machine
learning
estimated

28. • central sensing point
• easy to install
• low cost
• can observe every fixture
HydroSense
25

29. • central sensing point
• easy to install
• low cost
• can observe every fixture
HydroSense
25
40#
50#
60#
70#
80#
Cold Line Pressure
(Hose Spigot)
0 94.5
time (s)
psi
open close

30. HydroSense
26

31. kitchen sink
upstairs toilet
template matching
unknown event
27
downstairs toilet

32. kitchen sink
upstairs toilet
template matching
unknown event
27
downstairs toilet

33. feasibility study
• 10 homes
• staged calibration
• ~98% accuracy
Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water
activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244.
Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged
experiments. Pervasive and Mobile Computing, (2010).
28

34. 70
50
30
pressure(psi)
70
50
30
pressure(psi)
initial study: staged events
kitchen sink kitchen sink
29

35. 70
50
30
pressure(psi)
70
50
30
pressure(psi)
natural water use
30

36. how well does HydroSense work
in a natural setting?

37. longitudinal evaluation
32

38. 33

39. 34

40. 35

41. 36

42. totals
days
33
33
30
27
33
156
events
2374
3075
4754
2499
2578
14,960
events/day
71.9
93.2
158.5
92.6
78.1
95.9
compound
22.2%
21.8%
16.6%
32%
21.3%
21.9%
data collection
Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage
Events in the Home. Pervasive Computing, Springer (2011), 50–69.
most comprehensive labeled dataset
of hot and cold water ever collected
37

43. bathroom sink
natural water usage
70
50
30
pressure(psi)
70
50
30
pressure(psi)
8AM
2 minutes
38

44. kitchen sink kitchen sink
toilet
bathroom sink
natural water usage
70
50
30
pressure(psi)
70
50
30
pressure(psi)
8AM
2 minutes
38

45. kitchen sink kitchen sink
toilet
bathroom sink
natural water usage
70
50
30
pressure(psi)
70
50
30
pressure(psi)
template matching: 98% 74%
10 fold cross validation
35%
8AM
2 minutes
minimal
38

46. need a more realistic approach
templates feature vectors
matching parametric model
minimize training
39

47. need a more realistic approach
templates feature vectors
matching parametric model
minimize training
39

48. 10 psi
7.32 psi
15 Hz
200 ms
feature vectors: dense features
43

49. 10 psi
7.32 psi
15 Hz
200 ms
feature vectors: dense features
xd
1
44

50. feature vectors: sparse features
unlabeledinstances
xd
1
45

51. feature vectors: sparse features
codebook
x1
x56
x132
x240
0
0...
0
0...
0
0...
0
0...
xd
1
46

52. feature vectors: sparse features
codebook
xd
1
xs
1

53. 70
50
30
pressure(psi)
70
50
30
pressure(psi)
xd
1
xs
1 xs
2
xd
2 xd
3
xs
3
xd
4
xs
4
xd
5
xs
5
xd
6
xs
6 xs
7
xd
7
feature vectors: sequence

54. templates
matching parametric model
feature vectors
Traditional Methods
KNN
SVM
Decision Trees
CRF
DBN (i.e., HMM)
Ensemble Methods
KNN-subspace
Bagged Trees
Stacking Methods
TB+CRF
SVM+CRF
TB+DBN
51

55. minimal training set
0
10
20
30
40
50
60
70
80
90
100
NN KNN TM HMM KNN-sub SVM HMM-TM CRF TB SVM+CRF TB+CRF
valvelevelaccuracy(%)
error bars=std err.
1-2 labels per valve
52
densefeatures
sparsefeatures

56. 55%
valve
fixture
64%
category
78%
supervised results summary
53

57. fixture level confusions
KitchenSink
MasterBathroomSink
SecondaryBathroomSink
SecondaryBathroomToilet
MasterBathroomToilet
dishwasher laundry
MasterBathroomBath/Shower
MasterBathroomBath/Shower
WashingMachine
Dishwasher
54

58. accuracy
trusted
not trusted
how accurate should the system be?
how can we be sure the user trusts the system?
highly critical
noticeable
Lim, B. and Dey, A. Investigating intelligibility for uncertain context-aware applications. Proceedings of the 13th international
conference on ubiquitous computing, (2011), 415.
~80%
~99%
55

59. accuracy
trusted
not trusted
category 78%
~80%
~99%
85%
90%
80%
goalsminimal
55%valve
64%fixture
laundry
dishwasher
noticeable
10% 8% 0%
56

60. time
psi
fill
fill
fill
fill
cycle
cycle
cycle
57

61. 0 0.5 1 1.5 2 2.5 3 3.5 4
46
48
50
52
54
56
58
60
hours
psi finding the dishwasher
0.5 1 1.5 2 2.5 3 3.5 4
hours
template
59
dishwasher

62. 0 0.5 1 1.5 2 2.5 3 3.5 4
46
48
50
52
54
56
58
60
hours
psi finding the dishwasher
0.5 1 1.5 2 2.5 3 3.5 4
hours
template
X
n
tn
59
dishwasher

63. 0 0.5 1 1.5 2 2.5 3 3.5 4
46
48
50
52
54
56
58
60
hours
psi finding the dishwasher
0.5 1 1.5 2 2.5 3 3.5 4
hours
template
X
n
tn
X
n
pn
59
dishwasher

64. 0 0.5 1 1.5 2 2.5 3 3.5 4
46
48
50
52
54
56
58
60
hours
psi finding the dishwasher
0 0.5 1 1.5 2 2.5 3 3.5 4
hours
template
pressure difference
time difference
X
n
tn
X
n
pn
59
dishwasher

65. 0 0.5 1 1.5 2 2.5 3 3.5 4
46
48
50
52
54
56
58
60
hours
psi finding the dishwasher
0 0.5 1 1.5 2 2.5 3 3.5
46
48
50
52
54
56
58
60
hours
psi
template
pressure difference
time difference
X
n
tn
X
n
pn
59
dishwasher

66. 0 0.5 1 1.5 2 2.5 3 3.5 4
46
48
50
52
54
56
58
60
hours
psi finding the dishwasher
pressure difference
time difference
X
n
tn
X
n
pn
59
dishwasher

67. 0 0.5 1 1.5 2 2.5 3 3.5 4
46
48
50
52
54
56
58
60
hours
psi finding the dishwasher
pressure difference
time difference
X
n
tn
X
n
pn
X
n
tcyclelaundry machine
59
laundry
dishwasher

68. laundry
dishwasher
true
positives
false alarms precision
73% 15 90%
75% 16 89%
43% 58 85%
39% 119 82%
75% of all showering
category
60

69. accuracy
trusted
not trusted
category 78%
~80%
~99%
85%
90%
80%
goalsminimal
55%valve
64%fixture
laundry
dishwasher
noticeable
10% 8% 0%
61

70. leveraging unlabeled data
labeled unlabeled
classifier classifier
feature set 2
high confidence high confidence
agree?
feature set 1
self labeled
multi-view classification
62

71. training 48%
49%
labeled unlabeled
multi-view classification
self labeled
TB
SVM+CRF
55%
53%
feature split: hot sensor vs. cold sensor
88%
90%TB
SVM+CRF
99%
dense features
sparse features
63
training
training
training

72. 0 5 10 15 20
46
48
50
52
54
56
58
60
time of day (hours)
psi
kitchen sink, hot master bathroom toilet
multi-view classification
64

73. semi-supervised learning
rule based classifier
0 0.5 1 1.5 2
46
48
50
52
54
56
58
60
hourspsi
self labels
expert
knowledge
virtual
evidence
65

74. A B
C Dvirtual
evidence
1
semi-supervised learning
virtual evidence
kitchen sink kitchen sink
P=1
bath sink bath sink
P=1
toilet toilet
P=1
P=0.01
otherwise
66
arg max
A,B
P(A)P(B|A)P(C = c|A)P(D = d|B)P(ve|A, B)

75. semi-supervised learning
virtual evidence
67

76. semi-supervised learning
virtual evidence
68

77. semi-supervised learning
virtual evidence
69

78. semi-supervised learning
virtual evidence
70

79. semi-supervised learning
virtual evidence
71

80. semi-supervised learning
results
0
10
20
30
40
50
60
70
80
90
100
TM HMM SVM TB SVM+CRF TB+CRF HMM-VE-Co
valvelevelaccuracy(%)
10 fold cross validation
72

81. room shower.
57
6.6
0.6
1.1
4.2
56
1.7
0.8
6.4
4.6
31
0.8
82
1.5
9.1
4.3
14
1.5
2.6
4.2
2.0
6.1
1.1
8.8
1.5
28
2.0
81
10
9.1
2.0
15
1.0
3.0
22
6.1
6.7
8.5
2.0
6.5
0.6
33
3.3
1.0
0.8
0.5
0.7
17
5.0
39
1.0
1.5
0.6
0.8
24
4.4
0.7
2.5
6.8
1.1
11
6.5
65
2.4
5.4
0.8
3.1
7.9
2.5
6.0
0.9
7.8
5.7
14
3.0
68
1.0
1.6
3.5
1.5
3.9
1.6
9.5
4.8
4.3
1.4
83
0.8
1.4
17
1.9
3.1
3.5
1.9
91
2.5
0.8
11
17
6.4
42
2.4
60
4.3
1.0
3.4
4.8
7.9
0.6
3.7
7.5
1.6
86
1.6
8.4
1.5
1.7
0.7
4.5
1.7
5.7
1.0
6.8
0.6
3.0
3.1
1.9
85
6.4
2.2
6.1
0.7
66
3.2
1.1
38
14
9.1
43
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
r close,1
open,2
k close,3
open,4
r close,5
open,6
k close,7
open,8
t close,9
open,10
close,11
open,12
close,13
open,14
open,15
close,16
open,17
dishwasher
laundry
dishwasher
laundry
73

82. room shower.
57
6.6
0.6
1.1
4.2
56
1.7
0.8
6.4
4.6
31
0.8
82
1.5
9.1
4.3
14
1.5
2.6
4.2
2.0
6.1
1.1
8.8
1.5
28
2.0
81
10
9.1
2.0
15
1.0
3.0
22
6.1
6.7
8.5
2.0
6.5
0.6
33
3.3
1.0
0.8
0.5
0.7
17
5.0
39
1.0
1.5
0.6
0.8
24
4.4
0.7
2.5
6.8
1.1
11
6.5
65
2.4
5.4
0.8
3.1
7.9
2.5
6.0
0.9
7.8
5.7
14
3.0
68
1.0
1.6
3.5
1.5
3.9
1.6
9.5
4.8
4.3
1.4
83
0.8
1.4
17
1.9
3.1
3.5
1.9
91
2.5
0.8
11
17
6.4
42
2.4
60
4.3
1.0
3.4
4.8
7.9
0.6
3.7
7.5
1.6
86
1.6
8.4
1.5
1.7
0.7
4.5
1.7
5.7
1.0
6.8
0.6
3.0
3.1
1.9
85
6.4
2.2
6.1
0.7
66
3.2
1.1
38
14
9.1
43
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
r close,1
open,2
k close,3
open,4
r close,5
open,6
k close,7
open,8
t close,9
open,10
close,11
open,12
close,13
open,14
open,15
close,16
open,17
DW
Shower
Shower
CW
dishwasher
laundry
dishwasher
laundry
73

83. semi-supervised learning
leveraging the homeowner
which labels are needed most?
can we leverage multi-view models?
74

84. semi-supervised learning
leveraging the homeowner
can we leverage multi-view models?
• select low confidence examples
• ask homeowner for label
75

85. semi-supervised learning
leveraging the homeowner
can we leverage multi-view models?
• select low confidence examples
• ask homeowner for label
AT&T LTEAT&T LTE 5:23 PM
did you just use water?
YesNo
75

86. semi-supervised learning
leveraging the homeowner
can we leverage multi-view models?
• select low confidence examples
• ask homeowner for label
AT&T LTEAT&T LTE 5:23 PM
did you just use water?
YesNo
75
AT&T LTEAT&T LTE 5:23 PM
Fixture Selection
Do Not Disturb
Select Notification Times
OFF
Master Toilet
Recently Used:
Half Bath Toilet
Dishwasher
Master Sink Select Temp.
Kitchen Sink Select Temp.
Master Shower Select Temp.
More
Half Bath Sink Select Temp.

87. simulating labels from homeowner
AT&T LTEAT&T LTE 5:23 PM
did you just use water?
YesNo
• ask for two labels every other day
• one morning and one evening
• only from 8AM-9PM
• randomly ask for previous event
76
AT&T LTEAT&T LTE 5:23 PM
Fixture Selection
Do Not Disturb
Select Notification Times
OFF
Master Toilet
Recently Used:
Half Bath Toilet
Dishwasher
Master Sink Select Temp.
Kitchen Sink Select Temp.
Master Shower Select Temp.
More
Half Bath Sink Select Temp.

88. 10 15 20 25 30 35 40 45
0.65
0.7
0.75
0.8
0.85
Co−Labeling in H1
Number of Labels
ValveLevelAccuracyofCoLabel−HMM
Co−Labeling
Random Labeling
co-labeling
random labeling
iteration 1
iteration 3
iteration 5
iteration 10
simulating labels from homeowner
co-labeling for H1minimaltrainingset totals
days
33
33
30
27
33
156
events
2374
3075
4754
2499
2578
14,960
events/day
71.9
93.2
158.5
92.6
78.1
95.9
compound
22.2%
21.8%
16.6%
32%
21.3%
21.9%
77

89. totals
totals
0 1 2 3 4 5 6 7 8 9 10 11 12 13
60%
70%
80%
90%
100%
valve fixture category
error bars=std err.
co-label iteration
accuracy
78

90. usion is the secondary bathroom shower for the master bathroom shower.
-374
8.5
1.5
-3
6.1
72
1.1
0.8
5.0
0.7
1.5
0.6
-7
18
92
0.6
5.4
2.6
4.9
3.4
2.3
3.5
0.9
5.1
-7
18
0.9
92
1.4
4.9
0.7
4.4
1.4
1.4
7.3
1.0
0.7
3.2
-753
2.1 -1
4.2
67
0.6
2.4
0.8
31
2.5
5.2
-5
1.2
5.0
0.6
18
4.2
88
1.8
7.5
1.5
1.6
5.7
1.0
-5
1.2
0.5
5.3
4.6
10
1.8
89
1.7
1.7
1.4
0.7
3.2
-2
2.9
0.6
81
12
-2
0.8
94
0.6
8.1
-1
6.7
68
-2
2.8
42
-1
1.9
2.9
0.5
25
0.8
95
0.7 -1
2.2
3.1
10
1.3
96
6.6 -290
-3
1.1
0.7
59
18 -3
0.9
0.9
15
60
er close,1
open,2
k close,3
open,4
er close,5
open,6
k close,7
open,8
et close,9
open,10
r close,11
open,12
k close,13
open,14
open,15
e close,16
open,17
dishwasher
laundry
dishwasher
laundry
79

91. implications for homeowner
week one
• homeowner installs system
• 1-2 examples per fixture
• show sparse classes:
dishwasher, shower, laundry
80

92. implications for homeowner
week one
• homeowner installs system
• 1-2 examples per fixture
• show sparse classes:
dishwasher, shower, laundry
week two
• 2-4 labels, every 2 days
• fixture category: 85%
80

93. implications for homeowner
week one
• homeowner installs system
• 1-2 examples per fixture
• show sparse classes:
dishwasher, shower, laundry
week two
• 2-4 labels, every 2 days
• fixture category: 85%
week three
• 9-12 more examples
• fixture: 82%
80

94. implications for homeowner
week one
• homeowner installs system
• 1-2 examples per fixture
• show sparse classes:
dishwasher, shower, laundry
week two
• 2-4 labels, every 2 days
• fixture category: 85%
week three
• 9-12 more examples
• fixture: 82%
end of week three
• fixture: 87%
• valve: 80%
80

95. summary contributions
• comprehensive disaggregated dataset
• multi-view classification
• expert knowledge
• compressed sensing
• framework for virtual evidence in IMS
• co-labeling with multi-view
• idea: inception to industry ready
81

96. how can indirect sensing and machine
learning be used to reduce our
environmental footprint?

97. disaggregated hot and cold water sensing with minimal calibration
semi-supervised training for infrastructure mediated sensing
UbiComp Lab
electrical
engineering
computer
science and engineering
University of Washington
eclarson.com
eclarson@uw.edu
@ec_larson
Eric C. Larson

98. disaggregated hot and cold water sensing with minimal calibration
semi-supervised training for infrastructure mediated sensing
UbiComp Lab
electrical
engineering
computer
science and engineering
University of Washington
eclarson.com
eclarson@uw.edu
@ec_larson
Eric C. Larson