Do disaggregated electricity bills really help people to save energy?

Jack Kelly
jack.kelly@imperial.ac.uk

(Swipe or press right-arrow on your keyboard to change slides)

Outline

  1. Introduction to energy disaggregation
  2. Introduction to systematic reviews
  3. Methodology
  4. The studies
  5. Findings
  6. Gaps in our knowledge: Suggestions for future research
  7. Conclusions

Energy Disaggregation

Aggregate Energy Bill

Itemised Energy Bill

The many names of 'energy disaggregation'

  1. NILM: Non-Intrusive Load Monitoring
  2. NALM: Non-intrusive Appliance Load Monitoring
  3. NIALM: Non-Intrusive Appliance Load Monitoring

The Energy Disaggregation community

Graph from Oliver Parson, Overview of the NILM field, blog post, 2015

Bidgely raised $16.6 million in 2015

Why bother with disaggregation?

GB Smart Meter Roll-out

  • All homes to have a smart meter by 2020.
  • These reports whole-house power demand every 10 seconds to home area network (HAN).
  • DECC's business case assumes that smart meters will drive savings of £4.6 billion due to reduced energy consumption (across both electricity and gas).

Use-cases for energy disaggregation

  • Many use-cases
  • This talk is about one use-case:
    • Can disaggregated energy feedback help people to reduce energy consumption more effectively than aggregate energy data alone?

How might disaggregated data reduce energy demand?

1977: the Twin Rivers program

Socolow, The twin rivers program on energy conservation in housing: Highlights and conclusions,
Energy and Buildings, 1977

2:1 range in energy consumption between identical houses

Socolow, The twin rivers program on energy conservation in housing: Highlights and conclusions,
Energy and Buildings, 1977

"Information deficit"
and
"rational resource managers"

People self-report that they want disaggregated energy data

But do people save energy when given disaggregated data?

Why reduce energy consumption?

2015 Paris agreement on Climate Change

"[Hold] the increase in the global average [surface] temperature to well below 2 °C above pre-industrial levels and to pursue efforts to limit the temperature increase to 1.5 °C above pre-industrial levels"
United Nations Framework Convention on Climate Change, COP 21, Paris Agreement, 2015-12-11

Background image from The Guardian/Francois Guillot/AFP/Getty Images

Background image from phys.org/Gregory Heath/CSIRO

Fossil-fuel emissions estimated to be compatible with 2 °C (RCP2.6)

Background image from phys.org/Gregory Heath/CSIRO

Past and future changes in CO2 and mean temperature

Clark et al., Consequences of twenty-first-century policy for multi-millennial climate and sea-level change,
Nature Climate Change, 2016

Past and future changes in global mean sea level

Clark et al., Consequences of twenty-first-century policy for multi-millennial climate and sea-level change,
Nature Climate Change, 2016

My Work

The Computer Science of disaggregation

Recurrent Neural Nets

Systematic reviews

  • Common in medicine, social sciences etc.
  • Aim to collect all papers matching a defined search criteria
  • Quantitative summary of each paper and biases
  • Quantitative synthesis of all results
  • May include a "meta-analysis"
  • Distinct from "narrative" reviews

Background image from UCSF

Methodology

Research questions

  1. Can disaggregated energy data help an already-motivated sub-group of the general population (‘energy enthusiasts’) to save energy?
  2. How much energy would the general population save if given disaggregated data?
  3. Is fine-grained disaggregation required?
  4. For the general population, does disaggregated energy feedback enable greater savings than aggregate data?

Literature search

  1. Three search engines: Google Scholar, the ACM Digital Library and IEEE Xplore
  2. Search terms:
    • ‘disaggregated AND [energy|electricity] AND feedback’
    • ‘N[I|A|IA]LM AND feedback’
  3. Searched papers' bibliographies
  4. Send draft literature review to authors for comments

The studies

12 groups of studies identified

Findings

  • Mean energy reduction = 4.5%
  • Weighted by number of participants
  • Full meta-analysis probably not possible

Biases

Opt-in bias

  • All 12 studies suffer from 'opt-in' bias
  • Subjects self-selected to some extent
  • Subjects probably more interested in energy than the average person
  • Very likely to be a strong positive bias

The Hawthorne Effect

  • 8 studies did not control for Hawthorne
  • Hawthorne effect is illustrated by Schwartz et al. 2013:
    • Randomised controlled trial
    • 6,350 participants split into 2 groups: control & treatment
    • Treatment received weekly postcard saying: "You have been selected to be part of a one-month study of how much electricity you use in your home... No action is needed on your part. We will send you a weekly reminder postcard about the study..."
    • Treatment group reduced energy consumption by 2.7%!
  • Failure to control for Hawthorne very likely to be strong positive bias

Other biases

  • 6 studies used attention-grabbing displays
  • Home-visits
  • 10 studies were short (4 months or less)
  • Cherry-picking statistical analyses and comparison periods?
  • 8 studies used sub-metered data, hence avoiding mistrust from participants
  • Publication bias?

Can disaggregated electricity feedback enable "energy enthusiasts" to save energy?

  • Very likely. For example...

Home Energy Analytics (HEA) studies

  • 1,623 users
  • Up to 44 months
  • Average reduction of 6.1%
  • Top quartile (310 "super-enthusiasts") reduced by 14.5%

2014 PG&E study

  • 1,685 users: half got IHD; half got Bidgely
  • 3 months
  • No significant reduction across all 1,685 users
  • But users who selected a time-of-use tariff saved 7.7% (142 IHD; 136 Bidgely)

How much energy would the whole population save?

  • No "perfect" correction for opt-in bias
  • Consider study in Sweden (Vassileva et al. 2012):
    • 2,000 households given access to website analysing their aggregate energy demand
    • Only 32% accessed the website. They saved 15%.
    • Those who did not access website did not reduce energy.
    • Average saving = 32% x 15% = 5%

How much energy would the whole population save?

  • Average opt-in rate = 16%
  • Average saving across population = 16% x 4.5% = 0.7%

Is "fine-grained" disaggregation necessary?

Is "fine-grained" disaggregation necessary?

Home Energy Analytics (HEA) studies

  • Average reduction of 6.1%
  • Coarse-grained disaggregation may be sufficient
  • But no studies directly compared fine-grained against coarse-grained.

Aggregate versus disaggregated feedback

  • 4 of the 12 studies directly compared disaggregated against aggregate feedback
    • 3 studies found aggregate to be more effective
    • 1 study found aggregate to be equally effective
    • 2 field trials & 2 lab experiments

The 2 field trials...

Sokoloski 2015

Sokoloski's results

Sokoloski's results

Sokoloski's results

Energy reductions:

  • IHD: 8.1% (statistically significant)
  • Disaggregation: 0.5%
  • Control: -2.5%

Sokoloski's results

Findings from surveys:

  • Follow-up survey revealed that the disag group were not significantly more likely to be willing to replace large, inefficient appliances compared to controls or IHD group.
  • Neither controls nor the disag group significantly increased their perception of control (initial survey versus follow-up).
  • IHD group did increase their perception of control.

Sokoloski's results

Findings from surveys:

  • Users viewed their devices:
    • 0.86 times per day for disag users
    • 8.16 times per day for IHD users
  • Returning devices:
    • 2 of 7 (29%) wanted to return disag device
    • 2 of 30 (7%) wanted to return IHD

PG&E 2014 trial

  • 1,685 PG&E customers
  • 1,001 from SmartRate & 584 from time-of-use users
  • additional no-contact controls
  • Half got IHD & half got Bidgely
  • 3 months
  • Users choose intervention
  • Did not tease apart consumption of IHD vs Bidgely
  • Churchwell et al., HAN Phase 3 Impact and Process Evaluation Report, technical report by Nexant, 2014

PG&E 2014 trial results

  • No significant energy reduction across all 1,001 SmartRate users or 208 EV TOU users
  • 7.7% energy reduction for time-of-use users (142 IHD & 136 Bidgely users)
  • IHD users significantly more likely to report taking actions to reduce electricity usage and to use their device to deduce power demand of individual appliances(!)

PG&E 2014 trial results

  • Most common complaint from Bidgely users was about the disag feature. Several users did not trust the disag data; or were unsure whether users should assist the algorithm by turning loads on or off; or thought categories were too few or too broad; or didn't like that they couldn't add new disag categories.
  • IHD more successful in communicating power demand right now

PG&E 2014 trial results

Frequency of viewing devices

PG&E 2014 trial results

Percentage of customers saying they saved energy

PG&E 2014 trial results

Reported actions taken in response to feedback

Bidgely have redesigned their website since these studies

Suggestions for future studies

  • Compare aggregate versus disagg (both on an IHD)
  • Compare 2 groups:
    1. Aggregate on an IHD
    2. Aggregate (on an IHD) + disagg (on a website)
  • Compare fine-grained disag versus coarse-grained disag

Conclusions

  • There are many uses for disag data! This talk just considered one use!
  • Available evidence suggests that aggregate feedback is at least as effective as disag feedback
  • Disag feedback might drive savings of 0.7% - 4.5% in general population
  • Disag feedback might drive larger savings in "energy enthusiast" populations
  • Fine-grained disag may not be necessary
  • But! Lots of gaps in our knowledge. Cannot robustly falsify any hypotheses yet.

Conclusions

Users might become more interested in disag feedback if:

  • Energy prices increase
  • Concern about climate change deepens
  • Disag accuracy increases or if designers communicate uncertain estimates
  • Lots of ideas in the literature about how to improve disag feedback. e.g. disag by behaviour; or display feedback near appliances; or provide better recommendations etc.

This presentation is based on my paper:
"Does disaggregated electricity feedback reduce domestic electricity consumption? A systematic review of the literature",
In 3rd International NILM Workshop, Vancouver, Canada, 14-15 May 2016.