Disaggregate
Disaggregate Smart Meter data
|
This class does most of the work behind the "graphs and spikes" disaggregation approach. In particular, this class has responsibility for maintaining the "powerStateGraph" and the "disaggregation tree". More...
#include <PowerStateGraph.h>
Classes | |
struct | Disag_edge_writer |
used for write_graphviz for DisagGraph. More... | |
struct | Disag_vertex_writer |
used for write_graphviz for DisagGraph. More... | |
struct | DisagVertex |
A vertex for the directed acylic graphs used by getStartTimes() More... | |
struct | Fingerprint |
struct | LikelihoodAndVertex |
struct | PowerStateEdge |
struct | PowerStateVertex |
struct | PSG_edge_writer |
struct | PSG_vertex_writer |
struct | TimeAndPower |
A simple struct for pairing timestamp and meanPower . Used in Fingerprint struct. More... | |
Public Member Functions | |
PowerStateGraph () | |
void | update (const Signature &sig, const bool verbose=false) |
Update or initialise Power State Graph. | |
void | writeGraphViz (std::ostream &out) |
Information about an entire device 'fingerprint' found in the aggregate data. | |
const std::list< Fingerprint > | disaggregate (const AggregateData &aggregateData, const bool keep_overlapping=false, const bool verbose=false) |
This is the main public interface to the disaggregation algorithm. | |
void | setDeviceName (const std::string &_deviceName) |
const Statistic< double > & | getEnergyConsumption () const |
Private Types | |
typedef boost::adjacency_list < boost::vecS, boost::vecS, boost::directedS, PowerStateVertex, PowerStateEdge > | PSGraph |
A graph where each vertex(node) is a Statistic<Sample_t> and each edge is a PowerStateEdge . | |
typedef boost::graph_traits < PSGraph >::vertex_iterator | PSG_vertex_iter |
typedef boost::graph_traits < PSGraph >::edge_iterator | PSG_edge_iter |
typedef boost::graph_traits < PSGraph >::out_edge_iterator | PSG_out_edge_iter |
typedef boost::property_map < PSGraph, boost::vertex_index_t >::type | PSG_vertex_index_map |
typedef boost::property_map < PSGraph, boost::edge_index_t > ::type | PSG_edge_index_map |
typedef boost::adjacency_list < boost::setS, boost::vecS, boost::bidirectionalS, DisagVertex, double > | DisagTree |
Tree structure used to keep track of all the possible solutions during disaggregation. | |
typedef boost::graph_traits < DisagTree > ::out_edge_iterator | Disag_out_edge_iter |
typedef boost::graph_traits < DisagTree >::edge_descriptor | Disag_edge_desc |
Private Member Functions | |
PSGraph::vertex_descriptor | updateOrInsertVertex (const Signature &sig, const Statistic< Sample_t > &postSpikePowerState, const Statistic< Sample_t > &betweenSpikesPowerState, const bool verbose=false) |
Attempts to find an existing vertex which is statistically similar to stat . If an existing similar vertex is found then the vertex's stats are updated with the new data points. If a similar vertex is not found, and new vertex is inserted. Either way, a vertex_descriptor is returned to the new or existing similar vertex. | |
PSGraph::vertex_descriptor | mostSimilarVertex (bool *success, const Statistic< Sample_t > &stat, const double ALPHA=0.0000005) const |
Find the vertex statistically most similarto stat . | |
const bool | rejectSpike (const Statistic< Sample_t > &before, const Statistic< Sample_t > &after, const bool verbose=false) const |
Decides whether or not to reject this spike based on the before and after statistics. | |
void | printSpikeInfo (const std::list< Signature::Spike >::iterator spike, const size_t start, const size_t end, const Statistic< Sample_t > &before, const Statistic< Sample_t > &after, const Signature &sig) const |
Useful for diagnostics. | |
void | updateOrInsertEdge (const PSGraph::vertex_descriptor &beforeVertex, const PSGraph::vertex_descriptor &afterVertex, const size_t samplesSinceLastSpike, const double spikeDelta, const bool verbose=false) |
Update or Insert a new edge into powerStateGraph. | |
void | updateEdges (const Signature &sig) |
Update (or create) directional edges between power states (vertices) so the edges are consistent with the observed transitions between power states in sig . | |
const Fingerprint | initTraceToEnd (const AggregateData::FoundSpike &spike, const size_t deviceStart, const bool verbose=false) |
void | traceToEnd (DisagTree *disagTree, const DisagTree::vertex_descriptor &vertex, const size_t prevTimestamp, const bool verbose=false) const |
void | addItemToEdgeHistory (const PSGraph::edge_descriptor &edge) |
const bool | edgeListsAreEqual (const std::list< PSGraph::edge_descriptor > &a, const std::list< PSGraph::edge_descriptor > &b, const bool verbose=false) const |
Check if edge list a and edge list b are equal. | |
std::list < PSGraph::edge_descriptor > | getEdgeHistoryForVertex (const DisagTree &disagTree, const DisagTree::vertex_descriptor &startVertex) const |
Trace the disagTree backwards. | |
void | findListOfPathsThroughDisagTree (const DisagTree &disagTree, const DisagTree::vertex_descriptor vertex, const LikelihoodAndVertex lav, std::list< LikelihoodAndVertex > path=std::list< LikelihoodAndVertex >(0)) |
Trace the tree downwards from vertex recursively finding every path which successfully completes (i.e. reaches an off state). | |
const Fingerprint | findBestPath (const DisagTree &disagTree, const size_t deviceStart, const bool verbose=false) |
Iterates through each path in listOfPaths to find the one with the highest likelihood. | |
void | removeOverlapping (std::list< Fingerprint > *disagList, const bool verbose=false) |
void | displayAndPlotFingerprintList (const std::list< Fingerprint > &fingerprintList, const std::string &aggDataFilename) const |
const size_t | indexOfNextSpike (const std::list< Signature::Spike > &spikes, std::list< Signature::Spike >::iterator spike, const Signature &sig) const |
Determine the index of the next spike after spike . | |
Private Attributes | |
PSGraph | powerStateGraph |
store the power state graph learnt during training. | |
PSGraph::vertex_descriptor | offVertex |
size_t | totalCount |
the total number of times any edge has been traversed during training. | |
AggregateData const * | aggData |
std::list < PSGraph::edge_descriptor > | edgeHistory |
a "rolling" list storing the previous few edges we've seen. | |
std::list< std::list < LikelihoodAndVertex > > | listOfPaths |
Statistic< double > | energyConsumption |
Energy consumption in Joules obtained from training signatures. | |
std::string | deviceName |
All PowerStateGraphs are associated with a single device. | |
Static Private Attributes | |
static const size_t | EDGE_HISTORY_SIZE = 5 |
Set to 0 to disable. | |
Friends | |
std::ostream & | operator<< (std::ostream &o, const PowerStateGraph &psg) |
This class does most of the work behind the "graphs and spikes" disaggregation approach. In particular, this class has responsibility for maintaining the "powerStateGraph" and the "disaggregation tree".
Definition at line 26 of file PowerStateGraph.h.
typedef boost::graph_traits<DisagTree>::edge_descriptor PowerStateGraph::Disag_edge_desc [private] |
Definition at line 231 of file PowerStateGraph.h.
typedef boost::graph_traits<DisagTree>::out_edge_iterator PowerStateGraph::Disag_out_edge_iter [private] |
Definition at line 230 of file PowerStateGraph.h.
typedef boost::adjacency_list< boost::setS, boost::vecS, boost::bidirectionalS, DisagVertex, double > PowerStateGraph::DisagTree [private] |
Tree structure used to keep track of all the possible solutions during disaggregation.
Implemented using the Boost Graph Library.
Definition at line 228 of file PowerStateGraph.h.
typedef boost::property_map<PSGraph, boost::edge_index_t >::type PowerStateGraph::PSG_edge_index_map [private] |
Definition at line 138 of file PowerStateGraph.h.
typedef boost::graph_traits<PSGraph>::edge_iterator PowerStateGraph::PSG_edge_iter [private] |
Definition at line 134 of file PowerStateGraph.h.
typedef boost::graph_traits<PSGraph>::out_edge_iterator PowerStateGraph::PSG_out_edge_iter [private] |
Definition at line 135 of file PowerStateGraph.h.
typedef boost::property_map<PSGraph, boost::vertex_index_t>::type PowerStateGraph::PSG_vertex_index_map [private] |
Definition at line 137 of file PowerStateGraph.h.
typedef boost::graph_traits<PSGraph>::vertex_iterator PowerStateGraph::PSG_vertex_iter [private] |
Definition at line 133 of file PowerStateGraph.h.
typedef boost::adjacency_list< boost::vecS, boost::vecS, boost::directedS, PowerStateVertex, PowerStateEdge > PowerStateGraph::PSGraph [private] |
A graph where each vertex(node) is a Statistic<Sample_t>
and each edge is a PowerStateEdge
.
See boost::graph bundles tutorial
Definition at line 115 of file PowerStateGraph.h.
PowerStateGraph::PowerStateGraph | ( | ) |
Definition at line 17 of file PowerStateGraph.cpp.
References offVertex, and powerStateGraph.
void PowerStateGraph::addItemToEdgeHistory | ( | const PSGraph::edge_descriptor & | edge | ) | [private] |
Definition at line 28 of file PowerStateGraph.cpp.
References EDGE_HISTORY_SIZE, and edgeHistory.
Referenced by updateOrInsertEdge().
const list< PowerStateGraph::Fingerprint > PowerStateGraph::disaggregate | ( | const AggregateData & | aggregateData, |
const bool | keep_overlapping = false , |
||
const bool | verbose = false |
||
) |
This is the main public interface to the disaggregation algorithm.
Each candidate solution is represented as a tree. Each vertex on this tree is a spike found in the aggregate data. The edge weights are the mean of the probability density functions for the spike size and the timing. If an edge described in PowerStateGraph (from training) cannot be found in the aggregate data then this candidate is discarded. If we successfully get from vertex0 back to vertex0 then the shortest path through the tree is calculated and saved.
e
which connects vertex0
(offVertex) to vertex1. Look through the AggregateData searching for any spike within a certain number of standard deviations of e.delta
. aggregateData | A populated array of AggregateData |
keep_overlapping | Should we keep or remove overlapping candidates? |
Definition at line 612 of file PowerStateGraph.cpp.
References aggData, PowerStateGraph::Fingerprint::avLikelihood, PowerStateGraph::PowerStateEdge::delta, displayAndPlotFingerprintList(), PowerStateGraph::PowerStateEdge::duration, Utils::fatalError(), AggregateData::findSpike(), AggregateData::getFilename(), initTraceToEnd(), Statistic< T >::mean, offVertex, powerStateGraph, and removeOverlapping().
Referenced by main().
void PowerStateGraph::displayAndPlotFingerprintList | ( | const std::list< Fingerprint > & | fingerprintList, |
const std::string & | aggDataFilename | ||
) | const [private] |
Definition at line 691 of file PowerStateGraph.cpp.
References GNUplot::PlotVars::data, DATA_OUTPUT_PATH, deviceName, GNUplot::PlotVars::inFilename, Utils::openFile(), GNUplot::PlotVars::outFilename, GNUplot::plot(), GNUplot::PlotVars::plotArgs, Utils::size_t_to_s(), GNUplot::PlotVars::title, GNUplot::PlotVars::xlabel, and GNUplot::PlotVars::ylabel.
Referenced by disaggregate().
const bool PowerStateGraph::edgeListsAreEqual | ( | const std::list< PSGraph::edge_descriptor > & | a, |
const std::list< PSGraph::edge_descriptor > & | b, | ||
const bool | verbose = false |
||
) | const [private] |
Check if edge list a
and edge list b
are equal.
Definition at line 418 of file PowerStateGraph.cpp.
References EDGE_HISTORY_SIZE, and powerStateGraph.
Referenced by traceToEnd(), and updateOrInsertEdge().
const PowerStateGraph::Fingerprint PowerStateGraph::findBestPath | ( | const DisagTree & | disagTree, |
const size_t | deviceStart, | ||
const bool | verbose = false |
||
) | [private] |
Iterates through each path in listOfPaths
to find the one with the highest likelihood.
To be called after listOfPaths
has been populated by findListOfPathsThroughDisagTree().
Definition at line 1060 of file PowerStateGraph.cpp.
References PowerStateGraph::Fingerprint::avLikelihood, PowerStateGraph::Fingerprint::duration, PowerStateGraph::Fingerprint::energy, energyConsumption, listOfPaths, Statistic< T >::normalisedLikelihood(), PowerStateGraph::Fingerprint::timeAndPower, and PowerStateGraph::Fingerprint::timestamp.
Referenced by initTraceToEnd().
void PowerStateGraph::findListOfPathsThroughDisagTree | ( | const DisagTree & | disagTree, |
const DisagTree::vertex_descriptor | vertex, | ||
const LikelihoodAndVertex | lav, | ||
std::list< LikelihoodAndVertex > | path = std::list<LikelihoodAndVertex>(0) |
||
) | [private] |
Trace the tree downwards from vertex
recursively finding every path which successfully completes (i.e. reaches an off state).
path | Deliberately called-by-value because we want a copy. |
Definition at line 866 of file PowerStateGraph.cpp.
References PowerStateGraph::LikelihoodAndVertex::likelihood, listOfPaths, and PowerStateGraph::LikelihoodAndVertex::vertex.
Referenced by initTraceToEnd().
list< PowerStateGraph::PSGraph::edge_descriptor > PowerStateGraph::getEdgeHistoryForVertex | ( | const DisagTree & | disagTree, |
const DisagTree::vertex_descriptor & | startVertex | ||
) | const [private] |
Trace the disagTree backwards.
Definition at line 1152 of file PowerStateGraph.cpp.
References EDGE_HISTORY_SIZE, and offVertex.
const Statistic< double > & PowerStateGraph::getEnergyConsumption | ( | ) | const |
Definition at line 42 of file PowerStateGraph.cpp.
References energyConsumption.
Referenced by Device::trainPowerStateGraph().
const size_t PowerStateGraph::indexOfNextSpike | ( | const std::list< Signature::Spike > & | spikes, |
std::list< Signature::Spike >::iterator | spike, | ||
const Signature & | sig | ||
) | const [private] |
Determine the index of the next spike after spike
.
Definition at line 171 of file PowerStateGraph.cpp.
References Array< T >::getSize().
Referenced by update().
const PowerStateGraph::Fingerprint PowerStateGraph::initTraceToEnd | ( | const AggregateData::FoundSpike & | spike, |
const size_t | deviceStart, | ||
const bool | verbose = false |
||
) | [private] |
deviceStart | The possible time the device started. |
Definition at line 796 of file PowerStateGraph.cpp.
References EDGE_HISTORY_SIZE, findBestPath(), findListOfPathsThroughDisagTree(), PowerStateGraph::LikelihoodAndVertex::likelihood, AggregateData::FoundSpike::likelihood, listOfPaths, offVertex, powerStateGraph, AggregateData::FoundSpike::timestamp, traceToEnd(), and PowerStateGraph::LikelihoodAndVertex::vertex.
Referenced by disaggregate().
PowerStateGraph::PSGraph::vertex_descriptor PowerStateGraph::mostSimilarVertex | ( | bool * | success, |
const Statistic< Sample_t > & | stat, | ||
const double | ALPHA = 0.0000005 |
||
) | const [private] |
Find the vertex statistically most similarto stat
.
success
is also used as a return parameter.success | return parameter. Did we find a satisfactory match? |
stat | stat to find in graph vertices |
ALPHA | significance level (what constitutes as a "satisfactory" match?) |
Definition at line 277 of file PowerStateGraph.cpp.
References Utils::highest(), Statistic< T >::mean, powerStateGraph, Statistic< T >::tTest(), and Utils::within().
Referenced by updateEdges(), and updateOrInsertVertex().
void PowerStateGraph::printSpikeInfo | ( | const std::list< Signature::Spike >::iterator | spike, |
const size_t | start, | ||
const size_t | end, | ||
const Statistic< Sample_t > & | before, | ||
const Statistic< Sample_t > & | after, | ||
const Signature & | sig | ||
) | const [private] |
Useful for diagnostics.
Definition at line 134 of file PowerStateGraph.cpp.
Referenced by update().
const bool PowerStateGraph::rejectSpike | ( | const Statistic< Sample_t > & | before, |
const Statistic< Sample_t > & | after, | ||
const bool | verbose = false |
||
) | const [private] |
Decides whether or not to reject this spike based on the before
and after
statistics.
Definition at line 189 of file PowerStateGraph.cpp.
References Utils::highest(), Statistic< T >::mean, Statistic< T >::stdev, and Utils::within().
Referenced by update().
void PowerStateGraph::removeOverlapping | ( | std::list< Fingerprint > * | disagList, |
const bool | verbose = false |
||
) | [private] |
Remove any overlapping list entries and leave the one with the highest likelihood
disagList | Input and output parameter |
Definition at line 746 of file PowerStateGraph.cpp.
Referenced by disaggregate().
void PowerStateGraph::setDeviceName | ( | const std::string & | _deviceName | ) |
Definition at line 1171 of file PowerStateGraph.cpp.
References deviceName.
Referenced by Device::Device().
void PowerStateGraph::traceToEnd | ( | DisagTree * | disagTree_p, |
const DisagTree::vertex_descriptor & | disagVertex, | ||
const size_t | prevTimestamp, | ||
const bool | verbose = false |
||
) | const [private] |
Trace from startVertex to the off state in PSGraph
disagTree_p | input and output parameter |
prevTimestamp | timestamp of previous vertex |
Definition at line 909 of file PowerStateGraph.cpp.
References aggData, EDGE_HISTORY_SIZE, edgeHistory, edgeListsAreEqual(), AggregateData::findSpike(), offVertex, powerStateGraph, and AggregateData::readingGoesBelowPowerState().
Referenced by initTraceToEnd().
void PowerStateGraph::update | ( | const Signature & | sig, |
const bool | verbose = false |
||
) |
Update or initialise Power State Graph.
Definition at line 50 of file PowerStateGraph.cpp.
References Signature::Spike::compareIndexAsc(), edgeHistory, energyConsumption, Signature::getDeltaSpikes(), Signature::getEnergyConsumption(), Signature::getID(), Array< T >::getSize(), indexOfNextSpike(), J_PER_KWH, offVertex, printSpikeInfo(), rejectSpike(), Statistic< T >::update(), updateOrInsertEdge(), and updateOrInsertVertex().
Referenced by Device::trainPowerStateGraph().
void PowerStateGraph::updateEdges | ( | const Signature & | sig | ) | [private] |
Update (or create) directional edges between power states (vertices) so the edges are consistent with the observed transitions between power states in sig
.
Basic strategy is to:
sig
, extract the most salient and put the spikes into temporal order. delta
and duration
stats for the edge and update if necessary. If no edge exists then create one with the necessary delta
and duration
, with a hard-coded stdev. Definition at line 494 of file PowerStateGraph.cpp.
References Signature::Spike::compareIndexAsc(), Signature::getDeltaSpikes(), mostSimilarVertex(), offVertex, powerStateGraph, and totalCount.
void PowerStateGraph::updateOrInsertEdge | ( | const PSGraph::vertex_descriptor & | beforeVertex, |
const PSGraph::vertex_descriptor & | afterVertex, | ||
const size_t | samplesSinceLastSpike, | ||
const double | spikeDelta, | ||
const bool | verbose = false |
||
) | [private] |
Update or Insert a new edge into powerStateGraph.
Definition at line 329 of file PowerStateGraph.cpp.
References addItemToEdgeHistory(), edgeHistory, edgeListsAreEqual(), powerStateGraph, Utils::sameSign(), totalCount, and Utils::within().
Referenced by update().
PowerStateGraph::PSGraph::vertex_descriptor PowerStateGraph::updateOrInsertVertex | ( | const Signature & | sig, |
const Statistic< Sample_t > & | postSpikePowerState, | ||
const Statistic< Sample_t > & | betweenSpikesPowerState, | ||
const bool | verbose = false |
||
) | [private] |
Attempts to find an existing vertex which is statistically similar to stat
. If an existing similar vertex is found then the vertex's stats are updated with the new data points. If a similar vertex is not found, and new vertex is inserted. Either way, a vertex_descriptor is returned to the new or existing similar vertex.
sig | source of the raw data |
verbose | cout debugging messages? |
Definition at line 221 of file PowerStateGraph.cpp.
References mostSimilarVertex(), offVertex, and powerStateGraph.
Referenced by update().
void PowerStateGraph::writeGraphViz | ( | std::ostream & | out | ) |
Information about an entire device 'fingerprint' found in the aggregate data.
Produce a graphviz output.
Definition at line 576 of file PowerStateGraph.cpp.
References powerStateGraph.
Referenced by Device::trainPowerStateGraph().
std::ostream& operator<< | ( | std::ostream & | o, |
const PowerStateGraph & | psg | ||
) | [friend] |
Definition at line 1176 of file PowerStateGraph.cpp.
AggregateData const* PowerStateGraph::aggData [private] |
Definition at line 285 of file PowerStateGraph.h.
Referenced by disaggregate(), and traceToEnd().
std::string PowerStateGraph::deviceName [private] |
All PowerStateGraphs are associated with a single device.
Definition at line 309 of file PowerStateGraph.h.
Referenced by displayAndPlotFingerprintList(), and setDeviceName().
const size_t PowerStateGraph::EDGE_HISTORY_SIZE = 5 [static, private] |
Set to 0 to disable.
Definition at line 287 of file PowerStateGraph.h.
Referenced by addItemToEdgeHistory(), edgeListsAreEqual(), getEdgeHistoryForVertex(), initTraceToEnd(), and traceToEnd().
std::list< PSGraph::edge_descriptor > PowerStateGraph::edgeHistory [private] |
a "rolling" list storing the previous few edges we've seen.
Definition at line 288 of file PowerStateGraph.h.
Referenced by addItemToEdgeHistory(), PowerStateGraph::PSG_edge_writer::operator()(), traceToEnd(), update(), and updateOrInsertEdge().
Statistic< double > PowerStateGraph::energyConsumption [private] |
Energy consumption in Joules obtained from training signatures.
Definition at line 306 of file PowerStateGraph.h.
Referenced by findBestPath(), getEnergyConsumption(), and update().
std::list< std::list<LikelihoodAndVertex> > PowerStateGraph::listOfPaths [private] |
Definition at line 304 of file PowerStateGraph.h.
Referenced by findBestPath(), findListOfPathsThroughDisagTree(), and initTraceToEnd().
PSGraph::vertex_descriptor PowerStateGraph::offVertex [private] |
Definition at line 279 of file PowerStateGraph.h.
Referenced by disaggregate(), getEdgeHistoryForVertex(), initTraceToEnd(), operator<<(), PowerStateGraph(), traceToEnd(), update(), updateEdges(), and updateOrInsertVertex().
PSGraph PowerStateGraph::powerStateGraph [private] |
store the power state graph learnt during training.
Definition at line 277 of file PowerStateGraph.h.
Referenced by disaggregate(), edgeListsAreEqual(), initTraceToEnd(), mostSimilarVertex(), operator<<(), PowerStateGraph(), traceToEnd(), updateEdges(), updateOrInsertEdge(), updateOrInsertVertex(), and writeGraphViz().
size_t PowerStateGraph::totalCount [private] |
the total number of times any edge has been traversed during training.
Definition at line 281 of file PowerStateGraph.h.
Referenced by updateEdges(), and updateOrInsertEdge().