Defined using SignalKG (SKG) ontology.
The Signal Graph captures information needed to reason about the different types of signals (audio, vision, etc.) observed by sensors and possible causes of these observations. Entities perform Actions, Actions create Signals, and the presence of a signal is determined by a Classifier.
Defined using W3C Semantic Sensor Network (SSN/SOSA) ontology.
Sensors are described using the Semantic Sensor Network ontology, and linked to signals in the Signal Graph via the sosa:observes property.
Defined using RealEstateCore (REC) ontology (with Pose properties)
The smart building that sensors observe is defined using the RealEstateCore ontology.
Concrete entities to simulate are linked to the entity category in the Signal Graph (e.g. attacker) via the skg:ConcreteEntity property.
Concrete actions to simulate are linked to the action category in the Signal Graph via the skg:actionCategory property. They are also associated with a particular room or item in the Building Graph via the skg:concreteActsOn property.
The simulation is used to collect example Sensor Observations that could occur under a particular scenario. The observations are represented using the W3C Semantic Sensor Network (SSN/SOSA) ontology. In the next step, we will attempt to infer what took place given only the Sensor Observations (and our understanding of possible underlying causes of those signals specified in the Knowledge Graphs in the previous tab).
(3D Model credits: Generic passenger car pack by Comrade1280 is licensed under Creative Commons Attribution)
A Bayesian Belief Network, shown below, was generated from Knowledge Graphs in Tab 1 (the specific simulation scenario selected in Tab 2 was not used in construction of the Bayesian Belief Network). The Bayesian Belief Network models the probability that an entity (if present) will perform an action, the probability that an action (if it occurs) will result in a signal, and a signal (if emitted) will be detected by a sensor (taking into account the distance of the sensor from the source, any barriers between the source and the sensor that may block the signal, and the sensitivity of the classifier).
You can interactively condition the Bayesian Belief Network on a known observation by clicking on that value to fix (e.g. '1' or '0' in one of the node boxes).
Key: a_ = action, s_ = signal emitted due to action, r_ = received signal strength at location of sensor, d_ = detected signal (after classification)
(Bayesian Belief Network visualisation created using jsbayes-viz)
We can attempt to infer what took place (in the simulation) given the Sensor Observations (at the bottom of Tab 2), by conditioning the Bayesian Belief Network on those observations. You will notice that the bottom nodes (representing our observations) become fixed, and that the nodes at the top (the posterior probability of entities being present or not) are updated given our observations. The inference is approximate, so the values may fluctuate slightly if rerunning the computation.