In this section, we evaluate the performance of the KF and PF using the information position provided by the RPE and DPE localization algorithms. We apply the proposed approach in the localization process, where several distance estimates are used to verify its performance.
Methodology
The evaluation methodology is divided into five phases, as shown in Fig. 5. First, there is a newly deployed sensor network with some beacon nodes, where most of the nodes do not know their position (unknown nodes), so this network must be prepared to track the target. In the second phase, a localization algorithm is applied (RPE or DPE), and during this step, several distance estimates can be used to reduce the localization errors and improve the tracking accuracy, following the proposed approach (see Sect. 3). In the third phase, nodes know their position, so when three or more nodes detect the target, they compute its position with multilateration. In the fourth phase, the nodes send the target position to the sink node. In the final phase, the sink node predicts the next target position and reduces the measurement noise, performing the tracking algorithm (Kalman filter or Particle filter). While there are measurements, the target tracking continues (back to phase three).
The experiments were performed by simulation (implemented in Java), where the sensor field is composed of \(n\) sensor nodes, with a communication range of \(r_c\), that are distributed in a two-dimensional squared sensor field \(Q = [0, s] \times [0, s]\). As a proof-of-concept, we consider symmetric communication links, i.e., for any two nodes \(u\) and \(v\), \(u\) reaches \(v\) if and only if \(v\) reaches \(u\). Thus, we represent the network by the Euclidean graph \(G = (V, E)\) with the following properties:
-
\(V = \{v_1, v_2, \ldots , v_n\}\) is the set of sensor nodes;
-
\(\langle i,j \rangle \in E\) iff \(v_i\) reaches \(v_j\), i.e. the distance between \(v_i\) and \(v_j\) is less than \(r_c\).
To detect the target, we use the binary detection model [11, 34]. In this model, for a given event \(e\) (target presence), every sensor \(v\), whose distance \(d\) between it and the target is smaller than a detection radius \(r_d\), assuredly detects the event. Then, the probability of a sensor node to detect an event is defined as
$$\begin{aligned} P(v,e) = {\left\{ \begin{array}{ll} 1,&\text{ if} d \le r_d \\ 0,&\text{ otherwise} \end{array}\right.}. \end{aligned}$$
(8)
The default network configuration is composed of \(n = 150\) sensors nodes randomly distributed on a \(Q = [0,70] \times [0,70]\) m\(^2\) sensor field. The communication and detection ranges are \(r_c = r_d = 15\)m for every node. This configuration defines a network density of 0.03 nodes/m\(^2\), which is sufficient for the majority of nodes to have their locations estimated by both RPE and DPE algorithms. Oliveira et al. [24] use this same configuration in their experiments to evaluate the localization systems. Therefore, we adopted this configurations to estimate the nodes position and tracking the target.
Node locations are estimated by RPE or DPE. In the RPE algorithm, 5 % of the nodes are beacons, while DPE always uses four beacons. To simulate the inaccuracies of the distance estimations, usually obtained by RSSI, time of arrival (TOA) and time difference of arrival (TDoA) [8, 9], each range sample is disturbed by a zero-mean Gaussian variable with standard deviation equals to 5 % of the distance. This assumption is reasonable and leads to non-Gaussian errors in the localization algorithms [25]. During the localization algorithm, we vary the number of distance estimates used by the Kalman Filter as 1, 10, 20, 50, 100, and 200 measurements for each reference.
The target tracking is performed with Kalman or Particle filters. The Kalman filter has its linear system equations represented by
$$\begin{aligned} {\left\{ \begin{array}{ll} x_{k+1} \!=\! \left[ \begin{array}{c} px_{k+1} \\ py_{k+1} \\ vx_{k+1} \\ vy_{k+1} \end{array} \right] \!=\! \begin{bmatrix} 1&\quad \! 0&\quad \! T&\quad \! 0 \\ 0&\quad \! 1&\quad \! 0&\quad \! T \\ 0&\quad \! 0&\quad \! 1&\quad \! 0 \\ 0&\quad 0&\quad 0&\quad 1 \end{bmatrix} \!\times \! \left[ \begin{array}{c} px_k \\ py_k \\ vx_k \\ vy_k \end{array} \right] \!+\! w_k \\ y_k = \begin{bmatrix} 1&\quad 0&\quad 0&\quad 0 \\ 0&\quad 1&\quad 0&\quad 0 \end{bmatrix} \times \left[ \begin{array}{c} px_k \\ py_k \\ vx_k \\ vy_k \end{array} \right] + z_k \end{array}\right.} \end{aligned}$$
(9)
in which \(x\) represents the state of a discrete-time \(k\), composed by the position (\(px\), \(py\)) and velocity (\(vx\), \(vy\)); \(y\) is a measurement value; \(w\) and \(v\) represent the process and measurement noise, respectively.
The Particle filter uses 1,000 particles. This value was set based on a previous empirical tests that showed that more than 1,000 particles do not improve tracking significantly. The Particle filter used in the experiments is represented by Algorithm 1.
For illustration purposes, the Particle Filter algorithm presented considers only one dimension, in which \(x\) is the position, \(v\) is the velocity and \(w\) is the weight of each \(N\) particles in a discrete-time \(k\); \(y\) is the input measurement value. First, the algorithm randomly distributes the particles (line 2). The particle propagation and the calculus of their importance consider the distance from each particle to the measurement position (lines 4–10). The normalization process (line 12) prepares the particles weights for the resampling process (lines 14–21). Finally, the prediction of the position is calculated (line 23).
For the sake of simplicity, we consider an uniform movement, so that the movement is modeled by a linear system, suitable for both Kalman and Particle filters. The target trajectory is composed of 1,000 points to consider a significative sample. The distance between the points of the trajectory is 0.1 m (uniform motion) to keep the target within the area monitored. The interval between each measurement is \(T=1\)s. The maneuvers of the target are determined by an angle randomly generated within \(-25^\circ \) and \(25^\circ \) every 25 steps.
In Figs. 6–12, each point is plotted as an average of 100 random topologies to ensure a lower variance in the results. The error bars represent the confidence interval of 99 %.
Simulation results
Target tracking behavior
To illustrate the behavior of the tracking algorithms, we show some snapshots in this section. In these snapshots, the RPE algorithm could not find the location of two nodes (from 150 nodes), and the average error of the node locations is 3.49 m. Adopting the same instance, the DPE algorithm managed to estimate the location of every node, and the average location error is 2.56 m.
These two scenarios are compared with the ideal setting, in which the localization system is perfect. For all cases, the performance of Kalman and Particle filters are presented. The results are summarized in Table 1.
The Fig. 6a–c shows a target moving through the sensor field (red line). Orange points represent measurements and blue points are the results of Kalman Filter tracking algorithm. Figure 6d–f shows the error calculated from real target and Kalman Filter estimation for each measured point. Figure 6g–l represents the same case using the Particle Filter tracking algorithm instead. These figures illustrates the influence of the localization errors caused by each algorithm. In general, the greater the localization error, the greater the tracking error, independent of the tracking algorithm. The influence of localization errors is clearly visible in the region around point (20, 15) in Fig. 6c, in which localization errors lead to a wrong track estimation.
Tracking with Kalman Filter has better results when node’s location information are ideal, or when they are estimated by the DPE. However, when the node locations are estimated by the RPE, the Particle Filter presents the best results. The reason is that the Particle Filter is less affected by measurement errors, this fact becomes clear in the following sections.
As a general conclusion, Fig. 6 shows that both filters successfully reduce the errors resulting from the estimation of the target location, but errors resulting from the localization algorithms are not significantly filtered.
Costs and benefits of multiple distance estimates.
More distance estimates during the localization process can reduce the localization and tracking errors. However, it is necessary to send additional packets, i.e., more resources will be consumed to get this benefit. Therefore, in this section we evaluate the costs and benefits of using several distance estimates. This analysis is important, since it helps define how much you should spend for a given performance in the target tracking.
Both the RPE and DPE have the communication complexity of \(O(n)\), where \(n\) is the number of nodes. The Fig. 7a shows the number of packets sent when the number of distance estimates increases. Using \(k\) distance estimates causes each beacon and settled node to broadcast its position \(k\) times, increasing the communication complexity to \(O(kn)\). This figure also shows that the RPE sends fewer packets than the DPE. This occurs because of the network density used in the experiment that lead RPE, in some topologies, to estimate fewer node positions than DPE, so these nodes do not broadcast their location information, reducing the number of packets sent.
The Fig. 7b shows the improvement in accuracy of tracking when the number of distance estimates increase. Using 10 distance estimates already provides a significant improvement when compared to results obtained with only 1 estimation. In the target tracking using the RPE, the Particle Filter has better results, because this algorithm reduces a small fraction of non-Gaussian noise introduced by localization algorithm. In target tracking with DPE, the Kalman Filter becomes more accurate with more than 10 distance estimates, because the localization error is pretty low, so that the Kalman Filter starts to operate under ideal conditions.
It is not feasible to use a very high number of distance estimates, because the benefit achieved becomes lower in comparison with the required cost. Using 10 distance estimates is enough to achieve improvements of 50 % with a reasonable cost. Until 50 distance estimates can significantly reduce the tracking error, but most estimates (100 and 200) lead to a little improvement of results with high cost.
Impact of distance estimation inaccuracy
Distances estimated by sensor nodes are not perfect. Depending on the monitored environment, the associated errors can be greater, which affects the tracking performance. In general, these errors can be modeled by a zero-mean Gaussian variable, in which the standard deviation is a percentage of the actual distance [4].
Thus, to evaluate different situations, we vary the standard deviation from 0 to 15 % of the distance (for RPE and DPE estimation processes). A standard deviation of 0 % corresponds to a perfect distance estimate. Deviations between 0 and 8 % can represent the estimates obtained by techniques that use time of arrival of the signal as TOA and TDoA, which have errors smaller than 1 m. While larger deviations represent errors obtained by more imprecise methods, like RSSI.
Figure 8a–d presents the error performance by varying the distance estimation inaccuracy and the number of estimates, showing 3D graphs with the combination of RPE and DPE with KF and PF. Figure 8e, f shows the cases of 1 and 50 distances estimates, respectively.
When the distance estimation inaccuracy is low (between 5 and 10 % of the distance), the accuracy improvement of the target tracking using DPE is negligible, regardless of the number of distance estimates used. When this imprecision is high (between 15 and 30 %), with 50 distances estimates or more, note that the average error converges to 1 m (Fig. 8b, d). However, with RPE, the improvement is noticeable even when the distance estimates inaccuracy is low (Fig. 8a, c).
It is also interesting to note in Fig. 8e, f that the Particle filter outperforms the Kalman Filter, especially when RPE is chosen as the localization algorithm. The reason is that the non-linearity and non-Gaussian nature of the Particle Filter results in reducing a small fraction of the non-Gaussian noise introduced by the localization algorithm.
Impact of the network density
The impact of the density of the network is evaluated by increasing the number of nodes in the same sensor field, so that the network density varies from 0.03 to 0.07 nodes/m\(^2\). The smallest density used in this experiment allows both the RPE and DPE algorithms to estimate the location of the most of the sensor nodes.
In this case, Fig. 9 shows that, for the DPE algorithm, the target tracking error remains constant independent of the network density. The reason is that the same beacon structure is used regardless the network density. However, for the RPE algorithm, the number of beacons increase with the network density, because we ensure that 5 % of the nodes are beacons. As a result of the increasing number of beacons, the tracking error reduces accordingly.
Multiple distance estimates are important for improving the target tracking accuracy when the RPE is used, especially in sparse networks, as show in Fig. 9a and c. With the DPE instead, the network density does not interfere in the target tracking error. Therefore, for 50 distance estimates or more the average error converges to 0.7m (Fig. 9b, d).
Figures 9(e and f) show that with a single distance estimate during the location process, the Particle Filter is slightly better with both RPE and DPE. The reason is that it filters a small fraction of the non-Gaussian localization errors. When we use 50 distance estimates, the performance of the Kalman Filter becomes equivalent to the Particle Filter in the case of RPE and better in the case of DPE, since the Kalman Filter operates under ideal conditions with low localization errors.
The impact of the network scale
In this section, we evaluate how the network scale affects the combinations of localization and tracking algorithms.
In this context, we vary the number nodes from 100 to 350, while keeping a constant density of 0.03 nodes/m\(^2\). Therefore, the monitored area is resized according to the number of sensor nodes. As the percentage of beacons used by RPE is 5 %, the number of beacons also increases according to the number of nodes in this case. The DPE keeps using only a single structure of four beacons.
Figure 10 shows that increasing the network scale, the tracking errors with DPE increase accordingly. The reason is that a higher number of nodes generates a higher propagation of position error, since the same number of beacons is maintainedregardless the number of nodes. However, with the RPE, the tracking errors remains almost constant, because the number of beacons increase with the network scale (cf. [24]).
When there are many nodes in network (between 250 and 350), using multiple distance estimates in DPE improves significantly the target tracking accuracy, however they have little influence when there are few nodes (Fig. 10b, d). With RPE instead, the usage of multiple distance estimates is important for any number of nodes (Fig. 10a, c).
The impact of the number of beacons
The number beacons used by the DPE and RPE lead to different localization errors. Therefore, affecting the tracking solutions. For the DPE, Oliveira et al. [24] show that increasing the number of beacon nodes of the structure does not improve significantly the result of the localization. Therefore, this evaluation considers only the RPE algorithm.
In this experiment, the number of beacons is increased from 5 to 35 % of the total number nodes. A greater number of beacons means that the localization algorithm has more references for estimating the location of remaining nodes, which leads to smaller errors. Hence, the tracking error is also inversely proportional to the number of beacons. This behavior is depicted in Fig. 11.
When the amount of beacon nodes is significantly large, the localization error becomes so small that the Kalman Filter tends to have better results than the Particle Filter, it is up 20 % better with 1 estimate and up 10 % to 50 estimates (Fig. 11c, d). The reason is that the non-Gaussian errors resulting from localization systems are reduced in such a way that the Kalman Filter starts to operate under ideal conditions, which means it converges to the optimal solution for target tracking. When there are few beacons (10 %), multiple distance estimates improves significantly the target tracking accuracy. However, when more than 10 % of the nodes are beacons note that the average error converges to 0.6 m, both for target tracking with Kalman and Particle filters (Fig. 11a, b).
The impact of the beacon structure
As stated earlier, increasing numbers beacons by structure in the DPE does not improve the localization results. However, the DPE may benefit from multiple beacon structures [24]. Thus, to evaluate the performance of the target tracking algorithms with multiple beacon structures, we vary the number of such structures from 1 to 5. This experiment represents a situation for the DPE algorithm that is analogous to the previous experiment for the RPE algorithm.
By performing a single distance estimate in the localization process (Fig. 12(c)) and using a three-beacon structure, the tracking results with Kalman and Particle filters are very close. However, when more structures are used, the Kalman Filter tracking is favored. The opposite occurs when we use less than three-beacon structures. With 50 distance estimates, the Kalman Filter outperforms the Particle Filter with any number of beacon structures, because the non-Gaussian noise introduced by the location is very low (Fig. 12(d)). Besides, when we use of 50 distance estimates, the average tracking error converges to 0.6 m regardless of the number of beacon structures available (Fig. 12a, b).