Remove a n^2 operation in timestamp_extractor
The multinode timestamp estimator holds a deque of all the solved
timestamps, and removes them when enough time elapses. We were never
teaching the estimator that time had elapsed, so it was never expiring
entries. There is a linear step processing all old solved timestamps
looking for what we've written already, which was ending up as a n^2
operation. Drop the old timestamps like the real system...
This took a sort of 10 nodes with 10 GB of compressed events down from
days to complete to around an hour.
Change-Id: I126139ee3e572cd535127d50875bb80fd000dbdf
Signed-off-by: Austin Schuh <austin.linux@gmail.com>
diff --git a/aos/events/logging/timestamp_extractor.cc b/aos/events/logging/timestamp_extractor.cc
index f7f5a40..ed319db 100644
--- a/aos/events/logging/timestamp_extractor.cc
+++ b/aos/events/logging/timestamp_extractor.cc
@@ -161,6 +161,7 @@
if (!next_timestamp) {
break;
}
+ multinode_estimator.ObserveTimePassed(std::get<0>(*next_timestamp.value()));
}
LOG(INFO) << "Done";