Add retry logic to message bridge server
This makes it so that message bridge will attempt to retry failed sends
of any reliable connections. This helps to improve our guarantees around
reliable message channels. This is particularly relevant during startup
when there may be large bursts of reliable messages being sent.
Wrote a manual test that triggers the retry logic on my laptop. I have
not attempted to run the test on any other platforms, but am operating
on the assumption that it'll be absurdly flaky and so should only be run
by engineers wanting to exercise this logic themselves.
Change-Id: Ic4bdfe4799b902883e1626542dff77fcbf3fb913
Signed-off-by: James Kuszmaul <james.kuszmaul@bluerivertech.com>
diff --git a/aos/network/message_bridge_server.fbs b/aos/network/message_bridge_server.fbs
index e936828..b05d933 100644
--- a/aos/network/message_bridge_server.fbs
+++ b/aos/network/message_bridge_server.fbs
@@ -13,6 +13,8 @@
// Total number of messages that were dropped while sending (e.g.,
// those dropped by the kernel).
dropped_packets:uint (id: 2);
+ // Count of the total number of retries attempted on this channel.
+ retry_count:uint (id: 3);
}
// State of the connection.
@@ -62,6 +64,10 @@
// Statistics for every channel being forwarded to this node. Ordering is arbitrary;
// the channels are identified by an index in the ServerChannelStatistics.
channels:[ServerChannelStatistics] (id: 10);
+
+ // Total number of retries attempted on all channels. Typically due to kernel
+ // send buffers filling up.
+ retry_count:uint (id: 11);
}
// Statistics for all connections to all the clients.