Bandwidth estimator using a simple Kalman filter, Part 1 of 2

This here is a fairly compact C++ program I wrote that uses a simple technique to estimate a network circuits bandwidth. There's nothing new with the bandwidth estimation (there are more complex ones available), but I haven't seen this paired with an estimation algorithm (Kalman).

This post is going to be another one of the two parters since there was a bit of work to pull all this together. The first posting (this) covers the results and description of the technique, while the second will review the source code.

One of the problems with measuring bandwidth is that results are highly variable (i.e. noisy). This can be due to a number of reasons, but basically, packets travel to their target over several hops, each hop (or router, switch etc.) can introduce a variable amount of delay due to congestion, loading, system performance etc. Therefore, any single measurement of the bandwidth is unlikely to provide an accurate representation of the circuit bandwidth. In addition, bandwidth characteristics will change over time. Therefore, it's best to support a model that uses historical data to estimate the bandwidth without completely discounting large changes in the current measurement.

Simple C++ scope timing module

OK--I've been busy and tied down. And I've been trying to post an update on a weekly basis, so far so good this year. Except this is another short one. A longer posting is coming for next week though...

I've been using this little snippet of C++ code for ages, but it's nice to have the relatively simple source located at an easy to get location. Just uses the scope to determine the elapsed time in elapsed microseconds.

As you can see just requires a declaration that looks like:

Timer t("my time');

Which can be further simplified with a macro:

#define T Timer t(__PRETTY_FUNCTION__)

to:

T;

The implementation can (and has) been fancier. A static can be used to track nested or re-entry into blocks of code for easier consumption. That's great, however for multithreaded code a mutex needs to be thrown on the static which then perturbs the timing itself--so it's better to avoid unneeded complexity.

The header:

class Timer
{
public:
Timer(std::string id);
~Timer();

private:
struct timeval _start_time;
std::string _id;
};

The Source:

Timer::Timer(string id) : _id(id)
{
gettimeofday(&_start_time,NULL);
}

Timer::~Timer()
{
struct timeval end_time;
gettimeofday(&end_time,NULL);

unsigned long usecs = abs(_start_time.tv_usec - end_time.tv_usec);
unsigned long secs = end_time.tv_sec - _start_time.tv_sec;
if (_start_time.tv_usec >= end_time.tv_usec) {
secs = secs - 1;
usecs = 1000000 - usecs;
}

char buf[512];
sprintf(buf,"elapsed sec %d, usec %d",secs,usecs);
string msg = "timer(" + _id + "): " + buf;
string cmd = "echo '" + msg + "' >> /tmp/timer";
system(cmd.c_str());
}

Time printed is in elapsed time within the scope of the measurement. Obviously, this doesn't work well for scopes that are called multiple times, but is useful for outer scope timing. This can be easily modified to accumulate time though, at the expense of being thread unsafe.

Some time saving tips when setting up an Amazon EC2 network

I ran into several hiccups with my first foray into Amazon's EC2 and AMIs setting up a tiered server environment. As I hate making the same mistake twice (and love learning from others experiences) I put together a short list of "my" "lessons learned" when setting up an Amazon EC2 network for the first time. Which in retrospect seem somewhat obvious (but isn't that software development in general).

Growing a REST API, Part 2 of 2

OK, let's jump back into the fray here with part 2 (part 1 is here) of our discussion on the REST API for the Vyatta Appliance thingy. Now, we've already covered the API up to and including the Configuration mode. And found out that commands are relatively simple to construct. What's left you ask? The operational side of things and the weird red headed step-child interface: App mode.

Without further ado then...