Reviewer: Charles Wiseman
Date: 9-8-2005
How would you rate this paper, relative to others we have read? top 25%, but not top 10%
How would you rate your knowledge of the topic of this paper? familiar, but not expert
What problem or issue does the paper address? Why is it important?
This paper focuses on methods and tools for reporting end-to-end network congestion, and hence information loss. There are a couple of reasons that such tools are important. Notably, these kinds of tools can help us to understand what is happening in a network that we may not have full informational access to (e.g., we don't know what router capacities and queue lengths are). Given that more and more people are relying on such tools for measurement, it is clear that these tools need to be able to report accurate results.
What are the main contributions of the paper and why are they important?
One tool (that follows a specific methodology) that reports network congestion is evaluated. The results are then used to create a new tool (following a seemingly better methodology) that can report congestion more accurately.
How significant are these contributions relative to previous work?
While their results do show an improvement in loss event reporting, I'm not convinced that end result constitutes much of a contribution.
Give detailed comments justifying your view of the paper.
First off, I generally liked much of this paper. With a few additional references, I learned a few things from it, and so I can't complain too much.
Beyond that, however, I'm not sure that the paper really delivers on its promises. As I mentioned, the results at the end of paper comparing zing to badabing do indeed show that badabing is a more accurate tool. One question here that I don't the answer to is whether or not it is legitimate to compare only against zing. Is zing considered the best tool available before this paper? If not, then we need to see how other tools compare. At any rate, this improved accuracy from badabing isn't necessarily easy to come by. Basically, the problem is one of calibration. In order for the authors to achieve the results they did, they had to specifically tune badabing's parameters. This isn't trivial to do. In fact, they had to rely on knowing the real loss events. If their tuned values work under all traffic conditions, then there is no problem, and badabing is indeed a worthwhile contribution. But that doesn't seem likely, and isn't helped by the fact that small changes in the parameters can lead to large changes in the loss events reported (look at figure 9, and see that, for instance, changing tau from 20 to 40 ms with p=0.5 doubles the loss frequency).
There are other problems that caught my attention. As an example, at the end of the first paragraph of section 3, they use B in an expression without defining it (both Bin and Bout are defined, but not B). Something else that is wrong is in section 5.2.1. They assume that if Yi=11 then yi=11 or yi=00. In other words, if there is congestion in both time slots, then they will either report that congestion, or that there is _no_ congestion. However, there is nothing stopping one probe from seeing congestion and the other not. I thought that that was one problem they were trying to solve (i.e., that some flows see congestion while others don't when there is congestion). As it stands, they assume that this doesn't happen, which would lead to more reports of loss episodes and bring the average episode duration down.
For all that, I still think the paper is a pretty good one. There are other worthwhile contributions to be taken away from this paper. For one example, the ideas about probing with packet sequences, where there are many such sequences spaced out based on some probability, are good ones and could be applied elsewhere.