Reviewer: Michael Wilson
Date: 9-8-2005
How would you rate this paper, relative to others we have read? bottom 50%
How would you rate your knowledge of the topic of this paper? familiar, but not expert
What problem or issue does the paper address? Why is it important?
The authors note that measurement and estimation of packet loss characteristics is currently difficult. A good understanding of these characteristics is important for network protocols that need to deal with loss, particularly TCP's congestion control algorithms. Since this type of traffic constitutes the bulk of internet traffic today, the need to understand loss well is clearly important. In their experiments, the authors try to show that current methods for measuring loss are inadequate. While they successfully show that their method is more accurate, they fail to demonstrate the impact of this error in packet loss measurement in protocol design. This provides some question as to the importance of the paper.
What are the main contributions of the paper and why are they important?
The authors provide an algorithm for estimating the probability of accurate measurements. That is, from the context of the data, they estimate the actual values, based on the (logical) assumption that packet loss is a continuous event of some duration where some packets are lost but others pass unscathed. To obtain these measurements, the authors launch a tightly packed series of probes that comprise a measurement slot. If any of these packets is lost, it is assumed that the entire slot is part of a loss event of some duration. These results are compared to the results of neighboring slots as above. The technique successfully measures loss event frequency and duration very accurately, relative to other tools.
How significant are these contributions relative to previous work?
The authors do a good job of improving on the accuracy of the Zing tool. They do not directly compare to other tools, although they note that most other tools are based on the same principles. They also note that the literature is ripe with details on how to conduct loss measurements, but does not provide guidance on how to interpret the data. Their algorithm automatically interprets the data very concisely. This is a vast improvement over prior work.
Give detailed comments justifying your view of the paper.
I call into question the importance of the work. While it is always pleasant to have accurate data upon which to design new work, the author's data itself shows a reduced importance. For example, the authors use TCP congestion control as an excellent reason to need accurate packet loss measurement. Modern TCP congestion control is designed with the current understanding of packet loss. Under conditions where packet losses are intermittant on each flow and do not generally involve consecutive packet losses, fast recovery is a very efficient mechanism. In their tests of Zing, the authors note that consecutive packet losses were extremely rare. Thus, despite having good measurement tools, TCP's congestion control has been tuned to deal well with the set of real loss characteristics determined by inline snooping. TCP Congestion Control authors didn't need good measurements to design good protocols.