Paper name: Internet Research Needs Better Models

Authors: Sally Floyd Eddie Kohler

 

This paper provides a legitimate wake-up call to the networking research community to do some retrospection and evaluate their models and simulation methodology. Authors argue that “Internet research needs better models”, which is also the paper title. They claim that the results of the evaluation of most Internet relevant ideas remain highly sensitive to the models and parameters chosen. They give an example of TCP behavior and more particularly of phase effects in active queue management, in which they demonstrate that how a small difference in parameters can affect the results drastically. Therefore they argue that in any model, while some divergence from reality are acceptable, research community must understand which aspects affect fundamental system behavior and which aspects can safely be ignored.

Authors propose four network model principles and argue that any evaluation of research ideas must adhere to them, so that the different results remain comparable and useful.

 

In the discussion, however, there remained a discord about the real contribution of the paper. Most people were skeptical about the idea of having a single compliant model for research. Some believed that having diversity in network models actually results in better insights and further improvements.

 

Another point of disagreement was the argument made by authors that software infrastructure used for evaluations should be kept updated and available to the entire research community. Some people believed that it might be an over kill for people who publishes lot of papers. Some were even skeptical about the usage of such a software/tool infrastructure.

 

In conclusion I will say that while the general consensus was that the paper certainly provides a good wake-up call but doesn’t do any contribution as such. Few people ranked this paper among the bottom third, while most of them ranked it among middle third.