Research Seminar - February 15, 2002

Abstract:

In many other areas of science and engineering, proliferation of computers as research tools has resulted in the adoption of computer simulation as the most commonly used paradigm of scientific investigations. This together with a plethora of existing simulation languages and packages, has created a popular opinion that simulation is mainly an exercise in computer programming. In new computing environments programming can be minimised, or even fully replaced, by manipulation of icons (representing pre-built programming objects with basic functional blocks of simulation systems) on a computer monitor. One can say that we have witnessed another success of modern science and technology: an emergence of wonderful and powerful tools for exploring and predicting behaviour of such complex, stochastic dynamic systems as the Internet and related computer networks.

However this enthusiasm is not shared by all researchers in this area. An opinion is spreading that one cannot rely on the majority of the published simulation-based results on performance evaluation studies of telecommunication networks, including the Internet in particular, since they lack credibility. Some are talking about a deep credibility crisis. This opinion can be supported, for example, by the results of a survey of over 2200 publications on the Internet and other telecommunication networks in recent proceedings for the INFOCOM (an annual IEEE Int. Conference on Computer Communications), ACM/IEEE Trans. on Networking, IEEE Trans. on Communications, and Int. J. on Performance Evaluation.

We will discuss the main issues that influence the credibility of performance evaluation studies based on computer simulation, focusing on the quality of pseudo-random number generators and statistical accuracy of the final results. Finally, we will formulate guidelines that, if observed, could help to assure a basic level of credibility of simulation studies of telecommunication networks and, of course, any other stochastic dynamic system.