WIXLIB

Proceedings of the 2003 Winter Simulation ConferenceS. Chick, P. J. Sánchez, D. Ferrin, and D. J. Morrice, eds.MODELING AND SIMULATION BEST PRACTICES FOR WIRELESS AD HOC NETWORKSLuiz Felipe PerroneYougu YuanDept. of Computer ScienceBucknell UniversityLewisburg, PA 17837-2029, U.S.A.Dept. of Computer ScienceDartmouth CollegeHanover, NH 03755-3510, U.S.A.David M. NicolDept. of Elec. and Computer EngineeringUniversity of Illinois Urbana-ChampaignUrbana, IL 61801-2918, U.S.A.ABSTRACTIt is therefore not surprising that computer simulationhas been used to develop much of the knowledge we havetoday in the performance of protocols for wireless networks. The vast majority of this body of work has beendeveloped with the assistance of only a few simulators,namely OPNET, ns-2, and GloMoSim. While these toolsare household names in the wireless networking community,other simulators have been developed to address specificresearch questions or to take the capabilities of the toolin directions not covered by their predecessors. One suchexample is our own SSF-based Simulator for Wireless AdHoc Networks (SWAN) (Liu et al. 2001), developed atthe Institute for Security Technology Studies, at DartmouthCollege. This simulator is being constructed to study thesecurity exposures of protocols for wireless networks.In the development of SWAN, we have looked forstarting points in the literature and in existing open sourcesimulators. While much of what we have found was ofgreat assistance, not all that we discovered was positive.In what regards model development, there seems to be along lag between the time when a model is embraced bythe community and the time when that model becomeswell-understood. Recent developments in the analysis ofmobility models serve to corroborate this observation (Yoonet al. 2003, Bettstetter et al. 2002). Also, concerning thesimulation methodology used in experimental studies inthis community, we have come across evidence of ampleroom for improvement. Discounting the eventual cases ofless-than-rigorous analyses of simulation output that endup published, there are also instances when establishedsimulation techniques are ignored resulting in studies withcompromised credibility. Our goal in this paper is to proposesteps toward averting the crisis of credibility that has beenThis paper calls attention to important practices in the modeling and the simulation of wireless ad hoc networks. Wepresent three case studies to highlight the importance offollowing well-established simulation techniques, of carefully describing experimental study scenarios, and, finally,of understanding assumptions sometimes unstated in theframework of a simulator. The first case addresses theinitial transient problem inherent to mobility and trafficgeneration sub-models. We quantitatively demonstrate howthese transients can affect the simulation. Our second caseillustrates the fact that strong scientific contributions canonly be made via simulation studies when the models usedare unambiguously specified. The example we use are simulations with and without a model for the ARP protocol.Finally, our third case discusses the importance of understanding the simulation tool and any default values used formodel parameters. The example used relates to the use ofthe limited interference model.1INTRODUCTIONIt is widely known that comprehensive models for wirelessad hoc networks are mathematically intractable. On its own,each individual layer of the protocol stack may be complexenough to discourage attempts at mathematical analysis. Interactions between layers in the protocol stack magnify thiscomplexity. In order to better understand these protocols,many have resorted to an experimental approach with running networks, which can quickly prove to be cumbersomeand expensive. For these reasons, computer simulation hasbeen the tool of choice to study wireless networks in ascalable, controllable and repeatable environment.685

Perrone, Yuan, and Nicolnally, Section 5 presents final thoughts and directions forfuture research.building up around simulations of computer networks, aspointed out by Pawlikowski et al. (2002).The study presented in this paper results from effortsto validate the models created for SWAN and to establishour own experimental methodology. In order to constructconsistent and meaningful scenarios for our experiments, weset out to determine parameters values for each sub-modelin our simulations. Surveying the literature, we discoveredthat, while certain parameter settings have been widely used,the reason for their choosing is most often not explained andtheir effects not understood. As we attempted to evaluate themotivation for these choices, we have chosen to investigatehow they impact our simulations.Note that although this paper has points in commonwith works such as Takai et al. (2001) and Heidemannet al. (2001), our main goals are quite different. We do notintend so much to evaluate the effects of detail as to callattention to the necessity of precisely and completely statingthe simulation scenarios used in any experimental workswith wireless networks. This is not to say that the resultspresented here don’t shed light into the matter of analyzingthe sensitivity of wireless network simulation models totheir various parameters. We present these results partlyto demonstrate this sensitivity phenomenon. Our resultsillustrate that without the detailed description of experiments,it is unlikely that anyone will be able to successfully replicateor build upon them.Understandably, many publications reporting the simulation of wireless networks do not disclose all the parametersettings for their studies. The media used for scientific communications, conference proceedings or journals, does notallow one to dedicate much space to listing parameter values.The obvious solution, which has been gaining popularityquickly, is to make this kind of information available in aweb page associated with the paper (though it is not obvioushow to make the information persistent). Our work followsthis practice and the detailed descriptions of our experiments and result data are available at http://www.eg.bucknell.edu/ perrone/research/. We hopethat this data can help the users of our simulator to becomecognizant of the characteristics of our models.The remainder of this paper is structured as follows.Section 2 discusses the importance of “warming up” modelsto avoid transients in the simulations. Although this is anage old practice, it is hardly referred to in most experimental studies of wireless networks. Next, in Section 3, weillustrate the importance of a thorough structural description of the simulation model. The case studied considersthe sensitivity of the simulation to the use of the AddressResolution Protocol (ARP) in the protocol stack model ofthe wireless node. Section 4 illustrates the dangers of usinga wireless network simulator without fully understandingits implementation. The case studied showcases the effectsof the radio propagation model on networking metrics. Fi-2THE INITIAL TRANSIENT PROBLEMIt is surprising that although basic methodology for computersimulation was established decades ago, many recent studiesstill pay little attention to it. The vast majority of papers onthe simulation of wireless networks seems to ignore the factthat one or more of their sub-models do require warm-uptime to avoid transient behavior. In this section we exploretwo different ways in which the initial transient problemmanifests itself in the simulation of wireless networks. Thefirst regards sub-models in the simulation environment that“drive” the networking nodes. The second regards directlythe sub-models that compose the simulated networking node.Our first case study is illustrated by the application of therandom waypoint model (RWM). Although our case studyfocuses on RWM, the issues we investigate are relevant toother mobility models as well. We hope that efforts suchas ours will serve to motivate the further analysis of othermobility models either through experimental or mathematical means. Next, we present an experimental exploratoryanalysis of RWM’s sensitivity to a few parameters.The algorithm behind RWM is very simple and isbased on only three parameters: pause time, min speed,and max speed. The time a mobile remains stationary between spurts of movement is deterministic and specified bypause time. All mobiles start out paused and begin to moveat the same time: the end of the initial pause. Each mobile then chooses a destination (waypoint) independentlyand uniformly, and travels toward it with a speed sampled uniformly from min speed to max speed, inclusively.Upon reaching a waypoint, the mobile pauses again and thealgorithm repeats.As shown by Yoon et al. (2003), although RWM is themost popular mobility model in the simulation analysis ofmobile wireless networks, it has significant perils. First, ifmin speed is set to zero or a very small value, the instantaneous node speed (a metric that quantifies the aggregatelevel of mobility) converges to zero as time advances. Theconsequence is obvious and severe: after a sufficiently longtime, what is meant to be a mobile network becomes stationary. Under these conditions, the simulation analysisof protocols for mobile wireless networks is likely to produce misleading results. Second, their work also exposedthe fact that for RWM, the level of mobility goes throughoscillations before settling down onto a “steady state”.In general, if the statistics collected in a simulation runinclude the initial transient period, it is likely that the resultswill exhibit considerable error. This effect, known as theinitial transient problem, is classically mitigated with theapplication of data deletion: statistics collected during thetransient are discarded and therefore have no influence in686