WIXLIB

Diwi j Dci Ri (t) wcij ri j (t)(5)(6)where ·c is the average concerning time in variable Dci andwcij . Ri (t) and ri j (t) are stochastic variables assumed to beGaussian distribution without a time correlation. MD andMw are constant numbers that mean amplitude of fluctuationof Ri (t) and ri (t) respectively. Therefore, equation(1) isreplaced in the following equations using equations(3-4): ui Vi Dci G Ri (t)G R′i (t), t ui(7)whereN (8)j 1, j̸ iri j (t)u j .(9)Equation (7) is a Langevin equation depending on globalfeedback by parameter G. The set of individuals, that carryout interactions i.e. exchange a pheromon, and its numberof individuals are defined as h and Nh , respectively. Underthe condition that the number of individual is large, that iswcij 1/(Nh 1) 1/Nh ,i h, the following equation issatisfied:N ujFig. 4. Transition of probability distributions Peq at bw 0.1, bs 0.9,b 0.4, M 0.005, MD 0.01 and Dci 0.01. Global parameter G transitsto G 0, G 0.1 and G 0.1.Changes in G are assumed to be slower than the timeevolution of ui . This is the adiabatic approximation foranalysis. When Pi is the probability distribution of ui , FokkerPlanck of equation (7) and its equilibrium solution areexplained as follows using normalization constant A: Pi t ui( ()) Vi Dci Gui Pi (MD G)2 M 2 ui uiPeq A expN j 1, j̸ i G 0G 0.1((wcij u j d)R′i (t) (Dci Ri (t))G - 0.1(3)(4)〈Ri (t1 )Ri (t2 )〉 2MD δ (t1 t2 )〈ri j (t1 )ri j (t2 )〉 2Mw δ (t1 t2 ),G Peqratio which is evolutionarily-conserved. The equations ofevolution (1) include stochastic variables depending on botha average number of contacts and which the individual hascontacts with other one. The potential of activation for statetransition from worker to soldier depend on Vb Vw . Thefluctuations are needed for transitions to climb the potentialof activation.In order to separate stochastic factor from average amount,Di and wi j are replaced in the following equations: (Vi Dci Gui ) (MD G)2 M 2)(11)Figure 4 shows the transition of Peq according to the changesof global parameter G. If G increases or decreases, thebalance of potential curve between bw and bs changesbased on the effect of Dci G as show in equation(11). The gradient also becomes loose by the effect of (MD G)2 M 2 .Therefore the changes in parameter G gives the force for statetransitions and the mechanism of fluctuation adjustment.IV. S IMULATIONS OF D IFFERANTIATIONA. Simulation Setupwcij u j 〈u〉,(10)j 1, j̸ i,i hwhere 〈u〉 is the average of ui for i h. If the number ofsoldiers that is included in set h decreases (increases), 〈u〉decreases (increases) to activate (inhibit) transitions fromworker to soldier. However the transition from perfectlytransformed soldier to worker is unacknowledged in termite. Ri (t) includes fluctuations depending on G. R′ (t)i isstochastic variables give steady fluctuations to the systemand depending on bw ，bs , b and d. R′ (t)i is time average 0and Gaussian distribution without time correlation as well asR(t) and r(t). When the caste ratio converges to the optimumvalue expressed in equation(10), as G approaches a certainvalue determined by balances between the potential Vi andinteraction terms in equation (1), the effect of fluctuationpromoting to the system approaches a certain value. As showin equation (7), the amplitude of fluctuation for transition areamplified or attenuated by G.Computer simulation in this research is carried out atthe behavior level of individuals. The overviews of thesimulation is shown in figure 5. The number of individuals isfixed at N 100 at any time. The spatial size of the colonyand the individual are given as a square, 2 on a side, and acircle of 0.05 in diameter. The initial states of all individualsare set to ui 0.1, that means all individuals in the colonyare set as workers. The caste of individuals is determined asworker and soldier when ui 0.5 and ui 0.5, respectively.Termite actually seems to fuss around. However what groupof termite are doing is unknown for caste differentiation. Inthis research, therefore, the individual migration is assumedas consistently uniform linear motion. When an individualcollide with other one or colony edges, the contacts areassumed as perfectly elastic collision. By these assumptions,each individual behave as molecular motion in closed space.At t 200, predators that has no relationship with termiteindividuals are put into the colony. It means that an incursion

E B. Results simulations of differentiation D C 9QTMGT 5QNFKGT 2TGFCVQTFig. 5. Simulation view at N 100, bw 0.1, bs 0.9, b 0.4 andd 0.3. White circles and black circles indicate the position of workersand soldiers, respectively. Triangles indicate predators put in at t 200. (a)at t 0. (b) at t 199. (c) at t 205.Cuiijjiijuj GHQTG EQPVCEV%QPVCEV#HVGT EQPVCEV GHQTG EQPVCEV%QPVCEV#HVGT EQPVCEVDFig. 6. Interactions between individuals. Circles indicate worker and soldierindividual. Dotted arrowed lines indicate a velocity of individuals. (a) Eachindividual give u at one interaction. (b)Interactions between individuals andpredators.of predators is treated as one of the environmental variation.The predators also behaviors at random as well as termiteindividuals.Figure 6 explains interaction rules among individuals inthe colony. When individuals contact each other, the theymutually transfer the hormone through pheromone interaction as shown in figure 6 (a). In this simulation, the amountof hormone ui is subsequently updated by using equation(1)and (2) every iteration. At the same time, ui is updatedwith Di 0 in case that the individual is contacting withan predators.When individuals contact with an predators, there areno transfer of hormone as shown in figure 6 (b). In thiscase, ui is constantly updated using equation(1) and (2) withDi 0. This assumption means that soldiers interacting withpredators lose the effort for pheromone release, so that Gdecreases and this effect brings results of state transition asabove a mathematical analysis in section III.The worker individuals neglect the existence of predatorsand the soldier individuals follow predators that meanssoldier attack them. The soldiers continue to follow predatorsuntil it turns from predators by reactive force from otherindividuals. These behaviors are advisedly imitated fromobservations of real termite behaviors.Based on above these conditions, the time evolution of uiand 〈u〉 are evaluated by computer simulations.The simulation results are shown in figure 15. As show infigure 15 (a), the average 〈u〉 is adaptively changed on theboundary of t 200. The number of soldier individuals areNs 33 and Ns 49 at t 150 and t 400, respectively.Although actual amount of hormone can not be negative,some u j are negative in the results. In oder to realize thatall values of u j are located in positive area at any time, theparameter set needs to be changed, however, this problemseems to be nonessential in this paper. The mathematicalanalysis advanced in section III do not need any changes sincethere are no condition of positive and negative for u j . Asshown in figure 15(b-d), the distributions of ui dynamicallychange adapting to an incursion of predators. At t 205,the variance of the distribution increase. The distributioneventually converge to the state of figure 15(d).In addition, in order to estimate the effects of fluctuations,transitions of G and variances of ui included Nw are shownin figure 8(a) and (b), respectively. The G temporarilydecreases around the interval 200 t 220 and the variancealso increases . It means that the inherent fluctuations areincreased and decreased according to incursion predators toconverge to evolutionarily-conserved optimal caste ratio. Infigure 8 (a), G demonstrate rapid fluctuation after t 200,although the its average value are recovered to previousvalue. It is thought that the rapid fluctuation depends on aspatial constraint after predators putting into a colony. Dci ,R′i and Ri in equation (2) must change and the fluctuationcharacteristic seems to change from radical one. In oder toclarification these reason, the more developmental analysisare needed.V. C RICKET F IGHTING B EHAVIOR ANDN EUROMODULATORSA. Cricket Fighting BehaviorWhile pheromone behaviors of insects was once considered ”hard-wired” and specific pheromones thought tocause specific behavior, it is becoming clear that the insectpheromone behavior involves plasticity by modification, as incricket fighting behavior. Figure 9 diagrams the informationpathway in the cricket brain.A cricket’s body is covered with cuticular substancesconsidered a so-called pheromone, probably for individualidentification [26]．A male cricket encountering anothercricket first touches it with its antennae to determine itsgender, then attempts courtship behavior if the other cricketis female or fighting behavior if it is male (Fig. 10).Such fighting behavior usually continues 15 to 30 minutes[27]. If the loser later encounters the winner or senses thesame pheromone, it shows avoidance behavior i.e., experiencing defeat causes the cricket to select different behaviorfor the same stimulus adaptively based on the circumstances.Understanding the mechanism behind the switching of aprogram behavior could lead to clarify neural adaptation.

Time400(a)200Fig. 9.0010uiInformation pathway in the cricket brain1Num. of units(b)0010ui1Num. of units(c)0010ui1Num. of units(d)00ui1Fig. 10.Fig. 7. Simulation results at N 100, bw 0.1, bs 0.9, b 0.4 anditeration size, t 0.05. The number of predators, Ne 0 at 0 t 200),Ne 20 at 200 t 400. (a)Time evolution of ui and 〈u〉. The distributionof ui are shown by the gray scale map. (b)Distribution of u at t 150.(c)Distribution of u at t 205. (d)Distribution of u at t 400(b)00Fig. 8. (a)Transition of G given by equation (9). (b)Variance of ui (i w).2wThe variance is given by σw2 ΣNi 1 (ui 〈uw 〉) .Cricket behavior selectionB. Neuromodulators and NO/cGMP cascadeOrganic neural systems involve a variety of neurotransmitters and NMs. NO, for one, is thought to play an important role in selecting program behavior from polymorphiccircuits. NO diffuses in the brain at about 100 [µ m/s] inthree dimensions to control neurotransmitter emission [28].Due to diffusion and reverse propagation, NO is generallyconsidered to be involved in neural plasticity as the basis oflearning and memory [29].The NO/cGMP cascade is considered particularly closelyrelated to fighting behavior selection by crickets. Fightingbehavior is observed when an NO synthesis inhibitor is injected into the head of a crickets. When two males encounterone another, fighting starts as usual and soon ends. If thesecrickets again encounter each another after a 15-minute interval, even the defeated cricket may exhibit fighting behaviorinstead of avoidance, indicating appropriate behavior is notselected if the NO/cGMP cascade functions normally in thebrain. Even under these circumstances, the pheromone isidentified and NO is closely related to pheromone behavior

Fig. 12.Model of action selection from NO/cGMP cascadethe AL is expressed as vector N of (n 1) dimensions andthat at position x (0 x n, x Z) as N x .N [NFig. 11.NO/cGMP signaling pathwaymodified by experience [30].In the NO/cGMP signaling pathway (Fig. 11), theNO/cGMP cascade mediates the efficacy of different neuralpathways, or circuits, which in turn affect the cricket’sphysiological condition. One change octopamine (OA) levelsin the brain, for example [31],[32]. As recent research indicates (Aonuma, unpublished data), the amount of OA differssignificantly before and after fighting and in winner’s andloser’s OA levels. The NO/cGMP cascade is thus considereddeeply related to behavior selection through OA.Given this background, we constructed a dynamic hypothetical model, discussed below.VI. M ODELING C RICKET B EHAVIOR S ELECTIONIn attempting to model NO/cGMP cascade function incricket fighting behavior taking account with the relationshipbetween OA concentration and behavior selection. Manyneurons are complexly related in behavior generation andthese relationships are difficult to reproduce accurately. Inadaptive behavior selection from insect’s programmed behavior however, describing relationships between substancesis sufficient, so we express each substance as a module.A. Internal State Modeling of NO/cGMP CascadeNoting a cricket’s internal state based on amounts of NO,cGMP, and OA, we modeled behavior selection in fightingbehavior (Fig. 12).Simply put our model takes two states which are aggression and avoidance. Basing the internal state on theamount of OA, aggression causes the cricket to select fightingbehavior and avoidance causes it to stay away from aprevious winning cricket.Based on sensory information from its antennae, a cricketfirst generates NO, and pheromones from another individualincrease the NO concentration in the antennal lobe (AL).When NO is generated in the AL, the target range isconsidered the membrane surrounding the AL. If neuralcircuits are considered to consist of modules, no informationis needed about neuronal regarding the linkage in the ALand only distance information is important. The AL is thusconsidered as a dimension of n[µm]. The amount of NO in0N1· · · N n](12)NO is a radical that reacts with metal ions immediatelyafter diffusion throughout the brain, after which it disappears,presumably present for about 10 seconds. To express thiseffect, the equation of NO diffusion with constant γN anddiffusion constant D is expressed as follows: 2N N D γN N Nin Nout(13) t x2where, Nin and Nout represent the amount of NO generation and consumption. Because NO mainly remains insidethe membrane enveloping the AL, the following reflectingboundary is given as a boundary condition:N 0 N 1,N n Nn 1(14)The NO diffusion property in the cricket brain is thenmodeled based on equations (13) and (14).cGMP in the NO target cell is generated when solubleguanylate cyclase (SGC) and guanosine triphosphate (GTP)mutually react, and SGC is driven by diffused NO from NOgenerating cells (Fig. 11). cGMP affects the amount of OAfor behavior selection function. The amount of cGMP in NOtarget cell C is expressed as a scalar value using the constantγC based on the idea that excessive amounts of cGMP aredecomposed in the body: C γCC Cin Cout(15) twhere Cin and Cout represent amounts of cGMP generationand consumption in the NO target cell.The cricket selects behavior based on the amount of OA, Aand OA is generated by cGMP and used to generate behavior.Likewise, the amount of OA is expressed by using constantγA : A γA A Ain Aout(16) twhere Ain and Aout represent the amount of OA generationand consumptionThe dynamics of each substances are modeled Based onequations (13),(15) and (16).In physiological experiments, technical constraints limitthe ability to measure NO concentration in the AL, sowe measure the NO concentration leaking from AL. The

that total NO consumption is used to generate cGMP.xCin Nout(19)xBased on physiological experiments, OA is divided intothat which is NO-independent and that which is NOdependent. cGMP suppresses OA production, so cGMPconsumption is modeled as a smooth function.(a) NO to cGMPCout b (1 tanh(10 (C 0.64))/2)(20)If the amount of OA, A, is normalized to [0:b], the amountof NO-dependent is about 0.4b (Aonuma, unpublished data).Ain b 0.6 Cout(21)Fig. 13-(b) shows an example of the mapping functionusing equations (20) and (21).OA is used in the cricket, to obtain energy from fat. Usingthe constant c, Aout is:{Aout (b) cGMP to OAFig. 13.Functions in our modelincreased NO concentration in the AL is certain, but thedegree of increase is unclear, so we assume that the amountof NO generation Ninx at position x is determined for positionset I at the source: 0.0 if x ̸ I,1.5 if (x I f ighting),(17)Ninx 1.0 otherwise.This means that the amount of NO generated when thecricket fights becomes 1.5 times that of the normal amountwithout fighting.Given that NO is completely consumed in cGMP production, NO consumption depends only on the NO concentrax at position xtion, so, the amount of NO consumption Noutis determined for position set O at the source. According to[29], when the amount of NO in the brain is increased bya NO donor, the amount of cGMP in the NO target cell isx asmultiplied by NO consumption. Modeling property Nouta smooth function such as equation (18),{a (1 tanh(50 (N x 0.4)))/2 if x O,xNout 0otherwise.(18)x is expressedwhere a is a constant and the amount of Noutas [0:a]. Fig. 13-(a) shows an example of the mappingfunction.As stated, the amount of NO consumption in the NO targetcells is related to cGMP generation. Expressing the amountof cGMP in the NO target cell as scalar value Cin , Cin equalsthe total amount of NO consumption. Equation (19) indicatesc if f ighting,0 otherwise.(22)This indicates that fighting behavior consumes largeamounts of energy.B. Fighting Behavior and Behavior SelectionBonabeau et al. reported the need of probability factorsfor ranking in a group [33] and Ashikaga et al. modeled theeffects of breeding density and fighting behavior by introducing probability factors into behavior selection [34],[35].A deterministic method appears suitable for analyzing theinternal state of a single cricket.As stated, the NO/cGMP cascade in the cricket brainaffects its physiological condition changing OA levels. OAis a biogenic amine used to extract energy from fat.The simplest model assumes behavior selection based onlyon the amount of OA. When modeling behavior selection,however, fighting behavior itself must be modeled. Assumingthat a cricket selects behavior at each step ( t[sec]) duringfighting, probability P of fighting in the next step is expressedas follows:P sgn(A Θ)(23)where fighting behavior is selected when A is over threshold Θ and avoidance behavior is selected in other cases.We also assume that penalty c is given to a defeatedcricket after fighting. So, the post-fighting state is obtaineddeterministically if the pre-fighting state is given.VII. S IMULATIONS OF B EHAVIOR S ELECTIONThe behavior of the proposed model was verified incomputer simulation in which the internal states of cricketswere compared for changes between victory and defeat.

(a) The winnerFig. 14.Density of NO in antennal lobeA. Simulation SetupAs stated, one position was determined for NO generationand consumption, and I {50} and O {60} were set. Withdiffusion coefficient D 1000，γN 1.0 10 1 , γC γA 5.0 10 4 was set as the effect where NO is diffused anddisappears in about 10 seconds by reacting with metal ions.Based on physiological experiments, the stationary stateof cGMP was C 0.6 and that of OA A 0.8. To satisfythese values, a 1.82 10 2 and b 4.91 10 4 were set.Hypothesis that NO operates as a type of switch, stationarystates were ignored. The values were set n 100， t 0.001, Θ 0.5 and c 0.1. Figure 14 gives NO stateswhen the cricket is in an ordinary non-fighting state andat 30 seconds after the start of fighting (ordinary stateduring fighting). With these settings, fighting behavior clearlyincreases NO levels in the brain and affects cGMP and OAvalues.(b) The loserFig. 15.Simulation result of internal state after fighting (Θ 0.5)B. Results of Behavior SelectionIn simulated of a single fight between two crickets andthe internal state transition in the simulated winner andloser (Fig. 15). Fighting behavior starts at time 3000 andcontinues for 15 seconds in (a) and 30 seconds in (b).The proposed model takes either or state of aggressionor one of avoidance, determined by the amount of OA(threshold Θ 0.5). With aggression state leading to theselection of fighting behavior or avoidance behavior leadingto non-fighting.This model accurately represent behavior in which thecricket exhibits avoidance behavior after a loss and fightingbehavior after sufficient time has passed. The effective timeeffect lasts just over 30 minutes, similar to actual observations, required for a defeated cricket to exhibit fightingbehavior or for recovery of A 0.5. This shows that timeeffects can be matched with actual data by varying γC andγA .Results confirms whether another response can be expressed. When NO inhibitor is injected into the cricket’sbrain, fighting behavior selection increases even though thecricket has been defeated [27] so NO inhibition weakens thatthe memory of past experience. In simulation utilizing samemodel, if NO synthesis is inhibited (Ninx 0), C becomes 0,and A 1 indicating that recovery will become shorter (Fig.Fig. 16. Simulation result of internal state of loser after fighting (NOgeneration inhibit

behavior selection modeling in section V-VIII. Finally, we conclude in section VIII. II. TERMITE CASTE DIFFERENTIATION The termite (Order Isoptera, Class Insecta) is categorized into a eusocial insect that lives in a group based on kinship and forms a colony with a certain size. In a termite colony, there are several castes called worker .