Word-series high-order averaging of highly oscillatory differential equations with delay

We show that, for appropriate combinations of the values of the delay and the forcing frequency, it is possible to obtain easily high-order averaged versions of periodically forced systems of delay differential equations with constant delay. Our approach is based on the use of word-series techniques to obtain high-order averaged equations for differential equations without delay.


Introduction
We show that, for appropriate combinations of the values of the delay and the forcing frequency, it is possible to obtain easily high-order averaged systems of periodically forced systems of delay differential equations with constant delay.
The well-known theory of averaging [20,14] studies the reduction, by means of time-dependent changes of variables, of (nonautonomous) forced systems of differential equations to autonomous time-independent systems (averaged systems). Such a reduction is useful because autonomous systems are easier to analyze than their nonautonomous counterparts. In addition, the numerical integration of periodically or quasiperiodically forced systems may be expensive because typically integrators have to operate with step-sizes that are small with respect to the shortest period present in the forcing; in those cases, integrating an averaged version of the given system may be very advantageous.
In many occasions averaging is only carried out to first order. When the forcing is periodic this simply means replacing the right-hand side of the given oscillatory differential system by its average over one period of the forcing. In other cases, first order averaging is not enough and it is desirable to obtain averaged systems that provide more accurate approximations. For ordinary differential equations without delay the obtention of high-order averaging may be a daunting task, even with the help of a symbolic manipulator. The series of papers [6,7,8,9,15,10,18,19] has developed a technique that, in the absence of delays, makes it possible to compute high-order averaged systems through simple recursions. The technique is based on the theory of word series [16], formal series [21] that have numerous applications in the fields of deterministic or stochastic dynamical systems [17] and numerical integration [1,2].
The difficulties in obtaining high-order averaged systems are compounded if the system to be averaged has delays. In this paper we show that, for periodically forced differential systems with constant delay, it is possible to obtain high-order averaged systems by an application of the word-series results in [6,7,8,9,15,10,18,19]. The simple treatment presented here is only possible when the forcing period is a submultiple of the delay, a hypothesis whose scope is discussed in Section 3.
Section 2 recalls the word-series averaging of systems of differential equations without delay. Section 3 contains the main idea and Section 4 describes an application that shows how third-order averaging succeeds where second-order averaging fails. The final Section 5 concludes.

Preliminaries
In this section we summarize the word-series averaging of differential equations without delay [18]. This summary will provide the foundation to address later the case with delay.
Let us consider highly oscillatory initial value problems of the form Here the smooth function g(ξ, θ) is 2π-periodic in its second argument θ ∈ R, with Fourier expansion and the parameter Ω 1 is the angular frequency of the fast periodic forcing. The corresponding period is T = 2π/Ω.
In the theory of word series the set of indices Z in (2) is seen as an (infinite) alphabet and for n = 1, 2, . . ., the elements of Z n are regarded as words consisting of n letters of this alphabet. There is also an empty word ∅ with n = 0 letters. Given (1), to each word w one associates the corresponding word basis function g w , a map C D → C D . By definition, for words with one letter k ∈ Z, the word basis function g w (ξ) is the Fourier coefficient g k (ξ) in (2). For words with n > 1 letters, we define recursively where g k2···kn (ξ) is the Jacobian matrix of g k2···kn (ξ). For the empty word ∅, the basis function is the identity map ξ → ξ.
The set of all words (including the empty word) is denoted by W and C W refers to the set of all mappings δ : W → C. For δ ∈ C W and w ∈ W , δ w denotes the value of δ at w. To each δ ∈ C W we associate the corresponding word series (relative to the mappings g k in (2)); this is the formal series W δ (ξ) = w∈W δ w g w (ξ), whose terms are maps C D → C D . The complex numbers δ w are the coefficients of the series. As an example consider the case where δ ∅ = 1 and δ w = 0 for each nonempty word w; in this case W δ (ξ) = ξ.
As proved in [7,18] (see [19] for an alternative technique), there existβ ∈ C W and a 2π-periodic map θ ∈ R → κ(θ) ∈ C W such that the solution of (1) may be written as where Ξ(t) is the solution of In this way, the time-dependent change of variables ξ → Ξ in (4) transforms the given highly oscillatory initial value problem into the initial value problem (5) where there is no periodic forcing. Therefore (5) provides an averaged version of (1). In addition, the coefficients κ w (θ) are such that, at θ = 2kπ, k = 0, 1, 2, . . ., we have κ ∅ (θ) = 1 and κ w (θ) = 0 for each nonempty word w. This implies that, for those values of θ, the transformation ξ → Ξ is the identity map and it then follows that, in (4), ξ(t) = Ξ(t) at the strobocopic times, t = 0, T, 2T, . . . We then say that the change of variables is stroboscopic and that (5) is obtained from (1) through stroboscopic averaging. There are of course alternative forms of averaging; for instance one may impose the condition that the periodic change of variables ξ → Ξ has 0 average over one period, rather than being the identity at θ = 0. It is in order to point out thatβ and κ(θ) depend on Ω, but are otherwise universal in the sense that they do not change with the dimension D or with the choice of g(ξ, θ). Such a universality implies that they may be computed once and for all; averaging a new differential system requires the computation of new basis functions but not of new coeffcients. The valuesβ w and κ w (θ) for w ∈ W may be computed easily by recursion with respect to the number of letters in the word w; the reader is referred to [7,10,18] for details.
In general, the formal series Wβ(Ξ) in the averaged system (5) and the formal series in the change of variables (4) do not converge and have to be truncated and of course the truncation introduces an error, see [8,9]. For our purposes here we just mention that, if Wβ(Ξ) is truncated by eliminating all the terms in the series that correspond to words with more than n letters, one obtains an initial value problem whose solution Ξ(t) coincides at stroboscopic times with the solution ξ(t) except for an error of size O(1/Ω n ) as Ω → ∞. 1 The truncated averaged system with first order errors obtained by discarding contributions corresponding to words with two or more letters is found to be: as it may have been expected. The truncated averaged system with second order errors O(1/Ω 2 ) is given by [7,10,18] Note that, g 0 is by definition the word-basis function associated with the one-letter words 0. Furthermore, according to (3), the function g 0 g k is the word basis function associated to the word k0 and similarly g k g 0 , g k g −k are associated to 0k and −kk. Thus the right-hand side of (7) is indeed a word series (whose coefficients vanish for all words with 3 or more letters). The closed form expression for the third order averaged system for the problem (1) with general g may be seen in [7,10,18]. For order ≥ 4 it is not practical to find the general expression of the averaged system and then to apply it to the specific instance of (1) at hand. One should rather find recursively the word basis functions corresponding to the g of interest, perhaps with the help of a symbolic manipulator, see [15,19] for additional details.

Highly oscillatory problems with delay
We now consider the D-dimensional system with constant delay τ > 0 d dt where f (x, y, θ) is 2π-periodic in its third argument, with Fourier expansion and Ω is the angular frequency as above. Without loss of generality [22], it is assumed that the known function ϕ that specifies the initial history is independent of Ω. Reference [22] contains a list of references to problems of this form that appear in different applications.
The developments that follow are based on the standing hypothesis that the delay τ is an integer multiple of the period T , or in other words that t = τ is a stroboscopic time. Let us discuss this hypothesis. In some applications, the value of τ is given and . , x (L) ) of functions defined in the interval 0 ≤ t ≤ τ ; the four bottom subplots depict these in the case L = 3. Averaging the ordinary differential system without delay satisfied by the (x (0) , . . . , x (L) ) leads to a differential system without delay for the averaged solution (X (0) , . . . , X (L) ). The functions X ( ) (dotted lines) are patched together to get the solution X of the averaged delay problem that we wish to find.
the interest is in studying the behaviour of the solution of (8)-(9) for large Ω, but there is some freedom as to the choice of the specific value of Ω. In those cases, the theory below applies by choosing Ω in such a way that τ Ω/(2π) is an integer. Similarly, our theory applies to situations where the interest is in a given, large value of Ω and there is some freedom in the choice of τ to be used in the model. On the other hand, if the values of τ and Ω are fixed, the material in this paper does not apply. In those cases finding high-order averaged systems may be extremely complicated (see e.g. the computations in [22]).

The averaging procedure
Assume that (8)-(9) is to be studied in a bounded interval of the form 0 ≤ t ≤ Lτ for a suitable integer L > 0. (This hypothesis is made at this stage for mathematical convenience to avoid systems of infinitely many differential equations; the averaged systems that we will find will be valid for all t ≥ 0.) We introduce the functions (see Fig. 1).
and note that determining these functions is clearly equivalent to determining the solution of (8)- (9) in the interval 0 ≤ t ≤ Lτ . The x ( ) (t) with > 0 satisfy the conditions and the differential equations Even though the relations (13) may be reminiscent of a two-point boundary value problem, we are facing here an initial value problem: x (1) (t) is determined by solving the differential equation with = 1 with initial condition is determined by solving the differential equation with = 2 with initial condition x (2) (0) = x (1) (τ ), etc. By taking into account the periodicity of f (x, y, θ) as a function of θ and our standing hypothesis that τ is a multiple of the period, the last display may be written as Note that this would be a system of the form (1) for the D × L-dimensional vector (x (1) , . . . , x (L) ) if it were not for the fact that the right-hand side of the equation corresponding to = 1 contains x (0) , a known function of t given in (11). In order to have a system of the form (1), where the right-hand side only depends on t through the combination θ = Ωt, we proceed as follows. We introduce the 1 + D(L + 1)-dimensional vector of unknown functions of the variable t ξ = ( t, x (0) , x (1) , . . . , x (L) ), and add to (14) the differential equations (the dot represents differentiation) subject to the initial conditions The solution of the initial value problem (15)-(16) is obviously t = t and x (0) (t) = ϕ(t − τ ). Now (15) in tandem with (14) is a 1 + D(L + 1)-dimensional system of the form (1) that may be stroboscopically averaged by following the procedure described in the previous section. If X ( ) is the averaged counterpart of x ( ) , we are interested in the solutions of the averaged system that satisfy (cf. (13)) so that it is possible to define a continuous function X in the interval [−τ, Lτ ] by patching together the different X ( ) , thus undoing the process that we used to move from the solution x of (8)-(9) to the functions x ( ) in (11)-(12), see Fig. 1. In this way the averaged delay equation is obtained by patching the equations for the X ( ) . The whole procedure will be clear after we present the simple case of the first-order averaged system.

The first-order averaged system
If g denotes the right-hand side of the oscillatory system of differential equations (14)-(15) satisfied by ξ, the Fourier series for g(ξ, θ) (cf. (2)) has coefficients . . .
where the f k are the Fourier coefficients of f as in (10). According to (6), the firstorder averaged system of differential equation without delay in the interval 0 ≤ t ≤ τ is therefore given by (capital letters denote averaged dependent variables) This system is to be solved with the conditions T (0) = 0, X (0) (0) = ϕ(−τ ) and (17). Clearly T = t, X (0) (t) = ϕ(t − τ ) and therefore the function X in (18) satisfies the averaged delay problem (cf. (8)-(9)) d dt as one could have easily guessed. Note that L does not feature in the averaged problem, as we pointed out above.
The solutions x and X of (8)- (9) and (19)- (20) differ at stroboscopic times by O(1/Ω) as we prove next. That x(t) − X(t) = O(1/Ω) at stroboscopic times t ≤ τ follows by considering that, for those values of t, x(t) = x (1) (t), X(t) = X (1) (t) and x (1) (t) − X (1) (t) = O(1/Ω) because of the properties of stroboscopic averaging of ordinary differential equations without delay. For stroboscopic times t in the interval τ < t ≤ 2τ the situation is slightly more complicated because in addition to the error O(1/Ω) arising from truncating (5) there is an additional source of error: in this interval x(t) = x (2) (t + τ ), X(t) = X (2) (t + τ ) but x (2) (t) and X (2) (t) do not share the same initial value at t = 0. However the difference of initial values is , which we know is of size O(1/Ω). It follows that x(t) − X(t) = O(1/Ω) also at stroboscopic times τ < t ≤ 2τ . The iteration of this argument leads to the conclusion we seek, i.e. x(t) − X(t) = O(1/Ω) whenever t is a stroboscopic time. Of course the constant implied in the O notation of course depends on t (and in general will grow as t increases).

Second-order averaged system
If rather than using the truncated averaged system (6), we use (7), the procedure described above leads, after some simple algebra, to an averaged problem given by (20) and a delay differential equation of the form d dt with and valid for t ≥ τ . In these expressions ∂ x f k (x, y) and ∂ y f k (x, y) denote the Jacobian matrices of f k (x, y) with respect to x and y respectively. By arguing as in the first order case, one proves that at stroboscopic times the difference between the solution x(t) of (8)-(9) the solution X of (20)- (22) is O(1/Ω 2 ).

Third-and higher order averaged systems
It is still feasible to obtain in closed form the third-order averaged system for delay problems starting from the corresponding expression for the case without delay given in [7,10,18]. The expression of the averaged delay system one obtains in this way is lengthy and will not be reproduced here. In fact for order 3 or higher, the best approach is to find recursively the word basis functions for the system (14)- (15) for the specific f of interest. For an averaged system of order n, the right-hand side of the system has n expressions corresponding to the intervals 0 ≤ t < τ , . . . , (n − 2)τ ≤ t < (n − 1)τ , and t ≥ (n − 1)τ . (Recall that the second-order averaged system displayed above has two expressions.)

An example
As an illustration, we consider the system where α and β are parameters. There are two periodic forcing terms; the forcing A sin(ωt) is slow (i.e. the frequency ω is of moderate size) and B sin(Ωt), Ω 1, is fast. When there is no forcing (A = B = 0), the system represents a delayed genetic toggle switch, a synthetic gene regulatory network [12]. The forced system was studied in [11] as an instance of the emergence of vibrational resonance [13,15], i.e. the enhancement, due to the presence of the fast forcing, of the response of the system to the slow forcing.
In order to have a system of the form (8) it is necessary to rewrite A sin(ωt) as A sin(ωt), wheret is a new dependent variable defined by the differential equation (d/dt)t = 1 and the initial conditiont = 0 at t = 0 (this is of course the technique used above in (15)- (16) to deal with the slow dependence on t of the right-hand side of (14)). Thus only the fast forcing is averaged. After computing recursively the required word basis functions and coefficients, the third-order averaged system is found to be given by for 0 ≤ t < τ , and for t ≥ τ . (As pointed out above, in general, the third-order averaged system has an analytic expression in τ ≤ t < 2τ and a different analytic expression in t ≥ 2τ ; for the problem at hand those two expressions happen to coincide.) The second-order averaged system may be retrieved from the last displays by omitting the terms that have an Ω 2 factor in the denominator. We have integrated numerically the true dynamics and the second-and third-order averaged systems. The integrations were carried out with the Matlab function dde23 with relative and absolute tolerances 10 −8 and 10 −10 respectively. For the choice α = 2.5, β = 2, A = 0.1, ω = 0.1, B = 2.0, and τ = 0.5, with constant history u(t) = 0.5, v(t) = 2.0, −τ ≤ t ≤ 0 (an equilibrium of the unforced system), Table 4 presents the maximum error in u at stroboscopic times in the short interval 0 ≤ t ≤ 2; in agreement with the theory, the error behavior is O(1/Ω 2 ) for second-order averaging and O(1/Ω 3 ) for third-order averaging.
In Fig. 2, the parameters are α = 2.5, β = 2, A = 0.2, ω = 0.2, B = 2.0, Ω = 4π and τ = 0.5, with constant history u(t) = 2.0, v(t) = 0.5, −τ ≤ t ≤ 0 (again an equilibrium of the unforced system). The integration is perfomed in the interval 0 ≤ t ≤ 100. The numerical integration of the oscillatory problem needed 261.0 seconds, a quantity that has to be compared with the 2.1 and 2.6 seconds required to integrate the second-and third-order averaged systems respectively. In studies like those performed in [11], where the oscillatory systems has to be integrated in long time intervals for many different choices of the values of the parameters, the advantage of averaging is then clear.

Conclusion
We have showed that, when the delay is a multiple of the forcing period, it is possible to extend to periodically forced, constant delay problems, the word-series approach to the systematic derivation of high-order averaged systems. We have presented an example where the new technique has been applied to a system of interest in connection with the phenomenon of vibrational resonance. To conclude let us point out that, for ordinary differential equations, it is possible to compute numerically a stroboscopically averaged solution Ξ without the explicit knowledge of the corresponding averaged system; the information on the averaged system required by the integrator is derived by numerically simulating the oscillatory system (1) [3,4]. Such techniques have been extended to delay differential equations [22,5].