Comparison of Dynamic Games in Application to Safe Ship Control

Abstract The paper introduces methods of dynamic games for automation of ship control in the collision situation, the game control processes in marine navigation and the fundamental mathematical model of the game ship control. First, state equations, control and state constraints and then control goal function in the form of payments: the integral payment and the final one, have been defined. Multi-stage positional, and multi-step matrix, non-cooperative and cooperative, game and optimum control algorithms for a collision situation, have been presented. Te considerations have been illustrated with an exemplary computer simulation of algorithms to determine a safe own ship's trajectory in the process of passing the ships encountered in Kattegat Strait.


INTRODUCTION
The control of the ship's movement may be treated as a multilevel problem ( Fig. 1) which results from the division of the entire control system of ship -within the frame of shipping the cargo by ship's operator -into clearly determined subsystems to which appropriate layers of control are ascribed.

Fig. 1. Multilevel ship movement control system
This is connected both with a large number of dimensions of the control vector and status of the process, its random, fuzzy and decision-making characteristics -which are affected by strong interference resulting from sea current, wind and wave motion on the one hand, and a complex nature of the equations describing the ship's dynamics with nonlinear and non-stationary characteristics. The determination of the global control of the steering systems has in practice become too costly and ineffective.
The integral part of the entire system is the process of the ship's movement control, which may be described with appropriate differential equations of the kinematics and dynamics of a ship being an object of the control under a variety of the ship's operational conditions such as: -stabilisation of ship course or trajectory, -adjustment of ship's speed, -precise steering at small speeds in port with the use of thrusters or adjustable-pitch propeller, -stabilisation of ship's rolling, -commanding the towing group, -dynamic positioning of ship.
The functional diagram of the system corresponds to an actual arrangement of the equipment. The increasing demands with regard to the safety of navigation force the ship's operators to install integrated navigation systems on board their ships. By improving the ship's control, such systems rise the navigation safety of ship, a very expensive object, whose value includes also the shipped cargo, and improve the effectiveness of cargo shipping by sea [3].

SAFE SHIP CONTROL
The challenge in search for effective methods to prevent ship collisions has become crucial with the increasing of size, speed and number of ships participating in sea transport. An obvious contribution in increasing safety at sea has been first of all the application of radars and then the development of ARPA (Automatic Radar Plotting Aids) anti-collision system [4].
The ARPA system enables to track automatically at least 20 encountered j-objects, as shown in Fig. 2, in order to determine their movement parameters (speed Vj, course ψj) and elements of approach to the own ship  The risk value (1) is possible to be defined by referring the current situation of approach, described by parameters j min D and j min T , to the assumed evaluation of the situation as safe, determined by a safe distance of approach Ds and a safe time Ts -which are necessary to execute a collision avoiding manoeuvre with consideration of distance Dj to j-th met ship (Fig. 3).  The weight coefficients α 1 and α 2 are dependent on state of visibility at sea, the dynamic length L d and dynamic beam B d of the ship and a kind of water region. They are in practice equal to: The functional scope of a standard ARPA system ends with the trial manoeuvre by altering the course ψ ∆ ± or the ship's speed V ∆ ± selected by the navigator, as shown in Fig. 4. The problem of selecting such manoeuvre is very difficult as the control process is very complex due to its dynamic, non-linear, multi-dimensional, non-stationary and game making character.
In practice, methods of selecting a manoeuvre assume a form of appropriate steering algorithms supporting navigator decision in a collision situation. The algorithms are inserted into the memory of a Programmable Logic Controller PLC. This provides an option within the ARPA anti-collision system or a training simulator [11,13].
There are various methods to avoid ships collision. The simplest method is determination of the manoeuvre of a change in course or speed of own ship in relation to the most dangerous ship to be encountered. A more effective method is to determine safe trajectory of the ship. Most adequate to the real character of control process is determination of a game trajectory of the ship [5,8,16,19].

GAME CONTROL
The classical issues of the theory of the decision process in marine navigation include the safe steering of a ship. The problem of non-collision strategies in the steering at sea appeared in the Isaacs' work [9] called "the father of the differential games" and was developed by many authors both within the context of the game theory and also in the steering under uncertainty conditions [1,2,10,14].
The definition of the problem of avoiding a collision seems to be obvious, however, apart from the issue of the uncertainty of information which may be a result of external factors (weather conditions, sea state), incomplete knowledge about other objects and imprecise nature of the recommendations concerning the right of way contained in International Regulations for Preventing Collision at Sea COLREG [6].
The problem of determining safe strategies is still an urgent issue as a result of an ever increasing traffic of vessels on particular water areas. It is also important due to the increasing requirements as to the safety of shipping and environmental protection, from one side, and to the improving opportunities to use computer supporting the navigator's duties. In order to ensure safe navigation, the ships are obliged to observe legal requirements contained in the COLREG Rules.
However, the Rules refer exclusively to two ships under good visibility condition. In case of restricted visibility the Rules provide only recommendations of a general nature and they do not cover all necessary conditions of the real process. Therefore, in the light of the legal regulations, the real ship passing process occurs under the conditions of indefiniteness and conflict accompanied by an imprecise co-operation among the ships.
Consequently, it is reasonable for ship operational purposes, to present this process and to develop and examine methods for a safe steering of the ship, by applying the rules of the game theory.
A necessity to consider simultaneously the strategies of the encountered objects and the dynamic properties of the ships as the steering objects is a good reason to apply the differential game model, often called the dynamic game, for description of the processes [12,15,18].

PROCESSES OF GAME CONTROL
It is assumed that the dynamic movement of the ships in time occurs under the influence of the appropriate sets of steering: where: -a set of the own ship's strategies, Θ -a set of the j-th ship's strategies, -denotes course and trajectory stabilisation, -denotes the execution of the anti-collision manoeuvre in order to minimize the risk of collision, which in practice is achieved by satisfying the following inequality: -the smallest distance of approach of the own ship and the j-th encountered object, D s -safe approach distance in the prevailing conditions depending on the visibility conditions at sea, the COLREG Rules and the ship's dynamics. D j -current distance to the j-th object taken from the ARPA anti-collision system.
-refers to the manoeuvring of the ship in order to achieve the closest point of approach, for example during the approach of a rescue vessel, transfer of cargo from ship to ship, destruction of the enemy's ship, etc.). Using the adopted describing symbols we can distinguish the following type of steering the ship in order to achieve a determined goal: -basic type of steering-stabilization of the course or trajectory: Dangerous situations resulting from a faulty assessment of the approaching process by one of the party with the other party's failure to conduct observation -one ship is equipped with a radar or an anti-collision system, the other with a damaged radar or without such device.

BASIC MODEL
As the process of steering the ship in collision situations, when a greater number of objects is encountered, often occurs under the conditions of indefiniteness and conflict, accompanied by an inaccurate co-operation of the ships within the context of COLREG Regulations, therefore the most adequate model of the process which has been adopted , is a model of a dynamic game of j tracked ships , in general, as objects of steering.
The diversity of selection of possible models directly affects the synthesis of the ship's handling algorithms which are afterwards effected by the ship's handling device directly linked to the ARPA system and, consequently, determines the effects of the safe and optimum control.
The most general description of the own ship passing the j-number of other encountered ships is the model of a differential game of a j-number of objects, shown in Fig. 5.
The properties of the process are described by the state equation: where:   -speed of the j-th object, or force of the propeller thrust of the j-th object, (8) where: Values of coefficients of the process state equations (8) for a 12 000 DWT container ship are given in Tab. 1.

Tab. 1. Coefficients of basic game model equations.
In the case of j = 20 met ships, the basic game model is represented by i = 86 state variables of process control.
The constraints of the control and the state of the process are connected with the basic condition for the safe passing of the objects at a safe distance D s in compliance with COLREG Rules, which is generally expressed in the following form: The constraints referred to as the ships domains in the marine navigation, may take a shape of a circle, hexagon, parabola or ellipse, and be generated, for example, by an artificial neural network, as shown in Fig. 6, [2]. The synthesis of the decision-making pattern of the object control leads to the determination of the optimum strategies of the players who determine the most favourable, under given conditions, conduct of the process. For the class of non-coalition games, often used in the control techniques, the most beneficial conduct of the own control object as a player with j-th object is the minimization of its goal function in the form of the payments -the integral payment and the final one: (10) The integral payment represents loss of way by the ship while passing the encountered objects and the final payment determines the final risk of collision r j (t k ) relative to the j-th object and the final deflection of the ship d(t k ) from the reference trajectory.
Generally two types of the steering goals are taken into consideration -programmed steering u 0 (t) and positional steering u 0 [x 0 (t)]. The basis for the decision-making steering are the decision-making patterns of the positional steering processes, the patterns with the feedback arrangement representing the dynamic games.
The application of reductions in the description of the own ship's dynamics and the dynamics of the j-th encountered ship and their movement kinematics leads to synthesis of game ship control algorithms in collision situations.

Multi-stage positional game
The general model of dynamic game is simplified to the multi-stage positional game of j participants not co-operating with each other [7]. j  j  1  ,  j  2  ,  0  1  ,  0   j  2  ,  j  j  1  ,  j  0  2  ,  0  0  1 , 0 (11) The essence of the positional game is to subordinate the strategies of the own ship to the current positions p(t k ) of the encountered objects at the current step k. In this way the process model takes into consideration any possible alterations of the course and speed of the encountered objects while steering is in progress. The current state of the process is determined by the co-ordinates of the own ship's position and the positions of the encountered objects: (12) The system generates its steering at the moment t k on the basis of data received from the ARPA anti-collision system pertaining to the positions of the encountered ships: (13) It is assumed, according to the general concept of a multistage positional game, that at each discrete moment of time t k the own ship knows the positions of the objects.

State variables and control values are represented by:
The constraints for the state co-ordinates: (14) are navigational constraints, while steering constraints: (15) take into consideration: the ships' movement kinematics, recommendations of the COLREG Rules and the condition to maintain a safe passing distance as per relationship (6).
The closed sets  (16) are dependent, which means that the choice of steering u j by the j-th object changes the sets of acceptable strategies of other ships.

Multi-stage non-cooperative positional game algorithm pg_nc
The optimum steering of the own ship u relative to each j-th ship is determined and then the optimum positional strategy for the own ship ) p ( u 0 * -from the condition (10). (17) The function S 0 refers to the continuous function of the manoeuvring goal of the own ship, characterizing the distance S of the ship at the initial moment t 0 to the nearest turning point L k on the reference p r (t k ) route of the voyage.
The optimum control of the own ship is calculated at each discrete stage of the ship's movement by applying the Simplex method to solve the problem of the triple linear programming, assuming the relationship (17) as the goal function and the control constraints (9).

Multi-stage cooperative positional game algorithm pg_nc
The quality index of control for a cooperative game has the form:

Multi-step matrix game
When leaving aside the ship's dynamics equations, the general model of a dynamic game for the process of preventing collisions is reduced to the matrix game of j-participants non-co-operating with each other [17,20].
The state and steering variables are represented by the following values: includes the values of the collision risk rj determined from relation (1) on the basis of data obtained from the ARPA anti-collision system for the acceptable strategies uo of the own ship and acceptable strategies uj of any particular number of j -encountered objects.
The problem of determining an optimum strategy may be reduced to the task of solving Simplex dual linear programming method. Mixed strategy components express the distribution of probability

Multi-step non-cooperative matrix game algorithm mg_nc
As a result of using the following form for the control goal: the probability matrix P=[pj (u0,uj)] of using particular pure strategies may be obtained.
The solution for the control problem is the strategy representing the highest probability:

Multi-step cooperative matrix game algorithm mg_c
The quality index of control for a cooperative game has the form:

COMPUTER SIMULATION
Computer simulation of control game algorithms were carried out on an example of a real navigational situations of passing j = 25 encountered ships. The situations were registered , in Kattegat Strait on board r/v HORYZONT II, owned by Gdynia Maritime University, on the radar screen of the ARPA anti-collision system Raytheon ( Fig. 7 and 8). Examples of safe positional game trajectories are shown in Fig. 9 and 10.   The biggest impact on the amount of the final game payment considered as the final deviation from the reference trajectory, has a degree of co-operation of ships in avoiding collision.

CONCLUSIONS
The application of the models of a game theory for the synthesis of an optimum manoeuvring makes it possible to determine the safe game trajectory of the own ship in situations when she passes a greater number of the encountered ships.
The developed algorithms takes also into consideration the COLREGS Rules and the advance time of the manoeuvre approximating ship's dynamic properties and evaluates the final deviation of the real trajectory from the reference one.
The positional game control algorithms determine game and safe trajectory of the own ship with relation to all encountered ships.
The matrix game control algorithms determine game and safe trajectory of the own ship with relation to the most dangerous ship.
To sum up, it may be stated that the control methods considered in this study are, in certain sense, formal models for the thinking processes of a navigating officer steering own ship and making decisions on manoeuvres.
Therefore they may be applied to the construction of a new model of ARPA system containing a computer which supports the navigator's decision-making process.