Selection of extra objectives Using reinforcement learning in non-stationary environment: Initial explorations

Using extra objectives in evolutionary algorithms helps to avoid getting stuck in local optima and increases genetic diversity. We consider a method based on a reinforcement learning (RL) algorithm that selects objectives in evolutionary algorithms (EA) during optimization. The method is called EA+RL. In some researches, reinforcement learning algorithms for stationary environments were used to adjust evolutionary algorithms. However, when properties of extra objectives change during the optimization process, we propose that it is better to use reinforcement learning algorithms which are specially developed for non-stationary environments. We present an initial research towards EA+RL for a non-stationary environment. A new reinforcement learning algorithm is proposed to be used in the EA+RL method. We also formulate a benchmark problem with some extra objectives, which behave differently at different stages of optimization. Thus, non-stationarity arises. The new algorithm is applied to this problem and compared with the methods which were used in other researches. It is shown that the proposed method chooses the extra objectives which are efficient at the current optimization stage more often and obtains higher values of the target objective being optimized.