Levenberg-Marquardt Algorithm for Mackey-Glass Chaotic Time Series Prediction
Levenberg-Marquardt Algorithm for Mackey-Glass Chaotic Time Series Prediction
Junsheng Zhao,1 Yongmin Li,2,3 Xingjiang Yu,1 and Xingfang Zhang1
1School of Mathematics, Liaocheng University, Liaocheng 252059, China
2School of Science, Huzhou University, Huzhou 313000, China
3School of Automation, Southeast University, Nanjing 210096, China
Received 9 August 2014; Accepted 11 October 2014; Published 11 November 2014
Academic Editor: Rongni Yang
Copyright © 2014 Junsheng Zhao et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
For decades, Mackey-Glass chaotic time series prediction has attracted more and more attention. When the multilayer perceptron is used to predict the Mackey-Glass chaotic time series, what we should do is to minimize the loss function. As is well known, the convergence speed of the loss function is rapid in the beginning of the learning process, while the convergence speed is very slow when the parameter is near to the minimum point. In order to overcome these problems, we introduce the Levenberg-Marquardt algorithm (LMA). Firstly, a rough introduction is given to the multilayer perceptron, including the structure and the model approximation method. Secondly, we introduce the LMA and discuss how to implement the LMA. Lastly, an illustrative example is carried out to show the prediction efficiency of the LMA. Simulations show that the LMA can give more accurate prediction than the gradient descent method.
1. Introduction
The Mackey-Glass chaotic time series is generated by the following nonlinear time delay differential equation: ? ? ( ? ) = ? ? ? ? ( ? − ? ) 1 + ? ? ( ? − ? ) + ? ? ( ? ) , ( 1 ) where ? , ? , ? , and ? are real numbers. Depending on the values of the parameters, this equation displays a range of periodic and chaotic dynamics. Such a series has some short-range time coherence, but long-term prediction is very difficult.
Originally, Mackey and Glass proposed the following equation to illustrate the appearance of complex dynamics in physiological control systems by way of bifurcations in the dynamics: ? ? = ? ? ? ? ? 1 + ? ? ? − ? ? , ? , ? , ? > 0 . ( 2 )
They suggested that many physiological disorders, called dynamical diseases, were characterized by changes in qualitative features of dynamics. The qualitative changes of physiological dynamics corresponded mathematically to bifurcations in the dynamics of the system. The bifurcations in the equation dynamics could be induced by changes in the parameters of the system, as might arise from disease or environmental factors, such as drugs or changes in the structure of the system [1, 2].
The Mackey-Glass equation has also had an impact on more rigorous mathematical studies of delay-differential equations. Methods for analysis of some of the properties of delay differential equations, such as the existence of solutions and stability of equilibria and periodic solutions, had already been developed [3]. However, the existence of chaotic dynamics in delay-differential equations was unknown. Subsequent studies of delay differential equations with monotonic feedback have provided significant insight into the conditions needed for oscillation and properties of oscillations [4–6]. For delay differential equations with nonmonotonic feedback, mathematical analysis has proven much more difficult. However, rigorous proofs for chaotic dynamics have been obtained for the differential delay equation ? ? / ? ? = ? ( ? ( ? − 1 ) ) for special classes of the feedback function ? [7]. Further, although a proof of chaotic dynamics in the Mackey-Glass equation has still not been found, advances in understanding the properties of delay differential equations is going on, such as (2), that contain both exponential decay and nonmonotonic delayed feedback [8]. The study of this equation remains a topic of vigorous research.
The Mackey-Glass chaotic time series prediction is a very difficult task. The aim is to predict the future state ? ( ? + Δ ? ) using the current and the past time series ? ( ? ) , ? ( ? − 1 ) , … , ? ( ? − ? ) (Figure 2). Until now, there are many literatures about the Mackey-Glass chaotic time series prediction [9–14]. However, as far as the prediction accuracy is concerned, most of the results in the literature are not ideal.
In this paper, we will predict the Mackey-Glass chaotic time series by the MLP. While minimizing the loss function, we introduce the LMA, which can adjust the convergence speed and obtain good convergence efficiency.
The rest of the paper is organized as follows. In Section 2, we describe the multilayer perceptron. Section 3 introduces the LMA and discusses how to implement the LMA. In Section 4, we give a numerical example to demonstrate the prediction efficiency. Section 5 is the concl (...truncated)