Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting PM2.5

Neural Comput & Applic DOI 10.1007/s00521-015-1955-3 ORIGINAL ARTICLE Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting PM2.5 Bun Theang Ong1 • Komei Sugiura1 • Koji Zettsu1 Received: 11 November 2014 / Accepted: 5 June 2015  The Author(s) 2015. This article is published with open access at Springerlink.com Abstract Fine particulate matter (PM2:5 ) has a considerable impact on human health, the environment and climate change. It is estimated that with better predictions, US$9 billion can be saved over a 10-year period in the USA (State of the science fact sheet air quality. http:// www.noaa.gov/factsheets/new, 2012). Therefore, it is crucial to keep developing models and systems that can accurately predict the concentration of major air pollutants. In this paper, our target is to predict PM2:5 concentration in Japan using environmental monitoring data obtained from physical sensors with improved accuracy over the currently employed prediction models. To do so, we propose a deep recurrent neural network (DRNN) that is enhanced with a novel pre-training method using auto-encoder especially designed for time series prediction. Additionally, sensors selection is performed within DRNN without harming the accuracy of the predictions by taking advantage of the sparsity found in the network. The numerical experiments show that DRNN with our proposed pre-training method is superior than when using a canonical and a state-of-the-art auto-encoder training method when applied to time series prediction. The experiments confirm that when compared against the PM2:5 prediction system VENUS (National & Bun Theang Ong Komei Sugiura Koji Zettsu 1 Information Services Platform Laboratory, Universal Communication Research Institute, National Institute of Information and Communications Technology, 3-5 Hikaridai, Seika-cho, Kyoto, Soraku-gun 619-0289, Japan Institute for Environmental Studies. Visual Atmospheric Environment Utility System. http://envgis5.nies.go.jp/ose nyosoku/, 2014), our technique improves the accuracy of PM2:5 concentration level predictions that are being reported in Japan. Keywords Time series prediction  Deep learning  Pre-training  Recurrent neural networks  Elastic net  Fine particulate matter  Environmental sensor data 1 Introduction Air pollution remains a serious concern and has attracted the attention of industries, governments, as well as the scientific community. One type of air pollutant that has attracted immense attention is fine particulate matter or PM2:5 —particles \2.5 lm. PM2:5 is a widespread air pollutant, consisting of a mixture of solid and liquid particles suspended in the air. Thus, PM2:5 is a global issue that transcends geographical boundaries and calls for an interdisciplinary approach to solve a global problem, around which both industries and governments should play an active role. The environmental and health impacts [1, 2] of PM2:5 are well documented [3–5]. Organizations and governments such as the World Health Organization [6], the USA Environmental Protection Agency (EPA) [4], UK [7], Japan [8], to mention a few, have implemented policies to support clean air in their respective towns and cities [5]. Today, most of the major air quality indexes, such as the Pollutant Standards Index (PSI) or the Air Quality Index (AQI), take into account the concentrations of PM2:5 in their equations. These indexes were developed in order to provide the public with an indicator of how polluted the air 123 Neural Comput & Applic is, along with the health implications that each level may imply. Very often, recommendations are also provided to the public. In December 2012, the EPA decided to strengthened its air quality standards by revising the role of PM2:5 concentrations on the AQI. Concretely, the upper end of the range for the ‘‘Good’’ category has changed from the level of 15.0 lg per cubic meter (lg=m3 ) to 12.0 lg=m3 . This is a difference of only 3 lg=m3 . But in the eye of the EPA, this difference was enough to judge the previous value as not adequate to protect the public health, as required by law. Now, what is the validity of this new enforcement if the current PM2:5 prediction systems are not accurate enough to make the distinction between 12 and 15 lg=m3 ? In other words, what about the capability of the existing prediction models to meet the increasingly strict and sharp new standards? From a governmental point of view, the costs involved due to air pollution are huge. The National Oceanic and Atmospheric Administration (NOAA) of the US Department of Commerce estimates that exposure to poor air quality is responsible for as many as 60,000 premature deaths each year and that this amount could be reduced with better predictions [9]. It is also estimated that more effective prediction methods will save US$9 billion and 64,000 jobs over a 10-year period in the USA [9]. In China, an estimated 8572 premature deaths occurred in four major Chinese cities in 2012, due to high levels of PM2:5 pollution, and Beijing experienced a loss of US$328 million in the same year because of PM2:5 pollution [10]. Presently, the large majority of the models being in use to address PM2:5 in Japan are climate models based on Eulerian and Lagrangian grids or on Trajectory models [8]. However, an alternative to these expert models resides in artificial neural networks (NN), where high accuracy in prediction tasks has been reported [11]. In particular, a form of NN known as recurrent neural networks (RNN), in contrast with feedforward neural networks (FNN), has been shown to exhibit very good performance in modeling temporal structures [12] and has been successfully applied to many real-world problems [13]. However, it has been shown that shallow NN rapidly reach their limits due to their need for large amount of (labeled) data, which is going in contradiction with their inability to scale in complexity with the size of the network to handle the volume of data [14]. But recently, with the advent of open and big data and the alleviation of critical difficulties residing in training dense NN composed of many layers [15, 16], it has become possible to construct more complex and efficient networks. These complex networks are known as deep neural networks (DNN), and the training of such networks is often included in the appellation deep learning (DL). A review of the basic concepts of NN and DL is provided in Sect. 3. In this work, our ultimate goal is to compute PM2:5 concentration predictions in Japan using real sensor data and with improved accuracy over the currently employed prediction models. To do so, we introduce a deep RNN (DRNN) specifically designed for PM2:5 prediction that is enhanced with a new pre-training method (see Fig. 1), written DynPT for convenience. DynPT improves DL techniques on the task of time series modeling, which is a field that has not received much attention yet fr (...truncated)