Long short term memory deep net performance on fused Planet-Scope and Sentinel-2 imagery for detection of agricultural crop

PLOS ONE RESEARCH ARTICLE Long short term memory deep net performance on fused Planet-Scope and Sentinel-2 imagery for detection of agricultural crop Touseef Ur Rehman1,2☯, Maaz Alam1,2☯, Nasru Minallah1,2☯, Waleed Khan1,2☯, Jaroslav Frnda ID3,4☯*, Shawal Mushtaq2☯, Muhammad Ajmal5☯ a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Rehman TU, Alam M, Minallah N, Khan W, Frnda J, Mushtaq S, et al. (2023) Long short term memory deep net performance on fused Planet-Scope and Sentinel-2 imagery for detection of agricultural crop. PLoS ONE 18(2): e0271897. https://doi.org/10.1371/journal.pone.0271897 Editor: Manoj Kumar, Forest Research Institute Dehradun, INDIA Received: December 20, 2021 Accepted: July 10, 2022 Published: February 3, 2023 Copyright: © 2023 Rehman et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: The data has been uploaded to Zonedo. The link and description of the dataset is provided here with; https://zenodo.org/ record/6560241#.YoYyK6hByUk. Funding: This research was supported by the Ministry of Education, Youth and Sports of the Czech Republic conducted by VSB - Technical University of Ostrava, Czechia in the form of a grant to JF [SP2022/5]. 1 National Center for Big Data and Cloud Computing (NCBC), University of Engineering and Technology (UET), Peshawar, Khyber Pakhtoonkhwa (KP), Pakistan, 2 Department of Computer Systems Engineering, University of Engineering and Technology, Peshawar, Khyber Pakhtoonkhwa (KP), Pakistan, 3 Department of Telecommunications, Faculty of Electrical Engineering and Computer Science, VŠB – Technical University of Ostrava, Ostrava, Czechia, 4 Department of Quantitative Methods and Economic Informatics, Faculty of Operation and Economics of Transport and Communication, University of Zilina, Zilina, Slovakia, 5 Department of Agricultural Engineering Peshawar, University of Engineering and Technology, Peshawar, Khyber Pakhtoonkhwa (KP), Pakistan ☯ These authors contributed equally to this work. * Abstract In view of the challenges faced by organizations and departments concerned with agricultural capacity observations, we collected In-Situ data consisting of diverse crops (More than 11 consumable vegetation types) in our pilot region of Harichand Charsadda, Khyber Pakhtunkhwa (KP), Pakistan. Our proposed Long Short-Term Memory based Deep Neural network model was trained for land cover land use statistics generation using the acquired ground truth data, for a synergy between Planet-Scope Dove and European Space Agency’s Sentinel-2. Total of 4 bands from both sentinel-2 and planet scope including Red, Green, Near-Infrared (NIR) and Normalised Difference Vegetation Index (NDVI) were used for classification purpose. Using short temporal frame of Sentinel-2 comprising 5 date images, we propose an realistic and implementable procedure for generating accurate crop statistics using remote sensing. Our self collected data-set consists of a total number of 107,899 pixels which was further split into 70% and 30% for training and testing purpose of the model respectively. The collected data is in the shape of field parcels, which has been further split for training, validation and test sets, to avoid spatial auto-correlation. To ensure the quality and accuracy 15% of the training data was left out for validation purpose, and 15% for testing. Prediction was also performed on our trained model and visual analysis of the area from the image showed significant results. Further more a comparison between Sentinel-2 time series is performed separately from the fused Planet-Scope and Sentinel-2 time-series data sets. The results achieved shows a weighted average of 93% for Sentinel-2 time series and 97% for fused Planet-Scope and Sentinel-2 time series. Competing interests: The authors have declared that no competing interests exist. PLOS ONE | https://doi.org/10.1371/journal.pone.0271897 February 3, 2023 1 / 18 PLOS ONE Long short-term memory based deep neural networks for crops classification 1 Introduction From the inception of human civilization, agriculture has been regarded as the backbone of social and economic evolution. About 60% of the total population lives in rural areas and is highly dependent upon agricultural activities [1], contributing more than 24% to the Gross Domestic Product (GDP) of Pakistan and commissions half of the labour force. Moreover, it is also the largest source of foreign exchange earnings (https://www.pbs.gov.pk/content/ agriculture-statistics). However, the concerned agencies are facing difficulties in accurate crop monitoring and yield estimations [2] due to inaccurate and insufficient data from improper mechanisms. Making the task more challenging is the limited use of technology for generating seasonal crop statistics by the government, resulting in theft, overstocking and illegal trade. The currently deployed mechanisms in the country are limited to ground surveys and manual measurements, often proving very expensive due to a large number of human surveyors requirements. For policy-level decisions, cultivated land area and yield estimations are essential for determining the amount of food stored or exported to reduce food losses along the food supply chain [3]. Geographic Information System (GIS) has been adopted globally as a decision support system, for a variety of problems. Remote Sensing as a primary component of a GIS is the collection of Earth’s observational data through satellites and airborne sensors. Developed nations have already adopted such systems to surveil their valuable resources [4]. A number of remote sensing satellites are freely available for providing remote sensing data. Sentinel-2, Landsat (launched by United States Geological Survey, NASA), MODIS (Moderate Resolution Imaging Spectroradiometer) are a few of the satellites offering free access to remote sensing data. Each satellite offers unique set of features wiz. spectral, spatial and temporal resolution, number of channels and revisit time. Satellite remote sensing (RS) is considered to be a substantial technique for land cover classification and crop statistics generation [5], over a large geographical scale, providing periodically considerable observations regarding ground objects [6]. The surge of satellite data unlocked countless possibilities for land cover land use statistics and transformation of data into information. The common categorization of remote sensing data includes; Multi-spectral, Hyperspectral and Synthetic Aperture Radar (SAR). Multispectral sensors having a limited number of channels are frequently used for vegetation based studies, due to their simple nature, data availability, and fast processing, as compared to Hyperspectral (Having more tha (...truncated)