Model-Free Machine Learning in Biomedicine: Feasibility Study in Type 1 Diabetes
July
Model-Free Machine Learning in Biomedicine: Feasibility Study in Type 1 Diabetes
Elena Daskalaki 0 1
Peter Diem 1
Stavroula G. Mougiakakou 0 1
0 Diabetes Technology Research Group, ARTORG Center for Biomedical Engineering Research, University of Bern , Murtenstrasse 50, 3008 Bern , Switzerland , 2 Division of Endocrinology, Diabetes and Clinical Nutrition, Bern University Hospital “Inselspital” , 3010 Bern , Switzerland
1 Editor: Kathrin Maedler, University of Bremen , GERMANY
Although reinforcement learning (RL) is suitable for highly uncertain systems, the applicability of this class of algorithms to medical treatment may be limited by the patient variability which dictates individualised tuning for their usually multiple algorithmic parameters. This study explores the feasibility of RL in the framework of artificial pancreas development for type 1 diabetes (T1D). In this approach, an Actor-Critic (AC) learning algorithm is designed and developed for the optimisation of insulin infusion for personalised glucose regulation. AC optimises the daily basal insulin rate and insulin:carbohydrate ratio for each patient, on the basis of his/her measured glucose profile. Automatic, personalised tuning of AC is based on the estimation of information transfer (IT) from insulin to glucose signals. Insulinto-glucose IT is linked to patient-specific characteristics related to total daily insulin needs and insulin sensitivity (SI). The AC algorithm is evaluated using an FDA-accepted T1D simulator on a large patient database under a complex meal protocol, meal uncertainty and diurnal SI variation. The results showed that 95.66% of time was spent in normoglycaemia in the presence of meal uncertainty and 93.02% when meal uncertainty and SI variation were simultaneously considered. The time spent in hypoglycaemia was 0.27% in both cases. The novel tuning method reduced the risk of severe hypoglycaemia, especially in patients with low SI.
-
OPEN ACCESS
Data Availability Statement: All relevant data are
within the paper.
Funding: The authors received no specific funding
for this work.
Competing Interests: The authors have declared
that no competing interests exist.
Introduction
Type 1 diabetes (T1D) is a metabolic disease characterised by uncontrolled blood glucose
levels, due to the absence or malfunction of insulin. The Artificial Pancreas (AP) system aims to
simulate the function of the physiological pancreas and serve as an external automatic glucose
regulation system. AP combines a continuous glucose monitor (CGM), a continuous
subcutaneous insulin infusion (CSII) pump and a control algorithm which closes the loop between the
two devices and optimises the insulin infusion rate.
An important challenge in the design of efficient control algorithms for AP is the use of the
subcutaneous route both for glucose measurement and insulin infusion (sc-sc route); this
introduces delays of up to 30 minutes for sc glucose measurement and up to 20 minutes for
insulin absorption. Thus, a total delay of almost one hour restricts both monitoring and
intervention in real time. Moreover, glucose is affected by multiple factors, which may be genetic,
lifestyle and environmental. With the improvement in sensor technology, more information
can be provided to the control algorithm (e.g. more accurate glucose readings and physical
activity levels); however, the level of uncertainty remains very high. Last but not least, one of
the most important challenges emerges from the high inter- and intra-patient variability,
which dictate personalised insulin treatment.
Along with hardware improvements, the challenges of the AP are gradually being addressed
with the development of advanced algorithmic strategies; the strategies most investigated
clinically are the Proportional Integral Derivative (PID) [
1
], the Model Predictive Controller
(MPC) [
2
]-[
7
] and fuzzy logic (e.g. MD-Logic) algorithms [
8
]-[
9
]. A recent development has
been the bi-hormonal AP [
10
]-[
11
], which uses both insulin and glucagon. Comprehensive
reviews of the latest advancements and current challenges in AP can be found in [
12
]-[
15
]. The
increasing number of clinical trials has led to extensive in-hospital and, more recently, at-home
evaluation of the feasibility of AP outside the controlled hospital environment. Most studies
are restricted to the algorithmic evaluation of a patient cohort under uncertain conditions,
such as erroneous meal intake and insulin sensitivity (SI) changes (e.g. physical activity).
In spite of these promising results, none of the currently proposed control strategies is
intrinsically designed to handle uncertainties and personalisation. PID is designed for linear
systems, MPC solves an open-loop optimisation problem which has proved sub-optimal in the
presence of uncertainty [
16
] and MD-Logic is a rule-based approach directly subjected to the
experience of the designer. In the view of patient variabilit (...truncated)