 Research article
 Open Access
 Open Peer Review
 Published:
Utilizing distributional analytics and electronic records to assess timeliness of inpatient blood glucose monitoring in noncritical care wards
BMC Medical Research Methodology volume 16, Article number: 40 (2016)
Abstract
Background
Regular and timely monitoring of blood glucose (BG) levels in hospitalized patients with diabetes mellitus is crucial to optimizing inpatient glycaemic control. However, methods to quantify timeliness as a measurement of quality of care are lacking. We propose an analytical approach that utilizes BG measurements from electronic records to assess adherence to an inpatient BG monitoring protocol in hospital wards.
Methods
We applied our proposed analytical approach to electronic records obtained from 24 noncritical care wards in November and December 2013 from a tertiary care hospital in Singapore. We applied distributional analytics to evaluate daily adherence to BG monitoring timings. A onesample KolmogorovSmirnov (1SKS) test was performed to test daily BG timings against nonadherence represented by the uniform distribution. This test was performed among wards with high power, determined through simulation. The 1SKS test was coupled with visualization via the cumulative distribution function (cdf) plot and a twosample KolmogorovSmirnov (2SKS) test, enabling comparison of the BG timing distributions between two consecutive days. We also applied mixture modelling to identify the key features in daily BG timings.
Results
We found that 11 out of the 24 wards had high power. Among these wards, 1SKS test with cdf plots indicated adherence to BG monitoring protocols. Integrating both 1SKS and 2SKS information within a moving window consisting of two consecutive days did not suggest frequent potential change from or towards nonadherence to protocol. From mixture modelling among wards with high power, we consistently identified four components with high concentration of BG measurements taken before mealtimes and around bedtime. This agnostic analysis provided additional evidence that the wards were adherent to BG monitoring protocols.
Conclusions
We demonstrated the utility of our proposed analytical approach as a monitoring tool. It provided information to healthcare providers regarding the timeliness of daily BG measurements. From the real data application, there were empirical evidences suggesting adherence of BG timings to protocol among wards with adequate power for assessing timeliness. Our approach is extendable to other areas of healthcare where timeliness of patient care processes is important.
Background
Regular monitoring of blood glucose (BG) in hospitalized patients is an important component of inpatient diabetes mellitus (DM) care. The American Diabetes Association (ADA) recommends monitoring blood glucose (BG) four times per day (i.e., before meals and at bedtime) in hospitalized patients with DM. If the patient is fasting or receiving continuous enteral or parenteral nutrition, the recommended BG monitoring frequency is once every 4 to 6 h. BG monitoring is performed every 30 min to 2 hourly, if a patient is on an intravenous insulin infusion [1–4].
Timely measurement of BG facilitates the delivery of inpatient DM care, and allows treatment regimens to be revised to achieve optimal glycaemic control. For example, preprandial BG measurement supports clinical decisionmaking by enabling healthcare providers to prescribe an appropriate dose of supplemental insulin to correct for premeal hyperglycaemia. Therefore, a holistic assessment of the quality of inpatient DM care should include an evaluation of compliance to BG monitoring timings. However, methodologies assessing timeliness are not wellestablished.
A study by Buchs and colleagues assessed the compliance to BG monitoring protocols by using a pie chart to display the proportion of BG timings in a specific portion of time, such as, premeals and postmeals [5]. The majority of BG measurements (approximately 75 %) occurred before mealtimes over a 6month period. However, this method aggregated data over a long period which did not facilitate further evaluation of circumstances surrounding nonadherence, if any was detected. This would limit its utility in a realworld setting.
Continuous evaluation of BG measurements can now be conducted with the advent of electronic medical records [6]. We propose using distributional analytics as a surveillance tool to provide highresolution empirical evidence of adherence to clinical protocols where timeliness is an important factor [7]. Our proposed analytical approach will address the following: (i) assessment of daily adherence to a BG monitoring protocol by ward over a prespecified period, and (ii) detection of wards and days exhibiting potential nonadherence or changes in patient care processes.
Methods
We examined all pointof care (POC) BG measurements performed in 24 noncritical care wards in a 1000bed tertiary care hospital, National University Hospital, from November to December 2013. Capillary BG measurements were performed using POC glucose meter AccuChek Inform II (Roche, Basel Switzerland) and stored in a central laboratory database. The BG data downloaded from the central database contains deidentified patient identifier, BG value, patient location (i.e., ward), date and time of BG measurement. The specialty of the ward indicates the predominant type of patients in the ward. We did not collect additional data including mealtimes and patient demographics, which reflected the current setup of the laboratory database.
We analyzed all the POC BG timings during the 2month period. BG timings were converted to hours according to the standard 24h clock. In the hospital where we conducted the study, the mealtimes are targeted around 0800, 1200 and 1800 h, while the bedtime is targeted around 2200 h.
The majority of inpatients requiring BG monitoring are eating regular meals (i.e., not fasted). Hence, the ideal BG timings should be distributed with multimodes within a day, where the modes should occur before mealtimes and at bedtime. An extreme contrast would be a uniform distribution in the BG timing, which reflects a complete lack of adherence to ADArecommended timings for patients eating regular meals. Hence, a simple way to assess potential adherence to BG monitoring protocol is to perform a onesample KolmogorovSmirnov (1SKS) test that detects deviation of BG timings from a continuous uniform distribution between 0 and 24 h [8], where a significant pvalue (i.e., pvalue < 0.05) suggests potential adherence to protocol. For presentation purposes only, we ranked wards in a decreasing order according to the proportions of days with significant deviations from the uniform distribution over the 2month period and ties were broken with the median pvalue. We plotted the boxplots of pvalues for each ward stratified by medical specialty to investigate for potential differences between specialties.
To avoid false negatives with 1SKS test, we identified wards that are adequately powered by performing a Monte Carlo simulation study to estimate the power for each ward [9–11]. For each ward, we had generated 5,000 simulation iterations for each day. The BG timings were simulated via the inversion of the empirical cumulative distribution function (cdf) on that day and the total number of measurements simulated was the same as the observed number on that day [11, 12]. We used linear interpolation to obtain a continuous cdf for simulation purposes and recorded the simulated power for each day (i.e., the proportion of iterations with pvalues < 0.05) [13]. The simulation procedure for each BG timing in a day was as follows:

1.
We generated a random number from a uniform distribution, i.e., u ~ U(0, 1)

2.
We identified the closest observed BG timings interval, (t_{1,}t_{2}], on that day, such that, F(t_{1}) < u ≤ F(t_{2}), where F(·) is the empirical cdf of BG timings, then \( {t}_{simulate}=\left({t}_2{t}_1\right)\frac{uF\left({t}_1\right)}{F\left({t}_2\right)F\left({t}_1\right)}+{t}_1. \)
We gathered the estimated power from simulation across all days to compute the median and mean power for each ward. Wards with average or median power more than or equal to 90 % were considered adequately powered for assessing adherence. To understand the interplay among effect size, sample size and pvalue across all wards in the hospital, we also regressed –log_{10}(pvalue) of 1SKS test on standardized effect size and sample size, with interaction between the two standardized quantities. The effect size is the 1SKS test statistic, \( D={ \sup}_y\leftF(y)\frac{1}{24}y\right \), where y denotes the BG timing and D quantifies the magnitude of deviation between the empirical cdf of BG timing and the uniform distribution.
To complete the assessment of BG monitoring protocol with 1SKS, we corroborated the pvalue findings visually with the cdf plots to inspect for features that we would expect from a ward that had been compliant with the protocol. Testing against a uniform distribution is based on a simple assumption of an extreme nonadherence behavior. However, other nonadherent behaviours may exhibit distributions of BG timings that differ from a uniform distribution resulting in statistically significant findings with 1SKS tests too. To rely less on the parametric assumption of the uniform distribution, we also proposed to test the BG timing distribution of the current day against the previous day via a twosample KolmogorovSmirnov (2SKS) test [14, 15]. A significant pvalue from a 2SKS test suggests a statistically significant difference in BG timing distributions between two consecutive days.
So far, both the 1SKS and 2SKS tests were using BG timings from each day and two consecutive days respectively. By applying mixture modelling on all BG timings in the 2month period among wards with high power, we can deconvolute the overall distribution of BG timings into components. We assumed a mixture of normal distributions to model daily BG timings where the mean and variance parameters were fixed constants across all days, and we modelled the mixture probability of each component by day where day was a categorical variable. In this finite mixture model, we defined the conditional density as follows:
where y_{ ij } denotes the jth BG timing on the ith day for i = 1, …, 61, and j = 1, …, n_{ i } (where n_{ i } is the sample size on the ith day). Assuming there are K components, π_{ k } is the component probability assigned to the kth component, and f(⋅) denotes the normal density function with componentconstant mean, u_{ k }, and variance, σ _{ k } ^{2} , where μ = (μ_{1}, μ_{2}, …, μ_{ K }) ' and σ^{2} = (σ _{1} ^{2} , σ _{2} ^{2} , …, σ _{ K } ^{2} ) '. We assume a multinomial logit model for the component probabilities with independent variable, w_{ ij } = (I_{ij2}, …, I_{ ijd }, …, I_{ij61}) ', where I_{ ijd } takes a value of 1 if the ith day is the dth day in this 2month period:
where α = (α _{ k } ^{'} ) _{k = 1,…,K} ^{'} and α_{1}≡0. So the full loglikelihood function for all BG timing in a ward is:
We used the integrated classification likelihood (ICL) as the criterion for model selection. It is a more robust criterion than Bayesian information criterion (BIC) in the presence of violation in model assumptions [16]. Hence, if the wards with high power were adherent to the BG monitoring protocol, there should be only four components with small standard deviation (SD), which we had arbitrary taken it to be SD < 1, and the mean values of these components should be prior mealtimes and around bedtime. We summarized the mean and variance estimates for each component with SD <1 using the minimum, median and maximum statistics, and plotted the mean ± 1.96 × SD of each component for all wards. For the probability estimates, we first took the average probability estimates within each ward and reported their minimum, median and maximum for each component with SD < 1. For components with SD ≥ 1, their minimum, median and maximum statistics were reported collectively for mean, variance and probability estimates.
We used the R statistical software program to analyze the BG data. R packages, stats, Matching and flexmix were used to perform the simulation (R functions: runif and approx), KStests (R functions: ks.test and ks.boot) and mixture modelling (R functions: stepFlexmix and FLXPmultinom) [14, 17, 18].
Results
There was a total of 73,182 BG measurements in 23,221 patientdays from 3,673 patients during the period November to December 2013. Among the 23,221 patientdays, 9.4 % of patientdays had more than four BG measurements, 42.3 % of them had four BG measurements, and 48.3 % had less than four BG measurements.
Figure 1 ordered the boxplots of the 1SKS test pvalues for all 24 noncritical care wards by the proportions of days with significant pvalues within each specialty. The boxplots with dark gray shading corresponded to wards that were not adequately powered and these wards were either small wards, or obstetrics & gynecology (O&G) wards that had much fewer patients on BG monitoring. Wards with high power had median sample size ranging from 59 to 103 BG measurements in a day. When we examined the relationship of the pvalues with sample size and effect size across the 24 wards, a one unit increase in the standardized sample size alone could lead to an increase of 2.56 in –log_{10}(pvalues) keeping standardized effect size fixed at 1, which is about 99.7 % reduction in pvalues, and similarly a one unit increase in the standardized effect size alone could lead to an increase of 2.41 in –log_{10}(pvalues) keeping standardized sample size fixed at 1, which is about 99.6 % reduction in pvalues (see Table 1). Hence, the change in pvalue that was attributable to sample size and effect size respectively was comparable.
A significant 1SKS test pvalue only suggests the particular day is potentially adherent to protocol. To ascertain the adherence status of wards, we corroborated the pvalues with daily cdfs of BG timings. For the two highly ranked wards (i.e., Rank 1 and 2 in Fig. 2a and b respectively), we found that the majority of days had exhibited four pronounced steps before mealtimes and around bedtime suggesting adherence to BG monitoring protocols. This was also observed in the other wards with high power. There were only a few days that had fewer pronounced steps than the ideal BG timing distribution. When we further explored the days with fewer pronounced steps, we found that there were no BG records on prior days suggesting potential missing data influencing the BG timing distribution. The prevalence of days without any BG measurements were low. There were 31 instances where no BG measurements were available on an entire day among the 11 wards with adequate power in the 2month period (i.e., a total of 671 instances = 61 days × 11 wards) and 27 instances among 13 wards with inadequate power in the same period (i.e., a total of 793 instances = 61 days × 13 wards). For the two lowest ranked wards with low power (i.e., Rank 23 and 24 in Fig. 2c and d respectively), the lines were more jagged and less pronounced when compared to wards with high ranks. The wards with low ranks were O&G wards and patients from these wards were mostly fasting and hence monitored more frequently in every 4–6 h.
In Fig. 1, the majority of 2SKS test pvalues were insignificant. For the scenarios where two consecutive days had significantly different BG timing distributions, they contained the days with less than four pronounced steps observed in Fig. 2 previously. Further exploration among wards with high power by utilizing both the 1SKS and 2SKS pvalues within a two consecutive days moving window did not suggest frequent potential change from or towards nonadherence to protocol (i.e., a significant 2SKS pvalue, and only one significant 1SKS pvalue) as only three occurrences across all wards over the 2month period had transitions from nonadherence to adherence or vice versa.
The daily BG timings were expected to be distributed with four modes, i.e., before mealtimes and at bedtime, if the BG monitoring protocol was being adhered to. Among the wards with high power (≥90 %), there was evidence suggesting adherence to protocol. Hence, we modelled each ward individually using mixture models which deconvoluted the BG timing distribution into components. From the mixture modelling analysis, we consistently identified four components with SD estimates < 1. In Fig. 3, we visualized the four components identified for each ward with their mean and the interval corresponding to mean ± 1.96SD (indicating 95 % of the BG measurements were within this interval). Most of the wards were adherent to protocol because the majority of their measurements (i.e. ≥ 95 %) were taken before mealtimes (except lunch time) and around bedtime.
Among wards with high power, we found that the mean estimates of the component corresponding to before breakfast time, where the expected breakfast time was 8 am, the median of the mean estimates was about 46 min earlier than 8 am, and it ranged from 103 to 29 min earlier than 8 am (Table 2). For the component corresponding to before lunch time, where expected lunch time was 12noon, the median of the mean estimates was about 37 min earlier than 12noon and it ranged from 53 to 23 min earlier than 12noon. For the component corresponding to before dinner time, where the expected dinner time was 6 pm, the median of the mean estimates was 49 min earlier than 6 pm and it ranged from 59 to 38 min earlier than 6 pm. For the component corresponding to bedtime, where the expected bedtime is 10 pm, the median of the mean estimates was just 4 min before 10 pm and it ranged from 26 min before 10 pm to 16 min after 10 pm. The range of the mean estimates was the largest for the component corresponding to before breakfast time and it was almost twice as large when compared with the remaining three components.
As for the SD estimates, the median across the four components were between 0.25 and 0.34 h (i.e., for each component, 95 % of the BG measurements were approximately within an hour, or an hour and 20 min interval). For components with SD ≥ 1, the range of the total number of components were 1 and 3. These components may potentially represent the BG measurements taken from patients with hypoglycaemia, or patients who were fasting or receiving insulin infusions. The minimum SD for these components was around 1.6, suggesting 95 % of the BG measurements were within a time interval ≥ 6 h. The majority of the probability estimates for each component were distributed almost equally across the components, except for a few cases where the components with large SD had smaller probabilities when compared to the four components with small SD.
Discussion
In this paper, we applied distributional analyticsbased methodology, mixture modelling and visualization to assess adherence to an inpatient BG monitoring protocol in noncritical care wards. To alleviate the occurrence of false positives due to the interplay of small sample size and effect size in the 1SKS test, we performed a simulation study to identify wards where the daily total number of BG measurements had a mean or median power greater than or equal to 90 % over the 2month period (i.e., wards with high power). Restricting our assessment to these wards, we further corroborated the 1SKS test findings using: (1) visualizations with cdf plots, (2) 2SKS tests, and (3) mixture modelling.
The BG timing distribution of wards with high power exhibited four modes before three mealtimes and around bedtime. From the cdf plots, we observed four pronounced steps and from the mixture modelling, we observed exactly four components with small standard deviation. This phenomenon concurred with our expectation of a ward following ADA monitoring recommendations for patients eating regular meals. In particular, the cdf plots and the mean ± 1.96SD intervals from mixture modelling allowed us to identify the time interval where majority of BG measurements was taken, which was not possible using the pie chart with aggregated BG data [5]. These visualizations facilitate further investigations of nonadherence by healthcare administrators by identifying problematic timings during the day. This highlights the importance of pairing statistics with visualization to deliver actionable information. When we combined both the 1SKS and 2SKS tests within a 2day moving window over the 2month period, there was a low occurrence of change from or towards nonadherence to protocol.
Although we could only draw conclusions for wards with high power to minimize false negatives, we could accrue a sufficient number of BG timings for those wards with moderate power by aggregating the BG data over a twoday window. For example, the 6th medical ward in Fig. 1 had median power close to 90 % (i.e., 87.3 %) and median effect size close to the wards with high power. We proposed to combine the BG timing of the current day of interest with its previous day to obtain a larger sample size while assuming the patient care process was the same between two consecutive days. With a twoday window, we obtained six other wards with an average or median power greater than 90 %, and only the last ward in the first five specialties and the two O&G wards in Fig. 1 were not adequately powered. The results on adherence for these six additional wards with moderate power were similar to the 11 wards with high power. In particular, we identified 20 out of 360 instances = 60 2day windows × 6 wards were potentially nonadherent to BG protocol and there were no occurrence of change from or towards nonadherence to protocol.
As our proposed approach uses ward level as the unit of analysis, it captures the correlation of BG timings within a patient through the marginal distribution of BG timings from all patients in the ward. Although we had identified some days without BG measurements, these occurrences were low and sporadic, and therefore unlikely to affect our findings.
Our proposed approach provided an analytical way to alert healthcare administrators of potential nonadherence to protocol for a specific day and ward. However, to extend our approach to continuous surveillance through daily monitoring, we proposed to modify the criterion for wards with high power based on the estimated power for each day instead of the mean or median power over the entire 2month period. This will be useful for a large hospital and will open up the opportunity to monitor adherence on a daily timescale.
Conclusion
In this paper, we have proposed a way to assess adherence to BG monitoring protocols using electronic BG records. To reduce false negatives from our proposed assessment with 1SKS test, we used concepts from power calculation and simulation to determine the wards that are adequately powered with empirical data. By investigating the pvalues of KS tests, cdf plots and mixture modelling, we found that wards with high power were adherent to the BG monitoring protocol.
In summary, our approach leverages on the distributional analytics and the availability of electronic records of laboratory data to provide a practical surveillance tool for identifying potential nonadherence to clinical workflow. Our approach is also applicable to other areas of healthcare where timeliness of patient care processes is paramount, for example, medication administration or timed blood investigations.
Ethics approval and consent to participate
The investigations were carried out in accordance with the ethical codes and guidelines of the Nuremberg Code (1946), the Declaration of Helsinki (1964), the Belmont Report (1979) and the Singapore Guideline for Good Clinical Practice (1998), and were approved by the National Healthcare Group Domain Specific Review Board, including waiver of informed consent.
Consent for publication
Not applicable.
Availability of data and materials
The dataset supporting the conclusions of this article is available in: http://blog.nus.edu.sg/dasa/bgdataset1.
Abbreviations
 1SKS:

onesample KolmogorovSmirnov
 2SKS:

twosample KolmogorovSmirnov
 ADA:

American Diabetes Association
 BG:

blood glucose
 BIC:

Bayesian information criterion
 Cdf:

cumulative distribution function
 DM:

diabetes mellitus
 ICL:

integrated classification criterion
 O&G:

obstetrics & gynecology
 POC:

pointofcare
 SD:

standard deviation
References
 1.
Moghissi ES et al. American Association of Clinical Endocrinologists and American Diabetes Association consensus statement on inpatient glycemic control. Diabetes Care. 2009;32(6):1119–31.
 2.
Thomas P, Inzucchi SE. An internet service supporting quality assessment of inpatient glycemic control. J Diabetes Sci Technol. 2008;2(3):402–8.
 3.
Goldberg PA et al. “Glucometrics”Assessing the quality of inpatient glucose management. Diabetes Technol Ther. 2006;8(5):560–9.
 4.
Cook CB et al. Assessing inpatient glycemic control: what are the next steps? J Diabetes Sci Technol. 2012;6(2):421–7.
 5.
Buchs AE, Rapoport MJ. Institutional glucometrics to determine glucose control as practiced by general medicine wards. IMAJIsr Med Assoc J. 2010;12(8):463.
 6.
Cook CB et al. Inpatient pointofcare bedside glucose testing: preliminary data on use of connectivity informatics to measure hospital glycemic control. Diabetes Technol Ther. 2007;9(6):493–500.
 7.
Pivovarov R et al. Temporal trends of hemoglobin A1c testing. J Am Med Inform Assoc. 2014;21(6):1038–44.
 8.
Massey Jr FJ. The KolmogorovSmirnov test for goodness of fit. J Am Stat Assoc. 1951;46(253):68–78.
 9.
Goldstein R. Power and sample size via MS/PCDOS computers. Am Stat. 1989;43(4):253–60.
 10.
Manly BF. Randomization, bootstrap and Monte Carlo methods in biology. Boca Raton, Florida: CRC Press. 2006;70
 11.
Rubinstein RY and DP Kroese. Simulation and the Monte Carlo method. Hoboken, New Jersey: John Wiley & Sons 2011;707.
 12.
L’Ecuyer P. Nonuniform Random Variate Generations. In: International Encyclopedia of Statistical Science. Berlin, Heidelberg: Springer. 2011; p. 991995.
 13.
Davis P.J. Interpolation and approximation. Mineola, New York: Dover publications, Inc. 1975.
 14.
Abadie A. Bootstrap tests for distributional treatment effects in instrumental variable models. J Am Stat Assoc. 2002;97(457):284–92.
 15.
Sekhon, JS. Multivariate and propensity score matching software with automated balance optimization: the matching package for R. Journal of Statistical Software, Forthcoming, 2008.
 16.
Biernacki C, Celeux G, Govaert G. Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans Pattern Anal Mach Intell. 2000;22(7):719–25.
 17.
Leisch F. Flexmix: A general framework for finite mixture models and latent glass regression in R. 2004.
 18.
Team RC. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. p. 2014. ISBN 3900051070.
Funding
This work was supported by the Centre for Health Services and Policy Research SBRO14/NS01G from the National University Health Systems Pte Ltd, National University of Singapore (NUS) Startup Grant (WBS: R608000059133), and Ministry of Education Singapore Academic Research Funds Tier 1 grant (WBS: R148000195112).
Author information
Additional information
Competing interests
To the best of our knowledge, no conflict of interest, financial or other, exists. The authors declare that they have no competing interests.
Author’s contributions
YC processed the data, performed the data analysis, interpreted the findings and drafted the manuscript. SLK designed the study, participated in the data analysis and interpretation of the findings, edited the manuscript and cosupervised the project. EST, HLW and EYHK interpreted the findings and edited the manuscript. YN, MS and XD managed and processed the data, and edited the manuscript. CST conceived the project, participated in the data analysis and interpretation of findings, edited the manuscript and supervised the project. All authors reviewed the manuscript. All authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Chen, Y., Kao, S.L., Tai, E. et al. Utilizing distributional analytics and electronic records to assess timeliness of inpatient blood glucose monitoring in noncritical care wards. BMC Med Res Methodol 16, 40 (2016) doi:10.1186/s1287401601422
Received
Accepted
Published
DOI
Keywords
 Distributional analytics
 Timeliness
 Quality of care
 Diabetes mellitus
 Inpatient
 Electronic medical records