Introduction

According to DSM-V criteria, delirium refers to an acute onset and fluctuating neurological disturbance involving consciousness and cognition over a brief period of time, and occurs in the presence of an underlying medical condition.¹ The gold standard diagnosis of delirium is psychiatric evaluation. However, psychiatric evaluation is not feasible for routine monitoring of delirium in the pediatric intensive care unit (PICU) setting. Therefore, standardized and validated screening tools that enable rapid and reliable screening of patients in real-time are of the utmost importance.

Delirium occurs frequently in critically ill children. Although the prevalence of delirium in PICUs varies with regional and disease variations, an international large-sample study reported that one in four critically ill children had delirium.² Studies have shown that pediatric delirium is associated with longer intensive care unit (ICU) stay^3-5, increase in costs, prolonged mechanical ventilation and higher odds of death.^2-9 Therefore, rapid recognition of delirium is important to allow for effective intervention. In recent years, several screening tools have been proposed for children admitted to the PICU, including the Pediatric Confusion Assessment Method for Intensive Care Unit (pCAM-ICU)¹⁰, the Sophia Observation Withdrawal Symptoms-Pediatric Delirium (SOS-PD) scale¹¹ and the Cornell Assessment for Pediatric Delirium (CAPD).¹² The CAPD is a quick and easy bedside tool based on the nurses’ observational assessments during care.^3,13,14 It is designed to recognize all types of delirium for any pediatric age^2,5,15,16, regardless of the presence of developmental disabilities.^5,12,17 Since 2016, the European Society of Pediatric and Neonatal Intensive Care (ESPNIC) has recommended the CAPD as the assessment tool to diagnose delirium among children and infants (grade of recommendation: A).⁶ The CAPD has been internationally translated, adapted, and validated into Chinese, Japanese, Portuguese, Spanish, Italian and Danish.^14,18-23 In a nationwide study, 57.9% of PICUs in Türkiye did not use any delirium screening tool and the number of centers where delirium screening was routinely and regularly performed was only three (15.8%).²⁴

Currently, there is no validated Turkish screening tool for the diagnosis of pediatric delirium. The primary objectives of this study were to translate the CAPD from English to Turkish, ensure its cross-cultural adaptation, and to analyze the validity and reliability of the Turkish version. The secondary aim was to test the screening efficacy of the Turkish CAPD in specific subgroups, including developmental status and age categories.

Materials and Methods

Description of the CAPD

The CAPD is a screening tool for identifying symptoms of delirium. It consists of eight questions, each scored on a scale of 0 to 4, with responses ranging from ‘never’ to ‘always’. The total score ranges from 0 to 32, with a CAPD score of 9 or higher indicating the presence of delirium. Patients with a score of 9 or higher are categorized as “delirium present,” while those with a score below 9 are categorized as “delirium absent.” There are anchor points available to be used as a point of reference when scoring the CAPD in children under two years of age.¹⁵

Translation

The CAPD was translated following the International Society for Pharmacoeconomics and Outcomes Research’s (ISPOR) good practice guidelines²⁵ for translation and cross-cultural adaptation (Fig. 1). The following steps were followed in the process: 1. Preparation: Permission was obtained from the developers of the original scale during the translation process. The authors met with the original developers of the CAPD to understand the background and theory and to ensure the correct use and interpretation of the assessment tool. 2. Forward translation: The translation from English to Turkish was conducted by a translator familiar with medical terminology. 3. Back translation: The Turkish text was translated into English by a translator who is a native English speaker but had no prior knowledge of the CAPD. 4. Back translation review and harmonization: The original scale and back translation were compared by the developers. 5. Cognitive debriefing: Clinical applicability was reviewed by pediatric intensivists. 6. Review of cognitive debriefing results and finalization: The Turkish version was re-evaluated with the original author to ensure that it retained its original meaning. 7. Proofreading and final report: Grammatical errors were corrected and finalized.

Training

Training was provided by the original author of the CAPD to the PICU specialist leading the study. This specialist then organized training sessions for nurse leaders (a ‘train-the-trainers’ model). The nurses on the ward were then trained according to the original developer’s guidelines, with a maximum of three nurses attending each session. Thereafter, each PICU nurse had to complete three accurate CAPD assessments at the bedside in the presence of the supervisors, in order to be approved for independent delirium screening. Difficulties and questions in the assessment were clarified. Separately, the Child Psychiatry team organized a consensus session. A standardized psychiatric assessment form was created by consensus in accordance with the gold standard DSM-V criteria for pediatric delirium.

Assessing CAPD performance

This validation study was executed in a 32-bed medical-surgical PICU in a major urban academic medical center between November 2021 and August 2022. The study protocol was approved by the Institutional Review Board of Ankara City Hospital (No: E2-21-731, Date: 14.07.2021).

All patients who were admitted to the PICU for any reason and who had been in hospital for more than 48 hours were included in the study. Informed consent forms were obtained from the families during hospitalization. Patients without informed consent were excluded from the study. Thirty-two beds were numbered and a ’True Random Number Generator⁽’ tool (Randomness and Integrity Services LTD., Dublin, Ireland, www.random.org) was used to assign four bed numbers for each scheduled study day. If the same participant was part of the random selection, the participant had a maximum of four assessments. The definition of “significant clinical developmental delay” is on the basis of clinical evaluation and/or parental report of developmental problems affecting the child’s behavior or communication capability. Children with a history of mild or transient developmental problems (e.g., needing occupational therapy or having motor or speech delays) were not included in this category.¹² Demographic and clinical data were recorded on each subject. The patient’s level of sedation was assessed with the Richmond Agitation and Sedation Scale (RASS)²⁶ every four hours. If any of the selected beds had a patient with a RASS score less than -3 for the entire shift (i.e.: unarousable to verbal stimulation), CAPD scoring was not performed. Otherwise, the CAPD scores were assessed twice daily by the bedside nurse (Supplementary Tables S1, S2).

A set of double-blind matched nurse assessments were performed. The bedside nurse completed the CAPD as a paper checklist. Then, the supervisor nurse recorded the CAPD assessment. This was followed by an assessment by 2 blinded psychiatrists. If the child was diagnosed with delirium by either psychiatrist, the healthcare team was notified so that appropriate treatment could be given. Once inter-rater reliability was established, for the remainder of the study each patient had one CAPD and one psychiatric assessment. In our study, the DSM-V criteria were used as the gold standard for confirming the diagnosis of delirium, and the CAPD’s validity was evaluated by comparing its results with this diagnostic standard.

Statistics

Patient characteristics including demographics, reasons for admission, and level of sedation were summarized using descriptive statistics. Interrater reliability was quantified using Cohen’s κ coefficient.^27,28 The receiver operating characteristic (ROC) analysis was performed to find the optimal CAPD cut-off score; subsequently, sensitivity and specificity were calculated for the overall sample. In addition, in order to explore CAPD performance in subgroups, validity measures were described by age groups and developmental status. All confidence intervals (CI) were adjusted for the possible correlation between observations within subjects using a ratio estimator method.^29,30 Analyses were performed in R version 4.4.3 (R Foundation for Statistical Computing, Vienna, Austria).

Results

A total of 77 patients were enrolled in the study and 120 paired CAPD and psychiatric assessments were completed. One patient did not complete any study assessments. The remaining 76 patients completed at least one paired assessment (CAPD plus psychiatric evaluation). Thirty-eight nurses and three psychiatrists participated in study assessments.

Characteristics of the participants

Among the participants, 42 (55.3%) were male, and 37 (48.6%) were younger than 24 months. The age distribution of the participants is reported in Table I. Additionally, 22 participants (28.9%) had developmental delay. The diagnoses at the time of admission to the PICU and the underlying conditions can also be seen in Table I. Thirty-four participants (44.7%) received noninvasive ventilation, 23 participants (30.3%) received invasive mechanical ventilation, and 4 participants (5.3%) received supplemental oxygen. Fifteen participants (19.7%) were not receiving any form of respiratory support. In addition, 35 participants (46.1%) received sedation, 18 of whom (51.4%) received dexmedetomidine. The incidence of delirium was 25.0% (n=19 patients) according to the gold standard psychiatric assessment for the diagnosis of delirium. Of these cases, 10 (52.6%) patients had hyperactive delirium, 8 (42.1%) patients had hypoactive delirium, and 1 (5.3%) patient had a mixed type of delirium.

Criterion Standard and CAPD Performance

Inter-rater Reliability

The first 33 psychiatric assessments were performed independently by two psychiatrists, each blinded as to the other’s assessment. The concordance between the psychiatrist assessments was excellent, with a strong and reliable agreement (Cohen’s Kappa = 0.86, 95% CI: 0.72-0.99). The first 64 CAPD scores were performed independently by two nurses, each blinded as to the other’s assessment. The inter-rater reliability of nurses was 0.74 overall (Cohen’s Kappa = 0.74, 95% CI, 0.57-0.91). When each CAPD item was assessed individually, the inter-rater reliability ranged from 0.76 to 0.84 (indicating substantial to near-perfect agreement), except for item 6 (Cohen’s Kappa = 0.58, 95% CI: 0.39-0.77; Table II).

The sensitivity and specificity for the 120 CAPD assessments were 100% and 95%, respectively, using a cut-off point of 9 or higher, consistent with the original CAPD validation study. In our study, the optimal cut-off that maximizes sensitivity and specificity is a CAPD score of 11. Using this cut-off, sensitivity is 100% and specificity is 98.02% (Supplementary Figure S1).

The gold standard psychiatric diagnosis showed a 96% agreement with the CAPD screening test. Among the participants, there were five false positives (children with a CAPD score of 9 or higher in whom the psychiatrist did not diagnose delirium).

Subgroup analyses of CAPD performance against the gold standard of psychiatric diagnosis showed high sensitivity (100%) and specificity (96%) for patients with developmental delay. CAPD performance showed high sensitivity (100%) and specificity (96%) in participants aged less than 2 years. In infants <6 months of age, sensitivity remained at 100% and specificity decreased slightly (93%).

Discussion

Without routine screening, the diagnosis of delirium – especially in children – is often missed. The absence of a pediatric delirium screening tool in the native language complicates the recognition, prevention, and appropriate treatment of pediatric delirium. Therefore, it is imperative to have a linguistically and culturally appropriate delirium screening tool available for use in Turkish PICUs.

The CAPD is the first screening tool for delirium to be translated into Turkish. This study shows that the Turkish version of the CAPD is both valid and reliable, and results are consistent with the gold-standard psychiatric assessment for delirium. Implementing routine screening for delirium in Turkish PICUs will increase awareness of pediatric delirium, and may reduce morbidity and improve care nationwide.

In many prospective studies, a CAPD score of 9 or above has been used as a definition for delirium.^{2,6,12,14,17,18,20,22,31-33} Our data were in agreement with these previous reports, with a sensitivity of 100% and specificity of 95% when using the same cut-off. In this study, we found that a cut-off score of 11 further increased specificity (98.02%) without sacrificing specificity (still 100%). Further studies in Turkish PICUs will be necessary to replicate this finding before changing the CAPD cut-off score.

In practice, the diagnosis of delirium is challenging in extremely young and/or developmentally delayed children.¹⁵ Similar to the English version of the CAPD¹², we demonstrated good validity (100% sensitivity and 96% specificity) in detecting delirium in children younger than 2 years of age, with decreased specificity of the CAPD (93%) in patients younger than 6 months. However, in contrast with the reported literature, we demonstrated excellent specificity (96%) even in children with developmental disabilities. Kaur et al. showed an increase in specificity and positive diagnostic value from 66% to 97% and from 47% to 89%, respectively, when combining CAPD with RASS fluctuations in children with developmental delay compared to CAPD alone.¹⁷ We were aware of this finding when planning our current study and added a footnote to the final version of the Turkish CAPD: ’Fluctuation in RASS values (during at least six hours of follow-up) is a typical finding for delirium’, along with a checklist for RASS fluctuation (obvious/occasional/never). During the preparatory training of nurses, we stressed the significance of RASS fluctuations as indicative of changes in the patient’s level of consciousness. We believe that attention to RASS fluctuations when scoring the CAPD may have contributed to the nurses’ ability to identify delirium with high sensitivity and specificity, even in children with developmental disabilities.

The processes of translation and cross-cultural adaptation are complex and challenging. It is important to retain the characteristics of the original version while taking into account the regional use of expressions. The inter-rater reliability of the CAPD scores among nurses was 0.74 (Cohen’s Kappa = 0.74, 95% CI, 0.57-0.91). This indicates substantial agreement among nurses overall. For each individual item in the CAPD, inter-rater reliability ranged from 0.64 to 0.84 (substantial to near-perfect agreement), except for item 6 (“Is the child inconsolable?”), which showed only moderate agreement (Cohen’s Kappa = 0.58, 95% CI: 0.39-0.77). This is consistent with Hoshino et al.’s Japanese study which showed that item 6 had the lowest inter-rater agreement (Cohen’s Kappa 0.67).²² Japanese and Turkish belong to the same language family (Altaic language family) and the Altaic language family has a different sentence structure than the Euro-Indian language family. As a result, this relationship may be indicative of a translation problem. It is also notable that 4 out of the 5 patients who were falsely identified as delirious by the CAPD were of Syrian origin and did not speak Turkish. This language barrier made it difficult to calm the children. This may have led to decreased interrater reliability of item 6, as only some nurses spoke Arabic. To improve agreement for item 6, a discussion was held with the evaluators and it was suggested that wording for this item should be clarified. In our view, the translation of question 6 should be changed to “Çocuğu sakinleştirmek zor mu? (Is it difficult to calm the child?)”. This suggested modification may improve agreement between assessors and make communication more effective. We suggest that future studies in Turkish (and other) PICUs should consider evaluating the interrater reliability of item 6 with this alternate wording. Therefore, it is crucial to translate and validate original screening tools into various native languages according to guidelines.

There are some limitations to this study. It was conducted in a single institution. It needs to be replicated in a multi-institutional study. Patients with significant language barrier had a higher false positive rate, reflecting the difficulty in assessing these individuals. Further research is needed to replicate and address the best diagnostic approaches in this population, and the possibility of a higher CAPD cut-off point needs to be investigated in larger studies.

Conclusion

In conclusion, the Turkish version of the CAPD has high sensitivity and specificity compared to the criterion-standard psychiatric assessment. The Turkish CAPD is a valid and reliable screening tool for detecting and monitoring pediatric delirium in children of all ages and developmental stages.

Ethical approval

The study protocol was approved by the Institutional Review Board of Ankara City Hospital (date: 14.07.2021, number: E2-21-731).

Author contribution

The authors confirm contribution to the paper as follows: Study conception and design: EU, NOK, GD, EA, MO, YM, CT ; data collection: EU, NOK, EA, MO, EK, YM, GD ; analysis and interpretation of results: EU, NOK, EA, MO, EK, YM, GD; draft manuscript preparation: EU, NOK, CT. All authors reviewed the results and approved the final version of the manuscript.

Source of funding

The authors declare the study received no funding.

Conflict of interest

The authors declare that there is no conflict of interest.