Validation and reliability for the updated REAP-S dietary screener, (Rapid Eating Assessment of Participants, Short Version, v.2)

Abstract

Background

The American Heart Association (AHA) chose the REAP-S dietary screener in 2020 as one of three US dietary screeners recommended for integrating dietary assessment into clinical care. The REAP-S v.2 is an updated version that is aligned with the 2020–2025 US Dietary Guidelines and is easily incorporated into electronic medical records and taught to medical students.

Methods

The University of New England, Institutional Review Board, approved the study protocol. We evaluated the reliability and validity of the REAP-S v.2 scale by having first-year medical students (n = 167) complete both the REAP-S v.2 and a three-day food record and then analyzing their data with the following statistical techniques: Internal consistency was measured using Cronbach's alpha. Construct validity was assessed with exploratory factor analysis. Criterion validity was evaluated using analysis of variance (ANOVA) that explored the associations between REAP-S v.2 scale item responses and selected nutrient estimates from the food record analyses. The hierarchical cluster analysis classified healthy and unhealthy diet grouping under each subscale. Further using these groupings, cut points for "good" and "bad" diets for each of the three main REAP-S v.2 subscales (Food Sufficiency/Food Insufficiency; Healthy Eating Pattern and Low Nutrient Density Foods) were calculated using receiver operating characteristics (ROC) analysis. Students analyzed their three-day food intake records using an online USDA application called SuperTracker.

Results

The Cronbach’s alpha measuring internal consistency was acceptable for the overall scale at 0.71. The exploratory factor analysis extracted three factors that roughly paralleled the three main subscales, suggesting construct validity. Most selected food record-derived nutrient values were significantly associated with scale items confirming criterion validity. The score cut points suggest that dietary counseling might be needed at ≤ 8, ≤ 10, and ≤ 16 for the above subscales.

Conclusion

The REAP-S v.2 is intended for clinicians to use as a brief dietary screener with their patients. Tested in a population of first-year medical students, the REAP-S v.2 brief dietary screener showed acceptable internal consistency, criterion, and construct validity. It is easily scored and incorporated into the electronic medical record.

Background

In 2020 the American Heart Association (AHA) recommended that clinicians use one of three dietary screeners to evaluate patients’ dietary habits regarding cardiovascular disease risk reduction [1]. The original REAP-S (Rapid Eating Assessment for Participants, short version, [2]) was one of the AHA’s three recommended dietary screening tools. In this paper, we present validity and reliability analyses for an updated and revised version of the REAP-S dietary screener.

Eleven million deaths worldwide in 2017 were linked to people eating poor diets high in sugar, salt, and processed meat, which contributed to heart disease, cancer, and diabetes [3]. The AHA stated in their 2020 position paper on dietary screeners that “It is critical that diet quality be assessed and discussed at the point of care with clinicians and other members of the healthcare team to reduce the incidence and improve the management of diet-related chronic disease, especially cardiovascular disease” [1]. Suffering and expense caused by nutrition-related chronic diseases have significantly increased during this half-century, but physician competency in nutrition has not [4]. There are many complex reasons for this discrepancy. Funding models rewarded disease-oriented interventions and treatments rather than lifestyle interventions to maintain health and prevent disease, contributing to the “disturbing mismatch between the skills of physicians and the needs of patients” concerning nutrition [5].

A 2018 AHA science advisory, which focused on nutrition training for physicians, has optimistically stated: “Enhancing physician education and training in nutrition, as well as increasing collaborative nutrition care delivery by 21st-century health systems, will reduce the health and economic burdens from atherosclerotic cardiovascular disease to a degree not previously realized” [6].

A two-pronged approach is needed to equip physicians and other healthcare providers with the training and screening tool to assure that “diet quality (is) assessed and discussed at the point of care with clinicians and other members of the healthcare team to reduce the incidence and improve the management of diet-related chronic disease, especially cardiovascular disease” [7]. By gaining this experience early, physicians may be more likely to use a screener to assess patients’ dietary behaviors and refer them to registered dietitian nutritionists as needed. 2) Both medical systems and private practitioners should be encouraged to incorporate a simple dietary screener into an initial patient evaluation screening. The data are then entered into the patient’s electronic medical record.

The REAP-S is a brief dietary screening tool designed to provide clinicians who may want to take a quick snapshot of the strengths and weaknesses of a patient’s diet. It is intended to be administered before the patient sees the primary care provider (PCP) so the PCP can use the information to counsel or refer the patient as necessary. The screener can be easily entered into an electronic medical record system and scored automatically.

The original REAP-S dietary screener was first published in 2004 [2]. We recently updated it to 1) Reflect current recommendations in the Dietary Guidelines for Americans 2020–2025 [8], 2) Make it easier for patients to identify portion sizes [9], and 3) Provide clinicians empirically tested cut points for REAP-S v.2 scores. For clarity, we refer to the original REAP-S as REAP-S v.1 and the updated version as REAP-S v.2. This paper describes the validity and reliability study for REAP-S v.2.

Methods

Development of the REAP-S v.2 dietary screener

As previously mentioned, REAP-S v.2 was developed to give clinicians a rapid assessment of a patient’s dietary adequacy, including an evaluation of healthy eating and exercise patterns and the consumption of foods that should be limited (e.g., added sugars, saturated fats, sodium, and alcohol).

To do this, the questions in REAP-S v.1 were reorganized into these four subscales: Food Sufficiency/Food Insufficiency; Healthy Dietary Pattern; Low Nutrient Density Foods; and Exercise, and several questions were added or modified. The REAP-S v.2 instrument is presented in the supplemental material.

Questions in the Food Sufficiency/Food Insufficiency subscale (shown in blue) provide information on total caloric intake, protein intake, and calcium intake. Questions 1 and 4 are from REAP–S v.1. Questions 2 and 3 were added for protein and calorie intake information. In the Healthy Eating Pattern subscale (pink), Questions 5–7 were modified from REAP-S v.1 to include more descriptive examples of portion sizes. Questions 8–11 were added to the Healthy Eating Pattern subscale to include the intake of healthy fats found in vegetable oils, nuts and fish and the complex carbohydrates and fiber found in legumes, reflecting more recent dietary recommendations. In the Low Nutrient Density Foods subscale (green), the order of the questions was changed, and some of the questions were modified to include descriptive examples of portion sizes. Question 19 was added to provide information on alcohol intake. Question 20 in the Exercise subscale (yellow) was added to provide information on physical activity. We included question 21 from REAP-S v.1 to provide a segue for beginning the conversation on readiness to change. The subscale scores provide the clinician with the option to acknowledge areas of strength and focus on the most problematic areas for patients who are ready to make changes.

Study population

All students who participated in this study were in the first-year Osteopathic Medical Knowledge Course at the University of New England. Completion of the REAP-s v.2 and the three-day food record were part of a larger required project called the Health Promotion Project. The participant demographic data collected by the College of Osteopathic Medicine as part of the admissions process was used to describe the study population. The deidentified data included students’ gender, age, and self-reported race/ethnicity.

Study protocol

During the first week of the Health Promotion Project assignment, students were invited to participate in a study to validate a new dietary screener, REAP-S v.2. The course resource materials included a written description of the validation study. The data collected for the validation study were part of the University of New England College of Osteopathic Medicine curriculum activities for all students. Students who did not wish to participate in the validation study were given an opt-out form to complete and their data were not included. The University of New England Institutional Review Board approved the study protocol.

As part of the assignment, students were required to complete an assessment of their diet by keeping a three-day diet record consisting of two weekdays and one weekend day. Students entered their food records into an online United States Department of Agriculture (USDA) diet planning and tracking tool, SuperTracker [10], to calculate their individual nutrient and food group intake which were averaged over the three days of reported intake (Fig. 1). Students also completed the revised REAP-S v.2 dietary screener using an online form. SuperTracker analysis data and the REAP screener data were deidentified and entered into a Microsoft Excel spreadsheet [11]. No students chose to opt out of the study, although some did not complete all parts of the assignment. It wasn’t possible to do test–retest reliability because the students’ curriculum did not allow for it.

Fig. 1

Example of output from SuperTracker for a three day diet record

Statistical analysis

Participant characteristics, dietary data analyzed with SuperTracker, and responses to REAP-S v.2 were numerically summarized using descriptive statistics. The continuous variables were summarized using mean and standard deviation, while categorical variables were presented as frequency counts and percentages. The scale’s internal consistency was measured using Cronbach’s alpha, which assesses how individual items correlate with other items and the total scale.

The REAP-S v.2’s validity was assessed as follows. Exploratory factor analysis (EFA) was used to identify the construct validity (factors), nature, and number of constructs. The Kaiser–Meyer–Olkin (KMO) test was used to examine the sampling adequacy of the scale. A common factor analysis with an iterated principal factor extraction method and a Promax rotation was used, assuming the factors are correlated. Three factors were extracted based on the majority rule. The process included six methods: Eigenvalue > 1 criterion, Kaiser minimum Eigenvalue criteria, the cumulative proportion of variance (0.8), parallel analysis [12], minimum average partial test (MAP) [13], and the visual scree test [14]. The replicability of the estimates was examined using the Bootstrap estimates and 95% confidence interval procedure. The bootstrap analyses were performed based on 2000 replicated data.

Furthermore, an analysis of variance (ANOVA) was fitted to explore the associations between REAP-S v.2 scale items and selected macronutrients, minerals, and vitamins measured from the averaged three-day food records analyzed with SuperTracker. Some non-normally distributed variables were log-transformed to conform to a Gaussian distribution. A score from the analysis of participants’ dietary records for healthy versus unhealthy eating habits was not readily available; we used a nutrient marker provided by the participants’ SuperTracker analyses to derive an objective measure for healthy eating habits. A hierarchical cluster analysis was used on the selected nutrients for each REAP-S v.2 subscale to create two clusters representing healthy versus unhealthy eating behaviors. Each REAP-S subscale’s best optimal cut point scores were then ascertained by modeling the derived health status (healthy/unhealthy) as an outcome variable, using receiver operating characteristics (ROC) curves via a logistic regression model.

Missing Data: Participants (n = 9) who did not complete the REAP-S v.2 and Super Tracker were excluded from the analysis. Others with random incomplete information were imputed using a fully conditional imputation approach. All analyses were performed using SAS software version 9.4 [15].

Results

Population sample

In 2017, 178 students enrolled in the University of New England College of Osteopathic Medicine, of whom 176 participated in the study. Fifty-one percent (n = 90) were male; the average age was 24 years (min, max: 20, 39). About 68% were New England state residents, 29.7% were from other US states, and 2.2% were international. Racially, 71% self-reported as White, 25.8% as of Asian or Middle Eastern race, 2.8% were the underrepresented minority, and 5.6% were unknown. Of the 176 participants, nine did not complete the instrument, thus reducing the analysis sample to 167.

Reliability and validity of the scale

Descriptive statistics of the scale items are presented in Table 1. The average total (SD) REAP score was 39.08 (6.37), approximately the mid-point of the overall total score. The mean (SD) of Food Sufficiency/Food Insufficiency, Healthy Eating Patterns, Low Nutrient Density Foods, and Exercise subscale score in our study population were 9.42(1.83), 10.98(3.27), 16.81 (3.22) and 2.17(0.85), respectively. Nearly 28% (n = 46) of the sample reported a score suggestive of food insufficiency (Food Sufficiency / Insufficiency scale ≤ 8 scores); 87% (n = 143) reported unhealthy eating patterns (Healthy Eating Patterns ≤ 14 scores), 46% (n = 76) reported not consuming low nutrient density foods (Low Nutrient Density Foods ≤ 16 scores) and 25% (n = 41) reported less than 15 min a day of exercise on fewer than three days a week (Exercise ≤ 1 score). Ninety percent of the participants reported wanting to change eating and physical activity habits.

Table 1 Descriptive statistics: n = 167. Mean, Standard Deviations, Factor Loadings and Bootstrap estimates with 95% CI and Communality (h²) from Iterated principal factor analysis with Promax Rotation

The estimate of the internal consistency of the overall scale as measured by Cronbach’s alpha is 0.71, which is within acceptable limits for all study variables (i.e., α ≥ 0.70). Internal consistency estimates for the responses to the subscales; Food Sufficiency/Food Insufficiency, Healthy Eating Pattern and Low Nutrient Density Foods subscales were respectively α = 0.41, α = 0.65, and α = 0.62.

An exploratory factor analysis was conducted from the data collected from the 167 participants using the iterated principal factor analysis with Promax rotation. The KMO index measuring the sampling adequacy for each variable in the model was 0.67, indicating the sample was adequate to perform factor analysis. Three factors were extracted based on the majority criteria of extraction methods described in the statistics section. The rotated factor pattern (standardized regression coefficients or loadings) of items, factor structure (correlations), Bootstrap estimates of rotated factor pattern with 95% confidence intervals, and communality estimates are presented in Table 1. The rotated pattern matrix and rotated structure matrix support the three-factor solutions. The structure matrix values show the correlation between items and factors.

In contrast, the pattern matrix value presents a relationship between items and factors while holding other factors constant, hence the value differences. These three factors explained 87.49% of the total variance. The three factors loaded similarly to the theoretical constructs with minor changes. Factor 1, the Healthy Eating Pattern vector, consisted of nine items with factor loadings ranging from 0.31 to 0.59. Factor 2, the Low Nutrient Density Foods vector, had factor loadings from 0.43 to 0.56, and Factor 3, the Food Sufficiency/Food Insufficiency vector, had loadings from 0.33 to 0.66. Alcohol and feeling well did not play a major role in these constructs, possibly due to the age group of the study participants. The replicability of these factors was assessed through Bootstrap estimates with 2000 replications, which mainly showed similar trends. After ProMax rotation, the factors remain correlated. Table 2(a) presents inter-factor correlations between factor 1 vs. factor 2, factor 1 vs. factor 3, and factor 2 vs. factor 3 were 0.34, 0.25 and 0.04, respectively. Each item's proportion of variability explained by the factors assessed in terms of communality statistics ranged between 0.17 to 0.5, except for alcohol. The communality contribution for alcohol was very low, 0.06, consistent with its lack of loading on the factors. Table 2 (b-c) presents summary statistics for two types of factor scores: rotated factor pattern loadings and standardized loadings. From the results, though the two scores are different in metrics, the correlation table shows the factors provide similar results: factor 1 is correlated with two other factors, while factor 2 is uncorrelated with factor 3. Each subscale showed adequate psychometric properties independently and could be used separately.

Table 2 Inter-Factor correlation, descriptive statistics, and correlation between factor scores

Associations between the REAP-S v2. scale items and food record dietary data

We tested how well each scale item was associated with and predicted specific macronutrients, minerals, vitamins, and food groups estimated from the participants' three-day dietary self-report and compiled and analyzed with Super Tracker. Some non-normally distributed variables were log-transformed to conform to a Gaussian distribution. The association results for each scale item are presented in Table 3. Most macronutrients, minerals, vitamins, and food groups were statistically associated with scale items. These selected nutrients and food groups that were associated with each of the subscales were used to differentiate participants into cluster groups of healthy and unhealthy eating patterns using hierarchical cluster analysis. The dendrogram, a branching diagram, shows the relationship of similarity among participants based on the participant-reported SuperTracker nutrient variables values forming two clusters (healthy and unhealthy behavior) generated based on the cluster analysis (Supplemental Fig. 1a-c). Further, these groupings were tested against the REAP-S v.2’s subscale scores. The ROC analysis showed subscale cut points of ≤ 8, ≤ 10, ≤ 16, and ≤ 1, respectively, for the Food Sufficiency/Food Insufficiency subscale; Healthy Eating Pattern subscale; Low Nutrient Density Foods subscale; and Exercise to distinguish those who might benefit from nutrition counseling from those who should be congratulated on their healthy behaviors.

Table 3 Relationship between REAP-S v.2 scale items and selected nutrients

Discussion

The use of dietary screeners

Dietary behaviors are one of the most difficult human behaviors to measure accurately. Many individuals may not accurately recall what or how much they ate or drank. Social desirability may influence some to report an intake they perceive as healthier. Just the act of self-observation, as in the case of self-completed food records, may affect intake. Food intake from day to day and week to week may vary for many reasons, such as access to food, health, travel, changes in routine, etc. Several methods have been developed and tested that show good reliability and validity for measuring food intake for research studies. Some examples of these are semi-quantitative food frequency questionnaires and multiple-pass, multiple-day food recalls administered by a trained research dietitian. Krebs-Smith et al. has developed an excellent resource on dietary intake research tools. The resource is available online at the National Cancer Institute’s website at https://dietassessmentprimer.cancer.gov/ [16].

As mentioned earlier, dietary screeners, such as REAP-S v.2, are intended to give clinicians a quick and rough snapshot of a patient’s dietary patterns. As useful as dietary screeners are, they are not a substitute for nutrition research instruments.

Summary of results

The overall REAP-S v.2 screener showed an acceptable internal consistency comparable to the original REAP-S v.1 (Cronbach’s alpha of 0.72) using a similar sample of first-year medical students [17]. The total REAP score ranged between 25 to 55 (max = 80), while each subscale score range was food sufficiency/insufficiency score (max = 12): 3 to 12, healthy eating patterns (max = 21): 1 to 20, low nutrient density foods (max = 24): 6 to 24, and exercise (max = 3): 0 to 3. The standardized factor scores have a mean zero and variability closer to one.

The constructs identified in exploratory factor analysis paralleled the proposed three main subscales of the REAP-S v. 2: Factor 1, the Healthy Eating Patterns vector; Factor 2, the Low Nutrient Density Foods vector and Factor 3, the Food Sufficiency/ Food Insufficiency vector. The questions on feeling well and alcohol intake did not load well in these factors, which could be due to the specific sample group.

Criterion validity was evaluated by comparing selected nutrition data from averaged three-day food records to REAP-S v.2 scale items. Most nutrient data were significantly associated with scale items, confirming criterion validity. These selected nutrients were also used to develop mathematical cut-point summary scores for each of the three major REAP-S v.2 subscales: Food Sufficiency/Food Insufficiency; Healthy Eating Patterns; and Low Nutrient Density Foods; this allows clinicians to distinguish between healthy and unhealthy dietary patterns with the best sensitivity and specificity. In other words, the cut points allow clinicians to sum scale items within a subscale and use these subscale scores to determine whether a patient generally falls into the healthy or unhealthy dietary range, so the clinician can decide whether to counsel or refer a patient for nutrition counseling. Lower scores reflect poorer dietary patterns. The cut points are Food Sufficiency/Food Insufficiency—a score of ≤ 8; Healthy Eating Patterns – a score of ≤ 10; Low Nutrient Density Foods – a score of ≤ 16; and Exercise – a score of ≤ 1.

Comparison of REAP-S v.2 to other dietary screeners

We did a PubMed search for validated dietary screeners and found six recent reviews of dietary screeners, with the AHA’s 2020 position paper being the most current and relevant for the US population [1, 18,19,20,21,22].

Reliability and validity were among the theoretical and practice-based factors the AHA 2020 position paper used to evaluate dietary screeners. They included three types of validity: 1) Correspondence between screener scores and a "gold-standard" dietary assessment tool, such as a multiple-day food recall or a semi-quantitative food frequency questionnaire; 2) Generalizability within multiple populations; and 3) Correspondence between screener scores and a measured biomarker. REAP-S v.1 was, over time, validated in all three ways. REAP-S v.2, as a new instrument, has not yet been validated in multiple populations or against a biomarker. Because we retained much of the earlier approach, we suspect it will be generalizable to multiple populations and may well also correlate with biomarkers, such as serum ascorbate levels, as Johnston et al. found the REAP-S v.1 did [23].

However, in addition to validating REAP-S v.2 against the dietary recall, we validated it using several statistically rigorous techniques, such as factor analysis and construct validity. Most other dietary screeners have not undergone this degree of statistical validation. Further, we mathematically established through cluster analysis a marker for participants' objective healthy eating behavior and using ROC curves derived reasonable sensitive and specific cut points for the three main subscales.

The limitations of this study include: 1) The test population was a relatively homogenous, predominantly white population of first-year osteopathic medical students; 2) REAP-S v.2 has not yet been tested on diverse racial, age, educational level and socio-economic groups; 3) As mentioned earlier, REAP-S v.2 has also not yet been tested against a biomarker; 4) Food record data was collected and analyzed by the participants and not via an interview with an experienced research dietitian using the USDA five-step multiple-pass method for dietary recall [24], which may have led to less accurate food records. Another limitation is test–retest reliability assessment as this was not logistically feasible within this study population.

Conclusion

REAP-S v.2, an updated version of the AHA-recommended original REAP-S, appears to have adequate reliability and has passed multiple types of validity testing. It has been updated to 1) More closely conform to the Dietary Guidelines for Americans 2020–2025; 2) Be easier to administer to patients, including better descriptions of portion sizes for foods; and 3) Provide statistically computed cut points for subscales to assist clinicians in assessing which patients would benefit from nutrition or physical activity counseling.

With respect to clinical utility, REAP-S v.2 is easy to administer and score. It’s also easily converted to an electronic version that can be integrated into an electronic medical record system. It is freely available to clinicians, medical groups, researchers, and medical educators.