Relative validity of a semi-quantitative food frequency questionnaire for Singaporean toddlers aged 15–36 months

Toddlerhood is a critical period during the lifecycle. Defined here as children aged 12–36 months, this phase is marked by rapid growth, maturation of organs and increasing levels of physical activity [1]. Relative to their body size, toddlers have high nutritional requirements [1]. Any deficiencies or excesses in macro- and micronutrients that occur during this critical period can have lasting negative consequences later in life. Conditions such as iron deficiency and obesity are prevalent in developed and developing countries, and can often exist in parallel [2]. In addition to this, toddlers are establishing healthy eating habits as they transition from an infant diet to the family diet [3]. Therefore, insights into food and nutrient intakes of toddlers are extremely important. Dietary data collection can be integrated into clinical and epidemiological studies to understand the food and nutrient intake patterns of a population. Such information can help with the development of dietary guidelines, assess if children are meeting recommendations and if any diet-disease relationships exist.

Depending on study objectives, there are several different methods for collecting dietary information. These methods are similar in adults and children, however, with the exception of nutritional biomarkers, dietary information is obtained from a proxy (parent or guardian), especially if the child is under ten years of age [4]. The food record (FR) and FFQ are two examples of dietary assessment methods commonly used in epidemiological studies involving children [4].

The FR collects information current food intake and is used to estimate nutrient intakes [5]. Participants keep a diary of all foods and beverages consumed in a day, along with quantities that are estimated or weighed (WFR). Food records can be burdensome on participants due to the level of detail required and multiple days of recording. This can be especially challenging when toddlers are involved, as they may not eat the same foods as the rest of the family and different carers may be involved at various mealtimes, thus resulting in inconsistent reporting. For these reasons, the FR is one of the more tedious and expensive nutritional tools to implement and analyse [5, 6].

The FFQ differs significantly from the FR, as it retrospectively gathers information on habitual food intake. [7] The FFQ consists of a finite list of foods consumed by a particular population and participants indicate how often they consume these foods. Intake can also be crudely quantified [7]. The tool is inexpensive to administer; simple to complete and analyses is more straight forward. This makes it a useful tool in large population studies where the intention is to rank individuals according to their intakes and then seek associations between diet and disease [7]. Some limitations of the FFQ include: overestimation of nutrient intakes at the individual level; reliance on the user’s memory to recall past intake; its use is restricted to a specific population; it requires regular updating and it needs to be validated [5, 7].

There are a limited number of FFQs available for toddlers. Studies involving the development and validation of FFQs have been conducted in North America [8–15], Europe [16–23], Australia and New Zealand [24–26]. The age ranges included in these studies were 1.5–4 years. Even fewer FFQs are available for children in Asia [27, 28], of which none involve children less than three years of age. Therefore, presently, there is no simple tool for use in large epidemiological studies to understand the food and nutrient intakes of Asian toddlers.

To address this gap, a multi-ethnic sqFFQ for Singaporean toddlers was recently developed [29], but yet to be validated. The purpose of the present study was to validate this new tool for use among Singaporean toddlers aged 15–36 months. The most common reference method for the multi-nutrient validation of a sqFFQ designed for young children is the FR [30, 31]. The Singaporean toddler sqFFQ was assessed for its ability to rank and estimate nutrient intakes relative to the WFR for energy and 25 nutrients that are important for growth and development during this critical period.

Figure 1 illustrates the Bland-Altman plot for energy. The LOAs were wide, indicating large variability in the way the tools measured energy intake. The position of the midline indicated that the sqFFQ overestimated energy intake. This pattern was observed for all nutrients. Linear regression analyses revealed significant proportional bias for energy, SFA, DHA, sugars, fibre and vitamins A, B12 and E. This included both positive and negative trends with increasing intake.

Calibration also had varied effects on reproducibility. With the exception of total carbohydrates, reproducibility weakened for other nutrients, and SFA had a negative correlation (Table 2).

Calibration improved the ability of the sqFFQ to rank nutrient intake similarly to the WFR (Table 4). κ_w improved for energy and 64% of nutrients; energy and 16 nutrients had moderate agreement (0.41–0.60), and 9 nutrients had fair agreement (0.21–0.40).

Median intakes after calibration for all nutrients were very similar between the methods, with only phosphorus remaining significantly different (Table 3). For all nutrients, Bland-Altman analyses showed mean differences between the methods were now close to zero, with narrower LOAs. Proportional bias was still present for all nutrients, as illustrated visually in Fig. 2. However, overall, the magnitude was reduced and influenced mainly by extremes of intake.

This study aimed to validate a recently developed sqFFQ in its ability to rank and estimate nutrient intakes relative to the WFR, in multi-ethnic Asian toddlers. Results indicated that overall, the sqFFQ overestimated intakes of all nutrients when compared to the WFR. This finding was consistent with literature and most likely attributable to the format of the sqFFQ [26, 27, 48, 49]. With the traditional format of a sqFFQ, not only did parents have to think retrospectively about their child’s habitual intake, but they also had to consider frequency of intake, based on the portion size presented next to each food. For example, if the portion of food that their child consumed was smaller, or, larger than what was specified in the sqFFQ, then the frequency had to also be adjusted accordingly. This procedure had to be repeated for nearly 100 foods, which can be fatiguing for parents (the questionnaire took between 30 and 45 min to complete).

Typically, parents tended to overestimate intake of foods belonging to the breads and cereals, fruit, and meat/poultry/fish food groups. In this sample of children, traditional main meals usually consisted of composite dishes of multiple grains, meat and/or vegetables, and mixed fruit. In the sqFFQ, each type of rice/rice dish has a portion of ½ a bowl, while each meat item had a portion of 1 tablespoon. So, for example, if a child typically consumed ½ a bowl of rice consisting of equal amounts of two grains (brown and white rice) and ½ a tablespoon each of two meat items (pork and fish), 2–3 times a day, parents tended to place a tick in the 2–3 times a day column for each of the four items. This essentially doubled the child’s intake. Ideally, a lower frequency should have been selected to accommodate a smaller portion. This instruction was explained to parents during the initial face-to-face meeting and provided in writing, in the questionnaire. These were the kinds of responses that were flagged for review, and upon further explanation, often parents changed their response to a lower frequency to accommodate the specified portion. However, these types of instructions can be difficult to understand and a flaw of the sqFFQ design. One approach which may reduce this kind of error is to have participants choose a serving size on the FFQ, as well as a frequency, for each item. This would force participants to consider serving size, and it may minimise the need for participants to translate the child’s normal serving size and frequency into the set serving size and corresponding frequency on the FFQ. While answering two questions for each item that the child consumes may seem to increase the workload on the participant, this may in fact be easier than the present requirement.

Another approach would be to make the questionnaire interviewer-administered. It would allow a dialogue between the researcher and participant and the issue could have been addressed immediately, rather than up to two weeks after completion. This approach may not be feasible in large epidemiological studies where thousands of participants are involved. In this instance, a subsample of participants (dependant on study budget) should be interviewed for quality control purposes.

Digitalisation of the sqFFQ may also be an option, so that participants can access the questionnaire via an application on their smartphones or computer [50, 51]. This will provide the participant with a more interesting and interactive experience. For example, Chatbots could be used to clarify participants’ queries, flag unusual responses and prompt the user to think about their selection. Additionally, digital tools could potentially reduce the burden on researchers, if data entry can be replaced with nutrient intakes that are instantly calculated by the application, so that unusual nutrient intakes can be promptly identified and followed up.

Such technology will no doubt have its limits. The data would need to be reviewed for quality, as there is still reliance on the participants’ memory. Unusual responses cannot be eliminated and could potentially increase. How food portions are scaled could also be misleading [51]. The technology may not be accessible in all communities. Lastly, the time and cost invested in these technologies need to be considered as it is better suited to studies with long follow-up and frequent assessment time points.

In these analyses, it was found that total fat, SFA, MUFA, vitamin A and E had the weakest correlations and/or agreements between methods. Again, this was consistent with the literature, particularly for fats [26, 27, 48, 49]. There may be several reasons for this. Firstly, the addition of fats and oils are not typically measured during cooking. Secondly, these items are also not the main ingredient in recipes and could be subconsciously left out in recording. Lastly, such ingredients may not feature prominently in food prepared for this age group, or, food may be cooked in bulk and frozen into small portions making it very difficult to estimate how much was in each portion. Any of these reasons could create a significant discrepancy with the sqFFQ. PUFA and DHA on the other hand, were the only fatty acids to demonstrate much stronger correlations and agreements. It could be that follow-on and young child formulas (formulas for children above the age of one year) were the main contributors of DHA, and intake of this item was captured similarly by both methods. Likewise, calcium had high correlations and agreements, even before any adjustments.

The initial impression of the results indicated that the sqFFQ in its current form may not be suitable for ranking intake of all nutrients, as total fat, SFA, MUFA and vitamin A only had slight agreement. However, it must be noted that the tools are, in fact, measuring different aspects of the diet: habitual versus actual intake. So, some weaker agreements should be expected, relative to the WFR. Perhaps, if the WFR were completed for more days over a longer period of time, it would be more representative of present usual intakes, thus increasing the agreement between the methods. However, this could also reduce the quality of data from FRs as it is difficult to keep accurate food diaries for many days. Overall, the sqFFQ in its current form is suitable for ranking nutrient intakes.

The sqFFQ in its current form should not be used to estimate population intakes, as it would result in substantial overestimation. This finding was consistent with literature, regardless of participant age [16, 48]. If nutrient intakes need to be estimated, then this study demonstrated that the calibration step was effective in: bringing the sqFFQ values closer in range to the WFR values; strengthening agreements for at least two-thirds of nutrients and bringing the mean difference between the tools close to zero for all nutrients. Vitamin A was the only nutrient in the original sqFFQ whose median was not statistically different to the WFR median. Calibration resulted in the median difference becoming significantly different, however, it did improve correlations and agreements for this nutrient. Iodine was negatively impacted as agreement declined, although it remained within the category of “fair” agreement. Lastly, calibration had varied effects on reproducibility. With the exception of total carbohydrates, reproducibility weakened but remained acceptable and SFA had a negative correlation. We speculate weaker and negative correlations were an artefact of the calibration methods aimed at improving agreement between the sqFFQ and WFR. The adjustments via regression equations reduced the range of individual intakes, resulting in poorer apparent reproducibility of the sqFFQ for most nutrients. It must be noted, however, that reproducibility after calibration was only included here for demonstrative purposes. In an actual study, the reproducibility of a new tool only needs to be tested initially. If the calibration step is used to bring values closer to the reference method, repeating ICC is not necessary.

The ability to calibrate the sqFFQ values to bring values closer to the reference method is a particularly important finding for large studies aiming to estimate nutrient intakes. It is well understood that the use of FRs is expensive to implement in studies. Therefore, one option would be to administer the sqFFQ to the whole study group and then select a representative subsample (based on total sample size, age of subjects and study budget), to complete WFRs for internal calibration. This way, nutrient intakes can be estimated for a large sample, without significantly increasing costs and analysis time, compared to having the full sample complete WFRs. Alternatively, if a study had similar participants to the children in this study, then the coefficients provided in Table 5 can be used as a method of external calibration [52].

There are a few limitations to this study. Firstly, the use of non-probability sampling could have resulted in a non-representative sample. However, we were fortunate that the race distribution, parental education level and household income were all reflective of the current Singaporean population [32].

Secondly, weighed food records were kept for two days in this study, which was the minimum number of days reported in validation studies [34]. While two days was sufficient to capture micronutrient intakes in this age group, ideally, up to five days of dietary data would have more accurately accounted for the day-to-day variation in macronutrient intake and the three-month time frame of the sqFFQ [31, 53]. Based on feedback from our pilot study, where participants found keeping the 2-day WFR most challenging, reducing the burden on participants for our main study (to ensure retention and high quality data) was the primary reason for selecting the minimum two days for the reference method.

Presently, there are no recommendations as to whether the full sample was needed to assess reproducibility of the sqFFQ, or, if a proportion of sample was sufficient [34]. Due to the problems faced with recruitment with both the pilot and current study, 20% of the sample was randomly selected to complete the second sqFFQ. This was based on recent studies conducted in children and adolescents, where reproducibility was tested in 10–25% of the sample [27, 36]. While this approach resulted in high correlations (also an effect of the short timeframe between the questionnaires), the findings lacked power. It is therefore recommended that future studies endeavour to have the full sample complete the repeat questionnaire.

Lastly, the sqFFQ asked parents to report on habitual intake over the last three months. In hindsight, this would have been very difficult to estimate; we speculate that some parents may not have even considered this instruction at all. Given how much a toddler’s diet could change over three months, due to both developmental progression and inconsistencies related to illness or experimentation with new foods for example, a two-week retrospective time frame may be more realistic for the parents and produce more accurate results [16].

To the best of our knowledge, this is the first time that a toddler-specific sqFFQ, developed for a multiethnic Asian population, has been validated against a WFR for an extensive range of nutrients. It is also one of few FFQ validation studies using a range of methods in a systematic way, and therefore provides a model for the conduction of future toddler FFQ validation studies outside of Singapore. This tool will be useful in large epidemiological studies to determine dietary patterns, frequency of consumption of particular foods or food groups, or rank nutrient intakes to study diet-disease relationships. It is recommended that the sqFFQ is interviewer-administered, and only two weeks retrospective to minimise overestimation. While the tool in its current form is not suitable for estimating nutrient intakes of a population, including WFRs in a representative subsample within a study for calibration purposes can overcome this. This allows for more accurate estimation of nutrient intakes in large nutrition studies, without dramatically increasing the time and cost associated with implementing and analysing FRs.