Sampling, recruitment and participants
As studies involving the validation of nutritional tools among toddlers are limited in Asia, sample sizes used in other similar studies were used as a guide. Additionally, Cade and colleagues (2002), suggested at least 50–100 subjects are required for each demographic group, particularly if Bland Altman analyses and correlations are used; increasing the sample size beyond this would not strengthen correlations [31]. As the sqFFQ was designed as one questionnaire for a multiethnic sample (all races have access to many different types of foods and cuisines), a convenience sample of approximately 100 subjects and their primary caregiver was consecutively recruited over twelve months (December 2015 to November 2016). For inclusion into the study, toddlers had to be healthy, 15–36 months of age and of Chinese, Malay or Indian ethnicity (the predominant ethnic groups in Singapore) [32]. Children with any acute or chronic illnesses that affected food intake were excluded, as were children with one or both parents who did not meet the ethnicity criteria. This was to avoid over-representation of a minority group (3.2% of the population in 2016) [32], and also because the food list in the sqFFQ was based on food consumption of the three main races. Recruitment was from 15 months of age because the sqFFQ asks about habitual food intake over the last three months and only information from 12 months onwards was of interest.
Children were recruited via day-care centres. A convenience sample of 74 centres (35 government-based and 39 private) across the island were selected. Only 16 centres agreed to participate (reasons for non-participation included: too busy with administrative duties, committed to other studies, the principals felt they were not at liberty to authorise the study (headquarters had to be involved), or, simply not interested). These 16 centres provided approximately 260 children who met our inclusion criteria, of which, only 46 responded to the invitation letter (we are uncertain if invitation letters were distributed to all eligible children). To increase the speed of recruitment, the snowball technique was introduced, so current participants could refer others and research staff could ask colleagues and their friends to spread the word on the study. Sixty-six parents expressed interest via this method (n = 112).
Once the caregiver returned the participation form, the study research assistant arranged a face-to-face meeting to fully explain the purpose of the study, how to fill in a series of different questionnaires and obtain signed consent. Participants were given two weeks to complete the study components and received a $75 (Singapore dollars) shopping voucher for their participation.
Participants filled out a series of questionnaires in order to meet several study objectives. The questionnaires relevant to the objective of this study are described below.
Initial questionnaire
The initial questionnaire was completed during the first face-to-face meeting. This questionnaire aimed to capture information on each parent’s weight, height, education level and combined household income. Parents self-reported their child’s birth weight and length and current weight and length/height. Parents could use the most recent child weight and height/length measure noted in the child’s health book (if it was in the last 2 weeks); however, they were encouraged to have the child measured at a local clinic during the study period. Study staff did not do the measurements because it required the child to be present at the initial meeting, which added another level of complexity to the recruitment. As anthropometric data were not crucial to the validation analyses and mainly collected to describe the population, self-reported data were deemed sufficient.
Weighed food record
Parents were asked to record food intake for two non-consecutive days (one weekend, one weekday), as previously recommended [33,34,35]. Full instructions were given verbally during the initial meeting and detailed written instructions accompanied the WFR templates. Parents indicated the day, date, time of meals, meal occasion, description of all the foods offered, portion consumed, and place it was consumed. Extra pages were included for recipes and supplements used that day. Emphasis was placed on the level of detail required when describing the food types, recipes, cooking methods (including the addition of salt, seasonings, fats and oils) and brands. If the child was breastfed, mothers were asked to record the minutes the child latched on. Each participant was given digital kitchen scales which could register weights of 1 g to 5000 g (unnamed; model SF-2012) and they were showed how to tare the weight of plates/cups/bowls before weighing the food and weighing leftovers. In the instance where a meal was eaten away from the home, and the scales could not be used, parents were asked to describe portions in relation to standard cup and spoon measures, or, the standard bowl measure used in the sqFFQ (parents were shown what these were at the initial meeting). For meals consumed while the child attended day care, the research assistant obtained details from the facility, as meals are supplied by the facility. If another carer oversaw a mealtime, they were asked to fill out the details in the diary. All the food WFRs were reviewed in person or via phone call. At the end of the review, the parent was asked if the child’s intake was usual, more than usual or less than usual. If, after review, the record was still deemed as poor quality, then it was excluded from analyses.
Semi-quantitative food frequency questionnaire
The sqFFQ was an original design, with food lists and portion sizes developed in a previous study [29]. Briefly, the sqFFQ food list was derived by interviewing 30 mothers (ten from each ethnic group mentioned above) in a focus group setting. The mothers were asked about the child’s habitual intake and were also asked to complete 3-day food diaries. Over 500 different foods, typical portion sizes and utensils were reported from the interviews and diaries. It was decided that one food list would be used for all three races. This was because Singapore is a multicultural society and all races could easily access any type of food and cuisine. The final sqFFQ consisted of 99 items, including single and composite items, as well as items where foods of a similar type and nutrient profile were grouped together (for example in the vegetables section, vegetables were grouped as bulbs, tubers, root, stem, fruit, seeds, with examples provided for each; certain items in other food groups were separated based on their fat, sugar and fibre content). These 99 items were then divided across 11 food groups: breads and cereals, vegetables, fruits, legumes and nuts, meat/poultry/fish and alternatives, dairy and alternatives, snacks, fast foods, beverages (other than dairy and alternatives), salty and sweet seasonings including fats and oils used in cooking, and supplements. Within each group, an open question was included where participants could add other foods to the list. Portion sizes commonly used for toddlers were listed next to each item. Frequency responses started at “Never” and increased across 10 categories to a maximum of “>6 times per day”. Verbal and detailed written instructions were given, including illustrations showing the portion sizes referred to in the food lists and dimensions of common utensils. An appendix was included with photographs and descriptions of approximately 50 different foods listed in the sqFFQ to further guide parents. Reproducibility was assessed by asking participants to complete a second sqFFQ. As there were no guidelines indicating whether the full sample was needed for reproducibility, or, a proportion of the sample was sufficient, 20 % of the sample were asked to complete a second sqFFQ one to two weeks later [27, 34, 36].
The completed questionnaires were reviewed, particularly if the portions, when totalled, exceeded what was recommended for this age group.
Nutrient analysis
Energy and 25 nutrients were of interest in this study: protein, total fat, saturated fat (SFA), monounsaturated fat (MUFA), polyunsaturated fat (PUFA), docosahexaenoic acid (DHA), total carbohydrate (CHO), total sugars, fibre; vitamins: A, thiamine (B1), riboflavin (B2), niacin (B3), dietary folate equivalents (DFE), cobalamin (B12), C, E; minerals: calcium (Ca), iron (Fe), iodine (I), magnesium (Mg), phosphorus (P), potassium (K), sodium (Na) and zinc (Zn).
For the WFR nutrient values were determined with FoodWorks 8 Professional software package (Xyris Software Pty Ltd., Australia). This software linked several national databases available in Australia and allowed new foods to be added (38 generic food items and 19 follow-on and young child formulas were added). For foods specific to Singapore, the Singaporean Health Promotion Board nutrient database [37] was used to create and add new foods into the system (27 items in total). The Composition of Foods Integrated dataset by McCance and Widdowson (revised version) was also consulted [38]. When new foods or recipes were created, and information on all nutrients was not available, efforts were made to match it as closely as possible to an existing food in the database, based on ingredients and nutrient values. The software allowed each nutrient to be over-written with a new value, making it possible to “borrow” the missing nutrient value from a food already in the system. Where brands were given on a WFR, information was obtained from package labelling or company websites. (These were the main sources of information for formulas and supplements.) Breastfeeding was assumed to provide approximately 10 g breastmilk per minute. Per breast, feeds were capped at 10 min, since milk flow after this length of time was considered too slow to contribute nutritionally. Feeds shorter than 2 min were excluded for the same reason. If the next feeding session commenced within 30 min of the start of the previous feed, the duration was added to the first feed and capped at 10 min per breast [39,40,41].
For the sqFFQ, a reference spreadsheet was developed that included all the nutrient values for the portion specified for each item, using FoodWorks 8. The mean of up to five foods per single item was used to estimate nutrient values. For items which were a group of similar foods, for example, rice-based dishes or small flower fruits, up to five variations of each food in the group were averaged. Each frequency category was converted to a single number of serves per day. For example, 1–3 times a month was averaged to 2 serves/30.4 days = 0.065 serves per day. These were then multiplied by the portion of food to obtain nutrients per day for a particular food. The sum of all the foods in the list was the total intake. As a high proportion of children consumed vitamin and mineral supplements, each type was added to the food list as a new food. Additionally, four new foods were created as they could not fit into an existing category. These were muesli bar, breastmilk, dried seafood, and dried seaweed. Microsoft Office Excel 2010 (USA) was used to determine nutrient intakes per day, which was then exported to a statistical package for analyses.
Nutrients were not adjusted for energy intake. This was deemed unnecessary as the assessment of nutritional intake was not an aim of the present study.
Statistical analysis
Analyses were performed on IBM SPSS Statistics for Windows, version 23 (IBM Corp., Armonk, N.Y., USA). Anthropometry Z-scores were determined using the World Health Organisation (WHO) AnthroCalc v3.2.2. Data were checked for normality using the Smirnov-Kolmogorov test and visual checks of histograms. As 50% of data were skewed, a number of convenient Box-Cox transformations (cube, square, square root, cube root, natural log, inverse cube root, inverse square root, inverse, inverse square, inverse cube) were performed in an attempt to reduce the skewness to within a range of − 0.5 to 0.5. The cube root values were within this defined range and were used in Bland Altman, correlations and selected multivariable regression models, while raw values were used for other analyses described below. As there was no set method for validating the sqFFQ, a number of techniques documented in the literature were used in a series of steps [42]. A p-value of less than 0.05 was considered statistically significant.
Correlations between methods
Firstly, linear associations between the two methods were explored using Pearson correlations. Additionally, deattenuated Pearson correlations were used to account for variation in the diet with the formula:
$$ {\mathrm{r}}_{\mathrm{xy}}/\mathrm{sqrt}\left({\mathrm{r}}_{\mathrm{xx}}/{\mathrm{r}}_{\mathrm{yy}}\right) $$
where rxy was the correlation between the mean of the 2-day WFR and first (main) sqFFQ; rxx was the correlation between Day 1 and Day 2 of the WFRs and ryy was the correlation of the first and repeat sqFFQs [43]. Correlations between 0.30–0.49 were considered acceptable and 0.50–0.70 were good [44]. Only nutrients with deattenuated correlations ≥0.30 were included in subsequent analyses.
Reproducibility of the sqFFQ
Reproducibility of the sqFFQ was assessed using intra-class correlation (model: two-way mixed; type: absolute agreement; alpha = 0.05).
Agreement between methods
The Wilcoxon signed rank test was used to assess differences in median nutrient intakes by each method. Agreement was then assessed using weighted kappa (κw). Quadratic weights were used to assess the statistical significance of the agreement (if sqFFQ and WFR ranked the nutrient into the same quarter = 0 points; adjacent quarter = 1 point, 2-quarter difference = 4 points and extreme quarter = 9 points). This test is generally thought to be a more robust measure than simple percent agreement calculation, since κw takes into account the possibility of the agreement occurring by chance [45]. κw was calculated using an online tool developed by Lowry (1998), because SPSS did not have a κw calculator [46]. Agreements of < 0 indicated poor agreement, 0–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1.00 almost perfect [45].
Bland-Altman plots were constructed to assess the differences between the methods for each nutrient. Limits of agreement (LOA) were calculated as:
$$ {\displaystyle \begin{array}{c}\mathrm{Upper}\ \mathrm{limit}=\mathrm{mean}\ \mathrm{of}\ \mathrm{the}\ \mathrm{difference}+\left(1.96\times \mathrm{standard}\ \mathrm{deviation}\right)\\ {}\mathrm{Lower}\ \mathrm{limit}=\mathrm{mean}\ \mathrm{of}\ \mathrm{the}\ \mathrm{difference}-\left(1.96\times \mathrm{standard}\ \mathrm{deviation}\right)\end{array}} $$
(where difference refers to sqFFQ minus WFR for each nutrient), therefore indicating the range in which approximately 95% of data fall. Lastly, using linear regression analyses, the mean was regressed against the difference of the means to check for proportional bias [47].
Calibration of the sqFFQ
In the instance where there would be considerable under-, or, overestimation of nutrient intake measured by the sqFFQ, the last step in the validation process was to calibrate the sqFFQ nutrient values against the WFR values, so that it produced similar estimates to the WFR [48]. Multivariable linear regression analyses were used to determine the coefficients needed to derive new calibrated sqFFQ values, using a linear equation (independent variables: original sqFFQ nutrient values, age and/or sex; dependent variable: WFR nutrient values). To ensure the assumptions for multivariable linear regression were met (that is, residuals were normally distributed), certain dependent variable nutrients were entered into the model as cube root values. The final value calculated using the linear equation was back-transformed by cubing. Following this, correlations, comparison of medians, percentile ranks, κw statistics and Bland Altman plots were repeated with the new data to demonstrate improvement in the performance of the sqFFQ.