Study setting

The study was conducted in the Butajera District of southern Ethiopia, which is located approximately 130 km from Addis Ababa (the capital city of Ethiopia) in the Guraghe Zone in the Southern Nations Nationalities and People’s Region (SNNPR). The district houses a Rural Health Program (BRHP) (owned and operated by Addis Ababa University), which is a health and demographic surveillance system (HDSS) with a continuous registration of vital and migratory events among ten selected villages. The studied district was purposely selected for the benefit of a better sampling frame and research infrastructure.

Study design and period

A community-based, cross-sectional study design was employed between November and December 2013. We administered the survey questionnaire twice to the study participants, and the second survey questionnaire was administered to the study participants after 7 days of the first administration. This repeated survey was used to determine the reproducibility of the household food insecurity assessment tool (HFIAS).

Study population and sampling

The study included a total of ten HDSS villages, of which nine were rural and one was urban, and the study population included households residing in these villages.

The sample size for the study was estimated using the formula for a single population proportion. Assuming an 80% prevalence of household food insecurity [13], a 95% confidence level, a 4% margin of error, and a design effect of 2, the calculated sample size was 768 households. With an expectation of a 5% non-response rate, the final sample size required was approximately 800 households.

The final sample size was allocated to the ten HDSS villages proportionate to the number of households in each village. We then used BRHP data set as a sampling frame and applied a simple random sampling method to select study households within a given village.

Data collection

The HFIAS is composed of nine items, which are asked with a recall period of 1 month. For each item, there was a follow-up of the frequency of the occurrence question. The tool was also translated into the Amharic (local) language by one of the authors (SHG) and initially reviewed with research assistants who were residents in the study area.

Face validity

We discussed all nine questions independently with four urban and four rural households in the neighboring villages, basically aiming at whether the questions were clear, easily understandable, and had a minimal amount of multiple interpretations. We read the nine questions to the women, and the responses were recorded. This was followed by a question about how the women understood each question. For example, we asked: “What do you understand when I ask you the question: In the past four weeks, were you or any household member not able to eat the kinds of foods you preferred because of a lack of resources?” We compared their understanding with that of the primary aim of the questions, and when there was a difference between what they understood and what we were actually looking for, a discussion followed on how that question could best be framed to make it clearer and contextually appropriate. Lastly, together with these women, the nine questions were adapted through modification, rephrasing, and adding examples when necessary.

We collected food groups that a household had consumed over the preceding 24 h [14], with the household food intake structured using the consumption of 12 food groups/item. The food groups included meat, fish, vegetables, fruits, eggs, potatoes, and other roots/tubers, beans, cereals/breads, oil, fat or butter, sugar or honey, as well as other types of foods such as coffee and tea.

Moreover, a range of sociodemographic data about the respondents such as age, education, religion, marital status, and occupation was collected, in addition to household-level data such as ownership and size of land, type of house and construction materials, availability of fixed assets such as radio, television, phone, bed and chair, and other household items, possession of domestic animals, type of water source for drinking and cooking, and availability and type of latrine.

Interviews were conducted by 20 trained and experienced junior nurses who are residents of the local district and had similar data collection experiences. The work was monitored by six supervisors, and interviews were primarily conducted with women in the household, as women are commonly responsible for food preparation in the study area. If women were unavailable, another adult who was present and ate in the household the previous day was asked.

Quality control

Questionnaires were controlled for completeness and logical errors, and where errors were found, the questionnaires were redone. Consistency checks were done to improve the quality of the data, and inconsistent entries and responses were crosschecked with the questionnaires and corrected accordingly.

Ethical clearance

The study protocol was approved by institutional review boards from the Addis Ababa University, College of Health Sciences. The study was also approved by the Regional Committee for Medical and Research Ethics, Western Norway (REK Vest). Information on the research objective was read to the participants, verbal informed consent was received, and the privacy and confidentiality of respondents was also maintained.

Data entry and analysis

We used EpiData Version 3.1 for the data entry, and the data was exported to Stata 11.0 (StataCorp, College Station, TX) for cleaning and further analysis.

Household wealth was constructed through a principal component analysis (PCA) of the household-level data described above. The PCA was done independently for urban and rural samples, and the score was then used to assign sampled households into quintiles that indicate poorest, poor, medium, rich, and richest.

The results from HFIAS delineate households across the four levels of food insecurity, including food secure, mild food insecure, moderate food insecure, and severely food insecure. The procedure and steps used to assign households to one of the levels is described elsewhere [3].

Factor analysis

An exploratory analysis was conducted on the nine items, using a Horn’s parallel analysis (PA) to determine the number of factors to retain. PA is a Monte Carlo-based simulation method that compares observed eigenvalues with that obtained from uncorrelated normal variables.

Validation

We evaluated the validity of the nine-item food insecurity assessment tool based on the following recommended criteria employed by a few similar studies [9, 15, 16].

The first criterion is the value of the Cronbach’s alpha, which is a measure for internal consistency, approaching 0.85 for the two rounds of surveys. Secondly, we tested for parallelism on HFIAS item response curves across wealth status, which was done by comparing the likelihood of affirmative responses to the nine items across households’ wealth quintiles.

Thirdly, we evaluated the presence of a dose-response relationship between food insecurity level and the previous day consumption of certain food items. We also tested for a dose-response relationship between household wealth status and food insecurity levels and used the extended Mantel-Haenszel chi square for linear trend to check for dose-response relationships.

Additionally, the reproducibility between the first and second HFIAS scores (HFIAS overtime) was estimated by means of a paired t-test.

Sample characteristics

A total of 1,516 households (767 in the first round and 749 in the second round) were studied across the two rounds of data collection. The response rate for the first and second round of data collection was 96% and 98%, respectively, and we included 1,056 and 460 households from rural and urban villages, respectively.

Responses to the nine HFIAS items

Face validity

We conducted a total of eight interviews (four each among rural and urban households), with the main aim of these interviews being to get the wording right for the specific context of the use of the HFIAS. We found that four of the nine questions were not straightforward and needed to be modified in either way for a better understanding. We discussed with these nine women how to best frame these questions to help make them clearer and contextually appropriate without losing the main aim of these questions. The questions that needed modifications, rephrasing, or adding examples were items 2–4 and item 6 (Table 2). The remaining five questions were kept in their original form since they were not difficult to understand and did not have multiple interpretations.

According to these interviews, we found a higher likelihood of affirmative response to the item “unable to eat preferred food.” This item was restructured, and examples were necessary to reflect a preferred food according to the respondent’s own economy, rather than a general preference. Interviewees better understood the item “…limited variety of foods” if the item was translated with a core meaning of “monotonous diet” or “almost the same meal every time.” The item “…eat food that you really don’t want to eat” was also rephrased as food items that are not eaten under normal circumstances, but could be eaten during times of hardship, such as during a severe food crisis or a severe drought. Two to three meals per day were considered as a normal meal frequency in the study district. Hence, the item “…eat fewer meals a day” was modified as “…eat less than two-three meals per day.” In summary, the key adaptations we made included rephrasing, adding local (context-specific) examples, as well as specifying the name of the previous month. Furthermore, the importance of explaining the objective of the study in detail prior to initiating the interview process was found to be vital in obtaining a correct response.

Exploratory analysis

Internal consistency

Parallelism

Figures 1 and 2 show HFIAS item response curves across household wealth status in the urban and rural villages of the study district. HFIAS item response curves were parallel across wealth status in urban households, and with the exception of HFIAS item 4, we observed that the likelihood of affirmative responses decreased as the household wealth status increased. When compared to the urban sample, a similar but less clear pattern is observed for rural households.

Food insecurity level and food intake

Among rural sample households, a dose-response trend between the food insecurity level and the likelihood of the previous day’s consumption of food items such as eggs, milk, fish, cereals, and beans was observed. For example, the likelihood of the previous day’s consumption of milk among food-secure households was 39.8% compared with 11.6% for severely food-insecure households. Similarly, the likelihood of the previous day’s consumption of eggs among food-secure households was 10.2% as compared with 2.1% for severely food-insecure households. The observed trend between food-secure and food-insecure households was significant for eggs (p < 0.018), milk (p < 0.01), fish (p < 0.01), cereals (p < 0.01), and pulses (p < 0.01), with the result also indicating that some food items such as vegetables and roots were less sensitive to household food insecurity levels. These food items are common staples in the diet for the rural parts of the studied district.

Among urban sample households, with the exception of the consumption of fish, dose-response trends between food insecurity level and the likelihood of the previous day’s consumption of other food items were observed. Additionally, the observed trends between food-secure and food-insecure households were also significant for all the food items that showed a dose-response trend.

FI severity and household wealth status

We found a significant and positive dose-response trend (p < 0.01) between household wealth status and levels of food security among rural and urban samples. With the exceptions of minor deviations between the two lowest (poorest and poor) strata, an increase in household wealth has been accompanied by an improved household food security.

An inverse but significant dose-response trend between (p < 0.01) household food insecurity level and wealth status was found among the urban samples (Figure 3). In the urban samples, a decrease in household wealth is accompanied by a higher household food insecurity level, though we did not find a significant dose-response trend between the household wealth and food insecurity level among rural households.

FI over time (reproducibility)

The overall HFIAS score did not change over the two rounds of data collection. The HFIAS scores for the first and second round of data collection were 6.1 ± 4.5 and 6.3 ± 4.2, respectively (Table 1), while the HFIAS score for the urban sample did not show a statistically significant difference across the two rounds of data collection. However, we found an increase in the HFIAS score for rural samples during the second round of data collection, and the difference in HFIAS score between the two rounds was also statistically significant (mean difference: -0.58; 95% CI: -1.07, -0.083).

Limitations and strengths of the study

Our study has some limitations that warrant consideration. The study was done in one of the nine regions of Ethiopia where the applicability of the findings may be limited to similar groups in southern Ethiopia. Respondent understanding of the nine items and expectations of possible support may have also influenced the results, and respondents could lean towards affirmative responses. Furthermore, in the absence of an established gold standard for household food insecurity, it becomes difficult to discuss the external validity of the HFIAS.

Among the main strengths of this study are the inclusion of urban and rural residents, the employment of a relatively larger sample size, and the application of a simple random sampling method to recruit households. These could help strengthen the generalizability of the findings and its application to similar contexts. We also administered two rounds of surveys in the same households to evaluate the repeatability of the HFIAS. However, this validation study was done at the household level, so further work might be needed to evaluate whether the tool can perform well if applied at the individual level (e.g., with adolescents) to help measure FI.