Skip to content

Advertisement

BMC Nutrition

Research article
Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Formative evaluation of the usability and acceptability of myfood24 among adolescents: a UK online dietary assessments tool

  • Salwa A. Albar1, 2, 4, 5, 6, 7Email author,
  • Michelle C. Carter1,
  • Nisreen A. Alwan3,
  • Charlotte E. L. Evans1,
  • Janet E. Cade1 and
  • on behalf of the myfood24 consortium group
BMC Nutrition20151:29

https://doi.org/10.1186/s40795-015-0016-8

©  Albar et al. 2015

Received: 30 April 2015

Accepted: 29 September 2015

Published: 23 October 2015

Open Peer Review reports

Abstract

Background

Myfood24 is a new online 24 h dietary assessment tool developed for use among the UK population. Limited information is available on the usability and acceptability of such tools. Hence this study aims to determine the usability and acceptability of myfood24 among British adolescents (11-18y) before and after making the improvements.

Methods

A total of 84 adolescents were involved in two stages. In stage-I (beta-version of myfood24), 14 adolescents were recruited, 7 of whom (group-1) were asked to enter standardized tasks in a testing room with screen capture software. The remaining 7-adolescents (group-2) were asked to report their previous food intake using myfood24 at home. All participants then completed a usability and acceptability questionnaire. Stage-II was carried out after making amendments to the live-version of myfood24 in which 70 adolescents were asked to enter their food intake for two days and then complete the same questionnaire. Thematic analysis was conducted of observer comments and open-ended questions.

Results

Navigation, presentation errors and failure to find functions were the main usability issues identified in the beta-version. Significant improvements were found in the usability and acceptability of most functions after implementing certain features like a spell checker, auto-fill option, and adding ‘mouse hover’ to help with the use of some functions. Adolescents’ perceptions of searching food items, selecting food portion sizes and making a list function were significantly improved in the live-version. The mean completion time of myfood24 reduced from 31 (SD = 6) minutes in the beta-version to 16 (SD = 5) minutes in the live-version. The mean system usability score (SUS) of myfood24 improved from 66/100 (95 % CI 60, 73) in the beta-version to 74/100 (95 % CI 71, 77) in the live-version, which is considered as ‘good’. Of the adolescents in stage-II, 41 % preferred using myfood24 to the interviewer-administered 24 h recall because myfood24 was quicker, easier to use and provided the adolescents with privacy when reporting dietary intake.

Conclusion

Considering adolescents’ feedback has helped in improving the usability and acceptability of the final-version of myfood24. myfood24 appears to support adolescents’ need in reporting their dietary intake, which may potentially improve the overall quality of adolescents’ self-reported dietary information.

Keywords

Myfood24Usability and acceptability testingAdolescentsUK online 24 h dietary assessments toolWeb-based dietary assessments

Background

Collecting information on food and dietary intake provides valuable insights into the associations between diet and health, and helps to evaluate the impact of intervention programmes. Thus, high quality dietary assessment instruments are needed with high validity and reliability for all ages [1, 2]. Measuring adolescents’ dietary intake is still challenging and many sources of measurement error are reported [3]. Adolescents may be less interested, less motivated and less cooperative compared to other age groups with regard to reporting diet [3, 4]. Preliminary studies among adolescents suggest that the innovative use of new-technology may improve the accuracy of adolescents’ dietary information [5, 6].

Online dietary recall systems have been developed and tested in a number of countries based on their national food database and language, such as the Automated Self-Administered 24 h recall (ASA24) [7] and DietDay [8] in the US, and Web DASC in Denmark [9]. Furthermore, Young Adolescents’ Nutrition Assessment on Computer (YANA-C) was originally developed to collect dietary data among Belgian–Flemish adolescents, and another adapted version was developed to be used online: Children and Adolescents’ Nutrition Assessment and Advice on the Web (CANAA-W) [10]. There are also some tools available in the UK; for example, the Synchronized Nutrition and Activity Program (SNAPTM) for 7 to 15 year old children [11] and INTAKE24 for 11 to 24 year olds [12]. However, there is no British online 24 h dietary recall tool suitable for the whole population.

myfood24 (Measure Your Food on One Day) is a new and innovative self-administered online 24 h dietary recall/record tool designed to make the collection of multiple days’ worth of automated self-administered 24 h recall/records feasible among a wide variety of groups and settings in large scale epidemiological studies. The development of myfood24 was based on focus group evaluation of existing tools for different age groups, including adolescents [13], adults and older adults, thereby incorporating a wide age range of the British population [2]. Lessons were also learnt from the design of ‘My Meal Mate’ (MMM), a Smartphone app for weight loss [14] designed by the same research team.

Most of the available dietary assessment websites vary based on their features (website-functions and usability), food database and dietary assessment methods used (recall or record). myfood24 has been designed to balance the need for researchers to collect detailed dietary information and users’ desire for a quick and easy tool. Thus, to reduce the completion time, myfood24 implements some aspects of the Automated Multiple Pass Method (AMPM) [15], with an optional quick list as the first pass; a detailed food search; forgotten items; prompts for a limited number of foods; and final review before submission. The food database in myfood24 is unique and based on several food composition data sources (~3500 from British food composition tables; ~ 33,000 manufacturers’ items; 700 fast foods and ~4,600 supermarket items) to provide approximately 45,000 food items in the tool, with the potential to be updated. Moreover, one of the advantages of using myfood24 is that researchers can select between recall or diary option [16, 17].

Using new technology in health education and dietary assessment methods is in its infancy, therefore information regarding the process of developing and testing such tools is limited [18]. Successful web-based dietary assessment software needs to be intuitive, simple and engaging for users [2]. In order to identify these components, usability-testing is rated as the most effective method for creating greater strategic impact and enhancing the final product [19, 20].

The terms of usability and acceptability are usually used interchangeably in many different ways, as there are no absolute definitions for them. Usability is the overall technical term of the user experience, user friendliness and ease of use [21]. Usability means how well the users can use the system functions in terms of: easy to use and learn, efficient to use, easy to remember, few errors and pleasant to use [22]. Whereas acceptability can be defined as user’s willingness within a special target group to employ the tool for the tasks it is designed to support [23]. Usability and acceptability are the two leading criteria for successful design [21]. Usability-testing should be carried out in different stages, in the beta-version of the tool (the final version of the development process and before final amendment and public release) and after making changes to evaluate the usefulness of the modifications and the overall acceptability of the tool [24–27].

Although usability-testing has been used in various developed e-health software, such as smoking websites [28] and online physical activity tools [29], there is limited information published on the usability-testing of online dietary assessment websites [30]. Therefore, this study aims to identify usability and design issues associated with the completion of myfood24 (beta-version) and to determine tool acceptability among adolescents (11–18y) before and after making improvements (beta & live-version).

Methods

myfood24 features

myfood24 contains two main areas: the researchers area where researchers can customise the website to fit their study design by adding project related text and logo, personalized additional help, a tailored invitation, send reminder emails to participants, select recall or diary option and select whether to display nutrient summary for participants or not. In the participants’ area, participants can select food items from the search bar or create a recipe in the ‘recipe builder’. Users can filter food items by selecting any of the filter options: ‘recently used items’, ‘recipes’ that have been entered previously and ‘food group’. For instance, if a search for ‘apple’ is made, only categories containing that item will be displayed, such as apple (fruit), apple pie (dessert), apple juice (drink), or by brand.

After the food item is chosen, food portion size (FPS) options are given as a dynamic extension of the search result screen to enable a seamless (all-in-one) user experience. There are various options presented to cover different food types and to maximize participants’ abilities to determine their portion size. myfood24 has images for 100 different food types (most frequently consumed foods), each food has 7 portion sizes to select from. To maximum image coverage, images are assigned to all food items that look similar. Food photos were obtained from the Young Person’s Food Atlas Secondary [31]. Moreover, participants can alternatively enter a specific weight in gram/ml if they know the exact portion size or use standard pack size.

The selected food item and its portion size are added to the meal tracker display area (breakfast, lunch, evening meal, snacks and drinks). To enhance the completeness of reporting, a pop-up message appears on the screen after selection of some common foods (e.g. bread, cereal) and probing for food items (e.g. butter or margarine; milk) that are often eaten in combination. Moreover, if participants forget to add the portion size after entering the food item, the software is programmed to prompt the participant to check the entry. Before submitting the dietary data the review screen prompts participants to check their entries and answer additional questions regarding supplement intake and whether or not their food record is typical for a regular day. After submitting the diary it can no longer be edited by users. A ‘thank you’ message with an optional summary of energy and eight nutrient intakes (energy, macronutrients, saturated-fat, fibre, sugar and salt) is displayed. For enquiries relating myfood24 use in research, please visit www.myfood24.org.

Usability-testing techniques

Figure 1 illustrates the study design which consists of two stages: stage-I was conducted in the beta-version of myfood24 and stage-II was conducted after making the amendments in the live-version of myfood24.
Figure 1
Fig. 1

myfood24 usability and acceptability study design

Participants

It has been reported that 80 % of usability problems are uncovered with inclusion of five participants and 90 % with ten participants with each additional participant contributing fewer new problems [19, 32]. Usability-testing of the beta-version therefore requires approximately 10 to 20 participants to enable the vast majority of usability issues to be identified [33]. In stage-II, more participants were recruited to test the acceptability and feasibility of using myfood24 (live-version) in a larger sample to ensure a representative sample of adolescents from each age group and gender. Adolescents aged 11–18 years old were recruited from four different secondary/high schools from different areas in Leeds. Participants who did not speak English as their first language were excluded, as myfood24 uses only the English language. Experience in using a computer was not required. Written consent was obtained from all participants and parental consent was obtained for adolescents who were younger than 16 years old. The study was reviewed and received approval from the University of Leeds Research Ethics Committee (MEEC 11–046).

Procedure

Stage-I: myfood24 (beta-version)

The researcher informed the participants in all stages that the study was not meant to test their ability to use the computer, rather it was to test the quality and attributes of myfood24. There were two different groups. In group-1, the researcher identified a list of key tasks that all users should be able to perform on myfood24. These pre-defined tasks were designed in a scenario to test specific features of myfood24; for example, using the ‘make a list’ function, entering food recipes and selecting different portion size options, as well as correcting mistakes (Additional file 1). The standardised tasks contained a variety of foods from the most commonly consumed foods by adolescents [34]. The test was carried out in a meeting room at the University of Leeds and lasted around 60 min for each participant.

Screen capture software (Camtasia Studio version 8 (Techsmith, USA)) was used to record participants’ screens and verbal recordings whilst undertaking the user tests of myfood24. During the session, users were instructed to speak out loud about positive and negative experiences as they performed each task, and they were encouraged to complete the tasks by themselves. The researcher observed the users indirectly (because some participants may have felt uncomfortable) and reported the users’ behaviour in light of the task analysis criteria (Additional file 1). At the end, the participants filled in a usability-acceptability questionnaire, and then received £5.00 remuneration.

In group-2, remote usability-testing [35] (participants completing the test at home) was carried out to obtain a clear indication of how myfood24 would perform in a real life situation and to test the availability of different food items in the software. Participants were un-moderated and asked to complete one 24 h dietary recall using myfood24. Then complete the usability-acceptability questionnaire that was used with group-1. Users were also asked to provide written comments on any problems they faced.

Stage-II: myfood24 (live-version)

Based on users’ feedback from usability-testing in stage-I, amendments were made to develop the final version of myfood24 (live-version). Seventy adolescents were recruited and divided into 14 groups, each group containing five-participants. This was based on logistical reasons in order to manage the research in schools. In each group, 3 participants were assigned to start with myfood24 and 2 were assigned with the interviewer (MPR) to reduce bias due to a learning process in the second set of responses. They were asked to report their 24 h dietary recall in myfood24 (without any assistance from the researcher) and then attend an interviewer-administered 24 h recall on the same day for two non-consecutive days at school. After the second use of myfood24 participants were asked to complete the usability-acceptability questionnaire.

Usability-acceptability questionnaire

The questionnaire consisted of three parts. The first part covering demographic information and questions regarding participants’ self-defined attitudes towards new technology. The second part contained three open-ended questions to comment on myfood24 (my favourite thing about myfood24 was…, my least favourite thing… and which particular areas users think we need to address in detail within the ‘Help’ section…). Finally, participants answered questions regarding myfoof24’s acceptability and satisfaction. This contained eight statements with a five-point Likert scale (1 strongly-disagree to 5 strongly-agree), in addition to the System Usability Scale (SUS) (Additional file 1) [36, 37]. SUS is a validated, reliable and free tool to use (34). There were two questions added to the previous questionnaire in stage-II. To identify adolescents’ opinions about myfood24 compared to the interviewer-administered 24 h recall, they were asked to rank the ease of undertaking each method using a five-point Likert scale (1 very-easy to 5 very-difficult), choose their preferred method and give the reason why.

Data analysis

In stage-I, qualitative data from the transcribed verbatim of the screen record for each participant and the researcher’s observational notes (group-1), as well as users’ comments regarding issues they experienced while using myfood24 (group-2), were analysed in accordance with the principles of thematic analysis [38]. The different codes were sorted into potential themes and all the coded data extracts were gathered within these themes. The themes covered the key areas of myfood24 and are presented in Table 2. All findings from group-1 and group-2 were combined to report the full range of usability issues and users’ recommendations to improve the tool. Findings from the open-ended questions in the questionnaire were analyzed using thematic analysis [38] and some typical quotes were selected to represent different views of users’ acceptability.

In order to reduce the risk of bias, all qualitative data were coded by two researchers and conflicts in coding decisions were reviewed by all researchers and resolved by consensus. Furthermore, the overall completion time in myfood24 was calculated in both stages. The overall SUS for each respondent was calculated for both stages. SUS is a 10 part statement that consists of a 5-point Likert scale for each part. For statements 1, 3, 5, 7 and 9 the contribution for each is the scale position minus 1. For statements 2, 4, 6, 8 and 10 the contribution for each is 5 minus the scale position. The sum of the scores is then multiplied by 2.5 to obtain the overall value. The overall SUS score ranged from 0 (negative-views) to 100 (positive-views) [37]. Moreover, findings from the eight statements of the Likert scale regarding myfood24 acceptability and user satisfaction were calculated before and after making the improvements and the overall median and interquartile range (IQR) were calculated. A Mann–Whitney U-test (two-sample) was used to test the rank differences in users’ perceptions between myfood24’s beta-version and live-version. Unpaired t test was used to test the improvement in the SUS in the 1st and 2nd stage. Analyses were performed using the STATA statistical software release 11 (Stata Corporation) and the significance level was set at 0.05.

Results

Characteristics of participants

In total, eighty four participants took part in this study. Table 1 illustrates participants’ general characteristics at each stage. Most of the participants were of white ethnicity and their ages ranged from 11 to 18 years old. All users in stage-I and 69 (99 %) in stage-II had internet at home and 12 (86 %) and 59 (84 %), respectively, accessed the internet daily. On a scale of 1 to 10, 7 (50 %) of the participants gave themselves 9/10, 5 (36 %) gave themselves 10/10, and the rest rated themselves 8/10 in terms of their confidence in using technology. Similarly, in stage-II participants ranked themselves in the last three highest scales regarding confidence in using technology, 26 (37 %) gave themselves 10/10, and 20 (29 %) gave themselves 9/10.
Table 1

Sample characteristics by study stage, completion time and system usability scale (SUS) of myfood24

General characteristics

Stage-I (n.14) Beta-version of myfood24

Stage-II (n.70) Live version of myfood24

Age (mean, SD)

15.6 (2)

14.6 (2)

Gender (girls)

8 (57 %)

35 (50 %)

Ethnicity (white)

12 (86 %)

61 (87 %)

    (other)

2 (14 %)

9 (13 %)

Confident in using technologyb

  

    (10/10)

5 (36 %)

26 (37 %)

    (9/10)

7 (50 %)

20 (29 %)

    (8/10)

2 (14 %)

19 (27 %)

Access the internet (daily)

12 (86 %)

59 (84 %)

Access the internet at home

14 (100 %)

69 (99 %)

Completion time (mean, SD)

31.4 (9)

16.2 (4)

SUSa (mean, 95 % CI)

66 (60, 73)

74 (71, 77)

aSUS system usability score [36]

bfive participants did not answer the question

Time completion

The mean completion time of myfood24 (beta-version) in stage-I was 31.8 (SD = 3) minutes when using standardised tasks (group-1), and 31.0 (SD = 9) minutes for participants in group-2. After making the amendments to the beta-version of myfood24 the completion time was reduced to 16.2 (SD = 5) minutes in the live-version.

Usability of myfood24 (beta-version)

The thematic analysis from stage-1 revealed a number of key issues that needed to be addressed to enhance the overall utility of myfood24 among adolescents. The usability issue and participants’ comments/recommendations to enhance the tool are illustrated in Table 2. All reported issues were feedback to the software developers to inform the final development of the live-version. Snapshots of myfood24 have been annotated to give an example of the usability issues encountered during the test (Fig. 2).
Table 2

Usability problem of the beta-version of myfood24 and adolescents’ comments and recommendation

Functions

Problems

Users’ comments/recommendations

My profile (age, sex and general instruction)

• Missing the list of age and counties from drop down menu.

• aProvide missing action (age and list of counties

• All users read the instructions and find it easy to understand

Make a list function (area where users can report all food consumed)

• Not clear how to use it, users typed food name with its portion or brand so nothing comes after searching

• aAdd “mouse hover” to help in how to use it

• When users learnt how to use it, they found it useful as a reference point to remember what they ate

Food search bar (select food items from the search bar, filter the findings using category or brand)

• The search function is not reacting enough to find misspelled words

• aAdd spell checker and auto filer in the food search

• Spaces between words can be an obstacle when searching (e.g. Kit Kat and Kitkat)

• aAdd more synonyms for food name which can be written in different way

• Uncertainty in the results of food searching.

• aPresent the results in alphabetical order

• Brands and generic foods are mixed up, which hampers finding the right food (Fig. 2)

• aRe-programme the filtering system to be more effective

• Some users found the “search bar” superior to “make a list” function

Food portion-size (select one of three options: Standard pack size or select from food photos or enter exact g/ml)

• Not clear how to do the action correctly

• aMaking a tutorial video regarding how to select portion size.

• Average portion presented with photo portion size, so users were confused about which one to choose (Fig. 2)

• Label food photos with grams to help adolescents realize the average portion in relation to other photos. Users said that may help them make the right decision

• Uncertainty of some food portions, such as some drinks has one option while others have three options

• aDifferentiate visually between average and food photos by writing ‘OR’ or present them in different levels

• Free entering for ‘total gram’ does not work with some foods

• Entering foods portion-size becomes easy in the second use

Pop-up for missed items (Have you missed any of these items? Margarine, butter ..?)

• It seems unnecessary for certain foods and caused a bit of a hassle

• aIt may be better to appear once after each meal and at the end of the diary

• It would be easier if participants could click on the items in the prompt to take them straight to the food search bar

Recipe builder (create new recipe and select the portion from exist recipe)

• After submitting the recipe, there is no visual sign to find the saved recipe

• aMake enough visual sign (e.g. make the icon flashy)

• Not clear how to enter portion consumed, how to add it to the diary

• Need information in help option or in the instruction in “My profile”

Meal slot (Breakfast, Lunch, Evening, Snack, Drink)

• Having ‘drinks’ as an option confused the users ( they thought all drinks should be in this section (Fig. 2))

• Better to remove it, or mention when to use it in help section

• Not easy to move (swap) selected foods between meal slots

• It would be easier to drag and drop food to the right meal slot

Submission page (Prompts to check entries, answer additional questions: supplements intake and if their food record represent usual intake

• There were three submission buttons on the page which confused users about which one to press

• aKeep only one submission and make it visible for users

• There is no enough visual sign for the additional questions

• aThe question needs to be in different colour or bold

• Some users did not understand what was meant by ‘supplements’

• aAdd simple definition between brackets

Nutrient summary

• Bugs in the nutrients summary figures

• afixed bugs

• Users like to compare their intake with the recommendation

aaction made in myfood24 live-version

Figure 2
Fig. 2

Snapshot of food search bar and food portion size in myfood24 (beta-version). (Red text indicates areas identified for improvement)

From the open-ended questions, four themes emerged regarding the most favourite aspect of myfood24, which included design and layout, nutritional feedback, ‘easy to use’, and the availability of many options in the tool. Participant-6 said “finding out how my diet compared to the guide is very useful”. Participant-5 said “there are many options to enter the food diary and they were well integrated”. In contrast, many pop-up questions, technical issues and difficulties in selecting FPS were the least favourite aspects in myfood24. Participant-4 stated that “It took you a while to find the products”. Participant-2 indicated that “Choosing the FPS was not straight forward, as there are many options making it difficult to choose”. Participant-1 said “pop-up questions were unnecessary in certain foods and were slightly annoying”. Three themes were revealed regarding the areas adolescents thought should be addressed in detail in the ‘Help’ section. Two adolescents asked for more instructions to be added regarding how to use the ‘recipe builder’ and two mentioned adding a short video on how to enter cooked foods and select FPS. Most of them (10 (71 %)) stated that the tool was easy to use and that there was no specific area that needed to be addressed in the help functions. Participant-12 said “The website is easy to use and therefore doesn’t need any further instructions”.

Adjustments

A number of key changes were made to improve myfood24, including simplifying certain words, ensuring sufficient visual appeal for certain functions by changing the colour or font, and reducing the number of pop-ups for missed items to ensure a fast completion time. ‘Mouse hover’ was also added to help with the use of certain functions. Adding certain features to the food search, like a spell checker, auto-fill option, and presenting the results in alphabetical order, also took place (Table 2).

Acceptability of myfood24

Table 3 presents the findings from the 8 statements Likert scale before and after making the amendments in myfood24. With the beta-version, users assessed the time taken to complete myfood24 as being reasonable. They liked myfood24’s design and agreed that the terminology used was understandable (57 %, 72 %, 72 % respectively). Conversely, 50 % of adolescents disagreed with the simplicity of searching for food items in the database, and 57 and 50 % of them neither agreed nor disagreed with the simplicity of adding home recipes or selecting FPS respectively. However, there were significant improvements in adolescents’ perceptions regarding myfood24’s functions in the live-version compared to the beta-version, particularly in terms of time for completion, design and layout, searching food items, making a list function, selecting FPS and correcting mistakes. The average SUS score for the beta-version of myfood24 was 66/100 (95 % CI 60, 73) and this was increased to 74/100 (95 % CI 71, 77) in the live-version. There was a significant improvement between the mean SUS for the beta-version and live-version of myfood24 with a mean difference of 7.5 points (95 % CI: 0.2, 14.8; P < 0.04) with 80 % power to detect the changes.
Table 3

Users’ acceptability of myfood24 before and after making the amendments

Acceptability questions

Versiona

Acceptability (5-point Likert Items)

Median

IQRb

P-value c

Strongly disagree

Somewhat disagree

Neutral

Somewhat agree

Strongly agree

Time for completion was reasonable

B (n.14)

0

2 (14 %)

4 (29 %)

8 (57 %)

0

4

3,4

0.004

L (n.70)

1 (1 %)

4 (6 %)

9 (13 %)

26 (37 %)

30 (43 %)

4

4,5

Terminology used was easy

B (n.14)

0

1 (7 %)

3 (21 %)

5 (36 %)

5 (36 %)

4

3,4

0.716

L (n.70)

1 (1 %)

5 (7 %)

10 (14 %)

26 (37 %)

28 (41 %)

4

4,5

I like the design and layout of myfood24

B (n.14)

0

2 (14 %)

2 (14 %)

5 (36 %)

5 (36 %)

4

3,5

0.032

L (n.70)

0

4 (6 %)

13 (19 %)

33 (46 %)

20 (29 %)

4

4,5

Food searching was simple and efficient

B (n.14)

3 (21 %)

4 (29 %)

2 (14 %)

4 (29 %)

1 (7 %)

3

2,4

0.047

L (n.70)

0

12 (17 %)

24 (34 %)

25 (36 %)

9 (13 %)

3

3,5

Make a list function was useful

B (n.14)

1 (7 %)

3 (21 %)

4 (29 %)

5 (36 %)

1 (7 %)

3

2,4

0.002

L (n.64)

0

3 (5 %)

11 (17 %)

35 (55 %)

15 (23 %)

4

4,5

Selecting food portion size was easy

B (n.14)

1 (7 %)

1 (7 %)

7 (50 %)

5 (36 %)

0

3

3,4

0.045

L (n.70)

1 (1 %)

11 (16 %)

15 (21 %)

20 (29 %)

23 (33 %)

4

4,5

Add home cooked recipe was straight forward

B (n.14)

2 (14 %)

3 (21 %)

8 (57 %)

1 (1 %)

0

3

2,4

0.273

L (n.48)

2 (4 %)

8 (17 %)

16 (33 %)

16 (33 %)

6 (13 %)

3

3,4

Correcting my mistakes was easy

B (n.14)

0

2 (14 %)

5 (36 %)

4 (29 %)

3 (21 %)

3.5

3,4

0.026

L (n.66)

0

3 (5 %)

10 (15 %)

24 (36 %)

29 (44 %)

4

4,5

aB beta version, L live version

b(IQR) interquartile range

cMann–Whitney U-test comparing beta and live version results

Adolescents’ views on myfood24 & interviewer-administered 24 h recall

In stage-II, adolescents rated the ease of undertaking the two methods similarly; 32 (46 %) of them rated the interviewer-administered 24 h recall as very easy, 28 (40 %) as easy, 9 (13 %) neither easy or difficult, and 1 (1 %) difficult. Whereas 28 (40 %) rated myfood24 (live-version) very easy, 30 (43 %) easy, 9 (13 %) neither easy or difficult, and 3 (4 %) difficult. None of them rated any methods as very difficult. Moreover, 41 (59 %) of the adolescents stated that they preferred the interviewer-administered method and this was for two main reasons, namely “human interaction” and “easiness of completing the food diary” in terms of finding the exact food item, selecting FPS and being prompted to remember more food that was eaten. Most of adolescents who preferred the interview method believed that talking to an actual person would be much easier, friendly and more trustworthy as they depended on the interviewer’s knowledge and experience. Participant-37 said that “the interviewers were friendly and you can properly communicate with them”. Participant-45 said they were “better at jogging memory on what I have eaten and give more valid answers as to portion size to differentiate what size was eaten”. Participant-67 said “I didn’t have to do all the work” and another said that it provided “more help and explanation, reminded me when I forgot things”.

In contrast, 29 (41 %) adolescents preferred using myfood24. This was for three main reasons, namely confidentiality, simplicity of myfood24 and options availability, and instant nutritional feedback”. Participant-50 stated that “I prefer using myfood24 as it’s faster, easy and more efficient because some people like me don’t really like sharing what they eat”. Participant-59 mentioned that “It is easier to admit if you have eaten too much”. Participant-19 mentioned that “The website was quicker and easier to use and you can access the tool at any time”. Participant-69 said there was “No pressure to answer quickly” and that it was “easy to find food and FPS; interview feels too formal”.

Discussion

A number of usability issues were identified in the beta-version of myfood24. Most of these issues were related to navigation, presentation errors and failure to find functions. Consequently, the overall acceptability and SUS score was ok and the completion time was quite long, at 31 min. However, after addressing these issues by fixing the error and adding certain features to the food search, like a spell checker and auto-fill option, significant improvements were found in most functions. The completion time of myfood24 (live-version) was reduced to 16 min and the overall SUS was improved so as to be good. Adolescents rated the ease of using myfood24 similar to the interviewer-administered method and 41 % of adolescents preferred using myfood24 for its simplicity and options availability, as well as privacy in reporting dietary intake.

Few studies have formally tested the usability of online dietary assessment tools. Most of these published studies are generally focused on the description of development and functionalities of the system rather than detailing the methods used to assess the tool [30]. Other studies have only reported users’ acceptability with a comparison between using the system and using a traditional method, like YANA [39] and INTAKE24 [12]. Others, such as ASA24 [40] and CANAA-W [10], have used focus groups to evaluate the tool, making it challenging to compare the findings.

Most usability issues were related to searching for food items and selecting FPS, as they are the main functions of the tool. Similar to our findings with myfood24 (beta-version), in YANA [39], less than 20 % of adolescents strongly agreed with the simplicity of searching for foods and 29 % of children disagreed with the simplicity of finding of foods items in CANAA-W [10]. Although children liked food photos in CANAA-W, selecting FPS was sometimes difficult [10]. In ASA24, children were unable to understand what to do at a given point in time [40], and in SCRAN24 (developed into INTAKE24), adolescents were often confused and needed help due to the use of different interface screens and the many instructions to read [12].

Improvements were found in the usability and acceptability of most functions of myfood24 after implementing certain features such as a spell checker, auto-fill option, and adding ‘mouse hover’ to help with the use of some functions. Although pop-ups to prompt for forgotten food items seems to be a helpful function, some users in this study and CANAA-W study indicated that pop-ups can become irritating and they maintained that a reminder on the overview screen would be more helpful to remind them [10]. Limited number of children and parents recommended that starting with an example of how to use CANNA-W is a useful guide [10]. However, none of the adolescents used the help function in the second stage of this study after adding more information and video tutorials.

The maximum completion time in INTAKE24 reduced from 50 to 21 min after making the amendments in the proto-type-version, resulting in an average of 13.4 min (06.2 mini to 20.3 max) [12]. Similarly, the mean completion time of myfood24 reduced from 31 (SD = 6) minutes in the beta-version to 16 (SD = 5) minutes in the live-version. However, the average completion time for the online system should be considered as an estimate, as there are many other potential factors that can affect the completion time such as internet speed, the setting (school, home or others), how familiar the participant is with the system as well as the type of food participants have consumed (home cooked or ready meal for example). In the second stage, adolescents used myfood24 for two non-consecutive days which may have improved adolescents’ ability to use the tool. The setting also varied between stage 1 and 2 which could also have had an impact on completion time.

Traditional paper based dietary assessment method takes approximately 30 min to complete and a further hour to code the diary [41]. The overall SUS score of myfood24 improved from 66 (beta-version) to 74 (live-version) out of 100-points. Using the adjective rating scale produced by Bangor et al. [37], myfood24’s beta-version was associated with ‘Ok’ (can be defined as a marginal level but closer to acceptable level) and the live-version of myfood24 was associated with ‘Good’ as adjectives (an SUS score above 68 is considered to be above average and anything below 68 is considered to be below average). No results were found describing usability scales of other online or computerized dietary assessment tools. However, results from the SUS for a web-based physical activity intervention reported an average score of 73 (SD = 15), which is associated with ‘good’ usability [29].

Adolescents in this study ranked the ease of undertaking myfood24 (live-version) to be similar to the interviewer-administered 24 h recall, and 41 % of them stated that they preferred myfood24 as it was quicker, easy to use and provided adolescents with privacy when reporting their dietary intake. We did not find specific differences in user’s characteristics between adolescents who preferred myfood24 or interviewer-administer methods. Questions about risky or sensitive behaviour may be answered more truthfully when using computerised self-assessment tools [42]. Therefore, using new-technology with adolescents can be promising as it might help in reducing underreporting of energy-intake. Receiving an instant feedback was one of the best features adolescents liked in myfood24, and another study found that adolescents would more likely use the tool if they could receive instant feedback [43].

This study has strengths in terms of its design; it used standardised and remote usability-testing to cover a wide range of usability issues. Furthermore, the tool was evaluated before and after making the amendments, and one of the most established and validated usability metrics (SUS) was used to evaluate the tool and to our knowledge none of the available tools used usability metrics to evaluate dietary assessment websites’ usability and acceptability.

The study has a number of limitations. The live-version was only tested in a school environment rather than the home setting where participants are more likely to complete their dietary assessment in large studies. Exam and class pressures could have caused adolescents to rush during the second test and therefore introduce bias into the results. In accordance with previous research [44], Norman reports that website design and usefulness evaluation depend on users’ emotions and past experiences and it is therefore important to carry out testing with a diverse sample of participants. Participants in this study were mainly from a white ethnic background with smaller numbers from black and minority ethnic groups (BME) which may have affected the findings if different ethnic groups have different dietary patterns and food sources. Although this study sample is representative of the UK population where the adolescents were recruited; more research on the feasibility and acceptability of using myfood24 targeting different ethnic groups is warranted.

Conclusion

Usability-testing on a new online dietary assessment tool generated important information used to improve the usability and acceptability of the final-version of myfood24 among adolescents. myfood24 appears to support adolescents’ need in reporting their dietary intake, which may potentially improve the overall accuracy of adolescents’ self-reported dietary information. Further research is needed to determine socio-economic and ethnic differences in usability and acceptability of online dietary assessment tools.

Notes

Abbreviations

ASA24: 

Automated Self-Administered 24 h recall

SNAP: 

Synchronized Nutrition and Activity Program

MMM: 

My Meal Mate

AMPM: 

Automated Multiple Pass Recall

YANA: 

Young adolescents’ nutrition assessment on computer

CANAA-W: 

Children’s and Adolescents’ Nutrition Assessment and Advice on the Web

SUS: 

System Usability Scale

FPS: 

Food Portion Size

Declarations

Acknowledgments

This study was funded by the Medical Research Council (MRC) (ref: G1100235/1). S. A. Albar, is in receipt of a scholarship from King Abdul-Aziz University, Jeddah, Saudi Arabia. Authors would like to thank the integrated office systems development at Digital Equipment Co Ltd., Reading United Kingdom, who developed SUS as a part of the usability engineering programme and particularly Professor Phil Kortum and Aaron Bangor for their assistance. Also, our appreciation and thanks go out to all participants.

The myfood24 consortium

S. A. Albar1,2, N. A. Alwan3, H. Brown1, J. E. Cade1, M. Carter1, C. E.L. Evans1, D. C. Greenwood4, N. Hancock1, L. J. Hardie5, M. A. Morris1 , K. White5 from the University of Leeds, Leeds and, H. Ford6, G. S. Frost7, U. Z. Mulla6, A. Petropoulou, P. A. Wark6 from Imperial College, London, UK.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Nutritional Epidemiology Group, School of Food Science and Nutrition, University of Leeds, Leeds, UK
(2)
School of Food Science and Nutrition, King Abdul-Aziz University, Jeddah, Saudi Arabia
(3)
Academic Unit of Primary Care and Population Sciences, University of Southampton, Southampton, UK
(4)
Department of Primary Care and Public Health, Global eHealth Unit, London School of Public Health, Imperial College London, London, UK
(5)
Division of Biostatistics, Leeds Institute of Genetics, Health and Therapeutics, University of Leeds, Leeds, UK
(6)
Molecular Epidemiology Unit, Leeds Institute of Genetics, Health and Therapeutics, University of Leeds, Leeds, UK
(7)
Department of Investigative Medicine, Nutrition and Dietetic Research Group, Imperial College London, Hammersmith Hospital, London, UK

References

  1. Subar AF, Thompson FE, Potischman N, Forsyth BH, Buday R, Richards D, et al. Formative Research of a Quick List for an Automated Self-Administered 24-Hour Dietary Recall. J Am Diet Assoc. 2007;107(6):1002–7.View ArticlePubMedGoogle Scholar
  2. Carter M, Wark P, Alwan N, Evan C, Albar S, Greenwood D, et al. Devlopment of a UK online 24-hour dietary recall assessment tool:Myfood24. In: Hofman A, editor. European Journal of Clinical Nutrition; Aarhus, Denmark: European Journal of epidemiology, Springer. 2013. p. S50.Google Scholar
  3. Livingstone MBE, Robson PJ, Wallace JMW. Issues in dietary intake assessment of children and adolescents. Br J Nutr. 2004;92(S2):S213. doi:10.1079/bjn20041169.View ArticlePubMedGoogle Scholar
  4. Goodwin RA, BrulÉ D, Junkins EA, Dubois S, Beer-Borst S. Development of a Food and Activity Record and a Portion-Size Model Booklet for Use by 6- to 17-year Olds: A Review of Focus-Group Testing. J Am Diet Assoc. 2001;101(8):926–8. doi:10.1016/s0002-8223(01)00229-2.View ArticlePubMedGoogle Scholar
  5. Zhu F, Mariappan A, Boushey CJ, Kerr D, Lutes KD, Ebert DS, editors. Technology-assisted dietary assessment. 2008.Google Scholar
  6. Boushey CJ, Kerr DA, Wright J, Lutes KD, Ebert DS, Delp EJ. Use of technology in children/’s dietary assessment. Eur J Clin Nutr. 2009;63(S1):S50–7.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Subar AF, Kirkpatrick SI, Mittl B, Zimmerman TP, Thompson FE, Bingley C, et al. The Automated Self-Administered 24-Hour Dietary Recall (ASA24): A Resource for Researchers, Clinicians, and Educators from the National Cancer Institute. J Acad Nutri Diet. 2012;112(8):1134–7. http://www.ncbi.nlm.nih.gov/pubmed/22704899.View ArticleGoogle Scholar
  8. Arab L, Tseng C-H, Ang A, Jardack P. Validity of a multipass, web-based, 24-hour self-administered recall for assessment of total energy intake in blacks and whites. Am J Epidemiol. 2011;174(11):1256–65.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Biltoft‐Jensen A, Trolle E, Christensen T, Islam N, Andersen L, Egenfeldt‐Nielsen S, et al. WebDASC: a web based dietary assessment software for 8–11 year old Danish children. J Hum Nutr Diet. 2014;27(s1):43–53.View ArticlePubMedGoogle Scholar
  10. Vereecken C, Covents M, Maes L, Moyson T. Formative evaluation of the dietary assessment component of Children’s and Adolescents’ Nutrition Assessment and Advice on the Web (CANAA-W). J Hum Nutr Diet. 2014;27:54–65. doi:10.1111/j.1365-277X.2012.01290.x.View ArticlePubMedGoogle Scholar
  11. Moore HJ, Ells LJ, McLure SA, Crooks S, Cumbor D, Summerbell CD, et al. The development and evaluation of a novel computer program to assess previous-day dietary and physical activity behaviours in school children: The Synchronised Nutrition and Activity Program™(SNAP™). Br J Nutr. 2008;99(6):1266–74.View ArticlePubMedGoogle Scholar
  12. Food Standers Agency. Development of a web-based 24-hour dietary recall tool for use by 11–24 year olds: INTAKE24. Newcastle University2013. http://www.food.gov.uk/scotland/news-updates/news/2014/13135/intake24. Accessed 12/12/2013.
  13. Albar SA, Alwan NA, Evans CEL, Cade JE. Adolescents' preferences in developing a UK online dietary assessment tool (myfood24): focus group study. Proceedings of the Nutrition Society. 2014;73(OCE2):null-null. http://dx.doi.org/10.1017/S0029665114001311.
  14. Carter M, Burley V, Cade J. Development of ‘My Meal Mate’–A smartphone intervention for weight loss. Nutr Bull. 2013;38(1):80–4.View ArticleGoogle Scholar
  15. Raper N, Perloff B, Ingwersen L, Steinfeldt L, Anand J. An overview of USDA’s Dietary Intake Data System. J Food Compos Anal. 2004;17(3–4):545–55. doi:10.1016/j.jfca.2004.02.013.View ArticleGoogle Scholar
  16. Cade J, Hancock N, Carter M, McLoughlin C, Wark P, Hatherley A, et al. PP38 Development of a new UK food composition database. J Epidemiol Community Health. 2014;68 Suppl 1:A62–3. doi:10.1136/jech-2014-204726.134.View ArticleGoogle Scholar
  17. Carter M, Albar S, Morris M, Mulla U, Hancock N, Evans C, et al. Development of a UK online 24-hour dietary assessment tool: myfood24. Nutrients. 2015;7(6):4016–32. doi:10.3390/nu7064016.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Illner A-K, Harttig U, Tognon G, Palli D, Salvini S, Bower E, et al. Feasibility of innovative dietary assessment in epidemiological studies using the approach of combining different assessment instruments. Public Health Nutr. 2011;14(6):1055–63.View ArticlePubMedGoogle Scholar
  19. Nielsen J. Quantitative Studies: How Many Users to Test? 2006. http://www.nngroup.com/articles/usability-metrics/. Accessed 12/05 2012.
  20. Gray WD, Salzman MC. Damaged merchandise? A review of experiments that compare usability evaluation methods. Human–Comput Interaction. 1998;13(3):203–61.View ArticleGoogle Scholar
  21. Petrie H, Bevan N. The evaluation of accessibility, usability and user experience, The universal access handbook. 2009. p. 10–20.Google Scholar
  22. Khosrowpour M. Planing for Effective Web-Based Commerce Application Development, Managing Web-Enabled Technologies in Organizations: A Global Perspective. 2000.Google Scholar
  23. Dillon A. User Acceptance of Information Technology. Encyclopedia of Human Factors and Ergonomics. London: Taylor and Francis.: In W. Karwowski (ed). 2001.Google Scholar
  24. Nielsen J. Usability of websites for teenagers, Nielsen Norman Group. 2005.Google Scholar
  25. Nielsen J. Success Rate: The Simplest Usability Metric. In: Nielsen Norman Group. NN/g Nielsen Norman Group. 2001. http://www.nngroup.com/articles/usability-metrics/.Google Scholar
  26. Rubin J, Chisnell D. Handbook of Usability Testing: Howto Plan, Design, and Conduct Effective Tests. Wiley; 2008.Google Scholar
  27. Loranger H, Nielsen J. Teenagers on the Web: usability guidelines for creating compelling Websites for teens www.NN/g Nielsen Norman Group.: NN/g Nielsen Norman Group. Evidence-Based User Experience Reserearch, Training, and Consulting; 2013 [cited 2014 17/05/]. Available from: http://www.nngroup.com/articles/usability-of-websites-for-teenagers/.
  28. Stoddard JL, Augustson EM, Mabry PL. The Importance of Usability Testing in the Development of an Internet-Based Smoking Cessation Treatment Resource. Nicotine Tob Res. 2006;8 Suppl 1:S87–93. doi:10.1080/14622200601048189.View ArticlePubMedGoogle Scholar
  29. Bossen D, Veenhof C, Dekker J, de Bakker D. The usability and preliminary effectiveness of a web-based physical activity intervention in patients with knee and/or hip osteoarthritis. BMC Med Inform Decis Mak. 2013;13(1):61.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Ruggeri BFF, Voci SM, Borges CA, Slater B. Assessment of the usability of a nutritional epidemiology computerized system. Rev Bras Epidemiol. 2013;16:966–75.View ArticlePubMedGoogle Scholar
  31. Foster E, Hawkins A, Adamson A. Young Person's Food Atlas Secondary. London, UK Food Standards Agency Puplications 2010. Google Scholar
  32. Nielsen J, Landauer T. A mathmatical model of the finding of usability problems, Proceedings of ACM INTERCHI’93 Conference. 1993.Google Scholar
  33. Virzi RA. Streamlining the design process: Running fewer subjects. Proceedings of the human factors and ergonomics society annual meeting; SAGE Publications. 1990.Google Scholar
  34. Albar SA, Alwan NA, Evans CEL, Cade JE, editors. Most comonly consumed food by Britsh adolescents: a comparison of portion size anf contribution to total energy intake by age and gender. Ghent, Belgium: 2013 Annual meeting of the International Society for Behavioral Nutrition and physical Activity (ISBNPA) 2013; p. 22–5.Google Scholar
  35. Schade A. Remote Usability Tests: Moderated and Unmoderated. In: Evidence-Based User Experience Research, Training, and Consulting. NN/g Nielsen Norman Group. 2013. http://www.nngroup.com/articles/remote-usability-tests/. Accessed 10/01/2014.
  36. Brooke J. SUS-A quick and dirty usability scale. Usability Evaluation Industry. 1996;189:194.Google Scholar
  37. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: Adding an adjective rating scale. J Usability Stud. 2009;4(3):114–23.Google Scholar
  38. Green J, Thorogood N. Qualitative Methods for Health Research. 2nd ed. London: SAGE Publication Lted; 2010.Google Scholar
  39. Vereecken CA, Covents M, Matthys C, Maes L. Young adolescents’ nutrition assessment on computer (YANA-C). Eur J Clin Nutr. 2005;59(5):658–67. doi:10.1038/sj.ejcn.1602124.View ArticlePubMedGoogle Scholar
  40. Baranowski T, Islam N, Baranowski J, Martin S, Beltran A, Dadabhoy H, et al. Comparison of a web-based versus traditional diet recall among children. J Acad Nutr Diet. 2012;112(4):527–32.View ArticlePubMedPubMed CentralGoogle Scholar
  41. Liu B, Young H, Crowe FL, Benson VS, Spencer EA, Key TJ, et al. Development and evaluation of the Oxford WebQ, a low-cost, web-based method for assessment of previous 24 h dietary intakes in large-scale prospective studies. Public Health Nutr. 2011;14(11):1998–2005. doi:10.1017/S1368980011000942.View ArticlePubMedGoogle Scholar
  42. Ngo J, Engelen A, Molag M, Roesle J, Garcia-Segovia P, Serra-Majem L. A review of the use of information and communication technologies for dietary assessment. Br J Nutr. 2009;101 Suppl 2:S102–12. doi:10.1017/S0007114509990638.View ArticlePubMedGoogle Scholar
  43. Hongu N, Pope BT, Bilgiç P, Orr BJ, Suzuki A, Kim AS, et al. Usability of a smartphone food picture app for assisting 24-hour dietary recall: a pilot study. Nutr Res Prac. 2014;9(2):207–12.View ArticleGoogle Scholar
  44. Norman DA. Emotional Design – Why We Love (or Hate) Everyday Things. New York: Basic Books; 2004.Google Scholar

Copyright

© Albar et al. 2015

Advertisement

BMC Nutrition

ISSN: 2055-0928

Contact us