ORIGINAL ARTICLE

A systematic review of the application of interactive virtual reality to sport

David L. Neumann1,2 • Robyn L. Moffitt1,2 • Patrick R. Thomas2 •

Kylie Loveday1 • David P. Watling1 • Chantal L. Lombard1 • Simona Antonova1 •

Michael A. Tremeer1

Received: 26 April 2016 / Accepted: 10 July 2017 / Published online: 19 July 2017 ! Springer-Verlag London Ltd. 2017

Abstract Virtual reality (VR) technology is being increasingly used by athletes, coaches, and other sport-re-

lated professionals. The present systematic review aimed to

document research on the application of VR to sport to better understand the outcomes that have emerged in this

work. Research literature databases were searched, and the

results screened to identify articles reporting applications of interactive VR to sport with healthy human participants.

Twenty articles were identified and coded to document the

study aims, research designs, participant characteristics, sport types, VR technology, measures, and key findings.

From the review, it was shown that interactive VR appli-

cations have enhanced a range of performance, physio- logical, and psychological outcomes. The specific effects

have been influenced by factors related to the athlete and

the VR system, which comprise athlete factors, VR envi- ronment factors, task factors, and the non-VR environment

factors. Important variables include the presence of others

in the virtual environment, competitiveness, task auton- omy, immersion, attentional focus, and feedback. The

majority of research has been conducted on endurance sports, such as running, cycling, and rowing, and more

research is required to examine the use of interactive VR in

skill-based sports. Additional directions for future research and reporting standards for researchers are suggested.

Keywords Virtual reality ! Sport ! Exercise ! Systematic review

1 Introduction

The application of computer-based technology to sport is an area of intense interest. Such technologies include

computerised modelling, data acquisition and analysis,

mobile computers, and information technology networks (Baca et al. 2009). Virtual reality (VR) is another tech-

nology, and it was first applied to sport research in the

1990s, although there has been a resurgence of interest in recent years. VR refers to a computer-simulated environ-

ment that aims to induce a sense of being mentally or

physically present in another place (Baños et al. 2000; Sherman and Craig 2002). An important feature of VR is

that the individual can interact with the environment. In the

context of sport, interaction might occur through an exer- tion interface (Mueller et al. 2007). For example, physical

effort on a machine such as an ergometer can be related to

the speed of movement through a virtual race course. Motion capture video systems, infrared beams, and wear-

able sensors are other approaches that can be used to translate physical actions into virtual sport performance.

The key elements that define VR applications to sport

are the use of computer-generated sport-relevant content and a means for the athlete to interact with the virtual

environment. When defined in this way, the application of

VR to sport has a number of strengths. As noted by Hoffman et al. (2014), the VR environment can be con-

trolled and manipulated in specific and reproducible ways.

Hoffman et al. used these characteristics to train partici- pants to use a rowing race pacing strategy. VR can also be

used for assessment, to gain feedback on performance, and

& David L. Neumann [email protected]

1 School of Appled Psychology, Griffith University, Gold Coast, Queensland 4222, Australia

2 Menzies Health Institute Queensland, Gold Coast, Queensland 4222, Australia

123

Virtual Reality (2018) 22:183–198

https://doi.org/10.1007/s10055-017-0320-5

to practice specific skills. The VR environment does not

need to be limited to a single person. Other individuals may be present such as a coach, teammate, or competitor even if

they are physically located in another place. The ability to

connect with individuals via the Internet allows for inter- action without the need for travel. Finally, the increasing

availability of commercially produced software or full VR

systems avoids the need for specialised technical expertise and allows VR to be used in local gyms and at home.

The present study aimed to provide a systematic review of research on VR applications to sport. The PsycINFO,

SPORTDiscus, Scopus, Google Scholar, and Cochrane

Library databases were first searched for the existence of similar reviews. The search yielded systematic reviews on

VR in physical rehabilitation (e.g. Laver et al. 2015), VR in

psychological interventions (e.g. Meyerbröker and Emmelkamp 2010), and the use of exergames or active

videogames (e.g. Guy et al. 2011; Larsen et al. 2013; Peng

et al. 2013). The search helped to minimise overlap with existing reviews. Accordingly, the present review focused

on VR applications to sport and sport-related exercise with

healthy individuals. Studies were included if they were based on recognised sports even if those sports are used as

a component of physical conditioning or fitness programs

(e.g. cycling, running, rowing). As a result, this review focused on sport-based tasks as distinct from research with

interactive videogame systems that promote physical

activity through gameplay (i.e. exergames). The broad question examined in the present review was:

What is known about the application of VR to sport? In

particular, the review aimed to provide a definition of VR when used for sports. A further aim was to document the

aims, methods, and the broad findings from the research

conducted to date. Past research may be interpreted within the context of existing theories in sport and exercise, but of

particular focus in the present review were those factors

that are unique to VR applications to sport. The review also aimed to identify the gaps in the research to date and

develop recommended reporting standards for researchers

who apply VR to sport.

2 Literature review method

The literature search and selection method followed the

Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines (Liberati et al. 2009)

and the use of inclusion and exclusion rules described by

Meline (2006). Initially, the SPORTDiscus and PsycINFO databases were searched. The PsycINFO database includes

sport and exercise psychology journals, in addition to the

ACM Transactions on Applied Perception, the ACM Transactions on Computer–Human Interaction, and the

IEEE Transactions on Professional Communication. The

search was conducted using the terms: (sport* OR exercis* OR fitness OR physical train* or physical activit*) AND

(virtual realit* OR virtual environment* OR virtual world*

OR virtual system* OR virtual partner*). The search was limited to articles published from 1990 and up to the date

of the search (February, 2016) and included articles that

were in press. In addition, to identify any missed articles due to the inconsistent use of terms (e.g. virtual reality

versus virtual competitor) the reference lists of the articles selected for final inclusion from the database search were

examined. An examination was also made of the citations

of these articles, as collated from the Scopus database. The database search yielded 263 articles from the Psy-

chINFO database and 377 articles from the SPORTDiscus

database for a total of 640 articles. This reduced to 620 articles following removal of duplicates. A search of the

reference lists and citations yielded a further 66 unique

articles. Articles were screened for exclusion or inclusion by two individuals in a two-step process: title and abstract

(Step 1) and the full article (Step 2).1 The following

exclusion criteria were used: date (published before 1990), language (not published in English language), source (a

dissertation, thesis, abstract only, magazine article, or not a

peer-reviewed source), study type (a review, meta-analysis, commentary, letter to the editor, editorial report, or other

non-empirical article), no VR was used (a computer-gen-

erated environment was not used or there was no interac- tivity with the environment), population (the sample did

not include healthy human participants), task (the methods

did not include participation in a sport or a physical exer- cise that used equipment related to a sport or sports train-

ing), game (the task was based wholly on an exergame/

active videogame), rehabilitation (the purpose of the task was to rehabilitate those with physical injury), and measure

(performance, physiological, or psychological outcomes

were not the primary measures). Following the screening and selection process, 20 arti-

cles were included for full review. Of these articles, 18

were published in journals with journal citation metrics reported by the Web of Science database. The mean impact

factor (based on the most recent year) was 2.21 (range

0.06–4.47, SD = 1.21) indicating that the journals were largely of good quality although with some exceptions.

Consistent with this interpretation, the journal rankings

varied evenly across the full spectrum of Q1 (n = 5), Q2 (n = 5), Q3 (n = 4), and Q4 (n = 4). The articles were

coded by four authors and coding decisions were cross-

1 Cohen’s kappa for the decisions to exclude or include based on title and abstract (Step 1) was j = 0.64 and based on review of full article (Step 2) was j = 0.69, both of which fall within the guidelines for substantial agreement. Full agreement was reached at each step following discussion.

184 Virtual Reality (2018) 22:183–198

123

checked. Articles were coded for characteristics related to

the study (aims, type, location, conditions/groups, outcome measures, key findings), participants (sample size, age,

experience with sport), virtual reality technology (task

type, system, display features, point of view, others in the environment, immersion/presence measures), and sport

task (type).

3 Defining virtual reality in sport

VR when applied to sport may be defined as instances

when individuals are engaged in a sport that is represented in a computer-simulated environment which aims to induce

a sense of being mentally or physically present and enables

interactivity with the environment. This definition high- lights the computer-simulated nature and interactivity of

the virtual environment, which are key element of more

general definitions of VR (e.g. Baños et al. 2000; Sherman and Craig 2002). It also aims to highlight the application of

VR to sport from the perspective of the user (athlete).

Realistic responses to virtual environments are suggested to occur when the system induces a sense of presence and

the perception that the events are actually occurring (Slater

2009). In this respect, it is important that VR uses a computer-generated environment because this is a key

feature that allows for interactivity and the perception of

presence (Baños et al. 2000; Sherman and Craig 2002). In other words, the virtual environment or elements within it

will move or change in response to the actions of the

athlete. However, the method by which the virtual envi- ronment is presented to the athlete should not be specified

in the definition because it might impose technological

limitations to the application of VR to sport (see Steuer 1992).

In many applications outside of sport, the virtual envi-

ronment is displayed using a computer automatic virtual environment (CAVE) or head-mounted display (HMD).

The CAVE is composed of a large cube made up of display

screens that the user physically enters to become sur- rounded by the virtual environment. A HMD is a wearable

device that covers the eyes and thus removes vision of the

outside world. It has one or more small screens on which the virtual world is viewed in stereovision with a wide field

of view. The HMD is combined with head tracking to allow

the user to view areas of the virtual environment that are outside of the immediate field of view by turning their

head. Being a smaller, more portable, and a more afford-

able system, the HMD is more popular than the CAVE, although both may be regarded as sharing the same key

features of an immersive system (Slater 2009).

However, the potential applications for using CAVE and HMD systems can be limited for some types of sports.

A HMD may be impractical or potentially dangerous for

some sports. For example, running a race on a treadmill using a HMD can be hazardous because vision of the

moving treadmill is removed. The head movements and

sweating of the athlete can also make the HMD uncom- fortable to wear. Indeed, in no studies identified in this

review was a HMD system used despite researchers con-

sistently using the term virtual reality to describe their approach. The most common approach was a two-dimen-

sional depiction of the virtual environment using a com- puter screen or a projector. A computer screen or projector

has the advantages of ease of use and practicality with sport

but may induce less presence than a HMD or CAVE sys- tem. Further research is required to determine whether

there is significant difference in presence when a computer

screen or projector is used. Several instances can be identified in which researchers

used methodology that approximated the proposed defini-

tion of VR applications to sport. For example, some researchers have used a visual display that shows a video of

a real environment (e.g. Plante et al. 2006). Feltz et al.

(2011) conducted a series of studies that investigated the Köhler motivation gain effect with a plank exercise task.

These studies showed the participant via a video (i.e. not a

computer-generated avatar) and included a second indi- vidual shown on a second visual display without any

interaction. Videos of real environments and people may

have potential for VR applications to sport, but they must include elements of interactivity to fulfil the proposed

definition of VR. Similarly, other researchers have used

computer-generated environments to examine baseball batting (Ranganathan and Carlton 2007), handball goal-

keeping (Vignais et al. 2015), and soccer goalkeeping

(Stinson and Bowman 2014), but these did not allow for any interactivity with the environment and were not

included in the review. Thus, the present review was

focussed more specifically on interactive VR applications to sport. In some cases, it was also found that researchers

used a non-animated avatar against a blank screen (e.g.

Briki et al. 2013), but these do not meet the proposed definition because the methods did not simulate a real

environment.

Another important consideration for interactive VR applications to sport is the distinction between sport,

exercise, and exergaming. Sport may be defined as an

activity that requires motor skill and/or hand-eye coordi- nation combined with physical exertion and includes rules

and elements of competition (Australian Bureau of Statis-

tics 2008). Exercise, used synonymously with physical exercise, is a structured activity that may include repetitive

elements that is performed to maintain or improve physical

fitness (Australian Bureau of Statistics 2008). Exergame/ active videogame is a videogame played on commercial

Virtual Reality (2018) 22:183–198 185

123

game console systems (e.g. Xbox, Wii, PlayStation) that

combines gameplay with physical movements that are more than sedentary behaviour (Kim et al. 2014). Exercise

and exergames together represent a more general case of

enhancing physical activity and may not necessarily be based on a sport.

Exercises or exergames that are not based on a sport

clearly do not represent instances of VR applications to sport even if they incorporate a virtual environment.

However, investigators have used sport-related computer games, particularly those that run on a games console, in

research. Console games based on sports have been used to

examine skill acquisition and transfer in children (Rey- nolds et al. 2014) and adults (Tirp et al. 2015). However,

these applications lacked an appropriate exertion interface

(e.g. participants ran on the spot to simulate running in the game) or essential sporting equipment (e.g. no darts were

used in a dart game), and these aspects can make the task

substantially different to perform the sport in real life. VR has also been applied to exercise and improving physical

fitness. In several studies, researchers have used sport-re-

lated tasks such as cycling, running, and rowing (e.g. Murray et al. 2016). These applications have relevance to

sport performance particularly because many of these

studies have introduced elements of competition or pres- sure to meet team goals.

4 A conceptual framework for the application of virtual reality to sport

The application of VR to sport has taken many forms, with

various types of sport tasks, VR technologies, and types of

athletes used in the research. Some researchers have examined questions relating to the use of VR technology

itself, such as comparing outcomes when using VR and not

using VR (e.g. Annesi and Mazas 1997; Legrand et al. 2011; Mestre et al. 2011; Plante et al. 2003a), the effects of

immersion in the virtual environment (Ijsselsteijn et al.

2004; Vogt et al. 2015), and differences between computer- controlled and real virtual competitors (Snyder et al. 2012).

In contrast, other researchers have used VR technology as

part of a methodology to answer more general questions about factors related to sport performance. For example,

Oliveira et al. (2015) used a virtual partner as a means to

compare the effects of self-selected and externally imposed exercise intensity.

We developed a broad conceptual model that sum-

marises and provides a framework to interpret the research conducted to date. As shown in Fig. 1, the VR system

results in outcomes that occur concurrently or following

engagement in the VR sport task. The VR system is composed of four components. These are the VR

environment, the sport task, the athlete, and the non-VR

environment. Research on VR applications to sport have largely focussed on only the first three of these compo-

nents. The VR environment is the unique component for

VR applications to sport and is the focus of most research. The second component, the sport task used, will differ

according to the application and can vary between endur-

ance-type sports or skill-based sports. The third component relates to characteristics of the athlete, such as skill level

and competitiveness. The characteristics of the athlete may act independently or they may interact with other elements

of the VR system to influence outcomes. The fourth com-

ponent encompasses those aspects of the real-world envi- ronment in which the athlete completes the task. Ambient

temperature, humidity, and time of day are among the

relevant factors that can be present and influence outcomes. Finally, all four elements of the VR system will produce

outcomes that emerge on an ongoing basis when per-

forming the sport task (concurrent outcomes) or they may emerge at a later time (posttask outcomes). The posttask

outcomes may be short term or long term.

The four components of the VR system share elements in common with other models applied to sport and exercise

psychology. For example, Tenenbaum and Hutchinson

(2007) proposed that perceived effort and effort tolerance are determined by the individual (e.g. dispositions, task

familiarity, demographic characteristics), the task (e.g.

intensity, duration), and the environmental conditions (e.g. social, physical features) that are present in a given situa-

tion. These conditions are analogous to the three non-VR

components of the VR system as presented in Fig. 1. Such a similarity is to be expected because VR aims to simulate

a real environment. However, research on VR applications

to sport have not yet examined the effects of the real (non- VR) environment on performance. Instead, attention has

been directed towards variables related to the virtual

environment, such as immersion, presence, and interac- tivity with virtual others. Research supporting the con-

ceptual framework depicted in Fig. 1 is presented below.

5 The virtual reality system

5.1 Virtual reality environment and task factors

The first two components, the VR environment and sport task, may be considered together because they can be

closely linked. For example, a rower may complete a time

trial using a rowing ergometer. However, the ergometer is merely the exertion interface. It is transformed into a vir-

tual boat such that pulls on the ergometer handle are

depicted as movements of the virtual oars through the water. Increasing exertion on the task (e.g. rowing at a

186 Virtual Reality (2018) 22:183–198

123

higher intensity) will be reflected in changes in the virtual

environment (e.g. faster movement through the water and passing scenery). Thus, performance and other factors

related to the task will influence the virtual environment

and this relationship can be reciprocal. Research has shown that several characteristics of the VR

environment and the task influence outcomes. A summary of

the methodological approaches used to create the VR envi- ronment and task is provided in Table 1. As can be seen, the

sport tasks used most often have been cycling and running, but rowing, weightlifting, and golf have also been examined.

Cycling, running, and rowing are sports that contain ele-

ments of endurance and persistence. These sports are also relatively easy to translate into a virtual environment. The

exertion interface of the treadmill or ergometer can readily

monitor information with regard to the speed and other performance elements (e.g. cadence) and translate this

information into virtual movements. Interactivity is further

enhanced by including directional controls although few VR systems have been used which have this capability.

The VR software and display equipment used in

research has varied from commercially available products to those that are custom made. The virtual environment is

typically displayed on computer screens or projected

against a wall. A larger display or the inclusion of more multimodal elements of the environment will increase the

sense of immersion in the virtual world (Vogt et al. 2015)

and this can influence performance. Using a more immer- sive virtual environment during a cycling task (i.e. showing

the track from the point of view of the rider versus from a

birds eye view) has increased motivation and the speed of cycling in participants (Ijsselsteijn et al. 2004). Using a

virtual running task, over a third of participants have reported that the immersion induced by the VR environ-

ment is an important motivating feature (Nunes et al.

2014). There might be a dose-dependent relationship between the level of immersion induced by the VR system

and the magnitude of the resulting outcomes.

The presence of others in the virtual environment has also emerged as an important feature of the VR environ-

ment. Indeed, the presence of others may be even more

important than the capability of the VR system to induce feelings of immersion or the presence. In a survey study

examining golf play in a virtual environment, Lee et al.

(2012) distinguished between two types of presence: telepresence or the feeling of being physical immersed in

VR Display ■ screen ■ HMD ■ CAVE

Exertion interface

Athlete

Location

Social

CONCURRENT OUTCOMES

POSTTASK OUTCOMES

Short-term Long-term

Physiological

Psychological

Performance

Sport Pr

es en

ce o

f o th

er s

VR SYSTEM

Fig. 1 A model of interactive virtual reality (VR) in sport and sport-related exercise showing the relationship between components of the VR system, current outcomes, and posttask outcomes

Virtual Reality (2018) 22:183–198 187

123

the virtual environment and social presence or the feeling

of being with and communicating with others in the virtual environment. Social presence was shown to play a more

important role in perceived enjoyment, perceived value,

and behavioural intentions than telepresence. Further, unlike social presence, telepresence did not significantly

predict any of these outcomes.

The presence of others has also influenced motivation and performance for aerobic sport tasks. Using a running

task, Nunes et al. (2014) reported that participants pre-

ferred to run in the presence of virtual others than to run on the virtual course alone (Nunes et al. 2014). Irwin et al.

(2012) examined the Köhler motivation gain effect while

participants cycled in a virtual environment. Participants cycled at an intensity of 65% of heart rate reserve for as

long as they felt comfortable. Different groups of partici-

pants completed trials while cycling in the virtual envi- ronment alone or at the same time as another person (a

Table 1 Characteristics of the task and virtual reality system of studies investigating virtual reality in sport

Authors Years Task Sport equipment VR technology VR display

Point of view

Others in environment

Anderson- Hanley et al.

2011 Cycling Recumbent stationary bicycle

Netathlon riding software Laptop screen

Not specified

Some conditions

Anderson- Hanley et al.

2012 Cycling Recumbent stationary bicycle

Netathlon riding software Laptop screen

Not specified

Yes

Anderson- Hanley et al.

2014 Cycling Recumbent stationary bicycle

Netathlon riding software Laptop screen

Not specified

Yes

Annesi and Mazas

1997 Cycling Stationary recumbent bicycle

Tectrix VR bike Screen Not specified

Yes

Baños et al. 2016 Walking Treadmill Commercial VR exergaming platform) Projected Third Not specified

Chen et al. 2015 Weightlifting Dumbbells Cave automatic virtual environment (CAVE)

Projected Not specified

No

Hoffman et al.

2014 Rowing Indoor rowing ergometer

Not specified Screen First Not specified

Ijsselsteijn et al.

2004 Cycling Stationary racing bicycle

Tacx T19000i-magic’ VR trainer Projected First and third

Yes

Irwin et al. 2012 Cycling Stationary bicycle Expresso fitness bike system Screen Not specified

Some conditions

Lee et al. 2012 Golf Golf ball and clubs Golfzon managed and operated virtual golf simulator

Projected First Not specified

Legrand et al.

2011 Running or cycling

Treadmill and regular bicycle ergometer

Tacx I-magic fortius Projected Not specified

No

Mestre et al. 2011 Cycling Stationary bicycle Tacx VR trainer Screen Third Yes

Murray et al.

2016 Rowing Indoor rowing ergometer

Netathlon 2 XF software Projected Third Yes

Nunes et al. 2014 Running Treadmill Running wheel Screen Third Some conditions

Oliviera et al.

2015 Cycling Cycle ergometer CompuTrainer 3D software Screen Not specified

Yes

Plante et al. 2003a Cycling Stationary bicycle Trek extreme mountain biking and cycle Fx ITS-1 with ultra-coach VR lite

Screen Third Yes

Plante et al. 2003b Cycling Stationary bicycle Trek extreme mountain biking and cycle Fx ITS-1

Screen Third Yes

Sigrist et al. 2015 Rowing Indoor rowing ergometer

Cave automatic virtual environment (CAVE)

Projected Not specified

No

Snyder et al. 2012 Cycling Recumbent stationary bicycle

Cybercycle expresso S3R Screen Third Yes

Vogt et al. 2015 Cycling Cycle ergometer Custom made Projected First No

VR virtual reality

188 Virtual Reality (2018) 22:183–198

123

confederate) who the participant was informed had per-

formed moderately better than they did in a baseline trial. Cycling with the other person was either in a conjunctive

situation (a ‘‘team score’’ would be based on the rider who

quit the task first) or a coactive situation (no team part- nership). Task persistence was higher in the coactive sit-

uation than when cycling alone. Moreover, a further

enhancement of persistence was observed in the conjunc- tive situation, suggesting motivational gains when per-

forming a VR-based sport in a team situation. In the study by Irwin et al. (2012), the confederate was

shown via a video loop on another screen and not in the VR

environment. Murray et al. (2016) also examined the Köhler motivation gain effect in which the teammate was

present as a virtual partner in the virtual environment.

Female participants novice to rowing completed a rowing trial in the presence of a virtual teammate in a conjunctive

situation (the shortest distance rowed over a 9-min trial

would count as the team score) or in the VR environment alone. Prior to the trial, participants were informed that the

teammate had rowed 40% longer than them in an initial

baseline row. A Köhler motivation gain effect was found in that participants rowed further and had a higher heart rate

in the presence of a teammate than when rowing in the VR

environment alone. Moreover, the conditions did not differ in felt arousal, positive feelings, or ratings of perceived

exertion. The latter finding suggests that performance

improvements can be induced by a virtual partner in the absence of negative psychological costs.

The presence of others in a virtual environment can be

used to more directly induce a pressure to perform in a competitive situation. Using a sample of older adults,

Anderson-Hanley et al. (2011) compared cycling through a

virtual course either alone or in the presence of on-screen rider avatars. In the latter condition, participants were

explicitly asked to outpace the avatars. The introduction of

the on-screen avatars increased cycling power output when compared to solo cycling condition. However, this effect

was observed only in participants who were classified as

high in competitiveness based on a self-report question- naire. A limitation of this study was that all participants

completed the solo cycling condition first and the com-

petitive situation second. Nevertheless, the findings suggest that competitiveness is an important moderating factor in

responses to VR.

Similar outcomes to Anderson-Hanley et al. (2011) were reached in a study by Snyder et al. (2012) who compared

two competitive situations while participants cycled in a

VR environment. In the virtual condition, the participants were informed that the avatar of the other rider was con-

trolled by the computer. In a live rider condition, the par-

ticipants were introduced to a confederate and were informed that the avatar speed was controlled by the

cycling speed of the confederate. Cycling performance,

measured as watts generated, was higher for the live rider condition than in the virtual rider condition. Again, this

difference emerged only in participants high in competi-

tiveness. No differences between the rider conditions emerged for participants low in competitiveness.

Competitive situations can be constructed within a vir-

tual environment in various ways. Nunes et al. (2014) devised three competitive modes based on whether par-

ticipants competed against themselves (i.e. a prior perfor- mance), against an individual chosen for them who is

superior, or against any individual chosen by the partici-

pant. Using a VR running task, all types of competitive situations enhanced physical exertion (as measured by

heart rate) and self-reported motivation when compared to

running on the virtual course alone. Evidence was also found that participants who were not initially competitive

still felt pressure to outperform the on-screen avatars.

However, similar to the conclusions reached by Anderson- Hanley et al. (2011) and Snyder et al. (2012), participants

who had a stronger preference for competitive situations

showed the highest task performance. A different approach to the use of another individual in

the virtual environment was reported by Oliveira et al.

(2015). Participants completed two conditions of a VR cycling task. In one condition, the participant self-selected

the intensity of the cycling trial. In the other condition,

participants were asked to follow a virtual cyclist. The virtual cyclist was set to a speed that matched the self-

selected intensity condition. No significant differences

were found between conditions on physiological effort or affective responses. Typically, an externally imposed

intensity results in an increase in negative affect. The

findings thus suggest that this affective ‘‘cost’’ is mitigated when participants match the imposed pace of a virtual

partner. However, further research is required to confirm

these findings. For example, order effects may have been a factor because all participants completed the self-selected

condition first and followed by the externally imposed

intensity condition.

5.2 User (Athlete) factors

The third component of the VR system is the athlete who is

engaging in the virtual sport. The characteristics of the

athlete user have the potential to mediate or moderate the effects of VR on performance and psychological outcomes.

Athlete user factors may include physical characteristics,

expertise and experience, and psychological characteris- tics. As shown in Table 2, the participants recruited in

research to date have been relatively homogenous. The

typical participant has been a young adult sampled from Western countries who are novice to the sport. It has been

Virtual Reality (2018) 22:183–198 189

123

suggested that using novices is advantageous because it results in a sample that is more physiologically equivalent

and their performance is less likely to be influenced by

prior learning (Hoffmann et al. 2014). However, it reduces the generalisation of findings to participants that are

younger or older or who compete at the elite level.

It is surprising that most studies have not reported comparisons between males and females given the docu-

mented gender differences in not only sport performance

but also in experience with computerised environments (e.g. computer games). Plante et al. (2003a) included

gender as a factor when examining the effects of VR on

mood during cycling. Females showed a larger difference in reported relaxation between the VR alone (no cycling)

and both the cycling alone and cycling with VR conditions

when compared to males. Plante et al. (2003b) also used a cycling task and reported gender differences in ratings of

energy. Males reported higher energy when cycling alone, cycling with VR, or experiencing VR alone than in a

baseline control condition that did not involve VR or

cycling. In contrast, females reported more energy in cycling alone or cycling with VR than when VR was used

without cycling or in the baseline condition. While pre-

liminary, there is some suggestion that females may be influenced more by the VR environment than males.

The preferences of the individual user may be an

important psychological factor that moderates outcomes. Legrand et al. (2011) assigned participants to either a

cycling task alone (no VR input), a self-selected VR task

(either jogging or cycling), or an externally imposed VR task (either jogging or cycling). All conditions improved

positive affect and reduced negative affect when assessed

by pre- and posttask subjective measures. The in-task subjective measures showed that participants in the self-

Table 2 Sample size and participant characteristics of studies investigating virtual reality in sport

Authors Years N Gender Age (in years) Experience type Location

Anderson-Hanley et al. 2011 14 Both M = 78.51 Novice U.S.A.

Range 60–99

Anderson-Hanley et al. 2012 79 Both M = 78.76 Novice U.S.A.

Anderson-Hanley et al. 2014 30 Both M = 79.5 Novice U.S.A.

Annesi and Mazas 1997 39 Both M = 37.7 Novice U.S.A.

Baños et al. 2016 109 Both M = 11.86 Novice Spain

Range 10–15

Chen et al. 2015 11 Both M = 24.5 Novice U.S.A.

Hoffman et al. 2014 15 Males M = 24.1 Novice France

Ijsselsteijn et al. 2004 24 Both M = 41.3 Novice Netherlands

Irwin et al. 2012 58 Females M = 20.54 Novice U.S.A.

Lee et al. 2012 275 Both 80.4% fell within 30–49 Experienced South Korea

Legrand et al. 2011 131 Both M = 19.31 Experienced France

Mestre et al. 2011 6 Not specified Range 19–25 Novice France

Murray et al. 2016 60 Female M = 20.20 Novice Australia

Range 18–30

Nunes et al. 2014 12 Both M = 33.91 Novice Brazil

Range 22–52

Oliviera et al. 2015 17 Male M = 31 Novice Brazil

Range 18–40

Plante et al. 2003a 88 Both M = 38.10 Novice U.S.A.

Range 20–67

Plante et al. 2003b 121 Both M = 18.58 Novice U.S.A.

Range 17–27

Sigrist et al. 2015 24 Both M = 26.1 Novice Switzerland

Range 21–33

Snyder et al. 2012 23 Females M = 19.2 Novice U.S.A.

Range 17–22

Vogt et al. 2015 22 Both M = 30.27 Novice Germany

190 Virtual Reality (2018) 22:183–198

123

selected VR task reported higher pleasure than the cycling

alone or the externally imposed VR task, which themselves did not differ. Autonomy or the appropriate matching of an

individual to a preferred sport may thus be important for

mood benefits when using VR. As noted above, individual preferences for task intensity may be another factor in that

using VR technology may reduce the negative impact of

performing at an externally imposed intensity (Oliveira et al. 2015).

5.3 Non-VR environment factors

The final component of the VR system, the real-world environment, has received no attention in research con-

ducted to date. Researchers have used a controlled indoor

environment and have kept key variables like temperature, humidity, and time of day constant or allowed them to vary

at random. Tenenbaum and Hutchinson (2007) noted that

the environment can be divided into physical and social components and a similar distinction can be made here. In

particular, based on research showing that the presence of

others in the virtual environment can influence perfor- mance and psychological states, it would be expected that

the presence of others in the real environment will also

have an influence. Further research is required to examine the effects of environmental factors and to determine the

relative strength of these factors when present virtually

versus when present in reality.

5.4 Concurrent and posttask outcomes

A summary of the key research aims and outcomes is

shown in Table 3. The majority of the outcomes have been

observed concurrently with the task, but some have been observed posttask (i.e. short-term and long-term effects;

see Fig. 1). Concurrent outcomes are those that influence

ongoing behaviour (e.g. performance, persistence, affective states, perceived exertion). For example, VR tasks that

induce competitiveness may induce short-term increases in

performance if the individual is running at a pace slower than a virtual competitor (Nunes et al. 2014). Posttask

outcomes will influence behaviour at a later time and are

thus independent of the ongoing interaction with the VR system (e.g. meeting performance goals, competition out-

comes). For instance, Annesi and Mazas (1997) showed

that an exercise program that used a VR cycling task increased adherence to the exercise program relative to

cycling alone.

Outcomes may also be divided into those related to task performance, physiological effects, and psychological

processes. As shown in Table 3, performance outcomes in

past research include adherence (Anderson-Hanley et al. 2014; Annesi and Mazas 1997; Irwin et al. 2012), distance

travelled or speed in the virtual environment (Hoffmann

et al. 2014; Ijsselsteijn et al. 2004; Murray et al. 2016; Nunes et al. 2014; Snyder et al. 2012), physical intensity

exerted (Anderson-Hanley et al. 2011; Chen et al. 2015;

Snyder et al. 2012), in-task persistence (Irwin et al. 2012), and strategy (Hoffmann et al. 2014). Physiological out-

comes have included heart rate (Nunes et al. 2014; Snyder

et al. 2012), oxygen consumption and blood lactate level (Oliveira et al. 2015), muscle fatigue (Chen et al. 2015),

and electroencephalogram (EEG) amplitude and frequency (Vogt et al. 2015). Psychological outcomes may relate to

behavioural intentions (Lee et al. 2012), cognitive func-

tions (Anderson-Hanley et al. 2012), motivation (Ijssel- steijn et al. 2004; Nunes et al. 2014), perceived pressure

(Ijsselsteijn et al. 2004), attentional focus (Baños et al.

2016; Mestre et al. 2011), and various positive and nega- tive feeling states.

The application of VR to sport has resulted in several

beneficial outcomes. When compared to control conditions, tasks that incorporate VR have shown improved adherence

(Annesi and Mazas 1997), better race strategy performance

(Hoffmann et al. 2014), higher cognitive functioning (Anderson-Hanley et al. 2012), improved mood and

reduced tiredness (Plante et al. 2003b), increased workload

(Chen et al. 2015), and higher enjoyment (Mestre et al. 2011; Murray et al. 2016). However, the control condition

used in most research has involved performance of the

sport on its own. This approach may be questioned because it does not control for the presence of an external stimulus

during the task. It is possible that the VR environment may

produce its effects because it distracts and diverts attention away from the task (Baños et al. 2016; Mestre et al. 2011),

rather than because it induces a sense of the presence or

includes elements of interactivity, which are the key fea- tures of a VR environment.

It is also noteworthy that better performance or psy-

chological outcomes have not always resulted when VR is used (e.g. Lee et al. 2012; Legrand et al. 2011) suggesting

that other factors may moderate its effectiveness. As noted

above and shown in Table 3, these factors may relate to the VR system or user, such as level of immersion (Ijsselsteijn

et al. 2004), competitiveness (Anderson-Hanley et al. 2011;

Nunes et al. 2014; Snyder et al. 2012), social presence (Irwin et al. 2012; Lee et al. 2012; Murray et al. 2016),

self-selection of tasks (Legrand et al. 2011), attentional

focus (Mestre et al. 2011), and the mood altering effects of the task itself (Plante et al. 2003b).

Performance and psychological outcomes may result

from the additive or interactive effects of the VR system. For example, a high level of immersion will enhance

motivation and performance (Ijsselsteijn et al. 2004).

However, immersion may be increased in different ways. It can be enhanced by using a more realistic VR environment

Virtual Reality (2018) 22:183–198 191

123

T ab

le 3

C h ar ac te ri st ic s o f th e d es ig n , ai m s, co n d it io n s, m ea su re s, an d k ey

fi n d in g s o f st u d ie s in v es ti g at in g v ir tu al

re al it y in

sp o rt

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

A n d er so n -

H an le y

et al .

2 0 1 1

Q u as i-

E x p er im

en ta l

T o ev al u at e th e ef fe ct

o f

so ci al

fa ci li ta ti o n an d

co m p et it iv en es s o n

cy cl in g in

o ld er

ad u lt s

(1 ) S ta ti o n ar y cy cl in g w it h V R

(2 ) S ta ti o n ar y cy cl in g w it h V R

an d o n -s cr ee n co m p et it o rs

C o m p et it iv en es s an d cy cl in g

ef fo rt

N o

T h e in tr o d u ct io n o f co m p et it o r

av at ar s in cr ea se d cy cl in g

in te n si ty

m o re

fo r co m p et it iv e

o ld er

ad u lt s th an

fo r th o se

w h o

w er e le ss

co m p et it iv e

A n d er so n -

H an le y

et al .

2 0 1 2

E x p er im

en ta l

T o d et er m in e if v ir tu al

cy cl in g w o u ld

re su lt in

g re at er

ex ec u ti v e

fu n ct io n , an d in cr ea se

b ra in -d er iv ed

n eu ro tr o p h ic

g ro w th

fa ct o r

(1 ) S ta ti o n ar y cy cl in g (2 )

S ta ti o n ar y cy cl in g w it h

in te ra ct iv e V R

to u rs

E x ec u ti v e fu n ct io n an d o th er

co g n it iv e fu n ct io n m ea su re s,

B M I, b o d y co m p o si ti o n ,

st re n g th , en er g y ex p en d it u re ,

an d b ra in -d er iv ed

n eu ro tr o p ic

fa ct o r

N o

V R cy cl in g to u rs sh o w ed

g re at er

ex ec u ti v e fu n ct io n in g an d

n eu ro p la st ic it y th an

cy cl in g o n

it s o w n ; V R

cy cl in g to u rs

h ad

a 2 3 %

re la ti v e ri sk

re d u ct io n in

m il d co g n it iv e im

p ai rm

en t

A n d er so n -

H an le y

et al .

2 0 1 4

E x p er im

en ta l

T o d et er m in e if h ig h er

ex ec u ti v e fu n ct io n w o u ld

p re d ic t cy cl in g

b eh av io u r o v er

a 3 -m

o n th

fo ll o w -u p

p er io d

(1 ) S ta ti o n ar y cy cl in g (2 )

S ta ti o n ar y cy cl in g w it h

in te ra ct iv e V R

to u rs

E x ec u ti v e fu n ct io n , se lf -

ef fi ca cy , p er ce iv ed

b en efi ts

an d b ar ri er s to

ex er ci se , so ci al

su p p o rt , m o ti v at io n , co g n it iv e

im p ai rm

en t, an d p h y si ca l

il ln es s

N o

E x er ci se

se lf -e ffi ca cy

an d

d ec li n in g ex ec u ti v e fu n ct io n at

p o st -i n te rv en ti o n w er e

as so ci at ed

w it h m o re

fr eq u en t

ex er ci se

d u ri n g fo ll o w -u p

A n n es i an d

M az as

1 9 9 7

E x p er im

en ta l

T o te st th e ef fe ct iv en es s o f

V R

o n in cr ea si n g

ad h er en ce , at te n d an ce ,

an d fe el in g st at es

(1 ) U p ri g h t b ic y cl e (2 )

R ec u m b en t b ic y cl e (3 )

R ec u m b en t b ic y cl e w it h V R

A tt en d an ce , ad h er en ce , ex er ci se -

in d u ce d fe el in g s, an d se lf -

m o ti v at io n

N o

V R

cy cl in g w as

ef fe ct iv e in

m ai n ta in in g ad h er en ce

to re g u la r cy cl in g ; ex er ci se -

in d u ce d fe el in g s w er e n o t

af fe ct ed

b y V R

cy cl in g

B añ o s

et al .

2 0 1 6

Q u as i-

E x p er im

en ta l

T o d et er m in e if a V R

w al k in g ta sk

cr ea te s

at te n ti o n al

d is tr ac ti o n

fr o m

b o d il y se n sa ti o n s in

o v er w ei g h t ch il d re n

(1 ) G ro u p (o v er w ei g h t, n o rm

al w ei g h t) (2 ) C o n d it io n (n o V R ,

V R )

H R , at te n ti o n al

fo cu s, ex er ci se -

in d u ce d fe el in g s, ra ti n g s o f

p er ce iv ed

ex er ti o n , en jo y m en t,

an d p re fe re n ce

N o

V R

d ec re as ed

fo cu s o n b o d il y

se n sa ti o n s an d in cr ea se d

ex te rn al

fo cu s fo r o v er w ei g h t

ch il d re n ; en jo y m en t ra ti n g s

w er e h ig h er

in th e V R

co n d it io n

C h en

et al .

2 0 1 5

E x p er im

en ta l

T o d et er m in e if V R

im p ro v es

w ei g h tl if ti n g

p er fo rm

an ce

an d ra ti n g s

o f p er ce iv ed

ex er ti o n

(1 ) V R (n o V R , 3 -D

st er eo , 2 -D

st er eo ) (2 ) W ei g h t o f li ft (l o w ,

m o d er at e,

h ig h ) (3 ) H ei g h t o f

li ft (l o w , m o d er at e,

h ig h )

M u sc le

fa ti g u e,

p o w er –

fr eq u en cy , ra ti n g s o f p er ce iv ed

ex er ti o n , an d p er ce iv ed

w o rk lo ad

N o

B ic ep

m u sc le

ac ti v it y an d

w o rk lo ad

w as

h ig h er

in b o th

V R

co n d it io n s th an

n o V R

co n d it io n ; ra ti n g s o f p er ce iv ed

ex er ti o n d id

n o t d if fe r ac ro ss

co n d it io n s

H o ff m an

et al .

2 0 1 4

E x p er im

en ta l

T o d et er m in e if V R

u si n g

an av at ar

to tr ai n a ra ce

st ra te g y w o u ld

im p ro v e

en er g y m an ag em

en t an d

ra ce

o u tc o m es

(1 ) V R

w it h n o av at ar

(2 ) V R

w it h av at ar

u si n g a fa st -s ta rt

ra ce

st ra te g y

V en ti la to ry

an d en er g y

ex p en d it u re

v ar ia b le s, ra ce

ti m e,

p o w er

o u tp u t, p ac e,

S te p M ax , an d ra ce

st ra te g y

N o

T ra in in g w it h an

av at ar

to u se

a fa st -s ta rt ra ce

st ra te g y

im p ro v ed

ra ce

st ra te g y p ro fi le s

an d ra ce

ti m e p er fo rm

an ce

at p o st -t es t an d re te n ti o n te st

192 Virtual Reality (2018) 22:183–198

123

T ab

le 3

co n ti n u ed

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

Ij ss el st ei jn

et al .

2 0 0 4

E x p er im

en ta l

T o d et er m in e if im

m er si v e

V R

en v ir o n m en ts

an d a

v ir tu al

co ac h in cr ea se

m o ti v at io n to

cy cl e

(1 ) Im

m er si o n (h ig h , lo w ) (2 )

V ir tu al

co ac h (w

it h , w it h o u t)

In tr in si c m o ti v at io n , H R , an d

av er ag e sp ee d

T h e IT C

S en se

o f P re se n ce

In v en to ry

W h en

V R

w as

m o re

im m er si v e,

m o ti v at io n an d av er ag e sp ee d

w er e in cr ea se d ; th e v ir tu al

co ac h re d u ce d p er ce iv ed

p re ss u re , te n si o n an d co n tr o l

Ir w in

et al .

2 0 1 2

E x p er im

en ta l

T o d et er m in e if m o ti v at io n

to p er si st w o u ld

b e

in fl u en ce d b y th e

p re se n ce

o f a p ar tn er

in a

co n ju n ct iv e o r co ac ti v e

si tu at io n

(1 ) In d iv id u al

(2 ) C o n ju n ct iv e

(3 ) C o ac ti v e

P er si st en ce , se lf -e ffi ca cy ,

in te n ti o n to

ex er ci se , ra ti n g s o f

p er ce iv ed

ex er ti o n , an d

in te n ti o n to

ex er ci se

N o

V R

co m b in ed

w it h a p ar tn er

sh o w ed

g re at er

ta sk

p er si st en ce

in co n ju n ct iv e

co n d it io n s th an

co ac ti v e

co n d it io n s w it h b o th

h ig h er

th an

n o p ar tn er

L ee

et al .

2 0 1 2

S u rv ey

T o in v es ti g at e th e

p sy ch o lo g ic al

ef fe ct s o f

p re se n ce

an d im

m er si o n

in V R

(1 ) V R

co n d it io n

P er ce iv ed

en jo y m en t, p er ce iv ed

v al u e,

an d b eh av io u ra l

in te n ti o n

A q u es ti o n n ai re

d es ig n ed

to m ea su re

te le p re se n ce

an d so ci al

p re se n ce

S o ci al

p re se n ce

ra th er

th an

th e

V R

te ch n o lo g y it se lf w as

re sp o n si b le

fo r en jo y m en t,

p er ce iv ed

v al u e,

an d

b eh av io u ra l in te n ti o n

L eg ra n d

et al .

2 0 1 1

E x p er im

en ta l

T o ex am

in e re g u la r

ex er ci se

v er su s a V R

cy cl in g an d V R

ru n n in g

ta sk

an d al so

th e ef fe ct o f

im p o se d v er su s se lf -

se le ct ed

V R

ta sk s o n

af fe ct

an d v al en ce

(1 ) B ic y cl e er g o m et er

w it h n o

V R (2 ) P ar ti ci p an t ch o ic e o f

V R cy cl in g o r ru n n in g (3 )

E x p er im

en te r al lo ca te d V R

cy cl in g o r ru n n in g

P o si ti v e af fe ct , n eg at iv e af fe ct ,

v al en ce

N o

M o o d b en efi ts fo ll o w in g th e ta sk

w er e o b se rv ed

re g ar d le ss

o f

co n d it io n ; a se lf -s el ec te d V R

ta sk

re su lt ed

in h ig h er

p o si ti v e

v al en ce

d u ri n g th e ta sk

th an

w h en

th e V R ty p e o f ta sk

w as

ex te rn al ly

im p o se d

M es tr e

et al .

2 0 1 1

E x p er im

en ta l

T o te st th e ro le

o f V R an d

a v ir tu al

co ac h o n

at te n ti o n al

fo cu s,

p er fo rm

an ce , an d

en jo y m en t

(1 ) N o V R

(2 ) V R

(3 ) V R

an d

fo ll o w in g v ir tu al

co ac h p ac er

P er ce iv ed

ex er ti o n , p h y si ca l

ac ti v it y en jo y m en t, at te n ti o n al

fo cu s, an d p er fo rm

an ce

(s p ee d ,

p o w er , p ed al li n g fr eq u en cy ,

an d H R )

N o

D is so ci at iv e at te n ti o n al

fo cu s

w as

g re at er

in th e V R

co n d it io n s an d en jo y m en t w as

g re at er

fo r V R co n d it io n th an

N o V R co n d it io n an d g re at er

fo r V R w it h v ir tu al

co ac h th an

V R

M u rr ay

et al .

2 0 1 6

E x p er im

en ta l

T o d et er m in e if th e

p re se n ce

o f o th er s in

an im

m er si v e V R

af fe ct s

p er fo rm

an ce , m o ti v at io n ,

an d af fe ct

d u ri n g an

ae ro b ic

ro w in g ta sk

(1 ) N o V R (2 ) in d iv id u al V R (3 )

co m p an io n V R

D is ta n ce , p o w er , st ro k es

p er

m in u te , H R , ex er ci se

th o u g h ts ,

p er ce iv ed

b en efi ts

an d b ar ri er s

to ex er ci se , ra ti n g s o f

p er ce iv ed

ex er ti o n , af fe ct ,

ar o u sa l, in tr in si c m o ti v at io n ,

an d en jo y m en t o f ex er ci se

N o

In d iv id u al

an d co m p an io n V R

re su lt ed

in b et te r ro w in g

p er fo rm

an ce

an d m o re

en jo y m en t w it h o u t an

in cr ea se

in p er ce iv ed

ex er ti o n ;

co m p an io n V R g ro u p

ex ce ed ed

in d iv id u al

V R g ro u p

in d is ta n ce

tr av el le d an d H R

Virtual Reality (2018) 22:183–198 193

123

T ab

le 3

co n ti n u ed

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

N u n es

et al .

2 0 1 4

E x p er im

en ta l

T o d et er m in e if th er e is

a d if fe re n ce

in p er fo rm

an ce

d u ri n g a

ru n n in g ta sk

in th e

p re se n ce

o f a v ir tu al

co m p et it o r

(1 ) S in g le p la y er

o n ly

(2 ) S in g le

p la y er

co m p et it iv e ag ai n st

o n es el f (3 ) C o m p et it iv e m o d e

ag ai n st a su p er io r ad v er sa ry

(4 ) C o m p et it iv e m o d e ag ai n st

an ad v er sa ry

ch o se n b y

p ar ti ci p an t

P er fo rm

an ce

(s p ee d , h ea rt b ea t,

an d d is ta n ce ), p er ce iv ed

ex er ti o n , p re fe rr ed

ex er ci se

co n d it io n

N o

P ar ti ci p an ts

re p o rt ed

th e

co m p et it iv e m o d e w as

m o re

m o ti v at in g ; p er ce iv ed

ex er ti o n

an d p er fo rm

an ce

w as

h ig h er

in co m p et it o r co n d it io n s th an

in si n g le

p la y er

m o d e

O li v ie ra

et al .

2 0 1 5

E x p er im

en ta l

T o d et er m in e if se lf -

se le ct ed

o r im

p o se d

ex er ci se

in te n si ty

an d

d u ra ti o n p ro d u ce

b et te r

cy cl in g p er fo rm

an ce ,

af fe ct iv e re sp o n se s, an d

en jo y m en t

(1 ) S el f- se le ct ed

in te n si ty

an d

d u ra ti o n w it h si n g le

v ir tu al

cy cl is t (2 ) Im

p o se d in te n si ty

an d d u ra ti o n w it h ad d it io n al

v ir tu al

cy cl is t

H R , o x y g en

co n su m p ti o n , b lo o d

la ct at e,

co n ce n tr at io n , ra ti n g s

o f p er ce iv ed

ex er ti o n , af fe ct ,

ar o u sa l, an d en jo y m en t o f

ex er ci se

N o

N o si g n ifi ca n t d if fe re n ce s in

p er fo rm

an ce

o r p sy ch o lo g ic al

o u tc o m es

ac ro ss

co n d it io n s

w er e o b se rv ed ; th er e w as

a tr en d to w ar d s h ig h er

en jo y m en t in

th e im

p o se d

se ss io n

P la n te

et al .

2 0 0 3 a

E x p er im

en ta l

T o in v es ti g at e w h et h er

V R

en h an ce s th e p o te n ti al

p o si ti v e ef fe ct s o f

cy cl in g

(1 ) S ta ti o n ar y cy cl in g at

m o d er at e in te n si ty

w it h n o V R

(2 ) P la y in g a V R

co m p u te r

b ic y cl e g am

e w it h n o ex er ci se

(3 ) S ta ti o n ar y cy cl in g at

m o d er at e in te n si ty

w it h V R

M o o d , p er ce iv ed

ex er ti o n , so ci al

d es ir ab il it y , en jo y m en t, an d

H R

N o

W h en

cy cl in g w as

p ai re d w it h

V R

m o o d in cr ea se d an d

ti re d n es s d ec re as ed

w h en

co m p ar ed

to cy cl in g al o n e

P la n te

et al .

2 0 0 3 b

E x p er im

en ta l

T o in v es ti g at e w h et h er

V R

en h an ce s th e

p sy ch o lo g ic al

b en efi ts

o f

cy cl in g al o n e

(1 ) W at ch

v id eo

si m u la ti n g a

cy cl in g ex p er ie n ce

(2 ) P la y in g

a V R

co m p u te r b ic y cl e g am

e w it h o u t ex er ci se

(3 ) S ta ti o n ar y

cy cl in g w it h o u t V R (4 )

S ta ti o n ar y cy cl in g w it h V R

M o o d , p er ce iv ed

ex er ti o n , so ci al

d es ir ab il it y , an d H R

N o

C y cl in g b u t n o t V R

g av e m o o d

im p ro v em

en ts

d ir ec tl y

fo ll o w in g th e ta sk ; b o th

V R

an d cy cl in g d ec re as ed

ti re d n es s;

V R

sh o w ed

p sy ch o lo g ic al

b en efi ts

h o u rs

af te r cy cl in g fo r fe m al es

o n ly

S ig ri st

et al .

2 0 1 5

E x p er im

en ta l

T o te st th e ef fe ct

o f

co n cu rr en t au g m en te d

fe ed b ac k o n le ar n in g an d

p er fo rm

an ce

o f a V R

ro w in g ta sk

(1 ) V is u al

fe ed b ac k (2 )

A u d io v is u al

fe ed b ac k (3 )

V is u o h ap ti c fe ed b ac k

S p at ia l er ro r, te m p o ra l er ro r,

co m fo rt , u se fu ln es s an d

ap p li ca b il it y o f fe ed b ac k , an d

st ra te g y fo r re ca ll in g ta u g h t

m o v em

en t

N o

P er fo rm

an ce

w as

b et te r fo r al l

g ro u p s in

fe ed b ac k co m p ar ed

to n o -f ee d b ac k tr ia ls ;

au d io v is u al

fe ed b ac k p ro d u ce d

b et te r le ar n in g an d g re at er

co m fo rt th an

v is u o h ap ti c

fe ed b ac k

S n y d er

et al .

2 0 1 2

E x p er im

en ta l

T o ex am

in e th e in fl u en ce

o f a v ir tu al

v er su s li v e

co m p et it o r o n cy cl in g

in te n si ty

an d en er g y

ex er ti o n

(1 ) C o m p et it iv en es s le v el

(2 )

L iv e o r v ir tu al

co m p et it o r

C o m p et it iv en es s an d cy cl in g

in te n si ty

(e n er g y o u tp u t, H R ,

an d sp ee d )

N o

P ar ti ci p an ts

w h o w er e h ig h ly

co m p et it iv e p ro d u ce d g re at er

cy cl in g in te n si ty

w h en

co m p et in g ag ai n st a li v e v er su s

a v ir tu al

co m p et it o r

194 Virtual Reality (2018) 22:183–198

123

as done by Ijsselsteijn et al. (2004). It can also be enhanced

if the individual has a high trait level to feel a greater sense of the presence. Interactions between external and indi-

vidual factors may also influence outcomes. For instance,

the introduction of a virtual competitor (VR environment factor) can increase performance (Nunes et al. 2014),

although the increase may only be observed if the individual

is competitive (athlete factor) as demonstrated in research (Anderson-Hanley et al. 2011; Snyder et al. 2012). Further

research is required to examine other interactive effects.

6 Future research directions and recommendations

The present review has highlighted issues that warrant further investigation. Most research to date has focussed on

VR tasks that involve aerobic sports (cycling, running, and

rowing). More research is required on the effectiveness of a VR environment for learning or improving the mechanics

of skill acquisition and performance in skill-based sports

(see Sigrist et al. 2015 for an example). The capacity for VR environments to be created in specific and reproducible

ways can allow for the training and assessment of skills and

decision-making processes. Some of the factors identified as important with aerobic sports (e.g. attentional focus,

competitiveness) may also be important in skill-based

sports when VR is used. Research is required to examine the generality of effects

with VR. Studies should include more diverse populations,

particularly experienced and elite athletes, children, and the elderly. In addition, research has also not examined rela-

tionships between performance in VR and real-world

environments. Identifying how the two situations differ and how they are the same could inform how VR influences

performance and psychological states. The transfer of

performance from the virtual environment to the real world has also not been tested, yet it seems an essential

requirement if VR is to be used as a training approach for

sport. Further research is required that aims to directly

manipulate psychological processes. For example, it has

been suggested that VR environments induce a dissociative attentional focus and that this may be related to affective

responses (Mestre et al. 2011). Baños et al. (2016) applied

this concept by asking overweight and normal weight children to walk on a treadmill while focussing their

attention on their physical feelings or while focusing their

attention on a virtual environment. Ratings of enjoyment were higher for the VR condition than the self-focused

condition, although there were no differences in perceived

exertion or feeling states. The findings are promising but are in need of replication and extension. Past research withT

ab le

3 co n ti n u ed

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

V o g t et

al .

2 0 1 5

E x p er im

en ta l

T o te st th e ef fe ct

o f

ex er ci se

an d V R

im m er si o n o n co g n it iv e

p er fo rm

an ce

(1 ) S es si o n (a ct iv e cy cl in g ,

p as si v e au to m at ic

d ri v e)

(2 )

V R co n d it io n (f ro n t sc re en

o n ly , su rr o u n d w it h al l

sc re en s, co n tr o l w it h n o

sc re en s)

C o g n it iv e p er fo rm

an ce , H R ,

se n se

o f p re se n ce , an d E E G

am p li tu d e an d fr eq u en cy

Y es

S en se

o f p re se n ce

w as

as so ci at ed

w it h in cr ea se d E E G

ac ti v it y ; p re se n ce

w as

h ig h es t

in th e su rr o u n d V R

co n d it io n

d u ri n g ac ti v e cy cl in g ;

co g n it iv e p er fo rm

an ce

d id

n o t

d if fe r ac ro ss

co n d it io n s

V R v ir tu al

re al it y ; H R h ea rt ra te ; B M I B o d y M as s In d ex

Virtual Reality (2018) 22:183–198 195

123

non-VR tasks has also found that an external associative

focus enhances sport and exercise outcomes (e.g. Neumann and Heng 2011; Neumann and Piercy 2013). An external

associative focus involves focussing on the effects of

movements on the environment and the achievement of task goals (Neumann and Brown 2013; Stevinson and

Biddle 1999). Future research could thus use VR to induce

an external associative focus and examine its effectiveness in enhancing performance.

Further research is required to elucidate what factors are relevant to performance and affective outcomes.

Research using multiple measures or manipulations may

be particularly useful to determine the relative amounts of variance in performance attributed to different aspects of

the VR environment. In addition, different features of the

sport task should be varied. For example, intensity may be a particularly salient factor for aerobic sports. A higher

intensity level may switch attentional focus towards

internal physiological states (Stevinson and Biddle 1999) and result in individuals focusing attention away from the

VR environment. It may be possible to enhance atten-

tional focus on the virtual environment by requiring participants to follow a virtual partner as done by Oliveira

et al. (2015).

Finally, the nature of computer-based interactions is becoming more diverse and with a greater amount of

overlap between the different forms of technology and

their applications. The present review applied a definition of VR that required interactivity with the virtual envi-

ronment. However, it is acknowledged that researchers

are developing and testing systems that employ a virtual environment that the athlete responds to, even though the

behaviour of the athlete does not affect any feature of the

environment. For example, goalkeeping skills in penalty shots have been examined in both handball (Vignais et al.

2015), and soccer/football (Stinson and Bowman 2014).

In these applications, the goalkeeper viewed a virtual environment depicting an individual shooting a penalty

and was required to move their body in the predicted

direction of the ball. Their movements did not influence the action of the virtual penalty kick taker (e.g. moving

too early had no effect). Another instance that resembles

VR is the use of augmented reality. In such applications, a user has an indirect view of a physical, real-world

environment in which computer-generated input is added

to. The input may be visual, auditory, or other senses. This blending of real and virtual environmental elements

has yet to be extensively examined in sporting

applications. Based on the present review, recommendations can also

be made to ensure appropriate methodology and report in

studies. It is recommended that researchers:

1. Use the term virtual reality accurately and consistently

in reference to studies that have employed VR

technologies according to accepted definitions such as the one proposed here. The term should not be

confused with exergames, which refers to the more

general case of enhancing physical activity via inter- active computer game play. If interactivity with the

virtual environment is a particular feature that is to be

highlighted, such as in the present review, the term interactive VR may be used.

2. Report participants prior experience with VR in general

andwith the specificVR systembecause experience level may be an important factor that influences outcomes.

3. Use a measure of immersion or the presence as a

standard part of the protocol because these aspects are a core feature of VR, and the level of immersion has

emerged as an important factor that influences out-

comes (Ijsselsteijn et al. 2004; Vogt et al. 2015). Such measures include the Reality Judgement and Presence

Questionnaire (Baños et al. 2000) and the Presence

Questionnaire (Witmer and Singer 1998). 4. Provide full details of the VR system that is used.

These details include the name of the system or

software used, the participant point of view, the presence of others in the VR environment, the presence

of sounds in the VR environment, and the mechanisms

through which the participant interacts with the VR environment.

5. Report on relevant procedures that are important

psychologically, such as whether participants had choice over the type of VR task or discrete elements

within the task.

7 Conclusions

This review identified research studies that have investi-

gated the application of VR to sport. The research findings to date indicate that VR can be a promising adjunct to

existing real-world training and participation in sport.

A VR-based system for training and participation has several advantages such as enabling athletes to train

regardless of weather conditions, providing a means to

compete with others in a different geographic location, and allowing precise and replicable control over features of the

virtual environment. Future research would benefit from a

theoretical framework of VR application to sport (see Fig. 1). The present review has shown that the character-

istics of the individual user and system are important fac-

tors that can influence a range of performance, physiological, and psychological outcomes. By under-

standing the experience of when individuals are engaged in

196 Virtual Reality (2018) 22:183–198

123

sport within a VR environment, researchers, coaches, and

athletes will able to use the technology for the benefit of athletes and society in general.

References

Anderson-Hanley C, Snyder AL, Nimon JP, Arciero PJ (2011) Social facilitation in virtual reality-enhanced exercise: competitiveness moderates exercise effort of older adults. Clin Interv Aging 6:275–280. doi:10.2147/cia.s25337

Anderson-Hanley C, Arciero PJ, Brickman AM, Nimon JP, Okuma N, Westen SC, Molly EM, Pence BD, Woods JA, Kramer AF, Zimmerman EA (2012) Exergaming and older adult cognition: a cluster randomized clinical trial. Am J Prev Med 42:109–119. doi:10.1016/j.amepre.2011.10.016

Anderson-Hanley C, Arciero PJ, Barcelos N, Nimon J, Rocha T, Thurin M, Maloney M (2014) Executive function and self- regulated exergaming adherence among older adults. Front Hum Neurosci 8:989. doi:10.3389/fnhum.2014.00989

Annesi JJ, Mazas J (1997) Effects of virtual reality-enhanced exercise equipment on adherence and exercise-induced feeling states. Percept Motor Skill 85:835–844. doi:10.2466/pms.1997.85.3. 835

Australian Bureau of Statistics (2008) Defining sport and physical activity, a conceptual model. ABS Catalogue No. 4149.0.55.001

Baca A, Dabnichki P, Heller M, Kornfeind P (2009) Ubiquitous computing in sports: a review and analysis. J Sport Sci 27:1335–1346. doi:10.1080/02640410903277427

Baños RM, Botella C, Garcia-Palacios A, Villa H, Perpiñá C, Alcañiz M (2000) Presence and reality judgment in virtual environments: a unitary construct? Cyberpsychol Behav 3:327–335. doi:10. 1089/10949310050078760

Baños RM, Escobar P, Cebolla A, Guixeres J, Alvarez J, Francisco J, Botella C (2016) Using virtual reality to distract overweight children from bodily sensations during exercise. Cyberpsychol Behav Soc Netw 19:115–119. doi:10.1089/cyber.2015.0283

Briki W, Den Hartigh RJR, Markman KD, Micallef J, Gernigon C (2013) How psychological momentum changes in athletes during a sport competition. Psychol Sport Exerc 14:389–396. doi:10. 1016/j.psychsport.2012.11.009

Chen KB, Ponto K, Tredinnick RD, Radwin RG (2015) Virtual exertions: evoking the sense of exerting forces in virtual reality using gestures and muscle activity. Hum Factors 57:658–673. doi:10.1177/0018720814562231

Feltz DL, Kerr NL, Irwin BC (2011) Buddy up: the Köhler effect applied to health games. J Sport Exerc Psychol 33:506–526. doi:10.1123/jsep.33.4.506

Guy S, Ratzki-Leewing A, Gwadry-Sridhar F (2011) Moving beyond the stigma: systematic review of video games and their potential to combat obesity. Int J Hypertens. doi:10.4061/2011/179124

Hoffmann CP, Filippeschi A, Ruffaldi E, Bardy BG (2014) Energy management using virtual reality improves 2000-m rowing performance. J Sport Sci 32:501–509. doi:10.1080/02640414. 2013.835435

Ijsselsteijn W, de Kort Y, Westerink J, de Jager M, Bonants R (2004) Fun and sports: enhancing the home fitness experience. Lect Notes Comput Sci 3166:46–56. doi:10.1007/978-3-540-28643- 1_8

Irwin BC, Scorniaenchi J, Kerr NL, Eisenmann JC, Feltz DL (2012) Aerobic exercise is promoted when individual performance affects the group: a test of the Kohler motivation gain effect. Ann Behav Med 44:151–159. doi:10.1007/s12160-012-9367-4

Kim SY, Prestopnik N, Biocca FA (2014) Body in the interactive game: how interface embodiment affects physical activity and health behavior change. Comput Hum Behav 36:376–384. doi:10.1016/j.chb.2014.03.067

Larsen LH, Schou L, Lund HH, Langberg H (2013) The physical effect of exergames in healthy elderly–a systematic review. Games Health J 2:205–212. doi:10.1089/g4h.2013.0036

Laver KE, George S, Thomas S, Deutsch JE, Crotty M (2015) Virtual reality for stroke rehabilitation. Cochrane Database Syst Rev 2. doi:10.1002/14651858.CD008349

Lee HG, Chung S, Lee WH (2012) Presence in virtual golf simulators: the effects of presence on perceived enjoyment, perceived value, and behavioral intention. New Media Soc 15:930–946. doi:10.1177/1461444812464033

Legrand FD, Joly PM, Bertucci WM, Soudain-Pineau MA, Marcel J (2011) Interactive-Virtual Reality (IVR) exercise: an examina- tion of in-task and pre-to-post exercise affective changes. J Appl Sport Psychol 23:65–75. doi:10.1080/10413200.2010.523754

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta- analyses of studies that evaluate healthcare interventions: explanation and elaboration. Br Med J 339:b2700. doi:10.1136/ bmj.b2700

Meline T (2006) Selecting studies for systematic review: inclusion and exclusion criteria. Contemp Issues Commun Sci Disord 33:21–27

Mestre DR, Ewald M, Maiano C (2011) Virtual reality and exercise: behavioral and psychological effects of visual feedback. Stud Health Technol Inf 167:122–127. doi:10.3233/978-1-60750-766-6-122

Meyerbröker K, Emmelkamp PMG (2010) Virtual reality exposure therapy in anxiety disorders: a systematic review of process-and- outcome studies. Depress Anxiety 27:933–944. doi:10.1002/da. 20734

Mueller FF, Stevens G, Thorogood A, O’Brien S, Wulf V (2007) Sports over a distance. Pers Ubiquit Comput 11:633–645. doi:10. 1007/s00779-006-0133-0

Murray EG, Neumann DL, Moffitt RL, Thomas PR (2016) The effects of the presence of others during a rowing exercise in a virtual reality environment. Psychol Sport Exerc 22:328–336. doi:10.1016/j.psychsport.2015.09.007

Neumann DL, Brown J (2013) The effect of attentional focus strategy on physiological and motor performance during a sit-up exercise. J Psychophysiol 27:7–15. doi:10.1027/0269-8803/a000081

Neumann DL, Heng S (2011) The effects of associative and dissociative attentional focus strategies on muscle activity and heart rate during a weight training exercise. J Psychophysiol 25:1–8. doi:10.1027/0269-8803/a000011

Neumann DL, Piercy A (2013) The effect of different attentional strategies on physiological and psychological states during running. Aust Psychol 48:329–337. doi:10.1111/ap.12015

Nunes M, Nedel L, Roesler V (2014) Motivating people to perform better in exergames: Competition in virtual environments. In: Proceedings of the 29th annual ACM symposium on applied computing, ACM, New York. pp 970–975. doi:10.1145/ 2554850.2555009

Oliveira BRR, Deslandes AC, Nakamura FY, Viana BF, Santos TM (2015) Self-selected or imposed exercise? A different approach for affective comparisons. J Sports Sci 33:777–785. doi:10.1080/ 02640414.2014.968191

Peng W, Crouse JC, Lin JH (2013) Using active video games for physical activity promotion: a systematic review of the current state of research. Health Educ Behav 40:171–192. doi:10.1177/ 1090198112444956

Plante TG, Aldridge A, Bogden R, Hanelin C (2003a) Might virtual reality promote the mood benefits of exercise? Comput Hum Behav 19:495–509. doi:10.1016/S0747-5632(02)00074-2

Virtual Reality (2018) 22:183–198 197

123

Plante TG, Frazier S, Tittle A, Babula M, Ferlic E, Riggs E (2003b) Does virtual reality enhance the psychological benefits of exercise? J Hum Movement Stud 45:485–507. doi:10.1037/ e314842004-001

Plante TG, Cage C, Clements S, Stover A (2006) Psychological benefits of exercise paired with virtual reality: outdoor exercise energizes whereas indoor virtual exercise relaxes. Int J Stress Manag 13:108–117. doi:10.1037/1072-5245.13.1.108

Ranganathan R, Carlton LG (2007) Perception-action coupling and anticipatory performance in baseball batting. J Motor Behav 39:369–380. doi:10.3200/JMBR.39.5.369-380

Reynolds JE, Thornton AL, Lay BS, Braham R, Rosenberg M (2014) Does movement proficiency impact on exergaming perfor- mance? Hum Movement Sci 34:1–11. doi:10.1016/j.humov. 2014.02.007

Sherman WR, Craig AB (2002) Understanding virtual reality: interface, application, and design. Elsevier, San Francisco

Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P (2015) Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp Brain Res 233:909–925. doi:10.1007/s00221-014-4167-7

Slater M (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos T R Soc B 364:3549–3557. doi:10.1098/rstb.2009.0138

Snyder AL, Anderson-Hanley C, Arciero PJ (2012) Virtual and live social facilitation while exergaming: competitiveness moderates exercise intensity. J Sport Exerc Psychol 34:252–259

Steuer J (1992) Defining virtual reality: dimensions determining telepresence. J Commun 42:73–93. doi:10.1111/j.1460-2466. 1992.tb00812.x

Stevinson CD, Biddle SJH (1999) Cognitive strategies in running: a response to Masters and Ogles (1998). Sport Psychol 13:235–236. doi:10.1123/tsp.13.2.235

Stinson C, Bowman DA (2014) Feasibility of training athletes for high-pressure situations using virtual reality. IEEE Trans Vis Comput Gr 20:606–615

Tenenbaum G, Hutchinson JC (2007) A social-cognitive perspective of perceived and sustained effort. In: Tenenbaum G, Eklund RC (eds) Handbook of sport psychology, 3rd edn. Wiley, New Jersey, pp 560–573

Tirp J, Steingrover C, Wattie N, Baker J, Schorer J (2015) Virtual realities as optimal learning environments in sport—a transfer study of virtual and real dart throwing. Psychol Test Assess Model 57:57–69

Vignais N, Kulpa R, Brault S, Presse D, Bideau B (2015) Which technology to investigate visual perception in sport: video vs. virtual reality. Hum Mov Sci 39:12–26. doi:10.1016/j.humov. 2014.10.006

Vogt T, Herpers R, Scherfgen D, Strüder HK, Schneider S (2015) Neuroelective adaptations to cognitive processing in virtual environments: an exercise-related approach. Exp Brain Res 233:1321–1329. doi:10.1007/s00221-015-4208-x

Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a Presence Questionnaire. Presence-Teleop Virt 7:225–240. doi:10.1162/105474698565686

198 Virtual Reality (2018) 22:183–198

123

Practicum Experience Time Log and Journal Template

© 2012 Laureate Education Inc. 2

© 2014 Laureate Education, Inc. Page 1 of 3

sustainability

Review

The Intention to Use Fitness and Physical Activity Apps: A Systematic Review

Salvador Angosto 1 , Jerónimo García-Fernández 2,* , Irena Valantine 3 and Moisés Grimaldi-Puyana 2

1 Department of Physical Education and Sports, Faculty of Sports Sciences San Javier, University of Murcia, 30720 Santiago de la Ribera (Murcia), Spain; [email protected]

2 Department of Physical Education and Sports, Faculty of Educational Sciences, Universidad de Sevilla, 41013 Seville, Spain; [email protected]

3 Department of Sport and Tourism Management, Lithuanian Sports University, 44221 Kaunas, Lithuanian; [email protected]

* Correspondence: [email protected]; Tel.: +34-696-584-788

Received: 16 July 2020; Accepted: 15 August 2020; Published: 17 August 2020 !"#!$%&'(! !"#$%&'

Abstract: Recently the development of new technologies has produced an increase in the number of studies that try to evaluate consumer behavior towards the use of sports applications. The aim of this study is to perform a systematic review of the literature on the intention to use mobile applications (Apps) related to fitness and physical activity by consumers. This systematic review is a critical evaluation of the evidence from quantitative studies in the field of assessment of consumer behavior towards sport applications. A total of 13 studies are analyzed that propose models for evaluating the intentions to use fitness applications by sport consumers. The results revealed several key conclusions: (a) Technology Acceptance Model is the most widely used model; (b) the relationship between perceived utility and future intentions is the most analyzed; and (c) the most evaluated applications are diet/fitness. These findings could help technology managers to know the most important key elements to take into account in the development of future applications in sport organizations.

Keywords: physical activity; sport application; marketing consumption; technology acceptance model; smartphone app

1. Introduction

The constant technological evolution and the development of new mobile devices such as Smartphones or tablets o↵er a higher level of comfort and practical use, thus making this type of device the center of life for current consumers [1]. Globally, it is estimated that in 2019, there were 6.8 billion users worldwide and it is expected that in 2023 the number of users will increase to 7.33 billion [2]. In particular, 90% of the time dedicated to the Smartphone is for the use of mobile applications (Apps) [3].

Sustainability takes equal account of economic, environmental and social factors in any e↵ort to improve quality of life [4]. The dissemination and integration of information and communication technologies (ICT) and data management functionalities have been widely leveraged through the adoption of mobile devices, which allow people to participate in a larger way in society [5,6]. European Union (EU) policies emphasize the synergy between smart technologies and sustainable urban development because of the need for accurate, consistent and timely data for new policy formulation and the use of ICTs to facilitate service improvement [7,8].

The role of ICTs in sustainable development is clearly reflected in Goal 11 “make cities and human settlements inclusive, safe, resilient and sustainable” of the Sustainable Development Goals of the

Sustainability 2020, 12, 6641; doi:10.3390/su12166641 www.mdpi.com/journal/sustainability

Sustainability 2020, 12, 6641 2 of 24

United Nations Agenda 2030 [9], which considers ICTs as a means to advance human progress and knowledge in societies, to increase resource e�ciency, to promote economic development and protect the environment or to modernize industries on the basis of sustainable design [9,10]. Online tools and platforms contribute significantly to the repression of energy demands or pollution, promoting cities to a more environmentally sustainable economy [11]. Angleidou et al. [7] show that ICTs and the use of Apps help reduce the need for physical travel and the existence of physical workplaces.

An App is defined as “software applications usually designed to run on a Smartphone or tablet device and provide a convenient means for the user to perform certain tasks” [12] (p. 211). The increase is such that Blair [13] reported that the trade of Apps generates 189 billion dollars a year being used at least 11 times by 49% of users while 21% of the millennials open them at least 50 times a day.

Among them, health, fitness and physical activity Apps represent 5.18% of the total market [14], being used daily by 35% of people and several times a week by 40% [15]. In recent years, McKay, Wright, Shill, Stephens, and Uccellini [16] report a proliferation of Apps to improve health, including Apps to count the steps or promote physical activity in fitness centers, Apps to control diet and caloric intake or reduce poor habits such as smoking or alcohol consumption and improve mental health.

This increase in interest and number of Apps associated with physical activity could also have benefits for society. Therefore, the current situation of confinement caused by Covid-19 and consequently the reduction of physical activity, has encouraged di↵erent organizations such as the World Health Organization [17] to promote the need for physical activity at home. In fact, authors such as Banskota, Healy, and Goldberg [18] proposed di↵erent Apps as tools to maintain and improve physical and mental fitness in the Covid-19 pandemic. These Apps are linked to the fitness sector, revolutionizing the ways of doing physical activity and the relationships between fitness providers and consumers [19].

These new communication and prescription tools in sport could therefore have an impact on how organizations interact with consumers, with the appearance in recent years of studies that evaluate consumers’ motivations for using devices, the usefulness of Apps or consumers’ intentions to adopt them in di↵erent areas [20,21]. Particularly, researchers have begun to identify the factors that lead to the intention to use technologies, Smartphones and Apps in di↵erent sectors [22,23], but it is limited in the sports context.

Among the theories related to the intention to use of technologies, in the context of marketing we find the “Theory Acceptance Model” (TAM). This is the most used model by researchers to evaluate the intention to use of new technologies proposed by Davis [24]. TAM is an adaptation of the psychological theory, the “Theory of Reasoned Action” (TRA), which states that a person’s real behavior is determined by his or her intention to perform that behavior [25]. For instance, the TAM tries to explain how consumers use and accept new technologies based on two key beliefs, namely the usefulness of use and the ease of use that are predictive of consumers’ attitude towards future intention to use the new technologies [24]. Research based on TAM is one of the most widely used in professional settings because it focuses on the utilitarian aspect of the technology [26], with the intention of understanding the consumer’s intention to use it [22]. In particular, TAM has been used in di↵erent contexts such as finance, instant messaging, healthcare, gaming and tourism [27].

Although TAM has great robustness and applicability in terms of intention to use, attitude and perceived utility [27], di↵erent authors have developed new theories based on TAM such as the “Innovation and Di↵usion Theory” (IDT) [28] which considers that the user’s behavioral potential is driven by the user’s beliefs about innovation. Later, there is the “Unified Theory of Acceptance and Use of Technology Model (UTAUT)” [29] that proposes four constructs to develop TAM: performance expectation, social influence, e↵ort expectation and facilitation conditions. A second version of this model (UTAUT2) adds the constructs of hedonic motivation, price and habit, being adapted by Yuan, Ma, Kanthawala, and Peng [30] to measure the intention to use of health and fitness Apps. In addition, in sport, the “Sport Website Acceptance Model” (SWAM) is proposed by Hur, Ko, and Claussen [31] and is based on a framework of understanding how sport fans perceive and accept the websites of

Sustainability 2020, 12, 6641 3 of 24

their sport teams, how their level of participation and commitment to the sport team influences the intention to use the website and the actual consumption behavior they ultimately perform. Based on these models and theories, in recent years researchers have paid attention to the intention to use new technologies in di↵erent contexts such as e-payment, e-government, e-banking, retail or education [32–36]. Similarly, in academic sports literature there is also an increasing attention to the behavior of fans and consumers, with studies with di↵erent approaches such as motivation on sports websites [31], loyalty [37], participation, commitment and attributes [38,39], marketing opportunities [40], intention to use sports wearable [41], consumption of Smartphones and sports Apps [42], sports team Apps [1], fitness Apps [43,44] and sports products [45,46].

However, existing research does not provide clear results on what factors drive sports fans or consumers to use Smartphones or Apps and to benefit from new forms of experiences in sport [42]. In fact, the factors influencing the intention to use Smartphones and Apps di↵er depending on the types of products consumed and the marketing implications [47]. Therefore, while studies have been conducted on the intentions of use of technology, Smartphones and fitness Apps, there is not a review that captures the main findings of these studies. For this reason, the aim of this study is to conduct a systematic review of the literature on consumers’ intention to use Apps related to fitness and physical activity by consumers.

2. Materials and Methods

2.1. Search Strategy

The search terms for Smartphone use, Fitness and Sport Apps represented the concepts of App, Physical Activity and Use, with the search strategy for the di↵erent databases presented in Table 1. Di↵erent databases were selected to include a wide range of areas related to this interdisciplinary study, including sports science, marketing, health and psychology. The databases used were Web of Science, Scopus, SPORTDiscus (EBSCO), PsycINFO (Ovid), ABI/Inform (Ovied) and MEDLINE (Pubmed). The search was conducted between 18 March 2019 and 4 August 2020. The search covered all years and no limitations were placed on document type and language.

Table 1. Database search strategy.

Category Search Terms

App (“Smart phone *” or Smartphone * or smart-phone * or “cell * phone” or “cell-phone *” or “mobile phone *” or “mobile-phone” or “mobile device” or “mobile telephone” or * phone or Android * or iOS or app or apps or “mobile application *” or application)

Physical Activity

(“physical activit *” or exercise * or “active living” or walk * or “active transport” or “leisure activit *” or fitness or sport or “sport *” or “weight maintenance” or “maintaining weight” or “weight management”)

Use (“intention to use” or “app * usage” or “intent * to use” or usage or “behavioral intention *” or “behavior * change” or usability or “attitude toward” or consumption or Technology Acceptance Model)

Combination 1 and 2 and 3

* Truncation operator: word-based search.

2.2. Inclusion and Exclusion Criteria

For the purposes of this review, we included empirical papers in peer-reviewed journals, excluding dissertations and abstracts. Grey literature was not included, ruling out evaluation reports, annual reports, articles in nonpeer reviewed journals and other means of publication. The inclusion criteria for the articles in the search were: (i) journal articles; (ii) publications in English; (iii) use of any type of mobile application in the sports and fitness context; and (iv) measurement of the intention to use the App through a questionnaire. As exclusion criteria have been used: (i) Congress proceedings, book chapters, books or other types of publications; (ii) no mobile Apps were used in the sports context,

Sustainability 2020, 12, 6641 4 of 24

(iii) theoretical studies, qualitative approach or reviews; (iv) articles in a language other than English; and (v) duplicate articles.

2.3. Assessment of Methodological Quality

The risk of bias was assessed using a 20-item tool adapted by the authors to the context of sports marketing study typology in which there are no intervention processes on the subjects of the Consolidated Standards of Reporting Trials (CONSORT) checklist [48]. Each study was independently scored by two reviewers evaluating the di↵erent sections that make up the studies and scoring each item with 1 if the study satisfactorily met the criterion, and with 0 if the study did not satisfactorily meet the criterion or if the item was not applicable to the study. Disagreements between the reviewers were resolved by checking and discussing the original study until consensus was reached. Reviewer A is a researcher with extensive experience specializing in the field of sports management, fitness centers and development of new technologies. Reviewer B is a predoctoral fellow in sports management with focus research on methodological and statistical aspects. The results of assessment of methodological quality were shown in Appendix A.

2.4. Data Extraction and Synthesis

Figure 1 shows the Flow Diagram proposed by Moher, Liberati, Tetzla↵, and Altman [49] following the PRISMA methodology in all points that could be common to a systematic review of these characteristics. The initial database search returned 113,537 results, reduced to 36,105 once duplicates were eliminated. One reviewer conducted a full scan of the title, then an abstract review and finally a full text review using the inclusion and exclusion criteria. Among the articles that remained at the abstract level (n = 4), a second reviewer also examined the abstracts of the articles to confirm their eligibility, and there were no discrepancies with the first reviewer.

Sustainability 2020, 12, x FOR PEER REVIEW 4 of 25

any type of mobile application in the sports and fitness context; and (iv) measurement of the intention to use the App through a questionnaire. As exclusion criteria have been used: (i) Congress proceedings, book chapters, books or other types of publications; (ii) no mobile Apps were used in the sports context, (iii) theoretical studies, qualitative approach or reviews; (iv) articles in a language other than English; and (v) duplicate articles.

2.3. Assessment of Methodological Quality

The risk of bias was assessed using a 20-item tool adapted by the authors to the context of sports marketing study typology in which there are no intervention processes on the subjects of the Consolidated Standards of Reporting Trials (CONSORT) checklist [48]. Each study was independently scored by two reviewers evaluating the different sections that make up the studies and scoring each item with 1 if the study satisfactorily met the criterion, and with 0 if the study did not satisfactorily meet the criterion or if the item was not applicable to the study. Disagreements between the reviewers were resolved by checking and discussing the original study until consensus was reached. Reviewer A is a researcher with extensive experience specializing in the field of sports management, fitness centers and development of new technologies. Reviewer B is a predoctoral fellow in sports management with focus research on methodological and statistical aspects. The results of assessment of methodological quality were shown in Appendix A.

2.4. Data Extraction and Synthesis

Figure 1 shows the Flow Diagram proposed by Moher, Liberati, Tetzlaff, and Altman [49] following the PRISMA methodology in all points that could be common to a systematic review of these characteristics. The initial database search returned 113,537 results, reduced to 36,105 once duplicates were eliminated. One reviewer conducted a full scan of the title, then an abstract review and finally a full text review using the inclusion and exclusion criteria. Among the articles that remained at the abstract level (n = 4), a second reviewer also examined the abstracts of the articles to confirm their eligibility, and there were no discrepancies with the first reviewer.

Figure 1. PRISMA flow diagram. Source: Moyer et al. [49].

Sustainability 2020, 12, 6641 5 of 24

A form was developed for data extraction that included the following aspects: (a) year of publication; (b) country of study; (c) number of participants; (d) gender; (e) age of participants; (f) type of application evaluated; (g) theory used; (h) analyses performed; (i) variables included; and (j) main results. In order to homogenize the results of the di↵erent studies and to make the data more homogeneous, the confidence intervals of each correlation (CI 95%) and the e↵ect size with its confidence intervals (CI 95%) of each relationship were calculated through the Fisher’s Z statistics [50].

3. Results

3.1. Analysis of the Risk of Bias in Studies

To test quality, risk of bias analysis of the 19 studies evaluated in the research showed that only three studies had a high score of 15 points or more out of 20 total [1,45,51], most studies (n = 14) had a mean score between 10 and 15 points and only two studies had a score below 10 points [52,53]. It should be noted that none of the studies analyzed carried out a calculation of the sampling required for the generalization of the results, which could be due to the fact that all the studies carried out a selection of the sample for convenience within a certain population. There are also few studies that established criteria for inclusion in the sample to be selected (n = 5) and no study indicates the author who carried out each part of the research.

3.2. Summary of Reported Intervention Outcomes

Results of the descriptive data from the analysis of the articles can be seen in Table 2. The analysis shows that this topic is very recent within the context of sports marketing, with only 13 quantitative studies addressing the intention to use of sports applications by the sports consumer through the use of self-administered questionnaires and online. Of the articles analyzed, the majority were published in 2018 (n = 5) and 2020 (n = 5), followed by those published in 2017 (n = 4), three articles were published in 2015, while only one article was found in 2016 and 2019. Korea has been the country with the highest production with six articles, followed by the United States and Hong Kong with three publications, China had two studies and other countries such as Germany, India, Iran, South Africa and Taiwan each had one publication.

Analyzing the sample used in the di↵erent studies, there is a total of 16,025 subjects with an average sample of 843.42 subjects per study, with the Ndayizigamiye; Kante, and Shingwenyana study [54] having the smallest sample (n = 139) and Wei, Vinnikova, Lu and Xu study [55] having the largest sample with a total of 8840 subjects. Approximately a half of the studies (n = 8) used university students as a sample, followed by studies that considered users of sports applications (n = 4) and other studies took as their general population [54–56], a population of sports consumers [45], employees of a sports organization [57] and members of a fitness community [44,58]. Most studies had a higher proportion of females than males (n = 9), followed by studies that had parity in the sample (n = 5), four studies had a higher proportion of males while one study did not indicate the gender distribution of the sample [34]. Finally, all studies except Ha et al. [42] and Yoo et al. [53] reported some data on the age of the subjects. About half of the studies (n = 10) expressed age using a range, five studies showed age using mean (M = 24.58 years) and two studies did not specify the age [55,59]. The analysis indicated that mainly the study population are young subjects between 20 and 29 years old and all are over 18 years old except Lee, Kim and Wang [45] which also included 17-year-old subjects. Li, Liu, Ma and Zhang [46] sampled subjects over 25 years of age, while Huang and Ren [60] and Mohammadi and Isanejad [57] were at least 30 years old.

Sustainability 2020,12,6641

6 of24

Table 2.D

escriptive data

ofthe analysis

ofthe selected

studies.

A uthors

C ountry

Sam ple

A pp

Type T

heory D

ata A

nalysis M

ethods M

easure O

utcom es

Beldad &

H egner

[43] G

erm any

G erm

an’s app

user (n =

476) M

ale:50.0% Fem

ale:50.0% A

ge: 26.7±

5.0

Sport inform

ation TA

M C

ontentA nalysis

Trustin the

Fitness A

pp D

eveloper;D escriptive

SocialN orm

;Injunctive SocialN

orm ;Perceived

Ease ofU

se;Perceived U

sefulness;Intention to

C ontinue

U sing

a Fitness

A pp

Byun,C hiu,&

Bae [45]

K orea

K orean

consum ers

(n =

261) M

ale:4 9.1%

;Fem ale:50.9%

A ge:20–29

(29.9% );30–39

(34.9% );40+

(5.2% )

SportBrand TA

M C

ontentA nalysis

Perceived Enjoym

ent;Perceived Ease

ofU se;

Perceived U

sefulness;Intention to

use;A ctual

usage

C hen

& Lin

[44] Taiw

an

Fitness C

om m

unity (n =

994) A

ge:20� (10.06%

);20–29 (56.14%

);30–39 (1.83%

);40–49 (8.65%

);50–59 (3.32%

)

D iet/Fitness

TR A

M C

ontentA nalysis

H ealth

C onsciousness;O

ptim ism

;Innovativeness; D

iscom fort;Insecurity;Perceived

Ease ofU

se; Perceived

U sefulness;A

ttitude tow

ard U

sing A

pp;Intention to

dow nload

app

C hiu

& C

ho [61]

H ong

K ong

K orean

university students

(n =

204) M

ale:51.9% ;Fem

ale:48.1% A

ge:19–25 (71.8%

);26–30 (10.7%

);30+ (17.5%

)

H ealth

/Fitness TR

A M

D escriptive

C ontentA

nalysis

O ptim

ism ;Innovativeness;Insecurity;

D iscom

fort;Perceived U

sefulness;Perceived Ease

ofU se;Perceived

Enjoym ent;Intention

to use

C hiu,C

ho,& C

hi[56] H

ong K

ong

C hinese

population (n =

342) M

ale:45.6% ;Fem

ale:54.4% A

ge:20� (1.2%

);21–25 (14.9%

); 26–30

(35.4% );31–35

(29.8% );

36–40 (11.1%

);40+ (7.6%

)

H ealth

/Fitness EC

M D

escriptive C

orrelational C

ontentA nalysis

Investm entsize;Q

uality ofalternative;

C om

m itm

ent;C onfirm

ation ofexpectations;

Satisfaction;Perceived U

sefulness;C ontinuance

Intention

C ho,Lee,K

im ,&

Park [59]

K orea

U niversity

students (n =

294) M

ale:33.0% Fem

ale:67.0% A

ge:23.2 D

iet/Fitness TA

M C

orrelational C

ontentA nalysis

A ppearance

Evaluation;Fitness Evaluation;

A ppearance

O rientation;Fitness

O rientation;

Perceived U

sefulness;Intention to

U se

A pp

C ho,Lee,&

Q uinlan

[51] K

orea U

niversity students

(n =

508) M

ale:34.6% ;Fem

ale:65.4% A

ge:21.5 D

iet/Fitness TA

M D

escriptive C

ontentA nalysis

Subjective N

orm s;Entertainm

ent;R ecordability;

N etw

orkability;Perceived Ease

ofU se;Perceived

U sefulness;BehavioralIntention

to U

se

C ho

& K

im [52]

K orea

U niversity

students (n =

277) M

ale:34.3% ;Fem

ale:65.7% A

ge:22.5 D

iet/Fitness TA

M C

ontentA nalysis

Sm artphone

U se

E �

cacy;InternetInform ation

U se

E �

cacy;InternetInform ation

C redibility;

Perceived Ease

ofU se;Perceived

U sefulness;

BehavioralIntention

D him

an,A rora,

D ogra,&

G upta

[58] India

Indian fitness

lefts users

(n =

324) M

ale:54.0% ;Fem

ale:46.0% A

ge:20� (16.0%

);20–40 (80.0%

); 40+

(4.0% )

Fitness U

TA U

T2 D

escriptive C

orrelational C

ontentA nalysis

Perform ance

Expectancy;E ↵ortExpectancy;Self

E �

cacy;SocialInfluence;Facilitating C

onditions; H

edonic M

otivation;Price Value;Personal

Innovativeness;H abit;BehavioralIntention

Sustainability 2020,12,6641

7 of24

Table 2.C

ont.

A uthors

C ountry

Sam ple

A pp

Type T

heory D

ata A

nalysis M

ethods M

easure O

utcom es

H a,K

ang,& K

im [42]

K orea

U niversity

students (n =

226) M

ale:50.8% ;Fem

ale:49.2% A

ge:25.3

Sport Inform

ation TA

M D

escriptive C

ontentA nalysis

SportInvolvem ent;SportC

om m

itm ent;Social

Influence;PersonalA ttachm

ent;M edia

M ultitasking;Perceived

Enjoym ent;Perceived

Ease ofU

se;Perceived U

sefulness;U sage

Intention

H uang

& R

en [60]

H ong

K ong

C hinese

app users

(n =

449) M

ale:43.0% ;Fem

ale:57.0% A

ge:31.85± 6.9

Fitness TA

M R

egression

Instruction Provision;Self-M

onitoring; Self-R

egulation;G oalA

ttainm ent;Exercise

Self E �

cacy;Perceived U

sefulness;Perceived Ease

of U

se;Perceived Enjoym

ent;C ontinuance

Intention

K im

,K im

,& R

ogol[1] U

nited States

A pp

users (n =

233) M

ale:68.2% Fem

ale:31.8% A

ge:18–24 (46.8%

);25–34 (31.8%

);35–44 (13.7%

);45–54 (7.3%

);55+ (0.4%

)

SportTeam TA

M D

escriptive C

ontentA nalysis

Innovativeness;Perceived Ease

ofU se;Perceived

Enjoym ent;Perceived

Trust;Perceived U

sefulness; Intention;SportA

pps U

se

Lee,K im

,& W

ang [62]

U nited

States

C ollege

students (n =

267) M

ale:32.2% Fem

ale:67.8% A

ge:17–20 (48.3%

);21–25 (40.8%

);26–29 (8.2%

);29+ (2.6%

)

SportA pp

U TA

U T

C orrelational

C ontentA

nalysis

Entertainm entM

otivation;SocialU tility

M otivation;Perform

ance Expectancy;E

↵ort Expectancy;SocialInfluence;Intention

to M

obile Sports

A pps

U se

Li,Liu,M a,&

Z hang

[63] C

hina

SportA pp

users (n =

211) M

ale:45.02% Fem

ale:54.98% A

ge:25–30 (41.71%

);30–35 (47.87%

)35+ (10.43%

)

Social Fitness-tracking

U TA

U T2

C ontentA

nalysis A

ctivity A

m ountR

anking;A ctivity

Frequency R

anking;C onfirm

ation;U pw

ard C

om parison

Tendency;C ontinuous

Intention

M oham

m adi&

Isanejad [57]

Irán

Em ployers

SportO rganization

(n =

332) M

ale:37.3% Fem

ale:62.7% A

ge:30� (10.0%

);31–40 (44%

); 41–50

(38% );50+

(8% )

IT inform

ation TA

M D

escriptive C

orrelational C

ontentA nalysis

Technology A

nxiety;Technology Self-e�

cacy; Perceived

Enjoym ent;Perceived

Ease ofU

se; Perceived

U sefulness;U

ser Satisfaction;A

ttitude; Intention

to use

N dayizigam

iye; K

ante,& Shingw

enyana [54]

South A

frica

South A

frican population

(n =

139) M

ale:41.5% ;Fem

ale:58.5% A

ge:18–23 (57.15%

);24–29 (29.9%

);30–35 (7.5%

)

m H

ealth U

TA U

T C

orrelational C

ontentA nalysis

A w

areness,E ↵ortExpectancy;Facilitating

C onditions;Perform

ance Expectancy;Social

Influence;BehavioralIntention

Sustainability 2020,12,6641

8 of24

Table 2.C

ont.

A uthors

C ountry

Sam ple

A pp

Type T

heory D

ata A

nalysis M

ethods M

easure O

utcom es

W ei,V

innikova,Lu,& X

u [55]

C hina

C hinese

population (n =

8840) M

ale:4.78% ;Fem

ale:74.55% D

iet/Fitness U

TA U

T D

escriptive C

orrelational C

ontentA nalysis

Perceived Benefits;Perceived

Barriers;Perceived Threats;Self-E

� cacy;R

isk Perception;

Perform ance

Expectancy;W eightLoss

Intention; BehavioralIntention;U

se Behavior

Yoo,K o,&

Yeo [53]

K orea

U niversity

students (n =

1331) M

ale:65.9% ;Fem

ale:34.1% SportC

ontent TA

M C

ontentA nalysis

Perceived Trust;Perceived

U sefulness;A

ttitude; U

sing intention

Yuan,M a,

K hantaw

ala &

Peng [30]

U nited

States

U niversity

students (n =

317) M

ale:21.1% Fem

ale:78.9% A

ge:21 D

iet/Fitness U

TA U

T2 C

ontentA nalysis

Perform ance

expectancy;E ↵ortexpectancy;Social

influence;Facilitating conditions;Price

value; H

edonic m

otivation;H abit;Intention

to use

EC M

:Expectation-C onfirm

ation M

odel.

Sustainability 2020, 12, 6641 9 of 24

Regarding the type of App evaluated, six studies evaluated the intention to use diet and fitness applications [30,44,51,52,55,59], another five studies evaluated sports information Apps [42,43,53,57,62], two studies measured the intention of fans to use the sports team app [1,45], health and fitness app [56,61], or fitness [58,60] and one study evaluated a social fitness-tracking app [63] and mHealth related to promote physical activity [54]. The most widely used theory for the design and use of the mobile sports app intent of use assessment instrument was TAM (n = 10). Chen and Lin [44] and Chiu and Cho [61] used a variant of the TAM, Theory of Readiness and Acceptance Model (TRAM), three studies used UTAUT [54,55,62] and UTAUT2 [30,58,63], and an article with the Expectation-Confirmation Model (ECM) [56]. The most common method of analysis used was a content analysis by structural equations using the AMOS statistical package (n = 9) and the rest of the Partial Least Square studies. Seven studies also performed a correlation analysis of the data [54–59,62], and eight studies performed descriptive analysis in addition to content analysis [1,42,51,55–58,61].

The variables used by the di↵erent studies have been very varied, where the intention to use App has been found in all studies as a common factor. Considering that this systematic review study focused on studies based on TAM as the most commonly used theory in sports marketing studies, it implies that there are other common variables among most studies such as perception of usefulness (n = 13) and perception of ease of use (n = 10). Some studies have included other di↵erent perceptions by relating them to the previous ones and the intention to use, such as the perception of enjoyment [1,42,45,60,61] or the perception of trust [1,53]. The remaining variables used have been very diverse, with each study using di↵erent variables that can be seen in Table 2. However, the variable of social influence has received greater interest from researchers and has been considered in five studies [30,42,54,58,62].

Analyzing the quantitative data on the relationships between the most common variables associated with TAM (Table 3), the six studies that had a di↵erent theory such as UTAUT [54,55,62] or UTATUT2 [30,58,63] were excluded; however, the study that used ECM was included because there was a relationship between variables “perception usefulness” and “intention to use” [56]. The sample was very heterogeneous in terms of the results of the existing relationships and sample size in each study. In order to homogenize these results, the e↵ect size of each correlation was calculated. A total of seven relationships were identified between the di↵erent variables associated with TAM such as perception of ease of use (PEOU), perception of utility (PU), perception of enjoyment (PE), perception of trust (PT), intention to use (ITU) and actual usage (AU).

Sustainability 2020,12,6641

10 of24

Table 3.Q

uantitative data

on the

relationships betw

een “Theory

A cceptance

M odel”

(TA M

)variables in

the selected

studies.

A uthors

PE-PEO U

PEO U

-PU PE-IT

U PEO

U -IT

U PU

-IT U

PT-IT U

IT U

-A U

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

Beldad &

H egner

[43] -

0.180 ***

(0.092;0.266) -

0.440 ***

(0.365;0.510) 0.330

*** (0.246;0.408)

- -

Byun,C hiu,&

Bae [45]

0.584 ***

(0.498–0.659) 0.583

*** (0.497;0.658)

0.454 ***

(0.352;0.545) 0.157

* (0.036;0.273)

0.318 ***

(0.205;0.423) -

0.306 ***

(0.19;0.41)

C hen

& Lin

[44] -

0.650 ***

(0.613;0.685) -

0.310 ***

(0.253;0.365) 0.510

*** (0.463;0.555)

- 0.530

*** (0.484;0.573)

- 0.660

*** (0.623;0.694)

- 0.300

*** (0.24;0.36)

0.500 ***

(0.452;0.545) -

0.520 ***

(0.473;0.546)

C hiu

& C

ho [61]

0.453 ***

(0.337;0.556) 0.222

*** (0.087;0.349)

0.293 ***

(0.162;0.414) 0.213

** (0.078;0.340)

0.373 ***

(0.248;0.486) -

-

C hiu,C

ho,& C

hi[56] -

- -

- 0.299

** (0.199;0.392)

- -

C ho,Lee,K

im ,&

Park [59]

- -

- -

0.750 ***

(0.70;0.80) -

-

C ho,Lee,&

Q uinlan

[51] -

0.229 ***

(0.145;0.310) -

0.001 (�

0.086;0.088) 0.431

*** (0.357;0.500)

- -

C ho

& K

im [52]

- 0.580

*** (0.496;0.653)

- �

0.130 (�

0.244;� 0.012)

0.800 ***

(0.753;0.839) -

-

H a,K

ang,& K

im [42]

0.770 *

(0.711–0.818) 0.870

* (0.834;0.899)

� 0.08

(� 0.51;0.208)

0.350 *

(0.230;0.460) 0.090

(� 0.041;0.218)

- -

H uan

& R

en [60]

- -

0.233 ***

(0.144;0.319) 0.270

*** (0.182;0.354)

0.404 ***

(0.324;0.479) -

-

K im

,K im

,& R

ogol[1] -

-

0.313 **

(0.192;0.424) 0.163

(0.035;0.286) 0.628

*** (0.543;0.700)

0.387 *

(0.272;0.491) 0.787

*** (0.733;0.831)

0.331 ***

(0.212;0.440) 0.204

(0.078;0.324) 0.728

*** (0.661;0.783)

0.475 *

(0.369;0.569) 0.806

*** (0.756;0.847)

0.287 **

(0.165;0.400) 0.149

(0.021;0.272) 0.580

*** (0.489;0.660)

0.359 *

(0.241;0.466) 0.808

*** (0.758;0.848)

M oham

m adi&

Isanejad [57]

0.370 **

(0.273–0.459) 0.386

** (0.290;0.473)

- 0.173

** (0.067;0.283)

0.192 **

(0.086;0.293) -

-

Yoo,K o,&

Yeo [53]

- -

- -

0.454 ***

(0.410;0.496) 0.403

*** (0.357;0.447)

0.963 ***

(0.959;0.967)

Sustainability 2020,12,6641

11 of24

Table 3.C

ont.

A uthors

Fisher’s Z

(C I95%

) Fisher’s

Z (C

I95% )

Fisher’s Z

(C I95%

) Fisher’s

Z (C

I95% )

Fisher’s Z

(C I95%

) Fisher’s

Z (C

I95% )

Fisher’s Z

(C I95%

)

Beldad &

H egner

[43] -

0.18 (0.09;0.27)

- 0.47

(0.38;0.56) 0.34

(0.25;0.43) -

- Byun,C

hiu,& Bae

[45] 0.67

(0.55;0.79) 0.67

(0.55;0.79) 0.49

(0.37;0.61) 0.16

(0.04;0.28) 0.33

(0.21;0.45) -

0.32 (0.19;0.44)

C hen

& Lin

[44] -

0.78 (0.71;0.84)

- 0.32

(0.26;0.38) 0.56

(0.50;0.63) -

0.59 (0.53;0.65)

- 0.79

(0.73;0.86) -

0.31 (0.25;0.37)

0.55 (0.49;0.61)

- 0.58

(0.51;0.64) C

hiu &

C ho

[61] 0.49

(0.35;0.63) 0.23

(0.09;0.36) 0.30

(0.16;0.44) 0.22

(0.08;0.35) 0.39

(0.25;0.53) -

- C

hiu,C ho,&

C hi[56]

- -

- -

0.31 (0.20;0.41)

- -

C ho,Lee,K

im ,&

Park [59]

- -

- -

0.97 (0.86;1.09)

- -

C ho,Lee,&

Q uinlan

[51] -

0.23 (0.15;0.32)

- 0.001

(� 0.09;0.09)

0.46 (0.37;0.55)

- -

C ho

& K

im [52]

- 0.66

(0.54;0.78) -

� 0.13

(� 0.25;�

0.01) 1.10

(0.98;1.21) -

- H

a,K ang,&

K im

[42] 1.02

(0.89;1.15) 1.33

(1.20;1.46) 0.08

(� 0.05;0.21)

0.37 (0.23;0.50)

0.09 (�

0.04;0.22) -

- H

uan &

R en

[60] -

- 0.24

(0.14;0.33) 0.28

(0.18;0.37) 0.43

(0.34;0.52) -

-

K im

,K im

,& R

ogol[1] -

- 0.32

(0.19;0.45) 0.16

(0.04;0.30) 0.74

(0.61;0.87) 0.41

(0.28;0.54) 1.06

(0.93;1.19) 0.34

(0.214;0.47) 0.21

(0.08;0.34) 0.92

(0.80;1.05) 0.52

(0.39;0.65) 1.12

(0.99;1.24) 0.30

(0.16;0.42) 0.15

(0.02;0.28) 0.66

(0.53;0.79) 0.38

(0.25;0.51) 1.12

(0.99;1.25) M

oham m

adi& Isanejad

[57] 0.39

(0.28;0.50) 0.41

(0.30;0.51) -

0.17 (0.07;0.28)

0.19 (0.09;0.30)

- -

Yoo,K o,&

Yeo [53]

- -

- -

0.49 (0.44;0.54)

0.43 (0.37;0.48)

1.99 (1.93;2.04)

N ote:*p

0.05;**p 0.01;***p

0.001;PE:Perceived Enjoym

ent;PEO U

:Perceived Ease

ofU se;PT:Perceived

Trust;PU :Perceived

U sefulness;ITU

:Intention to

U se;A

U :A

ctualU sage.

Sustainability 2020, 12, 6641 12 of 24

The first relationship between the variables PE and PEOU was found in four di↵erent studies showing results of di↵erent degree of relationship, although all significant. In the study by Mohammadi and Isanejad [57], the e↵ect size was found to indicate a low-medium influence of PE on PEOU [Fisher’s Z = 0.39 (0.2; 0.50)], Chiu and Cho [61] found a medium influence [Fisher’s Z = 0.49 (0.35; 0.63)], while Byun et al. [45] found a medium-high influence of PE on PEOU [Fisher’s Z = 0.67 (0.55; 0.79)] and a very high influence of enjoyment in students on ease of use [Fisher’s Z = 1.02 (0.89; 1.15)]. Another relationship of perceptions and the most studied is between PEOU and PU. Although all the existing relationships were significant, they had di↵erent levels of relationship: for example, three studies had a low influence of PEOU on PU with values lower than 0.30 [43,51,61], while Mohammadi and Isanejad [57] found that PEOU had an influence on lower-middle on PU [Fisher’s Z = 0.41 (0.30; 0.54)]. Other studies found a high or very high influence between these two variables [Fisher’s Z� 0.66]. In the selected studies, no other studies were found where the di↵erent dimensions of TAM were interrelated.

The relationship of the di↵erent variables on the ITU is explained below. Firstly, four studies found an influence of PE on ITU where PE had a low influence [60,61], and moderate and significant influence on ITU in the di↵erent models evaluated with an e↵ect size range between 0.30 and 0.49 [1,45]. In contrast, Ha et al. [42] did not find that PE had an influence on ITU [Fisher’s Z = 0.08 (�0.05; 0.21)]. One of the most studied relationships is the PEOU with ITU, being tested in ten studies with contradictory results. Seven studies found a significant relationship, however, except Beldad and Hegner [43] which had a moderate influence [Fisher’s Z = 0.47 (0.38, 0.56)] and Chen and Lin [44] in their study evaluated two models with moderate-low influences [Fisher’s Z1 = 0.31 (0.25, 0.37); Fisher’s Z2 = 0.32 (0.26, 0.38)]. The rest had a low influence [45,57,60,61] or nonsignificant influence [1,51,52].

The relationship between PU and ITU was evaluated in all of the studies analyzed finding di↵erent levels of relationship in the studies. Most studies (n = 9) found a moderate and significant influence between both variables with a range of Fisher’s Z = 0.31–0.66 [1,43–45,51,53,56,60,61]. Some studies found a significant high to very high ratio with a large influence of PU on ITU based on e↵ect size values above 0.70 [52,59] and in two of the three models proposed by Kim et al. [1]. Mohammadi and Isanejad [57] found a low influence on sports organization employees [Fisher’s Z = 0.19 (0.09, 0.30) and Ha et al. [42] found no PU influence on ITU in university students on sports information Apps.

In turn, two studies evaluated the relationship between PU and ITU observing the existence of a moderate and significant influence between both variables in the three models proposed by Kim et al. [1] [Fisher’s Z = 0.38–0.52] and Yoo et al. [53] [Fisher’s Z = 0.43 (0.37; 0.48)]. Finally, some studies evaluated the relationship between ITU and AU found relationships where ITU had a moderate-low influence [45] [Fisher’s Z = 0.32 (0.19, 0.44)], while other studies found a moderate influence [44] and a very high influence [1,53].

3.3. Summary of Factors Relationship by Age

In order to carry out the analysis of the di↵erent relationships between the factors analyzed (Table 3) based on age, an analysis was made of the data shown by the di↵erent studies. The studies have been classified into three groups considering the mean age or the age range with the highest percentage. One group was established with six studies that had a higher proportion of subjects under 25 years [1,51–53,59,61], a second group formed by three studies with the age of the subjects between 25 and 30 years [42,43,56] and finally, a third group had a higher proportion of subjects over 30 years [45,57,60]. Chen and Lin’s study [44] was not classified as it indicated that their sample was mostly composed of subjects between 20 and 30 years of age.

The relationship between PE and PEOU showed that the study of group 25–30 years was very high and significant [42], compared to group under 25 and group over 30, which had very similar moderate and significant relationships with an e↵ect size range between 0.39 and 0.67 [45,57,61]. The relationship between PEOU and PU found that the studies with the lowest influence [43] and the highest influence [42] belonged to the group 25–30. The studies of the group over 30 had generally more influencing than the group under 25 who had the most studies with a low influence. However,

Sustainability 2020, 12, 6641 13 of 24

the study by Chen and Lin [44] reported that in their two models the relationship between PEOU and PU was high above 0.70, and significant. Concerning the PE relationship on ITU, the group over 30 had a moderate and significant overall influence [45,60], being higher than the group under 25 [1,61], while the study of the group 25–30 found no relationship between these two factors [42].

The group 25–30 revealed a significant influence of moderate level on the relationship between PEOU and ITU with e↵ect sizes higher than 0.37 [42,43], and the studies of the group over 30 found the existence of significant relationships with a low influence [45,57,60]. On the other hand, most studies of the group under 25 did not show the existence of significant di↵erences between these factors [1,51,52], except the study of Chiu and Cho [62] that had a low and significant influence. Chen and Lin [44] obtained moderate-low relations in their two models in the population between 20 and 30 years old. The most studied relationship in all studies was between PU and ITU, with a great variability in the results according to the di↵erent studies in each age group. Therefore, the larger influence has been found in the studies of the group under 25, with studies that had moderate [51,53,60], high [1,59] and very high [52] relationships, while one study of the group 25–30 observed a high influence [59] and two studies showed moderate influences [43,56]. However, the group 25–30 had one study in which no relationship was found between PU and ITU [42]. The group over 30 had all significant relationships with a variable low level [57] to moderate level [45,60]. In addition, Chen and Lin [44] reported a moderate influence of PU on UTI in their two proposed models. Finally, the relationship between ITU and AU has not been studied much, and no studies were found that took it into account in the group 25–30. Regarding the other two groups, the group under 25 obtained that the youngest ones presented a great influence between ITU and AU giving them significant and high relationships with an e↵ect size above one point [1,53]. In the study of group over 30, a moderate-low relationship was found [45], and Chen and Lin had moderate-significant relationships [44].

4. Discussion

The continuous technological advances have awakened the interest of marketing researchers in the intention to use Apps, especially in the field of sports. Walter [64] explained the existence of a trend towards increased interest by fitness consumers in using Apps for exercise control. Therefore, the aim of this study was to conduct a systematic review of the literature on consumers’ intention to use Apps related to fitness and physical activity. The result of the systematic search has been the existence of a remarkable interest in the subject, as the studies were found in the last four years; however, studies focused on sport Apps are still limited (n = 19) with very heterogeneous methodologies.

In the context of marketing, the existence of di↵erent theories to explain sports consumer behavior has been observed, with TAM being the most applied in the di↵erent studies found (n = 10) or some version derived from it such as TRAM [44,61], UTAUT in the study carried out by Lee et al. [62], Ndayizigamiye et al. [54] and Wei et al. [55], or UTAUT2 used by Dhiman et al. [58], Li et al. [46] and Yuan et al. [20]. One study used a di↵erent model, ECM, which was not derived from TAM [56]. Although most publications use the same base, no similar studies have been found that can be compared since the di↵erent authors have used the two basic constructs (PEOU and PU) of the theory proposed by Davis [24] and have incorporated other variables such as PE [1,42,45,57,60,61] or PT [1,53]. In addition, they have been relating these variables to the Apps ITU and AU [1,44,45,53] or to various other variables outside of TAM such as social influence [30,42,54,58,62], Health Consciousness [44], social norms [43] or sports context as sport involvement or sport commitment [42].

Most of the studies analyzed have tried to predict the influence of PEOU and PU on ITU, finding significant relationships between both constructs [42–45,51,52,57,61]. These relationships have been evaluated previously in the context of sport websites as a technological tool prior to the appearance of the Apps [31,38,39,65]. However, the findings found are di↵erent from those proposed by di↵erent studies that concluded with a greater influence of PU on PEOU in ITU [24,38,49,65–70] or where PEOU had no significant influence on ITU [1,51,52] or in the case of Ha et al. [42] where they found that PU

Sustainability 2020, 12, 6641 14 of 24

had no influence on ITU as opposed to PEOU. In particular, it should be noted that Davis [24] argued that PEOU is a secondary determinant construct of ITU in the perception of technology.

Furthermore, the use of other added beliefs such as PE or PT can also play a key role within the TAM, as PE not only has influence on PEOU [42,45,57,61] but also on ITU [1,45,60,61]. Ha et al. [42] found no influence of PE on ITU, but the authors felt that while it was not related to ITU, it was related to PEOU, allowing sports consumers to undermine the di�culty of using the technology [71]. These results highlight the hedonic motives of sports consumers who often overlook entertainment [26,65,71] and may make the App more interesting to sports fans if they find it fun and easy to use. On the other hand, PT has been found to be an important and influential factor on ITU [1,38,39,53,69]; this importance may be motivated by the fact that if a sports fan finds the information they receive useful and reliable they will show a greater interest in continuing to use the technology.

For example, in a study in Germany on the use of fitness Apps, it showed that the reasons for using them were to have the achievement of fitness goals and the enjoyment of being able to share the results obtained with their contacts [72], so gamification could have a direct relationship with their use [73]. Another aspect to take into account for consumers is that if they find the App tedious and requiring tedious procedures initially, it negatively a↵ects their intention to make use of the App when considering them complex [74]. On the other hand, Wang Egelandsdal, Amdam, Almli and Oostindjer [75] express that time and e↵ort may discourage users from using health Apps, thus arguing why PEOU can explain with more influence the ITU new technologies than PEU. A qualitative study by Tang, Abraham, Stamp and Greaves [76] revealed that the appearance and structure of a weight loss app’s interface could significantly influence the decision to use such technology. Sports providers should conduct better market research to better understand what fans want or expect in order to meet their expectations [77].

Analyzing the results of the relationships according to the age of the subjects in the sample, three di↵erent groups were established according to the average age of the sample or the age range with the highest percentage of representativeness. The first group consisted of studies with subjects under 25 years old [1,51–53,59,61], the second group consisted of participants aged between 25 and 30 years [42,43,56], a third group consisted of people over 30 years old [45,57,60] and one study was not grouped because the age of the sample ranged from 20 to 29 years [44]. Considering the di↵erent age groups and the relationships between the di↵erent factors, the results have shown that the group under 25 in the PU-ITU and ITU-AU relationships, this is, the studies with the larger young population, showed higher relationships of influence on the usefulness of the application for its use [1,51–53,59,60]. Young people were more likely to use fitness or sport Apps when they found them more functional, which meant that this intention had a great influence on the actual use of the app [1,53]. The studies in the group 25–30 and group over 30 found similar moderate relationships between PU-ITU, except for two studies in the group 25–30, one study observed a high influence between PU-ITU [59], and the study did not find PU to significantly influence UTIs [42].

On the other hand, the group 25–30 highlighted among the other two groups the relationships that have evaluated the PEOU factor, emphasizing the influence of this factor on ITU [42,43], while in the PEOU-PU relationship, the two studies of the group 25–30 that analyzed this relationship were the ones that had the greatest and least influence compared to the studies of the other groups [42,43]. The subjects in this intermediate group considered more the ease and intuition of the app to be able to perform it than the younger or older subjects. The group over 30 also found a moderate to high influence on the PEOU-ITU relationship [45,57]. However, the older group of studies were the ones that stood out in the PE influence on UTIs [45,60]; that is, an app that is fun will be more likely to be used. Finally, PE was shown to have a moderate influence on PEOU in the group under 25 [61] and group over 30 [45,57] and high influence in the group 25–30 [42].

Sustainability 2020, 12, 6641 15 of 24

Limitations and Future Research

Among the limitations of this review study could be started with the reduction of studies that are based on TAM and not the other existing theories or models. Likewise, a document search in the grey literature was not performed since the researchers marked a criterion that the studies should be collected in the di↵erent databases analyzed. Another aspect that could have conditioned the study was the focus only on studies that evaluated only fitness and physical activity Apps, discarding the studies that could evaluate the use of wearables or sports webs that would allow a broader analysis of existing studies on the intention to use in the sports context. Similarly, the years of research for the study could have been limited to the dates since the appearance of the Smartphone and the Apps, resulting in a large number of initial results that were not limited in time.

In addition, another of the limitations found is the lack of a common questionnaire that has been replicated in di↵erent studies and allows a better comparison of the di↵erent populations. This fact may be due to the fact that this line of research is very recent and very few studies have appeared that could lead to its replication and comparison of studies. In addition, all the studies choose the sample for convenience, whether the groups of university students or sports groups are easily accessible. The use of convenience sampling denotes a lack of application of more appropriate sampling that would allow the generalization of results to a larger universe. Some studies did refer to this limitation [44,78]. The age of the participants has been a widely reported limitation in the di↵erent studies [1,51,52,61,78] where the sample was mainly composed of young people born into the new digital society and whose beliefs may be very di↵erent from those of middle-aged adults who have not been raised in a digital society and who tend to have greater di�culties in starting to use the new technologies. Only three studies looked at a sample over 30 years old [45,57,60], while in the literature there is a study that evaluated the use of websites by people over 50 [79].

These di↵erences between age groups and therefore with digital gaps must also be approached from a sociodemographic point of view and with very di↵erent contexts. For example, Cho et al. [51] carried out a comparative study between the Korean and American populations, highlighting the existence of a limitation in this type of study due to the oversocialization of the individual characteristics of consumers and their contexts. Thus, some authors indicate that an individual’s socioeconomic status and educational level a↵ect their use of new technologies and Apps, since individuals with a higher level of education may have better digital skills [51]. In fact, they may have greater access to a larger number of technological devices, since not only Smartphones should be considered but also other devices such as tablets, laptops or even wearables [42]. This factor is important to take into account as currently, there are other technological devices that could use sports Apps (i.e., tablets, i-watch, etc.).

Similarly, the operating system could have an influence as not all Apps are available on the current large operating systems (Android and IOS). In addition, there are a multitude of di↵erent Apps that perform the same functions and many studies question the intention to use without determining what type of Apps the consumer uses. This fact could lead to a self-selection bias when filling out the questionnaire by the consumer considering the App that people use and not one common to the entire population [44].

With regard to other methodological aspects of the studies, it should be noted that all the studies found have had a cross-sectional approach, and no longitudinal study has been found to evaluate the evolution of the subject and his or her real intention to continue using said App. Chen and Lin [34] point out that there may be a discrepancy between the ITU of the App on the part of the consumer and the actual use that they make of this App afterwards. In general, this has been a relationship that some studies have considered [1,35,43]; however, as these are cross-sectional studies, it has not been possible to verify the change that they have had with respect to a lesser or greater use of the App. Likewise, not all studies have included other variables associated with TAM such as PE or PT which allow us to verify their influence on ITU in di↵erent groups and contexts.

Finally, this review has mainly focused on the search for quantitative studies, finding several qualitative studies but only two studies were found that used mixed methods to evaluate the ITU and

Sustainability 2020, 12, 6641 16 of 24

to be able to check what the user expresses in the questionnaire with the actual opinion about that App [75,80].

Future lines of research on the intention to use sports Apps or any other device should consider the inclusion of more variables of TAM in their model, as well as other variables specific to the Smartphone (social influence, attachment to the device, etc.), the sports context (motivation, commitment, participation) or variables traditionally linked to sports marketing such as satisfaction [44]. Therefore, common measuring instruments should be standardized to allow their application in di↵erent contexts. Similarly, the di↵erent age generations should be taken into account when evaluating and sampling in a population that includes consumers of di↵erent age ranges.

It would also be interesting to sample populations that do not refer to a single sport or discipline but rather have a variety of Apps from di↵erent sports or sports teams. Future work should also be carried out in a longitudinal way, being able to check whether the intention to use predisposed by the subject ends up being the real use of the application or not. Finally, it is interesting to contemplate studies that cover di↵erent social groups with cultural diversity analyzing not only the intention to use in the context itself but also in the individual characteristics of the consumer with di↵erent status and educational levels to evaluate the digital gap.

5. Conclusions

This systematic review responded to the need for a critical evaluation of existing research on the intentions of using sports Apps as this is an emerging field of research. The limited number of academic studies together with the deficiencies in some methodologies as can be seen in the risk analysis of research bias and the evidence found, has not allowed a more critical evaluation. These findings highlight the need for more rigorous and systematic research by researchers in the field, putting factors in common that allow a better evaluation of the context of the use of new technologies in the sports environment. At the same time, these findings have allowed the research team to identify a range of recommendations for sports organizations and researchers, which will help them to address future studies, and thus allow for a better growth and development of the evaluation of the intention to use Apps in sport.

Practical Implications

Sports organizations, sports marketing experts and new technology developers should make use of the considerations made in this study when developing or upgrading a sports App either for general sports information, a sports brand or exercise or health monitoring. Sports Apps have a great potential for the promotion and sponsorship of di↵erent products due to the potential use they have by consumers through advertising in these technologies. Suppliers of sports brands (retail, teams, etc.) should carry out an in-depth analysis of the fans by means of one of the tools evaluated that allow them to design the Apps according to the user’s expectations and to know what type of information or applicability they expect from it.

Another practical implication that is obtained is the possibility for sports organizations to develop Apps that have a gamification part that links the fan to consume the sports brand while he can enjoy certain benefits for using the organization’s App. They should also look for ways to engage the consumer with the App during the competition, i.e., generate exclusive content that can only be enjoyed by fans who have attended the live game, causing greater interest in the use of the App. Finally, to improve the quality of the Apps on Smartphones or other devices, industry and marketing professionals should examine all communication channels to ensure convenient access to the di↵erent content and services available.

Sustainability 2020, 12, 6641 17 of 24

In addition, the period of confinement caused by the COVID-19 pandemic, which has forced millions of people to stay at home, highlights the importance of this study to know the current situation on the topic of the intention to use fitness Apps by the population, especially for sports specialists and managers to know how to fit into the periods of new normality in which social distancing is forced. Although they have been in development for a long time, during the period of confinement there has been an increase in the o↵er of digital channels that help the guided practice of physical activity through the use of safe, simple and easy to implement programs and applications that cover activities related to cross-fit, yoga or dance activities for the general population or for the improvement of physical and mental well-being in older adults [18,81,82]. For experts in sports management or trainers, platforms such as Youtube or Zoom allow for individualized methods of physical exercise and real-time contact with the monitor, which allows for feedback to users regardless of where they are located [83]. Finally, Ammar et al. [84] suggest that future physical activity intervention to stay active during times of pandemic may be based on information and communication technologies, such as fitness Apps. Therefore, it is necessary to focus on a more in-depth study on the intention of the population to use these types of applications to promote active and healthy lifestyles.

Author Contributions: Conceptualization, S.A., J.G.-F., and M.G.-P.; methodology, S.A. and J.G.-F.; formal analysis, S.A.; investigation, S.A., J.G.-F. and I.V.; resources, I.V.; data curation, M.G.-P.; writing—original draft preparation, S.A., J.G.-F. and I.V.; writing—review and editing, S.A. and J.G.-F.; project administration, J.G.-F. and M.G.-P.; funding acquisition, J.G.-F. All authors have read and agreed to the published version of the manuscript.

Funding: This research was funded by University of Seville grant number 3840/0443/Study and design of technological consumer behavior in Spanish fitness centers, by Valgo Investment, S.L.U.

Conflicts of Interest: The authors declare no conflict of interest.

Sustainability 2020,12,6641

18 of24

A ppendix

A

Table A

1.A ssessm

entm ethodologicalquality

(part1).

ItemN Section

/Topic and

C hecklistItem

B eldad

& H

egner[43] B

yun et

al.[45] C

hen &

Lin [44]

C hiu

& C

ho [62]

C hiu

et al.[56]

C ho

et al.[59]

C ho

et al.[51]

C ho

& K

im [52]

D him

an etal.[58]

H a

et al.[42]

Title and

A bstract

1a Identification

ofthe type

ofstudy in

the title

1 1

1 1

1 1

1 1

1 1

1b Structured

sum m

ary ofobjective,m

ethods,results and

conclusions 0

1 1

1 1

1 1

1 1

0

Introduction B

ackground and

O bjectives

2a Scientific

background and

explanation ofrationale

1 1

1 1

1 1

1 1

1 1

2b Specific

objectives or

hypotheses 1

1 1

1 1

1 1

1 1

1

M ethods

Participants 3a

Eligibility criteria

for participants

1 1

0 0

0 0

1 0

0 1

3b Settings

and locations

w here

the data

w ere

collected 0

0 0

1 1

1 1

0 1

1 3c

A table

show ing

baseline dem

ographic characteristics

0 1

1 1

1 0

1 0

1 0

Sam ple

Size 4a

The sam

ple size

has been

determ ined

0 0

0 0

0 0

0 0

0 0

4b W

hen applicable,explanation

ofhow sam

ple size

w as

determ ined

0 0

0 0

0 0

0 0

0 0

Procedure

5 The

procedure has

su �

cientdetails to

allow replication,

including how

and w

hen they

w ere

actually adm

inistered 0

1 0

0 1

1 1

1 1

0

Instrum entorTools

6a C

om pletely

defined prespecified

prim ary

and secondary

outcom e

m easures,including

how and

w hen

they w

ere assessed

1 1

1 1

1 1

1 1

1 1

6b U

se ofvalidity

and reliability

tools. 1

1 1

1 1

1 1

1 1

1

Im plem

entation 7

W ho

m ade

each partofstudy

0 0

0 0

0 0

0 0

0 0

StatisticalM ethods

8a Statisticalm

ethods used

to analyze

the results

1 1

1 1

1 1

1 1

1 1

8b U

se ofM

ethods foradditionalanalyses

to objective

ofstudy 0

0 1

1 0

1 1

0 0

0

R esults

O utcom

es and

Estim ation

9 A

table orfigure

show ing

outputs ofanalysis

m ore

relevant ofstudy

1 1

1 1

1 1

1 1

1 1

Sustainability 2020,12,6641

19 of24

Table A

1.C ont.

ItemN Section

/Topic and

C hecklistItem

B eldad

& H

egner[43] B

yun et

al.[45] C

hen &

Lin [44]

C hiu

& C

ho [62]

C hiu

et al.[56]

C ho

et al.[59]

C ho

et al.[51]

C ho

& K

im [52]

D him

an etal.[58]

H a

et al.[42]

D iscussion

Interpretation

10 Interpretation

consistentw ith

results,balancing benefits

and harm

s and

considering other

relevantevidence 1

1 1

1 1

0 1

0 1

1

Lim itations

11 Study

lim itations,addressing

sources ofpotentialbias,

im precisions,etc.

1 1

1 1

1 1

1 0

1 1

PracticalIm plication

12 M

ain applicability

to results

ofstudy 1

1 1

1 1

1 1

0 1

1

O therInform

ation Funding

13 Sources

offunding and

other support,role

offunders 1

1 0

0 0

1 1

0 0

0

T O

TA L

12 15

13 14

14 14

17 9

14 12

Table A

2.A ssessm

entm ethodologicalquality

(part2).

ItemN Section

/Topic and

C hecklistItem

H uang

& R

en [60]

K im

et al.[1]

Lee et

al.[61] Lietal.

[63] M

oham m

adi& Isanejad

[57] N

dayizigam iye

etal.[54] W

eiet al.[55]

Yoo et

al.[53] Yuan

et al.[30]

Title and

A bstract

1a Identification

ofthe type

ofstudy in

the title

1 1

1 1

1 1

1 1

1

1b Structured

sum m

ary ofobjective,m

ethods,results and

conclusions 1

1 1

1 1

1 1

1 1

Introduction B

ackground and

O bjectives

2a Scientific

background and

explanation ofrationale

1 1

1 1

1 0

1 1

1 2b

Specific objectives

or hypotheses

1 1

1 1

0 1

1 1

1

M ethods

Participants 3a

Eligibility criteria

for participants

0 1

0 0

0 0

0 0

0 3b

Settings and

locations w

here the

data w

ere collected

1 1

0 1

0 1

0 0

1 3c

A table

show ing

baseline dem

ographic characteristics

1 1

1 1

1 0

1 1

0

Sam ple

Size 4a

The sam

ple size

has been

determ ined

0 0

0 0

0 0

0 0

0

4b W

hen applicable,explanation

ofhow sam

ple size

w as

determ ined

0 0

0 0

0 0

0 0

0

Sustainability 2020,12,6641

20 of24

Table A

2.C ont.

ItemN Section

/Topic and

C hecklistItem

H uang

& R

en [60]

K im

et al.[1]

Lee et

al.[61] Lietal.

[63] M

oham m

adi& Isanejad

[57] N

dayizigam iye

etal.[54] W

eiet al.[55]

Yoo et

al.[53] Yuan

et al.[30]

Procedure

5 The

procedure has

su �

cientdetails to

allow replication,

including how

and w

hen they

w ere

actually adm

inistered 0

1 0

0 0

0 0

0 0

Instrum entorTools

6a C

om pletely

defined prespecified

prim ary

and secondary

outcom e

m easures,including

how and

w hen

they w

ere assessed

1 1

1 1

1 1

1 1

0

6b U

se ofvalidity

and reliability

tools. 1

1 1

1 1

1 1

1 1

Im plem

entation 7

W ho

m ade

each partofstudy

0 0

0 0

0 0

0 0

0 StatisticalM

ethods 8a

Statisticalm ethods

used to

analyze the

results 1

1 1

1 1

1 1

1 1

8b U

se ofM

ethodsforadditionalanalysesto objective

ofstudy 0

0 1

0 0

0 0

0 1

R esults

O utcom

es and

Estim ation

9 A

table orfigure

show ing

outputs ofanalysis

m ore

relevant ofstudy

1 1

1 1

1 1

1 1

1

D iscussion

Interpretation

10 Interpretation

consistentw ith

results,balancing benefits

and harm

s and

considering other

relevantevidence 1

1 1

1 1

1 1

0 0

Lim itations

11 Study

lim itations,addressing

sources ofpotentialbias,

im precisions,etc.

1 1

0 1

0 0

1 0

1

PracticalIm plication

12 M

ain applicability

to results

ofstudy 1

1 1

1 1

0 1

0 0

O therInform

ation Funding

13 Sources

offunding and

other support,role

offunders 0

0 0

1 0

1 1

0 0

T O

TA L

13 15

12 14

10 10

13 9

10

Sustainability 2020, 12, 6641 21 of 24

References

1. Kim, Y.; Kim, S.; Rogol, E. The e↵ects of consumer innovativeness on sport team applications acceptance and usage. J. Sport Manag. 2017, 31, 241–255. [CrossRef]

2. Statista. Forecast Number of Mobile Users Worldwide from 2019 to 2023. Available online: https: //www.statista.com/statistics/218984/number-of-global-mobile-users-since-2010/ (accessed on 18 March 2020).

3. Cha↵ey, D. Mobile Marketing Statistics Compilation. Available online: https://www.smartinsights.com/mobi le-marketing/mobile-marketing-analytics/mobile-marketing-statistics/ (accessed on 18 March 2020).

4. Fuchs, C. Information technology and sustainability in the information society. Int. J. Commun. 2017, 11, 2431–2461. Available online: https://ijoc.org/index.php/ijoc/article/view/6827 (accessed on 17 April 2020).

5. Luque-Ayala, A.; Marvin, S. Developing a critical understanding of smart urbanism? Urb. Stud. 2015, 52, 2105–2116. [CrossRef]

6. Vanolo, A. Smartmentality: The smart city as disciplinary strategy. Urb. Stud. 2014, 51, 883–898. [CrossRef] 7. Angelidou, M.; Psaltoglou, A.; Komninos, N.; Kakderi, C.; Tsarchopoulos, P.; Panori, A. Enhancing sustainable

urban development through smart city applications. J. Sci. Technol. Polic. Manag. 2018, 9, 146–169. [CrossRef] 8. Maaroof, A. Big Data and the 2030 Agenda for Sustainable Development. 2015. Available online:

https://www.unescap.org/events/call-participants-big-data-and-2030-agenda-sustainable-development-ac hieving-development (accessed on 2 August 2020).

9. United Nation. Transforming Our World: The 2030 Agenda for Sustainable Development. 2015. Available online: https://sustainabledevelopment.un.org/post2015/transformingourworld (accessed on 3 August 2020).

10. United Nations. Habitat III Issue Papers, 21—Smart Cities (V2.0). 2015. Available online: http://habitat3.org /wp-content/uploads/Habitat-III-Issue-Paper-21_Smart-Cities-2.0.pdf (accessed on 1 August 2020).

11. International Telecommunications Union. An Overview of Smart Sustainable Cities and the Role of Information and Communication Technologies. 2014. Available online: www.itu.int/en/ITU-T/focusgroups/s sc/Pages/default.aspx (accessed on 3 August 2020).

12. Vodafone Group. Unifying Communications. Annual Report. 2015. Available online: http://www.annualre ports.com/HostedData/AnnualReportArchive/v/LSE_VOD_2015.pdf (accessed on 22 March 2020).

13. Blair, I. Mobile App Download and Usage Statistics. 2019. Available online: https://buildfire.com/app-statis tics/ (accessed on 19 March 2020).

14. Statista. Number of Mobile App Downloads Worldwide in 2017, 2018 and 2022 (in Billions). Available online: https://www.statista.com/statistics/271644/worldwide-free-and-paid-mobile-app-store-downloads/ (accessed on 20 March 2020).

15. Statista. How Often do you Currently Make Use of Sports and Fitness Apps? Available online: https://www.stat ista.com/statistics/639567/sports-and-fitness-app-usage-frequency-in-us/ (accessed on 21 March 2020).

16. McKay, F.H.; Wright, A.; Shill, J.; Stephens, H.; Uccellini, M. Using Health and Well-Being Apps for Behavior Change: A Systematic Search and Rating of Apps. JMIR 2019, 7, e11926. [CrossRef]

17. World Health Organization. #HealthyAtHome—Physical Activity. Available online: https://www.who.int/ news-room/campaigns/connecting-the-world-to-combat-coronavirus/healthyathome/healthyathome---p hysical-activity (accessed on 15 May 2020).

18. Banskota, S.; Healy, M.; Goldberg, E.M. Smartphone Apps for Older Adults to Use While in Isolation During the COVID-19 Pandemic. West. J. Emerg. Med. 2020, 21, 514–525. [CrossRef]

19. IHRSA. Fitness Apps are Revolutionizing the Industry. Available online: https://www.ihrsa.org/improve-yo ur-club/fitness-apps-are-revolutionizing-the-industry/ (accessed on 27 April 2020).

20. McLean, G.; Osei-Frimpong, K.; Al-Nabhani, K.; Marriott, H. Examining consumer attitudes towards retailers’m-commerce mobile applications–An initial adoption vs. continuous use perspective. J. Bus. Res. 2020, 106, 139–157. [CrossRef]

21. Stocchi, L.; Michaelidou, N.; Micevski, M. Drivers and outcomes of branded mobile app usage intention. J. Prod. Brand Manag. 2019, 28, 28–49. [CrossRef]

22. Gao, T.; Rohm, A.J.; Sultan, F.; Huang, S. Antecedents of consumer attitudes toward mobile marketing: A comparative study of youth markets in the United States and China. Thund. Int. Bus. Rev. 2012, 54, 211–224. [CrossRef]

Sustainability 2020, 12, 6641 22 of 24

23. Maghnati, F.; Ling, K.C. Exploring the relationship between experiential value and usage attitude towards mobile apps among the smartphone users. Int. J. Bus. Manag. 2013, 8, 1–9. [CrossRef]

24. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 318–339. [CrossRef]

25. Fishbein, M.; Ajzen, I. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research; Addison-Wesley: Reading, UK, 1975.

26. Ha, J.P.; Kang, S.J.; Ha, J. A conceptual framework for the adoption of smartphones in a sports context. Int. J. Sports Mark. Spons. 2015, 16, 2–19. [CrossRef]

27. Rivera, M.; Gregory, A.; Cobos, L. Mobile application for the timeshare industry: The influence of technology experience, usefulness, and attitude on behavioral intentions. J. Hosp. Tour. Technol. 2015, 6, 242–257. [CrossRef]

28. Lee, Y.; Hsieh, Y.; Hsu, C. Adding innovation di↵usion theory to the technology acceptance model: Supporting employees’ intentions to use e-learning systems. J. Educ. Technol. Soc. 2011, 14, 124–137. Available online: https://www.jstor.org/stable/jeductechsoci.14.4.124 (accessed on 20 April 2020).

29. Venkatesh, V.; Morris, M.G.; Davis, G.B.; Davis, F.D. User acceptance of information technology: Toward a unified view. MIS Q. 2003, 27, 425–478. [CrossRef]

30. Yuan, S.; Ma, W.; Kanthawala, S.; Peng, W. Keep Using My Health Apps: Discover Users’ Perception of Health and Fitness Apps with the UTAUT2 Model. Telemed. E-Health 2015, 21, 735–741. [CrossRef]

31. Hur, Y.; Ko, Y.J.; Valacich, J. Motivation and concerns of online sport consumption. J. Sport Manag. 2007, 21, 521–539. [CrossRef]

32. Singh, N.; Sinha, N. How perceived trust mediates merchant’s intention to use a mobile wallet technology. J. Retail. Consum. Serv. 2020, 52, 101894. [CrossRef]

33. Chen, L.; Aklikokou, A.K. Determinants of E-government Adoption: Testing the Mediating E↵ects of Perceived Usefulness and Perceived Ease of Use. Int. J. Public Adm. 2020, 43, 850–865. [CrossRef]

34. Hossain, S.A.; Bao, Y.; Hasan, N.; Islam, M.F. Perception and prediction of intention to use online banking systems. Int. J. Res. Bus. Social Sci. 2020, 9, 112–116. [CrossRef]

35. Kaushik, A.K.; Mohan, G.; Kumar, V. Examining the Antecedents and Consequences of Customers’ Trust Toward Mobile Retail Apps in India. J. Internet Commer. 2020, 19, 1–31. [CrossRef]

36. Almaiah, M.A.; Al-Mulhem, A. Analysis of the essential factors a↵ecting of intention to use of mobile learning applications: A comparison between universities adopters and non-adopters. Educ. Inf. Technol. 2019, 24, 1433–1468. [CrossRef]

37. Carlson, J.; O’Cass, A. Optimizing the online cannel in professional sport to create trusting and loyal consumers: The role of the professional sports team brand and service quality. J. Sport Manag. 2012, 26, 463–478. [CrossRef]

38. Hur, Y.; Ko, Y.J.; Claussen, C.L. Acceptance of sport websites: A conceptual model. Int. J. Sport Mark. Spons. 2011, 12, 209–224. [CrossRef]

39. Hur, Y.; Ko, Y.J.; Claussen, C.L. Determinants of using sports web portals: An empirical examination of the sport website acceptance model. Int. J. Sport Mark. Spons. 2012, 13, 6–25. [CrossRef]

40. Cilletti, D.; Lanasa, J.; Ramos, D.; Luchs, R.; Lou, J. Sustainability Communication in North American Professional Sport Leagues: Insights from web-site self-presentations. Int. J. Sport Commun. 2010, 3, 64–69. [CrossRef]

41. Kim, T.; Chiu, W. Consumer acceptance of sports wearable technology: The role of technology readiness. Int. J. Sport Mark. Spons. 2019, 20, 109–126. [CrossRef]

42. Ha, J.P.; Kang, S.J.; Kim, Y. Sport fans in a “smart sport” (SS) age: Drivers of smartphone use for sport consumption. Int. J. Sport Mark. Spons. 2017, 18, 281–297. [CrossRef]

43. Beldad, A.D.; Hegner, S.M. Expanding the Technology Acceptance Model with the Inclusion of Trust, Social Influence, and Health Valuation to Determine the Predictors of German Users’ Willingness to Continue using a Fitness App: A Structural Equation Modeling Approach. Int. J. Hum. Comput. Interact. 2018, 34, 882–893. [CrossRef]

44. Chen, M.F.; Lin, N.P. Incorporation of health consciousness into the technology readiness and acceptance model to predict app download and usage intentions. Internet Res. 2018, 28, 351–373. [CrossRef]

45. Byun, H.; Chiu, W.; Bae, J.S. Exploring the adoption of sports brand apps: An application of the modified technology acceptance model. Int. J. Asian Bus. Inf. Manag. 2018, 9, 52–65. [CrossRef]

Sustainability 2020, 12, 6641 23 of 24

46. Song, J.; Kim, J.; Cho, K. Understanding users’ continuance intentions to use smart-connected sports products. Sport Manag. Rev. 2018, 21, 477–490. [CrossRef]

47. Rohm, A.J.; Gao, T.T.; Sultan, F.; Pagani, M. Brand in the hand: A cross-market investigation of consumer acceptance of mobile marketing. Bus. Horiz. 2012, 55, 485–493. [CrossRef]

48. Schulz, K.F.; Altman, D.G.; Moher, D. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMC Med. 2010, 340, c332. [CrossRef]

49. Moher, D.; Liberati, A.; Tetzla↵, J.; Altman, D.G. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [CrossRef]

50. Lane, D. Fisher r-to-z Calculator. Available online: http://onlinestatbook.com/calculators/ (accessed on 17 May 2020).

51. Cho, J.; Lee, H.E.; Quinlan, M. Cross-National Comparisons of College Students’ Attitudes toward Diet/Fitness Apps on Smartphones. J. Am. Coll. Health 2016, 65, 437–439. [CrossRef]

52. Cho, J.; Kim, S.J. Factors of Leading the Adoption of Diet/Exercise Apps on Smartphones: Application of Channel Expansion Theory. J. Korea Soc. Internet Inf. 2015, 16, 101–108. [CrossRef]

53. Yoo, D.H.; Ko, D.S.; Yeo, I.S. E↵ect of user’s trust in usefulness, attitude and intention for mobile sports content services. J. Phys. Educ. Sport 2017, 17, 92–96. [CrossRef]

54. Ndayizigamiye, P.; Kante, M.; Shingwenyana, S. An adoption model of mHealth applications that promote physical activity. Cogent Psychol. 2020, 7, 1764703. [CrossRef]

55. Wei, J.; Vinnikova, A.; Lu, L.; Xu, J. Understanding and Predicting the Adoption of Fitness Mobile Apps: Evidence from China. Health Commun. 2020, 1–12. [CrossRef] [PubMed]

56. Chiu, W.; Cho, H.; Chi, C.G. Consumers’ continuance intention to use fitness and health apps: An integration of the expectation–confirmation model and investment model. Inf. Technol. People 2020. [CrossRef]

57. Mohammadi, S.; Isanejad, O. Presentation of the Extended Technology Acceptance Model in Sports Organizations. Ann. Appl. Sport Sci. 2018, 6, 75–86. [CrossRef]

58. Dhiman, N.; Arora, N.; Dogra, N.; Gupta, A. Consumer adoption of smartphone fitness apps: An extended UTAUT2 perspective. J. Indian Bus. Res. 2019, 12, 363–388. [CrossRef]

59. Cho, J.; Lee, H.E.; Kim, S.J.; Park, D. E↵ects of body image on college students’ attitudes toward diet/fitness apps on smartphones. Cyberpsychol. Behav. Soc. Netw. 2015, 18, 41–45. [CrossRef]

60. Huang, G.; Ren, Y. Linking technological functions of fitness mobile apps with continuance usage among Chinese users: Moderating role of exercise self-e�cacy. Comput. Human Behav. 2020, 103, 151–160. [CrossRef]

61. Chiu, W.; Cho, H. The role of technology readiness in individuals’ intention to use health and fitness applications: A comparison between users and non-users. Asia Pacific J. Mark. Logist. 2020. [CrossRef]

62. Lee, S.; Kim, S.; Wang, S. Motivation factors influencing intention of mobile sports apps use by applying the unified theory of acceptance and use of technology (UTAUT). Int. J. Appl. Sports Sci. 2017, 29, 115–127. [CrossRef]

63. Li, J.; Liu, X.; Ma, L.; Zhang, W. Users’ intention to continue using social fitness-tracking apps: Expectation confirmation theory and social comparison theory perspective. Inform. Health Soc. Care 2019, 44, 298–312. [CrossRef]

64. Walter, T. Worldwide survey of fitness trends for 2017. ACSM’s Health Fit. J. 2016, 20, 8–17. [CrossRef] 65. Hur, W.M.; Kim, H.; Kim, W.M. The moderating roles of gender and age in tablet computer adoption.

Cyberpsychol. Behav. Soc. Netw. 2014, 17, 33–39. [CrossRef] [PubMed] 66. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. User acceptance of computer technology: A comparison of two

theoretical models. Manag. Sci. 1989, 35, 982–1003. [CrossRef] 67. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. Extrinsic and Intrinsic Motivation to Use Computers in the Workplace.

J. Appl. Soc. Psychol. 1992, 22, 1111–1132. [CrossRef] 68. Hong, S.; Thong, J.Y.L.; Tam, K.Y. Understanding continued information technology usage behavior:

A comparison of three models in the context of mobile internet. Decis. Support Syst. 2006, 42, 1819–1834. [CrossRef]

69. Moon, J.W.; Kim, Y.G. Extending the TAM for a World-Wide-Web context. Inf. Manag. 2001, 38, 217–230. [CrossRef]

70. Venkatesh, V.; Davis, F.D. A theoretical extension of the technological acceptance model: Four longitudinal field studies. Manag. Sci. 2000, 46, 186–204. [CrossRef]

Sustainability 2020, 12, 6641 24 of 24

71. Sun, H.; Zhang, P. Causal relationships between perceived enjoyment and perceived ease of use: An alternative approach. J. Assoc. Inf. Syst. 2006, 7, 618–645. [CrossRef]

72. Klenk, S.; Reifegerste, D.; Renatus, R. Gender di↵erences in gratifications from fitness app use and implications for health interventions. Mob. Media Commun. 2017, 5, 178–193. [CrossRef]

73. Baptista, G.; Baptista, G.; Oliveira, T.; Oliveira, T. Why so serious? Gamification impact in the acceptance of mobile banking services. Internet Res. 2017, 27, 118–139. [CrossRef]

74. Gowin, M.; Cheney, M.; Gwin, S.; Wann, T.F. Health and fitness app use in college students: A qualitative study. Am. J. Health Educ. 2015, 46, 223–230. [CrossRef]

75. Wang, Q.; Egelandsdal, B.; Amdam, G.V.; Almli, V.L.; Oostindjer, M. Diet and physical activity apps: Perceived e↵ectiveness by app users. JMIR mHealth uHealth 2016, 4, e33–e47. [CrossRef] [PubMed]

76. Tang, J.; Abraham, C.; Stamp, E.; Greaves, C. How can weight-loss app designers’ best engage and support users? A qualitative investigation. Br. J. Health Psychol. 2015, 20, 151–171. [CrossRef] [PubMed]

77. Yoon, C.; Jeong, C.; Rolland, E. Understanding individual adoption of mobile instant messaging: A multiple perspectives approach. Inf. Technol. Manag. 2015, 16, 139–151. [CrossRef]

78. Cheung, D.S.T.; Or, C.K.L.; So, M.K.P.; Tiwari, A. Usability Testing of a Smartphone Application for Delivering Qigong Training. J. Med. Syst. 2018, 42, 191–199. [CrossRef]

79. Tseng, K.C.; Hsu, C.; Chung, Y. Acceptance of information technology and the internet by people aged over fifty in Taiwan. Soc. Behav. Per. Int. J. 2012, 40, 613–622. [CrossRef]

80. Kang, S.J.; Ha, J.P.; Hambrick, M.E. A mixed-method approach to exploring the motives of sport-related mobile applications among college students. J. Sport Manag. 2015, 29, 272–290. [CrossRef]

81. Chen, P.; Mao, L.; Nassis, G.P.; Harmer, P.; Ainsworth, B.E.; Li, F. Coronavirus disease (COVID-19): The need to maintain regular physical activity while taking precautions. J. Sport Health Sci. 2020, 9, 103–104. [CrossRef]

82. Nyenhuis, S.M.; Greiwe, J.; Zeiger, J.S.; Nanda, A.; Cooke, A. Exercise and Fitness in the age of social distancing during the COVID-19 Pandemic. J. Allerg. Clin. Immun. 2020, 8, 2152–2155. [CrossRef]

83. Ng, K. Adapted physical activity through COVID-19. Eur. J. Adap. Phys. Act. 2020, 13, 1. [CrossRef] 84. Ammar, A.; Brach, M.; Trabelsi, K.; Chtourou, H.; Boukhris, O.; Masmoudi, L.; Bouaziz, B.; Bentlage, E.;

How, D.; Ahmed, M.; et al. On Behalf of the ECLB-COVID19 Consortium. E↵ects of COVID-19 Home Confinement on Eating Behaviour and Physical Activity: Results of the ECLB-COVID19 International Online Survey. Nutrients 2020, 12, 1583. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Journal of Athletic Training 2020;55(9):902–910 doi: 10.4085/1062-6050-0540.19 ! by the National Athletic Trainers’ Association, Inc www.natajournals.org

Current Concepts

‘‘To Tech or Not to Tech?’’ A Critical Decision-Making Framework for Implementing Technology in Sport Johann Windt, PhD, CSCS*§; Kerry MacDonald, PhD†; David Taylor, MSc‡; Bruno D. Zumbo, PhD§; Ben C. Sporer, PhD*§; David T. Martin, PhD||

*Vancouver Whitecaps FC, BC, Canada; †Volleyball Canada, Ottawa, ON; ‡United States Olympic and Paralympic Committee, Colorado Springs, CO; §University of British Columbia, Vancouver, Canada; ||Australian Catholic University, Belbourne

The current technological age has created exponential growth in the availability of technology and data in every industry, including sport. It is tempting to get caught up in the excitement of purchasing and implementing technology, but technology has a potential dark side that warrants consideration. Before investing in technology, it is imperative to consider the potential roadblocks, including its limitations and the contextual challeng- es that compromise implementation in a specific environment. A thoughtful approach is therefore necessary when deciding

whether to implement any given technology into practice. In

this article, we review the vision and pitfalls behind technology’s

potential in sport science and medicine applications and then

present a critical decision-making framework of 4 simple

questions to help practitioners decide whether to purchase

and implement a given technology.

Key Words: analytics, measurement, wearable devices, global positioning systems

T echnology is here to stay—not just in sport but in virtually every discipline. This special issue focuses on training load, recovery monitoring, and manage-

ment, and in nearly every article, readers will find examples of how technology can be used in these areas. External loads can be monitored through global positioning systems (GPS), inertial measurement units (IMUs), optical tracking systems, and so on.1 Internal loads may be captured with heart-rate monitors, lactate measurements, and more.2

Recovery states may be measured with devices ranging from low-tech wellness surveys3–5 to more high-tech solutions, such as heart-rate variability6,7 or force-plate testing.8,9 Currently, we are seeing new technological solutions with potential sporting applications, such as implantable devices,10 markerless motion capture,11,12

breath analysis,13 smart garments, biomechanical insoles, and skin sensors.14 In this technological age, sports science practitioners must critically appraise the plethora of options available and make informed decisions about evaluating and adopting technology in their specific contexts. These context-specific questions demand a critical evaluation of the case for the intended use and the available evidence that supports (or does not support) technological implementa- tion. Our aim in this article is to provide a simple, foundational framework to aid practitioners in that critical decision-making process.

This article is divided into 3 parts: (1) a vision for what technology can provide and why we should be excited about its potential, (2) a warning about the potential dark side of technology and the pitfalls that can derail its successful implementation, and (3) a critical decision-

making framework consisting of 4 key questions to ask before purchasing a new technology.

PART 1: THE BRIGHT SIDE—A VISION FOR TECHNOLOGY

Excitement is the most appropriate response when taking an optimist’s view of technology in sport. In this section, we outline just a handful of benefits practitioners can expect from successful technological implementation.

Benefit 1: Improving and Off-Loading Data Collection—An Example From Pro Football

Technology can improve measurement precision and automate the process so that practitioners do not have to manually record data as they did in the past. For example, understanding the match demands to which athletes are exposed is foundational in the sport sciences.15–17 Football is no exception, and time-motion analyses to understand football players’ physical outputs (eg, total distance run in a match, time spent at different speeds) and physiological responses have been performed for decades.18–20 Before the technological advances that are commonplace today, these time-motion analyses were performed using tape-recorded commentaries, video recordings, and film analyses. All of these notational analysis processes were extremely time consuming, often limiting researchers’ ability to examine more than a small number of players in a defined number of matches.21 Technological advances, both through wearable devices22 and optical tracking systems,21 now provide these physical output measures to researchers and practitioners in near real time.23,24 Although these systems are not without

902 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

error and vary among technological providers and sys- tems,25 many provide more accurate physical output data than estimates derived from manual notational analyses based on video and can supply these data on all players simultaneously and in near real time. In this way, technology and its efficiency have off-loaded weeks and months of practitioner and researcher time. If used wisely, this regained time allows a deeper dive into the information and may inform practice more thoroughly. For example, advances in optical tracking and wearable technology have now allowed a better understanding of how physical exertion in football players relates to such contextual variables as player position, stage of play, and teams or players being in or out of possession.26–28

Benefit 2: Sport-Specific Load Measures—A Volleyball Journey

The prevalence of chronic injuries in volleyball players is known to be high.29,30 Of these injuries, jumper’s knee (ie, patellar tendinopathy) is the most common. The injury mechanism is fundamentally an overload of the extensor mechanisms of the knee joint.31 With the introduction of IMUs that measure athlete jump counts, the sport-specific load requirements of the knee extensors could be quantified for the first time without manual video annotation of jump counts for all players.32

One author (K.M.) used his dual roles as a researcher and volleyball coach to scientifically evaluate such a technology and implement simple heuristics to inform decision making in coaching. As a researcher, he performed validation work on a wearable IMU for measuring jump count32 to ensure that he could rely on the jump-count data being provided by the IMU software. Specifically, the jump counts from the accelerometers related very closely to the jump counts extracted from manual video notation, so he was comfort- able that the errors were few enough to reasonably inform practice. As a coach, measuring jump loads for all players in training and matches facilitated a better understanding of the position-specific training and match demands and individualized athlete load management throughout the season. After a retrospective assessment of training and match demands, he prospectively planned and prescribed individual jump loads. This prospective prescription may have improved the capacity of the players’ tendons to withstand the sports demands and prevent the development or flare-ups of such chronic injuries as jumper’s knee. In his case, this informed decision-making process helped to mitigate the prevalence of overuse injuries (zero practice sessions or games missed due to overuse injuries) in the team’s volleyball players and culminated in a national championship.

Although not all stories end in a championship, technological implementation allows for sport-specific and movement-specific load quantification that can inform practitioners’ workload, recovery, and return-to-sport decisions. A multitude of factors affect the onset of any injury, yet an informed approach to load is certainly a beneficial addition to any injury-mitigation strategy.

Benefit 3: A 3608 View of the Athlete

Performance is multifactorial and requires adequate physical, mental, technical, and tactical expertise to

compete at the elite level. The contribution of each element depends on the sport’s demands and the characteristics of the individual athlete. Furthermore, sport is dynamic and ever changing as athletes pursue multiple phases over the course of a single season, including training, competition, and recovery. Sleep, recovery, nutrition, social factors, and lifestyle can all affect athletes’ responses to and outcomes in training and performance.33,34

Technology allows for the rapid collection and analysis of data from many of these areas. The ability to integrate data streams enables practitioners to better understand how one factor affects another by providing a holistic perspec- tive of the athlete. It also permits information to be shared across disciplines, blending injuries with training load, medicine with physiology, and physical with technical and tactical performance. Where limitations once existed in storing and processing vast amounts of information safely and in a time-efficient manner, technological advancements have reduced many of the challenges involving costs, computing speed, and intelligence tools. Although pitfalls still exist (see the next section), technology allows practitioners and teams to provide holistic perspectives on athlete performance when using a strategic approach.

PART 2: THE POTENTIAL DARK SIDE—A WARNING TO THE WISE

The data life cycle may be summarized broadly as plan ! collect ! analyze ! communicate. Each subsequent step relates closely to the intended use of the information, as determined by a thoughtful plan underpinning the technology’s implementation. A problem at any stage of this life cycle can be fatal for the successful use of any technology. Failed technological implementations can have lasting ramifications, so considering the following potential pitfalls is important.

Pitfall 1: Not All Promises of Technology Are Kept

Underlying all stages of the data life cycle is a belief that the data are trustworthy enough to collect and interpret. However, some of the bold promises made by technology companies may not actually be true. In these instances, failed promises may result in poor data quality (eg, measurement error is too large) that challenges a practi- tioner’s ability to interpret any signal amid the noise. The failures could also stem from black-box algorithms that summarize the data and produce unactionable, uninterpret- able outputs.

Scientists have explored technological devices in an attempt to better understand the validity and reliability of the data from these emergent technologies. As such, different technologies are known to have inherent limita- tions: for example, the ability of GPS technology to accurately measure high-speed running velocities,22,35,36 the sensitivity of heart-rate variability measurement,37 and the subsequent requirement of rigid, standardized testing procedures or the effectiveness of wrist-based sleep monitoring compared with the criterion standard of polysomnography testing.38 Although these examples include some technological devices that have published validity-related evidence, it is important to note 2 items. First, none of these devices and the data they provide are perfect; all come with inherent measurement error. Second,

Journal of Athletic Training 903

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

most consumer devices have little scientific evidence for their accuracy, validity, and reliability,14 so a prudent practitioner should approach any new technological device with a healthy dose of skepticism.

Pitfall 2: Technology Transforms Into a Dust Collector If It Cannot Be Implemented

The best technologies are useless if they are not implemented in a way that informs decision making or changes practice. Given that technological implementation may require sizable investments of financial and human resources, understanding the burdens on time and staff resources that implementation will require is crucial. If the burden on staff is too great, practitioners may be stretched beyond their skill sets, be forced into uncomfortable situations, or have unrealistic time constraints placed on them, negatively affecting the feasibility and quality of data collection. If the staff are not educated about the potential benefits, lack the desire to collect the data appropriately, and do not believe the technology will provide useful information, the investment is flawed before it begins.

Pitfall 3: Technology Does Not Necessarily Provide the Right Data or the Raw Data

A high-tech solution and precise data collection do not inherently mean that the right data are being collected. What constitutes the right data in a particular setting depends on several contextual and organizational factors, such as the questions being asked by different practitioners within the organization and how the organization’s decision-making structure allows data to inform decisions.

Even if the right data are being collected, it is vital to understand what type of data the technology will provide. Many technologies come with a software package that delivers a dashboard or printable report of the data collected. It is important to consider whether these standard reports or dashboards analyze the data in a way that reflects the user’s needs and corresponds to the original plan. The analyze portion of the data life cycle implies that the technology analyzes the data in the way you need or that you can access and analyze the data yourself in accordance with those needs if the technology does not provide the answers you are seeking. In these instances, it is important to understand whether a technology provides access to the raw data so that you can perform the appropriate analyses in accordance with your original plan. If the technology does not provide such access and only reports summary findings based on proprietary algorithms, the ability of researchers and practitioners to analyze data in the ways they need may be compromised.

Pitfall 4: Technology Does Not Inherently Communicate a Message

Even when technology is introduced and data are collected consistently in an applied environment, the data have just made it through part of the data life cycle—they must still be analyzed and disseminated in accordance with the plan. Technology itself does not inherently communi- cate to decision makers. Although some technological devices are accompanied by software tools that provide reports or dashboards that summarize the underlying data,

the message delivered to decision makers must be readily interpretable by the end user, which can include high- performance team members, coaching staff, or manage- ment, and answer the specific questions that were planned. What appropriate communication and dissemination look like, therefore, depend on the intended use of the technology and the context in which it is implemented. Understanding end users’ requirements, interests, and necessary decisions is crucial so that information can be tailored into a clear, concise message. Crucially, these steps must be taken in each environment and are not accom- plished simply by having purchased a given technology.

PART 3: A CRITICAL DECISION-MAKING FRAMEWORK

Asking the right questions before jumping into a new technological investment can help guide practitioners and researchers to the vision of the technology while avoiding some of the common pitfalls. Unsurprisingly, several frameworks for integrating technology into sport have been proposed in the literature,39,40 and we strongly encourage readers to explore and critically think through other frameworks in addition to those presented here. In our critical decision-making framework, we pose 4 questions, all of which should be answered affirmatively before arriving at a decision to purchase a given technology (Figure 1). Each of these 4 questions, important follow-up questions, sources of evidence for finding appropriate answers, and key take-home messages are discussed in the following sections, detailed in Figure 1, and summarized in the Table.

Question 1: Would the Promised Information Be Helpful?

New technologies arrive on the market daily, many of which may pique the interest of curious and intelligent individuals. These technologies often come with bold claims, savvy marketing, and grand promises. In the sport sciences, these claims may include accurate injury prediction or ‘‘1-stop shops’’ for understanding an athlete’s fatigue and recovery status. We propose that the first question practitioners should ask when encountering a technology and engaging with such claims is would the promised information be helpful? The ability to extract new information can be exciting, yet this does not mean the information will help to inform decision making for practitioners in their specific contexts.

To answer this question, individuals should consider what specific question will be answered or which decision will be informed. This is an extremely important point, as technology must inform practice to be useful. When a new technological opportunity is considered, at least 1 key decision should be informed by the available information or 1 question should be answered. This is the premise of the first (and most important) stage of the data life cycle: planning. To plan effectively, practitioners must understand their specific contexts so they address relevant and pertinent questions that end users need to answer. Ideally, a need may already have been identified within the practice for more information to be collected, in which case investing in a technology that provides that specific data makes more sense. Considering 2 contexts from earlier in this article,

904 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

wearable technology may provide insight for team-sport practitioners, but this looks different in soccer, where GPS data may be most valuable to understand athlete distances and speeds, than volleyball, where an accelerometer that provides information about jump counts and distances may be deemed less important (and not measurable with GPS indoors). Note that in each of these instances, a need is a question that must be answered or a decision that must be informed, rather than a specific technology that the organization believes is in itself necessary.

Take-Home Suggestions. ! Start with the end in mind. Understand the decision you

want to inform and which data from the technology you

will need to extract and communicate in order to inform that decision.

! Explore existing paths. Is there an existing data stream you can use instead or a better alternative?

Question 2: Can You Trust the Information You Will Be Getting?

When researchers speak about technology, the discussion typically includes aspects of measurement error, reliability, responsiveness, and validity. Beyond all these technical terms, practitioners need to know whether they can take

Table. Unpacking Each Question Within the Critical Framework Through Follow-Up Questions, Sources of Evidence, and Take-Home Messages

Would the promised information be helpful?

Can you trust the information you

will be getting?

Can you integrate, manage, and analyze the data effectively?

Can you implement the technology in

your practice?

Follow-up questions

What question will you answer or what decision will you inform?

Has a need already been identified for the promised information?

How much validity-related evidence is available regarding the new technology?

Are you confident enough with the limitations of the technology to inform practice?

In what format and by what means is information from the technology delivered, and how much cleaning needs to be done to integrate it with other measurements?

Do you have the analytical resources to handle and analyze the data?

What burden is placed on athletes and practitioners to collect the data?

Does your culture allow for technology to be implemented and data to be collected, and will the technology affect the culture?

Does your context allow for data to inform and alter practice?

Sources of evidence

Understand the challenges of your own context.

Consult with other researchers and practitioners who have faced similar questions and challenges.

Scientific literature surrounding the validity- related evidence for the technology

Internal validation and reliability

Professional network

Data samples from the company, short-term trials

Internal discussions or methodologic and statistical consultancy

Professional network

Qualitative scientific evidence Internal communication (formal

and informal) and education

Take-home messages

Start with the end in mind. Evaluate the existing

environment and infrastructure to see whether you need new technology to get the information.

Evaluate continually. Consider the consequences. Pilot where possible. Partner where appropriate.

Plan ahead. Educate practitioners involved in

collection on proper formatting and process.

Automate processes where possible.

Audit data and proactively set up quality controls.

Understand the implementation context.

Look for invisible monitoring opportunities.

Build technological implementation into existing routines.

Figure 1. A critical decision-making framework for integrating technology in sport.

Journal of Athletic Training 905

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

information from a given technology and be confident in making a decision based on the evidence it provides. We believe the principles of unified validity theory can help guide researchers and practitioners in trying to answer this question of trustworthiness.

A Brief Overview of Unified Validity Theory. Ground- ed in the work of Loeveringer41 and Cronbach and Meehl,42

Samuel Messick posited and promoted a unified view of validity theory.43–45 In place of validity types, the following definition of unified validity theory was proposed by Messick and adopted in the Standard for Educational and Psychological Testing: ‘‘Validity is an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropri- ateness of interpretations and actions based on test scores or other modes of assessment.’’46

Unpacking this definition reveals 3 primary ways in which unified validity theory differs from other common and more traditional views of validity:

1. Validity is about claims and inferences that can be made, not about measures.

2. Validity evidence has multiple sources, and the aim is an integrated evaluative judgment (Figure 2).

3. Validity invites consideration of the consequences associated with technology and the data it provides.

How Does Validity Theory Translate to Adopting a New Technology? Unified validity theory provides prac- titioners and scientists with a lens to look through as they consider different measures. When we look through this lens, we see that thinking about measurements from technology is similar to thinking about science. The answers to most of our questions are more nuanced than yes versus no or valid versus invalid. Instead, we use terms such as it depends, to a degree, or in this specific context. This sets the stage for a practitioner considering a new technology to evaluate the technology on a continuum in terms of his or her specific context.

As an integrative, evaluative judgement, practitioners should examine all the available sources of evidence regarding a given technology and the specific metric it provides. Some answers may be found in the peer-reviewed literature, or the practitioner may have to pilot data internally. Reaching out to colleagues who have already

implemented these technologies may offer opportunities to discuss their internal validity-related evidence.

The consequences of testing are a final consideration that unified validity theory emphasizes and that practitioners should carefully consider. Implicit and explicit conse- quences are inherent when measuring and testing some- thing by implementing a technology. Athletes and practitioners will consider the quality being measured as important, athletes may train to improve that given quality, and decisions may be based more heavily on the provided data than on other pieces of information. These intended and unintended consequences can be positive or negative but should be considered carefully.

This process and evaluation must be performed on each of the different metrics that a practitioner hopes to use to inform decision making. Returning to our volleyball example, the IMU provides a more accurate measure of jump count than does video notation. However, this same IMU also provides measures of jump height and ground reaction forces. Although these measures may theoretically be linked to overuse injuries and performance, they should each be further investigated. In this case, video notation would not be the appropriate comparison measure, and more advanced biomechanical analyses and equipment would be preferable.

Ultimately, no technology, or the data it provides, is perfectly trustworthy. The practitioner faces this question: Given all the information at his or her disposal, are the limitations of the technology minimal enough that it can still inform decision making?

Take-Home Suggestions. ! Evaluate continually. View trust in one’s data as an

ongoing endeavor to judge how trustworthy the technol- ogy and data are on a spectrum using all the available sources of validity-related evidence.

! Consider the consequences of testing. What potential consequences, intended and unintended, could introduc- ing the technology have for athletes and practitioners?

! Pilot test where possible. If practitioners can gain early access to the technology before purchasing, they can conduct preliminary analyses of the data before purchas- ing.

! Partner where appropriate. When in-house expertise is not sufficient for examining certain aspects of the data,

Figure 2. Some sources of validity-related evidence.

906 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

collaborate with a research laboratory, university, or third-party company to facilitate analyses of the trustworthiness of the technology.

Question 3: Can You Integrate, Manage, and Analyze the Data Effectively?

A 3608 view of an athlete’s training, recovery, lifestyle, and so on is a major potential benefit of technological implementation. If a technology and the data it provides are deemed trustworthy enough to be added to this holistic athlete view, the next step is to understand how the data will be extracted, integrated with other data sources, and analyzed in a meaningful way.47 Different technologies provide different levels of granularity to the data, and the means of data extraction can vary from manual download- ing of files (eg, spreadsheets) to application programming interfaces that allow automated data extraction. The extracted data may also need additional cleaning before analysis and modification to be integrated with other data sources. It is vital to understand how much time will be needed to extract and clean the data; although these processes (eg, manual downloads and spreadsheet data management) may be common, they limit scalability and may preclude successful and sustainable implementation of a given technology.

A well-known truism in the data-science realm is that data scientists spend most of their time cleaning and preparing data for analysis. Given the complexity and challenges inherent in combining data from disparate technologies, practitioners must consider whether they have the expertise on their team or at their disposal to create a system that brings their data together. This may be accomplished through in-house data-science personnel, third-party athlete-management systems, or external con- sulting agencies. Without this expertise, it is very challenging to combine data sources to create a holistic athlete profile.

Once the data are collected and combined, analysis presents its own challenges. Over the last few decades, intensive longitudinal data have become increasingly common in elite sport settings. These data present a specific set of challenges and assumptions inherent to repeated-measures data, and in many instances, more sophisticated analyses are recommended to deal with these challenges.48,49 At least in the workload injury field, the authors of a methodologic review50 identified that statistical approaches to adequately address these challenges when investigating the question of how workload data relate to injury risk have not been applied to many intensive longitudinal data sets. Statistical in-house or outsourced expertise can help ensure that the statistical approaches applied are appropriate for the data complexity.

To understand the demands of accessing, cleaning, integrating, and analyzing the data that a new technology will provide, it is highly prudent to ask the company for a free trial and access to their data streams. Discussions with other practitioners in the industry who already use the technology may also be fruitful to paint a realistic picture of the data-management demands.

Take-Home Suggestions. ! Plan ahead: Obtaining data samples from prospective

companies ahead of time helps to ensure that the

processes and systems can be tested and evaluated before a technology is introduced.

! Educate: By training practitioners in basic principles of data collection,51 many of the data-cleaning challenges can be proactively prevented.

! Automate: Software solutions (eg, Alteryx [Irvine, CA], Fivetran [Oakland, CA], Matillion [Manchester, UK]) and open-source coding platforms (eg, R [Vienna, Austria], Python [Wilmington, DE]) can enable data scientists to streamline and automate many processes, thereby reducing the amount of time required to manually input, download, and edit data.

! Audit: Set up data-audit or quality-control checks to ensure the data are clean and appropriately combined and then respond appropriately when you find mistakes and outliers.

Question 4: Can You Implement the Technology in Your Practice?

The fourth and final question to consider is whether a technology can realistically be implemented in your specific sporting context. In the research arena, we know that injury-prevention protocols that are effective in a controlled trial setting may fail to deliver the same results in a real-world environment because the effects are largely dependent on the adoption and implementation of the program.43–45 In the same way, even a near-perfect technology may fail in an environment where it encounters an implementation problem.

Implementation failures may occur anywhere along the data-science pipeline. The challenges in data analysis were largely addressed in question 3, but implementation challenges are especially pivotal to consider at the data- collection and data-dissemination steps.

Implementation challenges in data collection stem from increased practitioner or athlete burden. The demand on staff and athletes alike to collect data can be significant. It is not uncommon for the rollout of a new technology to be one more item in a long list of responsibilities for staff. Athletes are also not always burden free when it comes to the implementation process. It is important to consider what the athletes will be asked to do, the collective burden of technology and data collection on athletes as a whole, and how the athletes will perceive the new technologies. One must consider the possible ramifications of this increased staff and athlete burden and whether the technology’s potential benefits outweigh the cost of implementation.

The flexibility, mentality, and willingness of the people in an environment to adopt new practices can determine whether implementation challenges in data-informed deci- sion making arise. Certain sports have already embraced technology, whereas others may be deemed resistant to technological innovations. It is therefore essential to consider if members of your sporting culture will accept a specific technology in their environment. For numerous reasons, an organization may not want the data or may not be keen to have the answers that the data could provide. Also, deep-rooted doubt in the reliability or validity of the data may prevail, despite the best available evidence. Many sporting cultures resist changing the way things have always been done, so technological implementation may be seen as changing or modifying their sport.

Journal of Athletic Training 907

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

The successful implementation of any technology requires careful consideration of the time and resources required from practitioners and athletes, the process and procedures that need to be in place to minimize the burden from the technology, and the communication and decision- making channels whereby the analyzed data will be delivered and used to inform practice.

Take-Home Suggestions. ! Understand and improve the implementation context.

Consider the burden and challenges on practitioners and athletes at all points across the data pipeline. Educating practitioners on the rationale and benefits associated with technology and empowering them in their roles may facilitate ‘‘buy-in’’ and more successful implementation.

! Look for invisible monitoring opportunities, which impose virtually no burden on athletes. The data collection is automated, and nothing else is needed from the athletes. These invisible opportunities, whatever they may be, will still require data aggregation and integration from staff and require an environment for informed decision making, but they may enhance implementation success when athlete buy-in is the primary barrier.

! Build technological implementation into existing and new routines and tasks to make it obvious that technology is being integrated and something new is being introduced.

The Final Question: Is the Technology Worth It?

The final step, which should only be considered seriously once all 4 previous questions have been answered affirmatively, is whether the technology is worth the investment. This is essentially a cost-benefit analysis comparing the expected net performance effect with the financial burden that the technology carries in the overall context of the other 4 questions in the framework.

On the Flip Side: Delivering With Technology

Although our focus in this article was predominantly on guiding researchers and practitioners to critically evaluate whether they should purchase a new technology, we believe it would be a mistake not to briefly address several key considerations for when that technology is introduced.

The following concrete recommendations may facilitate buy-in and increase the probability that technological implementation succeeds: (1) undersell and overdeliver; (2) allow athletes to provide the equivalent of informed consent before data collection; (3) try to include multiple key stakeholders in data-evaluation sessions so that the technology does not isolate support staff or create a ‘‘secret society’’; (4) continue to evaluate signal-to-noise ratios and provide clarity on accuracy and reliability; (5) allow complex analyses to take place behind closed doors but present simplified, clean data that support important messages to coaches and athletes; and (6) do not use data in ways that contribute to political agendas or undermine the integrity of colleagues.

Integrating advanced technology into high-performance sport can be challenging. Emotions, egos, time constraints, and unrealistic expectations can make it difficult to gain approval as well as purchase and implement new technology. Many young practitioners may purchase

technology in an attempt to win favor from upper management as they work to encourage enthusiasm, hope, and belief in players and stakeholders. Without being fully aware of the power of the placebo (that is, the excitement of buying something new), they may convince sports administrators to allocate significant funds for a speculative purchase. We hope that the critical framework provided in this article will help to prevent these types of poor decisions, yet we believe the placebo, or belief, effect is an important aspect of technological implementation to leverage.

Interestingly, it has been reported that some of the earliest placebo researchers examined the influence of fake advanced medical technology for treating pain. For those who believed new technology could take away pain, the sophisticated device that supposedly harnessed the power of special metals worked equally well when the devices looked like they were made of metal but were actually made of wood.52 Introducing advanced technology into a high-performance program may have similarly positive effects if it is sold and implemented the right way, with a collective belief among practitioners and athletes that the technology can have a positive effect. In contrast, organizational differences of opinion can dilute the power of the belief effect. When the support team is divided on the efficacy of using new technology, their conversations and attitudes can ultimately undermine buy-in from athletes. Technological implementation should ultimately be a collective team effort whereby stakeholders engage throughout the stages of the decision-making framework and as the technology is implemented.

CONCLUSIONS

Technology may help organizations reach their grandiose vision or drag companies down because of its pitfalls. We hope this critical framework will empower practitioners and organizations to make informed, wise decisions about whether a technology should be implemented. However, we acknowledge that this stepwise approach is an oversimpli- fication and over-regimentation of the technological- evaluation process. At times, a technology may simply cost too much, in which case the 4 questions are irrelevant. Several questions may be investigated at once, such as when a company provides a free trial. Although the framework may be applied differently than described in this stepwise presentation, we caution that all 4 questions are critical to answer affirmatively before an investment is made.

Three fundamental principles underpin this type of a framework: (1) proactivity allows practitioners to start with the end in mind and to plan ahead in considering how to solve implementation challenges across the data pipeline; (2) critical thinking informs how practitioners evaluate the trustworthiness of technology and its data, as well as the intended and unintended consequences of introducing the technology; and (3) collaboration, specif- ically internal collaboration, underpins the success of communication and data-informed decisions and external collaboration can be essential for piloting technologies where appropriate (eg, outsourcing validity-related, data management, or analytical work that is beyond an organization’s current capabilities). Each of these princi-

908 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

ples bodes well in critically thinking about technology and even more broadly for practitioner and organizational excellence.

The decision ‘‘to tech or not to tech’’ is critical but complex. It should be made through a careful evaluation of evidence related to the technology and the environment in which it will be deployed. Ultimately, it is a question that every organization and practitioner will face in today’s technologically driven age. Much like the technologies in question, no person or company will make all perfect decisions, but a thoughtful framework and critical approach can help them hit the target more often than not.

REFERENCES

1. Bourdon PC, Cardinale M, Murray A, et al. Monitoring athlete

training loads: consensus statement. Int J Sports Physiol Perform.

2017;12(suppl 2):S2161–S2170. doi:10.1123/IJSPP.2017-0208

2. Borresen J, Lambert MI. The quantification of training load, the

training response and the effect on performance. Sports Med.

2012;39(9):779–795. doi:10.2165/11317780-000000000-00000

3. Saw AE, Main LC, Gastin PB. Monitoring the athlete training

response: subjective self-reported measures trump commonly used

objective measures: a systematic review. Br J Sports Med.

2016;50(5):281–291. doi:10.1136/bjsports-2015-094758

4. Neupert EC, Cotterill ST, Jobson SA. Training-monitoring engage-

ment: an evidence-based approach in elite sport. Int J Sports Physiol

Perform. 2018;14(1):99–104. doi:10.1123/ijspp.2018-0098

5. Nässi A, Ferrauti A, Meyer T, Pfeiffer M, Kellmann M.

Psychological tools used for monitoring training responses of

athletes. Perform Enhanc Health. 2017;5(4):125–133. doi:10.1016/

j.peh.2017.05.001

6. Flatt AA, Esco MR. Smartphone-derived heart-rate variability and

training load in a women’s soccer team. Int J Sports Physiol

Perform. 2015;10(8):994–1000. doi:10.1123/ijspp.2014-0556

7. Buchheit M. Monitoring training status with HR measures: do all

roads lead to Rome? Front Physiol. 2014;5:73. doi:10.3389/fphys.

2014.00073

8. Gathercole R, Sporer B, Stellingwerff T. Countermovement jump

performance with increased training loads in elite female rugby

athletes. Int J Sports Med. 2015;36(9):722–728. doi:10.1055/s-

0035-1547262

9. Wu PP-Y, Sterkenburg N, Everett K, Chapman DW, White N,

Mengersen K. Predicting fatigue using countermovement jump

force-time signatures: PCA can distinguish neuromuscular versus

metabolic fatigue. PLoS One. 2019;14(7):e0219295. doi:10.1371/

journal.pone.0219295

10. Meng E, Sheybani R. Insight: implantable medical devices. Lab

Chip. 2014;14(17):3233–3240. doi:10.1039/C4LC00127C

11. van der Kruk E, Reijne MM. Accuracy of human motion capture

systems for sport applications; state-of-the-art review. Eur J Sport

Sci. 2018;18(6):806–819. doi:10.1080/17461391.2018.1463397

12. Grigg J, Haakonssen E, Rathbone E, Orr R, Keogh JWL. The

validity and intra-tester reliability of markerless motion capture to

analyse kinematics of the BMX Supercross gate start. Sports

Biomech. 2018;17(3):383–401. doi:10.1080/14763141.2017.

1353129

13. Bruderer T, Gaisl T, Gaugg MT, et al. On-line analysis of exhaled

breath. Chem Rev. 2019;119(19):10803–10828. doi:10.1021/acs.

chemrev.9b00005

14. Peake JM, Kerr G, Sullivan JP. A critical review of consumer

wearables, mobile applications, and equipment for providing

biofeedback, monitoring stress, and sleep in physically active

populations. Front Physiol. 2018;9:743. doi:10.3389/fphys.2018.

00743

15. Petersen CJ, Pyne DB, Portus MR, Dawson BT. Comparison of

player movement patterns between 1-day and test cricket. J Strength

Cond Res. 2011;25(5):1368–1373.

16. Gray AJ, Jenkins DG. Match analysis and the physiological

demands of Australian football. Sports Med. 2010;40(4):347–360.

doi:10.2165/11531400-000000000-00000

17. Michalsik LB, Madsen K, Aagaard P. Match performance and

physiological capacity of female elite team handball players. Int J

Sports Med. 2014;35(7):595–607.

18. Bangsbo J, Nrregaard L, Thors F. Activity profile of competition

soccer. Can J Sport Sci. 1991;16(2):110–116.

19. Reilly T. Energetics of high-intensity exercise (soccer) with

particular reference to fatigue. J Sports Sci. 1997;15(3):257–263.

doi:10.1080/026404197367263

20. Reilly T, Thomas V. A motion analysis of work-rate in different

positional roles in professional football match-play. J Hum Mov

Stud. 1976;2:87–97.

21. Barris S, Button C. A review of vision-based motion analysis in

sport. Sports Med. 2008;38(12):1025–1043. doi:10.2165/00007256-

200838120-00006

22. Cummins C, Orr R, O’Connor H, West C. Global positioning

systems (GPS) and microtechnology sensors in team sports: a

systematic review. Sports Med. 2013;43(10):1025–1042. doi:10.

1007/s40279-013-0069-2

23. Torreño N, Munguı́a-Izquierdo D, Coutts A, de Villarreal ES,

Asian-Clemente J, Suarez-Arrones L. Relationship between external

and internal loads of professional soccer players during full-matches

in official games using global positioning systems and heart-rate

technology. Int J Sports Physiol Perform. 2016;11(7):940–946.

doi:10.1123/ijspp.2015-0252

24. Li RT, Kling SR, Salata MJ, Cupp SA, Sheehan J, Voos JE.

Wearable performance devices in sports medicine. Sports Health.

2016;8(1):74–78. doi:10.1177/1941738115616917

25. FIFA Quality Performance Reports for EPTS. Football Technology.

FIFA Web site. https://football-technology.fifa.com/en/media-tiles/

fifa-quality-performance-reports-for-epts/. Accessed February 4,

2020.

26. Bradley PS, Lago-Peñas C, Rey E, Gomez Diaz A. The effect of

high and low percentage ball possession on physical and technical

profiles in English FA Premier League soccer matches. J Sports Sci.

2013;31(12):1261–1270. doi:10.1080/02640414.2013.786185

27. Bush MD, Archer DT, Hogg R, Bradley PS. Factors influencing

physical and technical variability in the English Premier League. Int

J Sports Physiol Perform. 2015;10(7):865–872. doi:10.1123/ijspp.

2014-0484

28. Gregson W, Drust B, Atkinson G, Salvo VD. Match-to-match

variability of high-speed activities in premier league soccer. Int J

Sports Med. 2010;31(4):237–242. doi:10.1055/s-0030-1247546

29. MacDonald KJ, Palacios-Derflingher LM, Emery CA, Meeuwisse

WH. The effect of injury definition and surveillance methodology

on measures of injury occurrence and burden in elite volleyball. Int

J Sports Med. 2018;39(11):860–866. doi:10.1055/a-0577-4639

30. Bere T, Kruczynski J, Veintimilla N, Hamu Y, Bahr R. Injury risk is

low among world-class volleyball players: 4-year data from the

FIVB Injury Surveillance System. Br J Sports Med .

2015;49(17):1132–1137. doi:10.1136/bjsports-2015-094959

31. Helland C, Bojsen-Mller J, Raastad T, et al. Mechanical properties

of the patellar tendon in elite volleyball players with and without

patellar tendinopathy. Br J Sports Med. 2013;47(13):862–868.

doi:10.1136/bjsports-2013-092275

32. MacDonald K, Bahr R, Baltich J, Whittaker JL, Meeuwisse WH.

Validation of an inertial measurement unit for the measurement of

jump count and height. Phys Ther Sport. 2017;25:15–19. doi:10.

1016/j.ptsp.2016.12.001

33. Mujika I, Halson S, Burke LM, Balagué G, Farrow D. An

integrated, multifactorial approach to periodization for optimal

Journal of Athletic Training 909

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

performance in individual and team sports. Int J Sports Physiol

Perform. 2018;13(5):538–561. doi:10.1123/ijspp.2018-0093 34. Kiely J. Periodization theory: confronting an inconvenient truth.

Sports Med. 2018;48(4):753–764. doi:10.1007/s40279-017-0823-y 35. Haugen T, Buchheit M. Sprint running performance monitoring:

methodological and practical considerations. Sports Med. 2015;46(5):641–656. doi:10.1007/s40279-015-0446-0

36. Coutts AJ, Duffield R. Validity and reliability of GPS devices for measuring movement demands of team sports. J Sci Med Sport.

2010;13(1):133–135. doi:10.1016/j.jsams.2008.09.015 37. Bellenger CR, Fuller JT, Thomson RL, Davison K, Robertson EY,

Buckley JD. Monitoring athletic training status through autonomic heart rate regulation: a systematic review and meta-analysis. Sports

Med. 2016;46(10):1461–1486. doi:10.1007/s40279-016-0484-2

38. Kelly JM, Strecker RE, Bianchi MT. Recent developments in home sleep-monitoring devices. ISRN Neurol. 2012:768794. doi:10.5402/

2012/768794 39. Torres-Ronda L, Schelling X. critical process for the implementa-

tion of technology in sport organizations. Strength Cond J. 2017;39(6):54–59. doi:10.1519/SSC.0000000000000339

40. Liebermann DG, Katz L, Hughes MD, Bartlett RM, McClements J, Franks IM. Advances in the application of information technology

to sport performance. J Sports Sci. 2002;20(10):755–769. doi:10. 1080/026404102320675611

41. Loevinger J. Objective tests as instruments of psychological theory. Psychol Rep. 1957;3(3):635–694. doi:10.2466/pr0.1957.3.3.635

42. Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychol Bull. 1955;52(4):281–302.

43. Messick S. Foundations of validity: meaning and consequences in psychological assessment. ETS Res Rep Series. 1993;1993(2):i–18.

doi:10.1002/j.2333-8504.1993.tb01562.x

44. Messick S. Validity of test interpretation and use. ETS Res Rep

Series. 1990;1990(1):1487–1495. doi:10.1002/j.2333-8504.1990.

tb01343.x

45. Messick S. The standard problem: meaning and values in

measurement and evaluation. Am Psychol. 1975;30(10):955–966.

doi:10.1037/0003-066X.30.10.955

46. American Educational Research Association, American Psycholog-

ical Association, and National Council on Measurement in

Education. Standards for Educational and Psychological Testing.

Washington, DC: American Psychological Association; 1999.

47. Dayal U, Castellanos M, Simitsis A, Wilkinson K. Data integration

flows for business intelligence. In: Kersten M, Novikov B, Teubner

J, eds. EDBT ’09: Proceedings of the 12th International Conference

on Extending Database Technology: Advances in Database

Technology. Saint Petersburg, Russia; March 22, 2009:1–11.

https://dl.acm.org/doi/abs/10.1145/1516360.1516362. Accessed

May 20, 2020.

48. Bolger N, Laurenceau J-P. Intensive Longitudinal Methods: An

Introduction to Diary and Experience Sampling Research. New

York, NY: Guilford Press; 2013.

49. Walls TA, Schafer JL, eds. Models for Intensive Longitudinal Data.

New York, NY: Oxford University Press; 2006.

50. Windt J, Ardern CL, Gabbett TJ, et al. Getting the most out of

intensive longitudinal data: a methodological review of workload–

injury studies. BMJ Open. 2018;8(10):e022626. doi:10.1136/

bmjopen-2018-022626

51. Broman KW, Woo KH. Data organization in spreadsheets. Am Stat.

2018;72(1):2–10. doi:10.1080/00031305.2017.1375989

52. Finniss DG. Placebo effects: historical and modern evaluation. Int

Rev Neurobiol. 2018;139:1–27.

Address correspondence to Johann Windt, PhD, CSCS, Vancouver Whitecaps FC, 3065 Wesbrook Mall, Vancouver, BC V6T 1Z3, Canada. Address email to [email protected].

910 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021