Read two articles and Answer the questions

Read Saito et al (2018) and identify the:

Research design

Variables (Dependent variable (outcome variable), Independent variable, Main independent variable, Mediating variable, Moderating & Intervening variables, Control variable, Extraneous or confounding variables) Language Learning
ISSN 0023-8333
EMPIRICAL STUDY
Motivation, Emotion, Learning Experience,
and Second Language Comprehensibility
Development in Classroom Settings:
A Cross-Sectional and Longitudinal Study
Kazuya Saito
,a Jean-Marc Dewaele,a Mariko Abe,b
and Yo In’namib
a
Birkbeck, University of London and b Chuo University
This study presents a cross-sectional and longitudinal analysis of how 108 high school
students in English as a foreign language (EFL) classrooms enhanced the comprehensibility of their second language (L2) speech according to different motivation, emotion,
and experience profiles. Students’ learning patterns were primarily associated with their
emotional states (anxiety vs. enjoyment) and secondarily with their motivational dispositions (clear vision of ideal future selves). Students’ anxiety together with weaker
Ideal L2 Self related negatively to their performance at the beginning of the project—
performance that they had achieved after several years of EFL instruction. Students’
enjoyment together with greater Ideal L2 Self predicted the extent to which they practiced
and developed their L2 speech within the 3-month framework of the project. Results
suggest that more frequent L2 use with positive emotions directly impacts acquisition,
Parts of this study were presented at the 2017 Japan Second Language Acquisition Research Forum.
We are grateful to anonymous Language Learning reviewers and Journal Editor, Pavel Trofimovich,
for their helpful comments on earlier versions of the article. We also acknowledge Hui Sun for
her help with data collection, coding, and analyses. This study was funded by a grant from the
Grant-in-Aid for Scientific Research in Japan (16H03455) and the Birkbeck College Additional
Research Fund.
This article has been awarded an Open Materials badge. Study materials are
publicly accessible in the IRIS digital repository at http://www.iris-database.org.
Learn more about the Open Practices badges from the Center for Open Science:
https://osf.io/tvyxz/wiki.
Correspondence concerning this article should be addressed to Kazuya Saito, Birkbeck, University of London, Department of Applied Linguistics and Communication (Room 334), 25 Russell
Square, London, United Kingdom WC1B 5DQ. E-mail: k.saito@bbk.ac.uk
Language Learning 68:3, September 2018, pp. 709–743

C 2018 Language Learning Research Club, University of Michigan
DOI: 10.1111/lang.12297
709
Saito et al.
Motivation and Emotion in L2 Speech Learning
which may in turn lead to the lessening of negative emotions and better long-term L2
comprehensibility.
Keywords second language speech; foreign language learning; motivation; emotion;
comprehensibility; development
Introduction
There is a theoretical consensus in the field of second language (L2) acquisition
that adult L2 learners are able to learn new sounds and improve oral proficiency
as a function of increased input and practice in the target language (e.g., Flege,
2016). Yet the final outcome of late L2 learning is subject to a great deal of
variability, especially in foreign language classroom settings, where the amount
of L2 input and output is limited (Muñoz, 2014). To investigate the source of
such individual differences, scholars have examined the role of L2 learners’
experience inside and outside classrooms in L2 oral proficiency development
(e.g., Saito & Hanzawa, 2016a). With respect to learner-internal variables,
much research attention has been given to the social and psychological dimensions of individual differences. For example, there has been extensive research
conceptualizing, surveying, validating, and refining different constructs of motivation, such as Ideal versus Ought-to L2 Self (Dörnyei, 2005), and emotion,
such as anxiety versus enjoyment (Dewaele & MacIntyre, 2014), which are
relevant to foreign language learners in various classroom contexts. It has been
recently proposed that motivation and emotion are intertwined because motivated actions entail certain types of negative and positive emotions (Teimouri,
2016).
However, a set of unanswered questions may contribute to our understanding of the mechanisms underlying successful foreign language learning: Are L2
learners’ motivational and emotional states, different L2 experience profiles,
and actual L2 speech development patterns related to each other? If so, then to
what extent and how? Using data from 108 Japanese English as a foreign language (EFL) students, the present study examined (a) how L2 motivation and
emotion orientations influenced students’ practice of the target language and
(b) how both learner-external (experience) and learner-internal (motivation,
emotion) variables interacted to ultimately impact overall comprehensibility of
their L2 speech over one 3-month academic term. The present study combined
a cross-sectional perspective, comparing students’ initial motivation, emotion,
and proficiency profiles at the start of data collection, and a longitudinal perspective, linking their motivation and emotion to any change in proficiency and
experience during the project.
Language Learning 68:3, September 2018, pp. 709–743
710
Saito et al.
Motivation and Emotion in L2 Speech Learning
Background
Learners’ Motivation, Emotion, and Behavior
Motivation is one of the most extensively researched topics in L2 research.
Motivation has been found to lead to different language learning behaviors in
various contexts (for a research synthesis, see Boo, Dörnyei, & Ryan, 2015).
The L2 Motivational Self System (Dörnyei, 2005) has established itself as
the main theoretical framework for analyzing the motivational dispositions of
L2 learners, especially in foreign language classrooms. In conjunction with
the possible selves theory and self-discrepancy theory in social psychology
(Higgins, 1987; Markus & Nurius, 1986), this model states that L2 learners’
clear vision of their future selves exerts a significant impact on their current
behaviors (i.e., whether, how often, and in what way they use/practice a L2) and,
by extension, on their achievement (i.e., the extent to which they can improve
their proficiency). Such possible selves can be conceptualized in terms of two
self guides: the Ideal L2 Self and the Ought-to L2 Self.
The Ideal L2 Self, which roughly corresponds to not only integrative but also
instrumental motives with a promotional focus, refers to the self-image of an
ideal L2 user that a learner wants to become (Dörnyei, 2005). If L2 learners find
that their current proficiency levels are distant from a desirable future level,
they aim to fill in the discrepancy by striving to use, practice, and improve
their L2 ability. In contrast, Ought-to Self relates to those characteristics that
learners believe they ought to have in order to meet certain expectations and
avoid negative outcomes in the future. These expectations are not learners’
own; rather, they are imported and imposed images of the future that learners
internalize to some extent. In the present study, we focused on the social aspect
of the Ought-to Self, assuming that L2 learners with stronger social Ought-to L2
Self make greater efforts to study the target language so as to achieve what their
social networks or communities (e.g., friends, family members) expect them
to achieve, that is, instrumental motivation with a prevention focus (Dörnyei,
Csizér, & Németh, 2006).
To further understand the mechanism underlying motivation effects in L2
learning, scholars have recently begun to emphasize the importance of including L2 learners’ emotional states in the L2 Motivational Self System because L2
learners’ perception of actual and future selves may trigger different emotional
reactions (Dörnyei & Ushioda, 2009; MacIntyre & Gregersen, 2012; Papi &
Teimouri, 2014; Teimouri, 2016). So far, much of the discussion has been
concerned with one kind of negative emotion in classroom settings—anxiety
(Gkonou, Daubney, & Dewaele, 2017). MacIntyre and Gardner (1989) argued
that situation-specific language anxiety builds gradually because repeated
711
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
experience of anxiety leads learners to associate the L2 language class with
anxiety. Moreover, although anxiety can fluctuate during a class, it is typically
linked to what L2 learners have experienced over a prolonged period of
learning, such as test scores, attitude toward the target language, and standing
relative to their peers in class (Horwitz, 2017). Anxiety has been linked to harsh
error correction (Gregersen, 2003) or incompatibility between teachers and
students (Gregersen, MacIntyre, & Maze, 2014) and thus has been identified as
having a debilitating effect on L2 learning and achievement (for reviews, see
Horwitz, 2017, and MacIntyre, 2017). Crucially, such negative emotions
have been claimed to influence the Ought-to Self aspect of motivation and
vice versa: Prevention-focused L2 learners feel anxious when they perceive
difficulty in achieving their obligations, duties, and responsibilities regarding
their foreign language learning (Papi & Teimouri, 2014).
More recently, several scholars have argued in favor of a more holistic
view of emotions, including the role of positive emotions in foreign language
classrooms (Dewaele & MacIntyre, 2014; MacIntyre & Gregersen, 2014). For
example, one kind of positive emotion—enjoyment—is believed to help L2
learners better attend to, process, and acquire a target language (Dewaele &
Alfawzan, in press; Dewaele & MacIntyre, 2014). Such a positive, acquisitionfriendly atmosphere can occur in certain classrooms where activities are adequately challenging (i.e., slightly beyond L2 learners’ competence), creative,
and unpredictable; where activities have clear benefits and purpose; and where
efforts are made by teachers to facilitate task completion while providing praise,
encouragement, and feedback in a humorous, constant fashion. Different from
anxiety, enjoyment may relate to the Ideal Self aspect of motivation because a
sense of elation arises when L2 learners achieve their more internalized futureself guides, that is, their hopes, aspirations, and ideals. There is also some
evidence that L2 learners with a more promotional focus tend to express more
positive emotions toward their own learning experience, peers, and teachers
(Teimouri, 2016). Dewaele and Dewaele (2017) carried out a survey study on
British students between 12 and 18 years of age. The results identified a slight
increase in L2 enjoyment over time while L2 anxiety remained constant. However, the learner-internal and learner-external variables predicting enjoyment
and anxiety did change dramatically over time, suggesting dynamic change
below the surface.
Over the past 20 years, much empirical research has been conducted to
expound the value of self guides as motivational orientations. The Ideal L2
Self has been found to have relatively strong relationships with L2 learners’
motivated behaviors (Kormos, Kiddle, & Csizér, 2011; Taguchi, Magid, & Papi,
Language Learning 68:3, September 2018, pp. 709–743
712
Saito et al.
Motivation and Emotion in L2 Speech Learning
2009). However, the associations between the Ought-to Self and L2 learners’
behaviors remain unclear (e.g., Csizér & Kormos, 2009; Lamb, 2012). From
a methodological perspective, these behaviors have been typically measured
via L2 learners’ intended learning efforts by rating relevant statements on
questionnaires (e.g., “I think that I am doing my best to learn English”). Some
researchers have cast doubt on whether such reported intentions reflect the
quantity and quality of L2 learners’ actual use (e.g., Ryan, 2008). Moskovsky,
Assulaimani, Racheva, and Harkins (2016, p. 643) pointed out that “ultimately,
[L2 acquisition] is about achievement, that is, about attaining an adequate level
of proficiency in the [target language]” and that “[t]herein lies the real test for
the theory—in the capacity of the self guides to predict L2 achievement” (see
also Ushioda, 2016).
To date, several empirical studies have explored the relationship between
EFL learners’ motivation and their L2 performance although no clear link
between motivation and achievement has been found (e.g., Lamb, 2012;
Moskovsky et al., 2016; Papi & Teimouri, 2014). Although these findings
have suggested that stronger self guides may not necessarily be linked to more
successful L2 learning, they need to be replicated with greater methodological
rigor. First, L2 learners’ proficiency in these previous studies has been typically
measured via general proficiency tests, final grade, or self-ratings at only one
data collection point. Such cross-sectional designs have allowed researchers to
explore the relationship among L2 learners’ motivation, emotion, and achievement. To our knowledge, however, few longitudinal studies have delved into
how such learner-internal variables impact L2 learners’ development over time
(cf. Nagle, 2018). This deficit in the research corresponds to the general lack of
longitudinal work in L2 research (Ortega & Byrnes, 2008). Furthermore, these
previous studies did not control for L2 learners’ diverse EFL experience—
another key variable affecting L2 speech learning in foreign language settings.
Even if some L2 students have stronger motivation and show more language
development, the extent to which such correlations could be tied to the way that
they have practiced the target language in EFL classrooms remains unclear.
Thus, more research is needed to clarify the relationships between L2 learners’
motivation and emotion orientations, their experience profiles, achievement,
and development.
Experience, L2 Speech Learning, and Individual Differences
L2 speech—the focus of the present study—is a multifaceted phenomenon
comprising a wide range of different linguistic skills. According to previous
literature on naturalistic L2 speech learning, learners tend to show a great deal
713
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
of improvement within a short period of immersion (e.g., first 3 to 4 months
of stay) in terms of global dimensions of speech, such as comprehensibility
and accentedness (e.g., Derwing, Munro, & Thomson, 2008), as well as its
temporal (e.g., Segalowitz & Freed, 2004), segmental (Saito & Munro, 2014),
and lexicogrammatical (Mora & Valls-Ferrer, 2012) aspects as long as learners
use the L2 as their main language of communication in various social settings
(for a review of experience effects in naturalistic L2 development, see Saito,
2015).
In contrast, foreign language classrooms have been referred to as minimum input environments (Larson-Hall, 2008) because learners in such contexts typically receive only a few hours of instruction per week without many
opportunities to use the target language for the purpose of communication. In
this regard, successful classroom L2 speech learning crucially depends on the
extent to which learners actively seek and utilize every possible opportunity to
practice the target language both inside and outside of classrooms. It is thus
unsurprising that learners within the same language classroom setting can have
different L2 learning experiences, resulting in varying amounts and kinds of
improvement.
Under minimum input conditions, few longitudinal studies have indeed
shown statistically significant improvement in oral proficiency for L2 learners
(e.g., Mora & Valls-Ferrer, 2012; Muñoz & Llanes, 2014; Segalowitz & Freed,
2004). In these studies, the oral performance of participants between data collection points has failed to reach statistical significance, arguably because the
researchers considered participants as a categorical group (without taking into
consideration individual differences), using statistical analyses, such as t tests
and analyses of variance (ANOVAs). However, the incidence and degree of foreign language learning success widely varies between individuals as a function
of many variables. By using variance-based analyses, such as correlations or
regressions, rather than means-based comparisons, such as t tests or ANOVAs,
previous studies have found the multivariate nature of L2 oral proficiency development in EFL classrooms to be influenced by age (Larson-Hall, 2008) and
length of learning (Muñoz, 2014) and by the nature of L2 use inside classrooms
(e.g., see Saito & Hanzawa, 2016b, for form- vs. meaning-oriented instruction)
and outside instruction (e.g., see Muñoz, 2014, for extracurricular activities).
Plonsky and Oswald (2017) provided further discussion about the selection of
appropriate statistical analyses for examining multivariate data.1
To disentangle the intricate connections between experience and classroom L2 speech learning, Saito and Hanzawa (2016a) examined the linguistic
and learner profiles of university-level Japanese students with similar EFL
Language Learning 68:3, September 2018, pp. 709–743
714
Saito et al.
Motivation and Emotion in L2 Speech Learning
backgrounds (6 years of foreign language education without any experience
overseas). The students’ oral proficiency attainment was significantly associated
with their EFL experience inside (e.g., pronunciation training) and outside (e.g.,
cram school)2 classrooms in high school but not in junior high school. These
results indicated that the pedagogical potential of foreign language learning can
be maximized by how students optimize their most immediate L2 experience
beyond the regular syllabus, bringing to light the importance of the quantity,
quality, and timing of experience—three key variables in usage-based accounts
of L2 learning (Ellis, 2006).
The Present Study
A number of L2 speech researchers have examined the types of L2 learners who ultimately attain high-level pronunciation performance after years of
immersion in naturalistic settings. One well-researched yet controversial variable for successful L2 speech learning is motivation. Although some studies
have shown that L2 learners with highly advanced oral proficiency are likely
to demonstrate a strong concern for nativelike pronunciation accuracy (e.g.,
Moyer, 1999), others have failed to find any such significant predictive power
of motivation for successful L2 pronunciation ability (e.g., Purcell & Suter,
1980). Rather, researchers have found the ultimate quality of naturalistic L2
pronunciation performance to be strongly determined by learners’ age and the
length and intensity of their immersion in a L2 speaking environment (e.g.,
Flege, Munro, & MacKay, 1995).
To our knowledge, very few studies have examined how inexperienced L2
learners with varied levels of motivation can differentially improve their L2 pronunciation, especially when they engage in different kinds of foreign language
classroom learning. One of these studies, Baker-Smemoe and Haslam (2013),
longitudinally investigated a total of 31 Chinese EFL learners’ L2 motivation
(elicited via a subsection of the Pimsleur Language Learning Aptitude Battery)
and their L2 oral proficiency development, finding significant associations
between the two. In another study, Saito, Dewaele, and Hanzawa (2017) devised
a tailored questionnaire for their target population (40 Japanese EFL university
students). The students’ different levels of context-specific motivation—
studying English for their future career development as a vague, long-term
goal in a globalized society, which could be captured through the construct of international posture (Yashima, Zenuk-Nishide, & Shimizu, 2004)—significantly
predicted the longitudinal development of the students’ L2 oral proficiency
over one academic term. Finally, over one academic year, Nagle (2018) tracked
the development of the oral proficiency (comprehensibility and accentedness)
715
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
and the motivational orientations (Ideal L2 Self, Ought-to L2 Self) of
26 American learners of Spanish. Whereas the results of the quantitative
analyses did not reveal significant links between motivation and L2 speech
learning, the study provided some qualitative evidence that the participants
uniquely allocated motivational resources according to their different,
individual learning contexts and objectives.
On the whole, these precursor studies have suggested some acquisitional
value for motivation, especially in classroom L2 speech learning. However,
the constructs of L2 motivation investigated in these studies did not build on
more recent frameworks of motivation and emotion research (Baker-Smemoe &
Haslam, 2013; Saito et al., 2017). Although Nagle (2018) focused on Dörnyei’s
L2 Motivational Self System, the statistical power of the data set could be considered relatively weak (26 learners). Therefore, previous findings (in terms of
the lack of significant motivation effects in particular) need to be interpreted
with caution. Additionally, no prior study took into account recent trends considering motivation and emotion as being interrelated with the concept of sociopsychological individual differences (Teimouri, 2016). Further, as observed
in previous L2 motivation research (e.g., Moskovsky et al., 2016), the interaction of motivation and acquisition was not probed in relation to the quantity,
quality, and timing of the participants’ L2 experience.
Focusing on first-year Japanese high school students with varied EFL backgrounds, the present study primarily aimed to investigate L2 motivation, emotion, and experience as key variables for explaining variance in the process and
outcome of L2 oral proficiency development (measured through comprehensibility). Specifically, we set out to answer two research questions:
1. To what extent do L2 learners’ motivation and emotion profiles predict the
way they practice the target language inside and outside the classroom?
2. To what extent do L2 learners’ emotion, motivation, and experience differentially predict their language development?
Our study departed from previous studies (Baker-Smemoe & Haslam, 2013;
Saito et al., 2017) and aimed to provide generalizable and interdisciplinary
insights to the existing EFL, motivation, and emotion research by assessing
learner-external and learner-internal variables using solid theory-driven instruments for experience (Saito & Hanzawa, 2016a), motivation (Dörnyei et al.,
2006), and emotion (Dewaele & MacIntyre, 2014).
The data from the motivation/emotion questionnaire were collected at the
end of the participants’ second term (T1) in December 2016, and the participants’ experience and speech performance were examined at both T1 and the
Language Learning 68:3, September 2018, pp. 709–743
716
Survey 1 (movaon,
emoon, previous & recent
experience)
Speech test 1
Survey 2 (experience during
Weeks 1–12)
Longitudinal
T1 (end of 2nd term)
Motivation and Emotion in L2 Speech Learning
Cross-seconal
Saito et al.
T2 (end of 3rd term)
Speech test 2
Figure 1 Summary of research timeframe.
end of their third term (T2) in March 2017. Using the T1 data only, we first
cross-sectionally examined how the participants’ motivation and emotion were
associated with their past (preschool, elementary, and junior high school) and
current (high school) L2 use inside and outside the classroom (Research Question 1) and how motivation, emotion, and experience related to their long-term
achievement after several years of EFL experience at the start of the project
(Research Question 2). Using the data from T1 and T2, we then explored longitudinally how the participants’ motivation and emotion at T1 predicted their
L2 use between T1 and T2 (Research Question 1) and how their motivation,
emotion, and their L2 use could affect their L2 speech development over time
(Research Question 2). Overall, we aimed to pinpoint the role of motivation,
emotion, and experience in L2 oral proficiency development within a specific
time framework—one 3-month academic term. The timeline of the study is
summarized in Figure 1.
Method
Participants
The target participants included a sample of Japanese EFL students. Although
122 students originally participated in the project, 14 were eliminated from the
main analyses because the quality of their speech samples was poor or because
they did not complete the speaking tests at both T1 and T2. In this study, all 108
participants included in the final analyses were first-year students 15 to 16 years
of age at the same prestigious high school in Japan (44 males, 64 females).
None had stayed abroad for more than 1 week except for short family trips.
However, their age of first exposure to L2 English varied widely (M = 10.0
years, SD = 3.1, range = 1–14). At this high school, these first-year students
717
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
were divided into three class groups irrespective of their proficiency levels.
Each group included 30–40 students and followed the same curriculum.
For the duration of the project (one academic term), all students were
required to participate in seven 50-minute EFL lessons per week. These classes
were taught by three different Japanese teachers who demonstrated near-native
L2 English proficiency. Different from other EFL studies, where participants
have varied substantially in the number and type of classes that they were taking
(e.g., see Muñoz, 2014, for undergraduate-level students), our participants’ EFL
curriculum was exactly the same in their first year of high school, that is, all took
the same seven English lessons per week followed by the same final exams. The
content, syllabi, and observations of participants’ EFL classes are described in
detail in Appendix S1 in the Supporting Information online; therefore, only a
brief description of the curriculum is provided here.
According to the standard syllabus and our classroom observations, the EFL
classes equally focused on writing, speaking, reading, and listening throughout
the term; the instruction was frequently delivered in English; and the students
were always encouraged to interact with their teachers and peers in English. As
we quantified later via the EFL Experience Questionnaire, the amount of L2
use (i.e., using L2 English to communicate with teachers and peers) was substantially different among the participants. Although some reported that they
actively and frequently used L2 English throughout classes, others reported
that they chose to remain silent (see below). In terms of participants’ general
L2 English proficiency, their self-reported general proficiency test scores (i.e.,
EIKEN Test in Practical English Proficiency) ranged from Grade Pre-2 to 2 at
the beginning of the project. This indicated that they could be considered A2
(basic) and B1 (independent) users according to the benchmarks of the Common European Framework of Reference for Languages (EIKEN Foundation of
Japan, 2017).
Measures of L2 Oral Proficiency
Given that few adult L2 learners have been reported to attain nativelike L2
oral proficiency and that accent is a normal characteristic of adult L2 learning (Flege et al., 1995), it has been claimed that enhanced comprehensibility
(i.e., ease of understanding), rather than accent reduction, should be considered as a realistic goal for L2 speech teaching and learning (e.g., Derwing &
Munro, 2015). To capture what native speakers do in real-life situations when
interacting with L2 users, many researchers have used listeners’ intuitive judgments for assessing the comprehensibility aspects of L2 speech (e.g., Isaacs &
Trofimovich, 2012). Unlike in high-stakes testing settings, where accredited
Language Learning 68:3, September 2018, pp. 709–743
718
Saito et al.
Motivation and Emotion in L2 Speech Learning
raters make careful evaluations with reference to multiple proficiency descriptors, untrained raters employed in L2 research make quick, intuitive assessments
of comprehensibility on a 9-point scale (1 = difficult to understand, 9 = easy
to understand) immediately after listening to L2 speech samples. Although
raters receive only minimal training or practice, they tend to show relatively
high interrater reliability (Cronbach α > .90), indicating the presence of a
shared notion of comprehensibility (Saito, Trofimovich, & Isaacs, 2017). Extensive research on comprehensibility assessments has suggested that native
raters selectively attend to those linguistic features that hinder successful communication or prompt understanding (for a list of communicatively important
linguistic features, see Derwing & Munro, 2015; Saito et al., 2017; Trofimovich
& Isaacs, 2012).
In the present study, participants’ oral proficiency was measured through
native speakers’ intuitive judgment of L2 comprehensibility for the following
reasons. First, comprehensibility serves as an adequate index to assess the extent
to which the beginner-to-intermediate Japanese students have reached the minimum phonological, lexical, and grammatical requirements for being easily
understood in L2 English, despite still sounding accented. Second, listenerbased comprehensibility judgments likely correspond to what matters for students in their future communicative settings, that is, making themselves easy
to understand to their interlocutors. Third, although L2 learners’ accentedness
(linguistic nativelikeness) is resistant to change even in naturalistic settings,
the comprehensibility aspect of L2 oral proficiency tends to improve under
naturalistic and foreign language learning conditions provided that learners use
the target language (Derwing & Munro, 2015; Saito, 2015).
To assess the overall comprehensibility of the Japanese participants’ oral
proficiency, five native speakers of British English were recruited in London,
UK. Comprehensibility is typically measured using the scalar judgments of
minimally trained native raters. Even though such comprehensibility assessments are highly intuitive in nature, raters’ familiarity with foreign accented
speech has been found to affect their judgments to some degree (e.g., Isaacs
& Thomson, 2013). Thus, the decision was made to recruit only experienced
raters (graduate students in an applied linguistics master’s degree program at
the time of the project) who reported extensive English teaching experience
(M = 11.2 years, range = 5–20) and L2 speech analysis experience (i.e., they
had previously participated in similar L2 comprehensibility judgment sessions).
None of the raters had studied Japanese or visited Japan before the project, but
they all reported high familiarity with Japanese-accented speech (M = 5.6,
range = 5–6) on a scale of 1 (not familiar) to 6 (very familiar).
719
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
Materials
To collect speech samples from a relatively large number of participants within
a short period of time (i.e., in a span of 10 days), all participants took part in the
Telephone Standard Speaking Test, an automated English speaking test (ALC
Press Inc., 2017), using their home phones or cell phones at their convenience
at both T1 and T2. They were explicitly asked to take the exam in a quiet room
in order to provide clear speech for proper ratings (as noisy samples negatively
affect raters’ assessments). The participants’ successful completion of the test
was monitored by the testing agency, and the test was used as part of their final
assessment by their teachers. Outside of rare cases where the participants failed
to finish the test for technical reasons (e.g., unstable cell phone signals) and
retook the test, they had only one opportunity to take the test.
During the test, which lasted 15 minutes, the participants responded to 10
recorded questions. For each question, they were given 45 seconds to elaborate
and complete their answers without any preparation time. To avoid misunderstanding of the task instructions, all instructions were delivered in both English
and Japanese. The 10 questions differed in terms of target grammar structure
(present/past tense vs. comparatives) and task type (providing narratives, descriptions vs. reasoning) that they elicited. For each question, the testing system
was programmed to ensure that the participants engaged in different topics (randomly picked from the databank of the test) at T1 and T2. The speaking test
was considered to elicit extemporaneous speech, given that it was impossible
for the participants to do any preparation in advance.
Participants’ responses to the seventh question were chosen for analysis in
the present study. This question measured their ability to describe and explain
events in the past. According to the test guideline (ALC Press Inc., 2017),
Question 7 was considered relatively difficult (given that participants had to
use the past tense consistently) and thus was expected to elicit much individual
variability among the participants. For each participant, a topic was randomly
chosen from 17 different alternatives (e.g., favorite movie, family trip, shopping). Following the standards of previous L2 speech literature (e.g., Derwing
& Munro, 2015), the first 30 seconds of each speech sample were extracted
and saved in a separate file and then normalized for peak intensity and perceived loudness. Furthermore, the researchers listened to all speech samples
and discussed whether the quality of each recording was adequate for comprehensibility judgments. Through these discussions, we decided to remove the
data for four participants whose T1 or T2 samples sounded slightly distorted
due to background noise as well as for two participants who did not successfully complete the assigned task (exhibiting more than 20 seconds of silence)
Language Learning 68:3, September 2018, pp. 709–743
720
Saito et al.
Motivation and Emotion in L2 Speech Learning
either at T1 or T2. Eight participants’ data were also removed because they did
not take the speaking test at either T1 or T2. Thus, the final data set for rating
consisted of the speech of 108 participants at T1 and T2.3
Procedure
All individual rating sessions took place in a quiet room at a university in
London. First, the five raters received a brief explanation of the construct of
comprehensibility (for training scripts, see Appendix S2 in the Supporting
Information online) and familiarized themselves with the task format, a monologue task based on one of 17 different topics. Second, the raters practiced
the procedure by rating the comprehensibility of three speech samples not included in the main data set. During the rating session, 216 speech samples
(108 participants × 2 testing sessions) were played in a randomized order via
Praat software (Boersma & Weenink, 2017), and the raters listened to the full
30 seconds extracted for each participant. Upon listening to each sample only
once, they assigned a comprehensibility score using a 9-point scale. To avoid
listener fatigue, the speech samples were divided into four different blocks
(54 samples per block) with a 10-minute break between the blocks. In total, the
entire session took approximately 3 hours per rater. Cronbach’s alpha analyses demonstrated relatively high interrater agreement among the five raters for
comprehensibility ratings on the 9-point scale (α = .90). By pooling the native
raters’ judgment scores, one mean rating score was given to each of the 108
participants at each testing time (T1, T2).
Measures of Experience
As operationalized in previous EFL studies (Larson-Hall, 2008; Muñoz, 2014;
Saito & Hanzawa, 2016a, 2016b), the language backgrounds of the participants
in our study were surveyed via a structured experience questionnaire that we
named the EFL Experience Questionnaire (see Appendix S3 in the Supporting
Information online). At T1, the participants filled in the first version of the
questionnaire to report how they had practiced L2 English prior to as well as
at the beginning of the project. The past-experience variables included age of
onset of L2 learning and length of learning (how many hours they had practiced
L2 English inside and outside classrooms at preschool, elementary school, and
junior high school). As for recent experience, the participants reported what
percentage of time they spoke L2 English during class and how many hours
they practiced English outside of the classroom per week. In the present study,
these extracurricular activities were divided into three subcategories: preparing
for classes, studying at cram schools, and engaging in conversational activities
721
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
with native speakers and English L2 users. The second experience questionnaire
was administered at T2 to probe the participants’ most current L2 use over the
term (T1→T2). Similar to the T1 questionnaire, the longitudinal experience
variables included (a) the ratio of their L2 use in the classroom and (b) the
number of hours that they spent preparing for classes, attending cram schools,
and participating in conversation activities with native speakers and English L2
users per week.
Measures of Motivation and Emotion
To survey the participants’ motivation and emotion orientations, a composite
questionnaire of 58 items was developed in Japanese and administered at T1
(see Table 4 for the items used in the final analysis). For each item, participants
rated the extent to which they agreed or disagreed by marking one of the responses on a scale ranging from 1 (strongly disagree) to 6 (strongly agree). The
first part of the questionnaire consisted of 40 items adapted from the Japanese
version of Taguchi et al.’s (2009) questionnaire, which was designed to measure multiple dimensions of motivation (e.g., integrativeness, instrumentality,
family influence, attitudes to L2 community and culture) based on the L2 Motivational Self System theory (Dörnyei et al., 2006). Among them, four items
corresponded to Ideal L2 Self and four items to Ought-to L2 Self. Therefore,
they were used as motivation measures in the present study. The second part of
the questionnaire featured 18 statements adapted from the Foreign Language
Enjoyment Questionnaire (Dewaele & MacIntyre, 2014) as well as Foreign
Language Classroom Anxiety Scale (Horwitz, Horwitz, & Cope, 1986). The
first 10 statements were designed to tap into two dimensions of enjoyment specific to foreign language learning—private and social enjoyment in a teachercontrolled environment. The remaining eight statements reflected the physical
symptoms of anxiety, nervousness, and lack of confidence related to foreign
language learning. Because two out of the eight statements were rephrased to
indicate low anxiety, their scores were reverse coded so that all the statements
equally indicated high anxiety.
Results
L2 Oral Proficiency and Experience
L2 comprehensibility development patterns appeared to be different across
individuals, but these changes took place mostly within a range of 3 to 6
on a 9-point scale ranging from 1 (difficult to understand) to 9 (easy to
understand). The mean comprehensibility rating at T1 was 4.55 (SD =
1.54, 95% confidence interval [CI] [4.25, 4.84]), and at T2 the mean was
Language Learning 68:3, September 2018, pp. 709–743
722
Saito et al.
Motivation and Emotion in L2 Speech Learning
Table 1 Descriptive statistics for Japanese participants’ past, recent, and current EFL
experience
Experience
Past EFL experience (total hours of L2 practice)
Before project
Preschool
Elementary school: inside EFL classroom
Elementary school: outside EFL classrooma
Junior high school: inside EFL classroom
Junior high school: outside EFL classrooma
Recent EFL experience at T1 (end of 2nd term)
L2 use during class (%)
L2 practice outside EFL classrooms (hours
per week)
Preparation for class (hours per week)
Cram school (hours per week)
Conversation activities with native and
nonnative speakers (hours per week)
Current EFL experience at T2 (end of 3rd term)
L2 use during class (%)
L2 practice outside EFL classrooms (hours
per week)
Preparation for class (hours per week)
Cram school (hours per week)
Conversation activities with native and
nonnative speakers (hours per week)
M
SD
Range
1,349.9
20.3
175.0
127.0
699.4
368.0
977.6
91.2
271.8
264.6
403.6
435.2
300–7,200
0–750
0–1,500
0–1,800
150–4,500
0–3,000
58.7%
8.0
16.8
4.4
23–95
1–21
6.7
0.7
0.4
3.4
2.1
1.3
0.5–19
0–10
0–8
55.2%
7.4
19.9
5.9
0–90
0–52
5.9
1.0
0.5
3.3
4.2
1.4
0–15.5
0–35
0–7
Note. a L2 practice outside school (e.g., at cram school).
4.43 (SD = 1.52, 95% CI [4.14, 4.72]). In terms of their previous EFL
experience, the participants substantially varied regarding how they practiced
English inside and outside of their classrooms throughout preschool, elementary school, and junior high school (see Table 1). Similarly, Table 1 shows that
the participants’ recent (T1) and current (T2) EFL experience profiles inside
and outside the classroom were subject to much individual variability. Whereas
some participants used English very often during class and made extra efforts
to practice through extracurricular activities, such as preparation for classes,
cram schools, and conversations with native speakers and L2 users, others did
not. Such EFL experience was quite comparable at the beginning (T1) and
end (T2) of the term. Because our data set was multivariate in nature and not
suitable for mean-based analyses (ANOVAs or t tests), following Plonsky and
723
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
Table 2 Descriptive statistics and reliability coefficients for motivation and emotion
variables
Variables
M
SD
α
95% CI
Ideal L2 Self
Ought-to L2 Self
Enjoyment
Anxiety
3.7
3.0
4.5
3.5
1.2
1.1
0.8
0.9
.82
.85
.83
.86
[3.5, 3.9]
[2.7, 3.2]
[4.4, 4.7]
[3.3, 3.7]
Note. CI = confidence interval.
Oswald (2017), we used variance-based analyses (correlation, regression) to
compare the participants’ varied oral performance at T1 and their gain scores
between T1 and T2 according to different experience, motivation, and emotion
profiles.
Motivation and Emotion
Just as researchers had found in previous studies (Dewaele & MacIntyre, 2014;
Taguchi et al., 2009), our Cronbach alpha analyses demonstrated high reliability
for Ideal L2 Self, Ought-to L2 Self, foreign language enjoyment, and anxiety
(see Table 2).
Next, we examined several underlying factors among eight items from
the motivation questionnaire (4 for Ideal L2 Self, 4 for Ought-to L2 Self) and
18 items from the emotion questionnaire (10 for enjoyment, 8 for anxiety).
We followed Loewen and Gonulal’s (2015) field-specific guidelines for
analyzing factorability and determining a threshold for factor loadings. First,
the participants’ motivation and emotion ratings were submitted to a factor
analysis with Promax rotation and the minimum Kaiser criterion eigenvalue
set to 1.0 for both data sets. For the motivation questionnaire, the factorability
of the entire data set was confirmed via two tests: a significant Bartlett’s test
of sphericity, X2 = 431.78, p < .001, and a Kaiser-Meyer-Olkin measure of sampling adequacy of .799. A decision was made to identify a two-factor solution, which accounted for 63.3% of the total variance in the participants’ motivation ratings. Corresponding to the original conceptualization of the motivation questionnaire (Taguchi et al., 2009), all the items for Ideal L2 Self and Ought-to L2 Self were clustered in two different groups. Therefore, Factor 1 was labeled as Ideal L2 Self and Factor 2 as Ought-to L2 Self (see Table 3). An additional factor analysis was conducted using the participants’ responses to the emotion questionnaire with Promax rotation and a Kaiser Language Learning 68:3, September 2018, pp. 709–743 724 Saito et al. Motivation and Emotion in L2 Speech Learning Table 3 Summary of a two-factor solution based on a factor analysis of the motivation questionnaire Items Ideal L2 Self I imagine myself as someone who is able to speak English. I can imagine a situation where I am speaking English with foreigners. I can imagine myself living abroad and having a discussion in English. Whenever I think of my future career, I imagine myself using English. Ought-to L2 Self My parents believe that I must study English to be an educated person. Learning English is necessary because people surrounding me expect me to do so. I have to study English, because if I do not study it, I think my parents will be disappointed with me. I study English because close friends of mine think it is important. Factor 1 Ideal L2 Self Factor 2 Oughtto L2 Self .929 −.052 .924 −.141 .910 −.035 .673 .368 .104 .750 .069 .749 −.230 .729 −.037 .705 Note. All loadings > .5 are highlighted in bold.
criterion eigenvalue of 1.0. The two tests again confirmed the factorability
of the data set: a significant Bartlett’s test of sphericity, X2 = 1051.82, p
< .001, and a Kaiser-Meyer-Olkin measure of sampling adequacy of .850. A three-factor solution was chosen because it explained 60.8% of the total variance in the participants’ emotion ratings. According to this model, the 10 items for enjoyment appeared to tap into two different aspects of the participants’ positive emotional states. Following Loewen and Gonulal’s (2015) suggestion, we assigned each item to the factor on which it loaded most highly. Five items that loaded onto Factor 1 greatly represented the social aspects of positive emotions (i.e., how positive students feel about their relationship with other peers). Thus, Factor 1 was labeled as Social Enjoyment. The other six items (loading onto Factor 2) highlighted the participants’ inner perception of positive emotions (i.e., how positive students feel about their own EFL learning in the classroom). Thus, Factor 2 was labeled as Private Enjoyment. Finally, seven out of eight statements for anxiety neatly loaded onto Factor 725 Language Learning 68:3, September 2018, pp. 709–743 Saito et al. Motivation and Emotion in L2 Speech Learning 3. Factor 3 was thus labeled Anxiety (see Table 4). This three-factor structure reproduced the one uncovered by Dewaele and MacIntyre (2016). To examine the relationship between the participants’ motivation and emotion, the resulting two motivation factor scores (Ideal L2 Self and Ought-to L2 Self) and three emotion factor scores (Social Enjoyment, Private Enjoyment, and Anxiety) were then used for the subsequent analyses. Pearson correlation analyses were performed to investigate the strength of the associations between the motivation and emotion factor scores. The participants’ Ideal L2 Self was strongly correlated with the degree of their Private Enjoyment, r = .48, p < .025 (Bonferroni corrected), and inversely with the degree of Anxiety, r = –.33, p < .025 (Bonferroni corrected). Ideal L2 Self was less strongly correlated with Social Enjoyment, r = .21, p < .051 (Bonferroni corrected). Conversely, the participants’ Ought-to L2 Self was unrelated to any aspect of their emotion at a p < .025 level (Bonferroni corrected). The results here suggested that when L2 learners clearly visualize and internalize their future selves (i.e., Ideal L2 Self), they likely experience fewer negative emotions (Anxiety) and more positive emotions about their own EFL learning (i.e., Private Enjoyment). Cross-Sectional Investigation of Motivation, Emotion, Use, and Achievement Motivation and Emotion Versus Past/Recent Use In this Pearson correlation analysis, we first examined the extent to which the participants’ motivation and emotion could be related to their L2 use prior to the project (preschool, elementary school, and junior high school) and their recent L2 use at the beginning of the project (high school). As shown in Table 5, the participants’ motivation factor scores demonstrated significant and marginally significant associations with their early EFL experience (Ideal/Ought-to L2 Self vs. total hours of L2 practice at preschool and elementary school). Their positive emotion scores were significantly and marginally correlated with how much they had recently engaged with the L2 inside and outside the classroom (Social/Private Enjoyment vs. L2 use during class, total hours of preparation for class). The relationship between their negative emotion scores and previous/recent experience profiles did not reach statistical significance in any context. Taken together, the results here indicated that motivation could be tied to previous experience, that positive emotion could be influenced by recent experience, and that negative emotion may be independent of particular EFL experience at specific time points (i.e., preschool vs. elementary school vs. junior high school). Language Learning 68:3, September 2018, pp. 709–743 726 Saito et al. Motivation and Emotion in L2 Speech Learning Table 4 Summary of a three-factor solution based on a factor analysis of the emotion questionnaire Factor 1 Factor 2 Social Private Enjoyment Enjoyment Items Enjoyment My class has a good atmosphere. My peers are nice. We laugh a lot in class. I always feel like there is a positive environment in my class. English class is fun. I’m a worthy member of my foreign language class. I don’t get bored in class. I enjoy my foreign language class. It’s cool to know English as a foreign language. In class, I feel proud of my accomplishments. Anxiety I feel confident when I speak in my foreign language class. I can feel my heart pounding when I’m going to be called on in my foreign language class. I start to panic when I have to speak without preparation in my foreign language class. I get nervous and confused when I am speaking in my foreign language class. Even if I am well prepared for my foreign language class, I feel anxious. I am embarrassed to volunteer answers in my foreign language class. I always feel that the other students in my class speak the foreign language better than I do. I don’t worry about making mistakes in my foreign language class. Factor 3 Anxiety .892 .804 .672 .650 −.086 −.183 .270 .093 .009 −.124 .102 −.003 .585 −.212 .466 .836 .027 −.070 .289 .454 .134 .629 .557 .533 .165 −.076 .346 .355 .531 −.028 .205 −.566 .376 −.007 .091 .832 −.348 .211 .761 −.095 −.136 .715 .031 .223 .708 −.446 .035 .626 .494 −.300 .617 −.035 −.394 .583 Note. All loadings > .5 are highlighted in bold.
727
Language Learning 68:3, September 2018, pp. 709–743
Saito et al.
Motivation and Emotion in L2 Speech Learning
Table 5 Correlations between previous and recent experience variables and
motivation/emotion factors
Experience
Ideal Ought-to Social
Private
L2 Self L2 Self Enjoyment Enjoyment Anxiety
Past EFL experience (total hours of L2 practice)
Before project
.13
.28∗∗
Preschool
.09
.23∗
∗
.23∗
Elementary school: inside
.23
EFL classroom
.21
.20
Elementary school: outside
EFL classrooma
Junior high school: inside
.02
.08
EFL classroom
.01
.25∗∗
Junior high school: outside
a
EFL classroom
Recent EFL experience at T1 (end of 2nd term)
L2 use during class (%)
.14
−.06
L2 practice outside EFL
.04
.17
classrooms (hours per
week)
Preparation for class (hours
−.01
.13
per week)
Cram school (hours per week)
.02
.12
Conversation activities with
.11
.02
native and nonnative
speakers (hours per week)
−.01
.06
.15
.10
.03
.16
.01
−.04
−.12
.08
.10
−.09
−.10
.01
.01
−.07
.08
.09
.22
.08
.24∗
.09
−.14
−.01
.26∗∗
.10
−.01
−.18
−.12
−.01
.01
.07
−.14
Note. a L2 practice outside school (e.g., at cram school). ∗∗ p < .02, Bonferroni corrected. ∗∗∗ p < .01, Bonferroni corrected. Motivation, Emotion, and Use Versus Achievement Next, we conducted a series of Pearson correlation analyses to explore the participants’ long-term achievement (i.e., their comprehensibility scores at T1) relative to their motivation, emotion, and experience profiles. The results, summarized in Table 6, show that the participants’ T1 performance was significantly and marginally correlated not only with their past (preschool) and recent (L2 use in classrooms) experience but also with their motivation factor (Ideal L2 Self), positive emotion factor (Private Enjoyment), and negative emotion factor (Anxiety). To further examine the potentially different contributions of the motivation, emotion, and experience factors to the participants’ oral performance Language Learning 68:3, September 2018, pp. 709–743 728 Saito et al. Motivation and Emotion in L2 Speech Learning Table 6 Correlations between motivation, emotion, experience and long-term achievement Comprehensibility at T1 Variables Motivation and emotion Ideal L2 Self Ought-to L2 Self Social Enjoyment Private Enjoyment Anxiety Past EFL experience (total hours of L2 practice) Before project Preschool Elementary school: inside EFL classroom Elementary school: outside EFL classrooma Junior high school: inside EFL classroom Junior high school: outside EFL classrooma Recent EFL experience at T1 (end of 2nd term) L2 use during class (%) L2 practice outside EFL classrooms (hours per week) Preparation for class (hours per week) Cram school (hours per week) Conversation activities with native and nonnative speakers (hours per week) r p .30 .01 .09 .24 −.34 .002 .962 .328 .014 10,000) and we used stratified sampling, with the categories of the sampling frame including (i) two learning contexts: university and secondary school; (ii) three geographical regions: eastern, central, and western; (iii) within secondary schools two types: urban and rural; (iv) within universities two types: key and ordinary; and (v) within universities two strands: English and non-English majors. The wealth of data obtained allow us to offer a balanced and empirically based overview that describes general trends and interesting contrasts within the English learning motivation of Chinese students. Downloaded from https://academic.oup.com/applij/article/37/4/495/1742097 by Columbia University in the City of New York user on 13 October 2020 Language Learning Motivation in China: Results of a Large-Scale Stratified Survey 496 LANGUAGE LEARNING MOTIVATION IN CHINA In the era of economic globalization, proficiency in English has been seen in China as a definite asset of considerable value both at an individual and a societal level. Accordingly, English is taught as a core subject of the national curriculum across the whole country from primary school until the second year in university. However, the uniform educational targets do not automatically result in an equally uniform learning environment: with a population of over a billion, China displays considerable regional differences. In the past 30 years, the eastern regions of the country have experienced faster economic growth, better developmental opportunities, and greater returns than the central and western regions (Chen 1987), and the gap in regional development is still increasing (Tu 1995). Economic disparities among regions, in turn, result in uneven allocations of education resources in these areas (Chen and Wu 2011), with English language provision being no exception to this trend (Wu 2001). Consequently and unsurprisingly, students in more developed parts of the country have been found to have better English proficiency than their peers elsewhere (Yan and Horwitz 2008). Within the regional differences, we find a further developmental contrast: the urban/rural disparity. Due to the scarcity in natural and social resources, priority has typically been given to urban areas over rural regions, and thus the gap between urban and rural areas in China has been widening (Yu 2006). This has had inevitable consequences for English language ...