1. Sheet V#1 scores (includes all the teachers who used version 1 texts both native and
non-natives) with a total of 22
participants.
2. Sheet V#2 scores includes all the teachers who used version 2 texts both native and
non-natives)
with a total of 22 participants.
3. Sheet Natives_V1 (all natives who used version 1 texts) with a total of 11
participants.
4. Sheet Non_natives_V1 (all non-natives who used version 1 texts) with a total of 11
participants.
5. Sheet Natives_V2 (all natives who used version 2 texts) with a total of 11
participants.
6. Sheet Non_Natives_V2 (all non-natives who used version 2 texts) with a total of 11
participants.
7. Sheet Natives_V1&2 includes all the natives together who used both version 1 and 2
with a total of 22 participants.
8. Sheet Non-Natives_V1&2 includes all the natives together who used both version 1
and 2 with a total of 22 participants.
9. Here, I have combined all the data for natives and non-natives for different
types of analyses.
I need to identify, analyze and test the inter-rater reliability of the teachers in the different
groups as explained above and the intra-rater reliability of each rater. I also need to identify
the leniency and harshness of the raters as well. I also want explanations for each test
explaining the results.
For the inter-rater reliability I need to measure the following:
? I need to calculate the percent agreement (measure the agreement of the multiple
raters)
? I also need to use Cohen’s and Fleiss’s Kappa statistics. Cohen’s Kappa when we
compare between the two ratings (paper-based and computer-based) and the ones
between natives and non-natives. Fleiss’s Kappa to compare between the multiple
texts and raters.
? I also need to use the Kendall’s Coefficient (Kendall’s W) to measure the strength
and the relationship between the multiple raters.
For intra-rater reliability, we need to treat the data once as a single index for the whole
raters and as individual raters.
? We can use Cohen’s kappa and the correlation coefficient (we have each participant
rated three texts twice, once on paper-based and on computer). The intra-class-
correlation (ICC).
? I also need to measure the leniency and harshness of the groups based on the
above divisions of sheets, where I also need graphs. I need the analysis of severity
and leniency to be shown between natives vs non natives in both versions and ways
of ratings.
Also, I need all the data files for all the analyses. Also, I want to know which softwares will
you use.
In summary, I need the following analyses:
Inter-rater:
? Cohen’s Kappa inter-rater
? Fleiss’s Kappa inter-rater
? Kendall’s Coefficient (Kendall’s W)
Intra-rater:
? Cohen’s Kappa intra-rater
? Intra-class correlation test (ICC)
? Multi-faceted rasch model (to analyze leniency and harshness)
I am working on an assignment in which we asked the teachers to rate 3 texts twice. First
they rated the three of them on computer and then the same teachers rated them on paper.
We have got two versions of texts, version 1 and versions 2. So a group of teachers rated
version 1 (which have 3 texts) and another group of teacher rated version 2 (which also have
3 texts).
We have got English native teachers and non native teachers. Group 1 of teachers, in which
there were both native and non native teachers, rated the version 1 of the texts. Group 2 of
teachers, which also have got both native and non native teachers, rated the version 2 of
texts.
The data is attached in the excel file. The following is the description of how I organised the
different data sheets in excel:
1. Sheet V#1 scores (includes all the teachers who used version 1 texts both native and
non-natives) with a total of 22 participants.
2. Sheet V#2 scores includes all the teachers who used version 2 texts both native and
non-natives) with a total of 22 participants.
3. Sheet Natives_V1 (all natives who used version 1 texts) with a total of 11
participants.
4. Sheet Non_natives_V1 (all non-natives who used version 1 texts) with a total of 11
participants.
5. Sheet Natives_V2 (all natives who used version 2 texts) with a total of 11
participants.
6. Sheet Non_Natives_V2 (all non-natives who used version 2 texts) with a total of 11
participants.
7. Sheet Natives_V1&2 includes all the natives together who used both version 1 and 2
with a total of 22 participants.
8. Sheet Non-Natives_V1&2 includes all the natives together who used both version 1
and 2 with a total of 22 participants.
9. Here, I have combined all the data for natives and non-natives for different
types of analyses.
I need to identify, analyze and test the inter-rater reliability of the teachers in the different
groups as explained above and the intra-rater reliability of each rater. I also need to identify
the leniency and harshness of the raters as well. I also want explanations for each test
explaining the results.
For the inter-rater reliability I need to measure the following:
●
●
I need to calculate the percent agreement (measure the agreement of the multiple
raters)
I also need to use Cohen’s and Fleiss’s Kappa statistics. Cohen’s Kappa when we
compare between the two ratings (paper-based and computer-based) and the ones
between natives and non-natives. Fleiss’s Kappa to compare between the multiple
texts and raters.
●
I also need to use the Kendall’s Coefficient (Kendall’s W) to measure the strength
and the relationship between the multiple raters.
For intra-rater reliability, we need to treat the data once as a single index for the whole
raters and as individual raters.
●
●
We can use Cohen’s kappa and the correlation coefficient (we have each participant
rated three texts twice, once on paper-based and on computer). The intra-classcorrelation (ICC).
I also need to measure the leniency and harshness of the groups based on the
above divisions of sheets, where I also need graphs. I need the analysis of severity
and leniency to be shown between natives vs non natives in both versions and ways
of ratings.
Also, I need all the data files for all the analyses. Also, I want to know which softwares will
you use.
In summary, I need the following analyses:
Inter-rater:
Cohen’s Kappa inter-rater
Fleiss’s Kappa inter-rater
Kendall’s Coefficient (Kendall’s W)
Intra-rater:
Cohen’s Kappa intra-rater
Intra-class correlation test (ICC)
Multi-faceted rasch model (to test leniency and harshness)
Participants
Text Number
Paper-based scores
computer scores
P10
P10
P10
P4
P4
P4
P5
P5
P5
P1
P1
P1
P14
P14
P14
P16
P16
P16
p18
p18
p18
p20
p20
p20
p21
p21
p21
P23
P23
P23
P25
P25
P25
P27
P27
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
8
6
6
7
7
6
8
7
8
7
6
7
7
5
4
8
7
6
6
4
5
7
7
6
9
5
4
9
7
6
10
7
9
8
4
7
6
7
8
6
7
8
7
8
8
8
7
7
7
6
9
8
8
7
5
4
7
6
6
9
4
5
9
7
7
10
7
8
9
6
P27
P29
P29
P29
p31
p31
p31
P32
P32
P32
P35
P35
P35
P37
P37
P37
P39
P39
P39
P41
P41
P41
P44
P44
P44
P46
P46
P46
P48
P48
P48
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
4
6
5
6
9
6
9
10
7
5
9
6
7
7
7
6
9
8
7
8
7
7
10
10
9
7
6
5
4
7
7
3
6
6
7
9
8
8
10
7
10
8
8
6
7
8
5
8
9
9
10
7
7
10
10
9
7
7
7
10
8
8
Participants
P13
P13
P13
P15
P15
P15
P6
P6
P6
P12
P12
P12
P11
P11
P11
P2
P2
P2
p9
p9
p9
p19
p19
p19
P22
P22
P22
P24
P24
P24
P26
P26
P26
P28
P28
P28
p30
p30
p30
P33
P33
P33
P34
P34
P34
P36
P36
P36
Text Number
Paper-based scores
computer scores
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
5
7
5
5
7
5
6
8
5
6
9
6
7
7
5
6
6
5
7
7
5
6
7
4
6
7
6
9
7
5
7
6
3
8
9
10
6
5
3
6
8
5
7
7
5
5
6
5
7
7
5
5
6
6
5
6
5
6
9
6
8
6
4
6
7
6
6
8
6
5
8
5
7
7
6
8
5
6
5
6
4
9
9
8
7
6
3
6
7
6
7
7
6
4
6
5
P38
P38
P38
P40
P40
P40
P43
P43
P43
P45
P45
P45
P47
P47
P47
P49
P49
P49
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
7
9
5
7
6
5
4
6
4
8
7
6
7
8
6
6
9
7
4
6
5
8
8
6
4
7
5
6
8
4
7
7
5
6
7
9
Participants
Text Number
Paper-based scores
computer scores
P10
P10
P10
P4
P4
P4
P1
P1
P1
P14
P14
P14
P32
P32
P32
P35
P35
P35
P37
P37
P37
P39
P39
P39
P41
P41
P41
P46
P46
P46
P48
P48
P48
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
8
6
6
7
7
6
7
6
7
7
5
4
10
7
5
9
6
7
7
7
6
9
8
7
8
7
7
7
6
5
4
7
7
7
6
7
8
6
7
8
8
7
7
7
6
10
7
10
8
8
6
7
8
5
8
9
9
10
7
7
7
7
7
10
8
8
Participants
P5
P5
P5
P16
P16
P16
p18
p18
p18
p20
p20
p20
p21
p21
p21
P23
P23
P23
P25
P25
P25
P27
P27
P27
P29
P29
P29
p31
p31
p31
P44
P44
P44
Text NumberPaper-based scorescomputer scores
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
8
7
8
8
7
6
6
4
5
7
7
6
9
5
4
9
7
6
10
7
9
8
4
4
6
5
9
6
9
10
10
9
8
7
8
9
8
8
7
5
4
7
6
6
9
4
5
9
7
7
10
7
8
9
6
3
6
6
7
9
8
8
10
10
9
Participants
P13
P13
P13
P15
P15
P15
P6
P6
P6
p9
p9
p9
P33
P33
P33
P36
P36
P36
P38
P38
P38
P40
P40
P40
P43
P43
P43
P47
P47
P47
P49
P49
P49
Text Number
Paper-based scores
computer scores
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
5
7
5
5
7
5
6
8
5
7
7
5
6
8
5
5
6
5
7
9
5
7
6
5
4
6
4
7
8
6
6
9
7
7
7
5
5
6
6
5
6
5
6
8
6
6
7
6
4
6
5
4
6
5
8
8
6
4
7
5
7
7
5
6
7
9
Participants
P12
P12
P12
P11
P11
P11
P2
P2
P2
p19
p19
p19
P22
P22
P22
P24
P24
P24
P26
P26
P26
P28
P28
P28
p30
p30
p30
P34
P34
P34
P45
P45
P45
Text Number
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
Paper-based scores computer scores
6
9
6
7
7
5
6
6
5
6
7
4
6
7
6
9
7
5
7
6
3
8
9
10
6
5
3
7
7
5
8
7
6
6
9
6
8
6
4
6
7
6
5
8
5
7
7
6
8
5
6
5
6
4
9
9
8
7
6
3
7
7
6
6
8
4
Participants
P10
P10
P10
P4
P4
P4
P1
P1
P1
P14
P14
P14
P32
P32
P32
P35
P35
P35
P37
P37
P37
P39
P39
P39
P41
P41
P41
P46
P46
P46
P48
P48
P48
P13
P13
P13
P15
P15
P15
P6
P6
P6
p9
p9
p9
Text NumberPaper-based scores
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
4
8
24
4
8
24
4
8
24
4
8
24
8
6
6
7
7
6
7
6
7
7
5
4
10
7
5
9
6
7
7
7
6
9
8
7
8
7
7
7
6
5
4
7
7
5
7
5
5
7
5
6
8
5
7
7
5
7
6
7
8
6
7
8
8
7
7
7
6
10
7
10
8
8
6
7
8
5
8
9
9
10
7
7
7
7
7
10
8
8
7
7
5
5
6
6
5
6
5
6
8
6
P33
P33
P33
P36
P36
P36
P38
P38
P38
P40
P40
P40
P43
P43
P43
P47
P47
P47
P49
P49
P49
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
6
8
5
5
6
5
7
9
5
7
6
5
4
6
4
7
8
6
6
9
7
6
7
6
4
6
5
4
6
5
8
8
6
4
7
5
7
7
5
6
7
9
Participants
P5
P5
P5
P16
P16
P16
p18
p18
p18
p20
p20
p20
p21
p21
p21
P23
P23
P23
P25
P25
P25
P27
P27
P27
P29
P29
P29
p31
p31
p31
P44
P44
P44
P12
P12
P12
P11
P11
P11
P2
P2
P2
p19
p19
p19
Text NumberPaper-based scores
computer-based scores
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
4
8
24
4
8
24
4
8
24
4
8
24
8
7
8
8
7
6
6
4
5
7
7
6
9
5
4
9
7
6
10
7
9
8
4
4
6
5
9
6
9
10
10
9
6
9
6
7
7
5
6
6
5
6
7
4
8
7
8
9
8
8
7
5
4
7
6
6
9
4
5
9
7
7
10
7
8
9
6
3
6
6
7
9
8
8
10
10
9
6
9
6
8
6
4
6
7
6
5
8
5
P22
P22
P22
P24
P24
P24
P26
P26
P26
P28
P28
P28
p30
p30
p30
P34
P34
P34
P45
P45
P45
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
6
7
6
9
7
5
7
6
3
8
9
10
6
5
3
7
7
5
8
7
6
7
7
6
8
5
6
5
6
4
9
9
8
7
6
3
7
7
6
6
8
4
Participants
Group
version
Text Number
Paper-based scores
P10
P10
P10
P4
P4
P4
P1
P1
P1
P14
P14
P14
P32
P32
P32
P35
P35
P35
P37
P37
P37
P39
P39
P39
P41
P41
P41
P46
P46
P46
P48
P48
P48
P13
P13
P13
P15
P15
P15
P6
P6
P6
p9
p9
p9
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
4
8
24
4
8
24
4
8
24
4
8
24
8
6
6
8
7
7
7
6
7
7
5
4
10
7
5
9
6
7
7
7
6
9
8
7
8
7
7
7
6
5
4
7
7
5
7
5
5
7
5
6
8
5
7
7
5
P33
P33
P33
P36
P36
P36
P38
P38
P38
P40
P40
P40
P43
P43
P43
P47
P47
P47
P49
P49
P49
P5
P5
P5
P16
P16
P16
p18
p18
p18
p20
p20
p20
p21
p21
p21
P23
P23
P23
P25
P25
P25
P27
P27
P27
P29
P29
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
12
1
5
6
8
5
5
6
5
7
9
5
7
6
5
4
6
4
7
8
6
6
9
7
8
7
8
8
7
6
6
4
5
7
7
6
9
5
4
9
7
6
10
7
9
8
4
4
6
5
P29
p31
p31
p31
P44
P44
P44
P12
P12
P12
P11
P11
P11
P2
P2
P2
p19
p19
p19
P22
P22
P22
P24
P24
P24
P26
P26
P26
P28
P28
P28
p30
p30
p30
P34
P34
P34
P45
P45
P45
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
Non-Native
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
12
1
5
12
1
5
12
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
4
8
24
6
9
6
9
10
10
9
6
9
6
7
7
5
6
6
5
6
7
4
6
7
6
9
7
5
7
6
3
8
9
10
6
5
3
7
7
5
8
7
6
Computer-based
7
6
7
8
6
7
8
8
7
7
7
6
10
7
10
8
8
6
7
8
5
8
9
9
10
7
7
7
7
7
10
8
8
7
7
5
5
6
6
5
6
5
6
8
6
6
7
6
4
6
5
4
6
5
8
8
6
4
7
5
7
7
5
6
7
9
8
7
8
9
8
8
7
5
4
7
6
6
9
4
5
9
7
7
10
7
8
9
6
3
6
6
7
9
8
8
10
10
9
6
9
6
8
6
4
6
7
6
5
8
5
7
7
6
8
5
6
5
6
4
9
9
8
7
6
3
7
7
6
6
8
4
Essay Writing Service Features
Our Experience
No matter how complex your assignment is, we can find the right professional for your specific task. Achiever Papers is an essay writing company that hires only the smartest minds to help you with your projects. Our expertise allows us to provide students with high-quality academic writing, editing & proofreading services.
Free Features
Free revision policy
$10Free bibliography & reference
$8Free title page
$8Free formatting
$8How Our Dissertation Writing Service Works
First, you will need to complete an order form. It's not difficult but, if anything is unclear, you may always chat with us so that we can guide you through it. On the order form, you will need to include some basic information concerning your order: subject, topic, number of pages, etc. We also encourage our clients to upload any relevant information or sources that will help.
Complete the order form
Once we have all the information and instructions that we need, we select the most suitable writer for your assignment. While everything seems to be clear, the writer, who has complete knowledge of the subject, may need clarification from you. It is at that point that you would receive a call or email from us.
Writer’s assignment
As soon as the writer has finished, it will be delivered both to the website and to your email address so that you will not miss it. If your deadline is close at hand, we will place a call to you to make sure that you receive the paper on time.
Completing the order and download