Skip to main content

Table 4 Intratater reliability for experienced raters with > 20 years of clinical experience and novice rater with ≤4 years of clinical experience

From: Visual assessment of movement quality: a study on intra- and interrater reliability of a multi-segmental single leg squat test

Raters PAa Kappab (CI 95%) Kappamaxc PId BIe PABAKf (CI 95%)
Experienced
 Rater 1
  Foot 1.0 1.0 (1.00–1.00) 1.0 0.91 0.00 1.00 (1.00–1.00)
  Knee 0.99 0.97 (0.91–1.00) 0.97 −0.06 0.02 0.97 (0.91–1.00)
  Pelvis 0.94 0.86 (0.73–1.00) 0.86 0.32 −0.06 0.88 (0.76–1.00)
  Trunk 0.95 0.89 (0.77–1.00) 0.96 0.40 0.02 0.91 (0.80–1.00)
  All segmentsg 0.97 0.93 (0.88–0.98) 0.98 0.39 −0.01 0.94 (0.90–0.98)
Experienced
 Rater 2
  Foot 1.0 1.0 (1.00–1.00) 1.0 0.91 0.00 1.00 (1.00–1.00)
  Knee 0.86 0.71 (0.52–0.89) 0.97 0.25 −0.02 0.72 (0.55–0.90)
  Pelvis 0.92 0.74 (0.51–0.96) 0.95 0.65 0.02 0.85 (0.71–0.98)
  Trunk 0.99 0.95 (0.85–1.00) 0.95 0.62 0.02 0.97 (0.91–1.00)
  All segmentsg 0.94 0.82 (0.73–0.91) 0.99 0.60 0.00 0.89 (0.83–0.94)
Novice
 Rater 3
  Foot 0.99 0.66 (0.02–1.00) 0.66 0.95 0.02 0.97 (0.91–1.00)
  Knee 0.92 0.72 (0.48–0.96) 0.94 0.68 −0.02 0.85 (0.71–0.98)
  Pelvis 0.89 0.17 (−0.22–0.55) 0.88 0.86 −0.02 0.79 (0.63–0.94)
  Trunk 0.92 0.69 (0.43–0.95) 0.94 0.71 −0.02 0.85 (0.71–0.98)
  All segmentsg 0.93 0.62 (0.45–0.78) 0.96 0.80 0.01 0.86 (0.80–0.92)
Novice
 Rater 4
  Foot 0.97 0.48 (−0.16–1.00) 1.0 0.94 0.00 0.94 (0.85–1.00)
  Knee 0.91 0.70 (0.47–0.92) 0.70 0.63 0.09 0.82 (0.67–0.96)
  Pelvis 0.91 0.69 (0.46–0.93) 0.90 0.63 0.03 0.82 (0.67–0.96)
  Trunk 0.92 0.82 (0.66–0.97) 0.82 0.40 0.08 0.85 (0.71–0.98)
  All segmentsg 0.93 0.75 (0.64–0.86) 0.83 0.65 0.05 0.85 (0.79–0.92)
Rater 1–4 PAa Overall kappah (CI 95%) Kappamaxc PId BIe PABAKf (CI 95%)
 All segmentsg 0.94 0.82 (0.77–0.86) 0.97 0.61 0.01 0.89 (0.86–0.91)
  1. aPA: Percent agreement
  2. bKappa: Cohens´ kappa, calculated by; \( \upkappa =\frac{P_o-{P}_c}{1-{P}_c} \)
  3. Where; Po (observed agreement)\( =\frac{a+d}{n} \) and Pc (chance agreement) \( =\frac{\left(\frac{f_1x{g}_1}{n}\right)+\left(\frac{f_2x{g}_2}{n}\right)}{n} \)
  4. cKappamax: Is calculated so that the proportions of positive and negative judgements by each rater (i.e. the marginal totals) are taken as fixed, and the distribution of paired ratings (i.e. the cell frequency a,b,c and d) is adjusted so as to represent the greatest possible agreement. That will say, the maximum possible agreement for either presence or absence of the disease is the smaller of the marginal totals in each case [39]
  5. dPI: Prevalence index, calculated by; \( PI=\frac{a-d}{n} \)
  6. eBI: Bias index, calculated by; \( BI=\frac{b-c}{n} \)
  7. fPABAK: Prevalence-adjusted bias-adjusted kappa, calculated by; PABAK = 2P0 − 1
  8. gAll segments: Denotes a merged kappa coefficient for the intrarater reliability of each segments together (foot, knee, pelvis and trunk)
  9. hOverall kappa: Presents an overall average kappa for the variable all segments for all raters comparing test occasion one and two. Calculated with Cohens’kappa