Skip Navigation



The European Journal of Orthodontics Advance Access published online on July 16, 2008
The European Journal of Orthodontics, doi:10.1093/ejo/cjn007
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
30/5/503    most recent
cjn007v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Al-Abdallah, M.
Right arrow Articles by O'Brien, K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Al-Abdallah, M.
Right arrow Articles by O'Brien, K.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2008. Published by Oxford University Press on behalf of the European Orthodontic Society. All rights reserved. For permissions, please email: journals.permissions@oxfordjournals.org.

Is the Royal London Space Analysis reliable and does it influence orthodontic treatment decisions?

M. Al-Abdallah, J. Sandler and K. O'Brien

School of Dentistry, University of Manchester, UK

Address for correspondence M. Al-Abdallah, Orthodontic Department, School of Dentistry, University of Manchester, Higher Cambridge Street, Manchester M15 6FH, UK, E-mail:mariam.al-abdallah{at}postgrad.manchester.ac.uk


   Summary
Top
 Summary
Introduction
 Materials and methods
 Results
 Discussion
 Conclusions
 References
 
The purpose of this study was to investigate the reliability of the Royal London Space Analysis (RLSA) and to evaluate its influence on orthodontic treatment decisions. Thirty-one case records were collected to represent various levels of crowding and different types of malocclusions. Seventeen examiners assessed these records and completed a data sheet that recorded information on their treatment decision. One month later, the examiners attended a course on the RLSA and then used the analysis to rescore the 31 cases. The models were also scored by the expert who led the course and these were then considered the ‘gold standard’ scores. After a further month, the examiners reapplied the RLSA and formulated a treatment plan for each set of patient records. A paired Student’s t-test and intraclass correlation coefficient (ICC) were used to assess the agreement in scoring RLSA, a paired sample t-test was used to compare the scores with the gold standard, and finally the reliability in treatment planning was determined using kappa ({kappa}) statistics.

The scores for lower arch crowding showed the highest inter-examiner agreement with an ICC of 0.93 whereas the lowest level of agreement was for upper arch space requirement with an ICC of 0.77. Intra-examiner agreement was generally high, particularly for the assessment of lower arch crowding (ICC = 0.93) and lower arch space requirement (ICC = 0.88). There was excellent validity for all the examiners against the gold standard scores with a paired samples correlation ranging between 0.96 for lower arch crowding and 0.79 for upper arch space requirement. The intra-examiner reliability in treatment decision was only moderate, with an average {kappa} value of 0.52 (maximum 0.82, minimum 0.24). Intra- and inter-examiner agreement for scoring the RLSA was acceptable. Nevertheless, the additional information obtained from the application of the RLSA did not have a substantial impact on the treatment decisions.


   Introduction
Top
 Summary
 Introduction
Materials and methods
 Results
 Discussion
 Conclusions
 References
 
Orthodontic treatment planning should follow a detailed and accurate assessment of space requirement and anchorage needs. Ideally, there should be a uniform, universally accepted method of measuring these components. Space assessment varies between a visual evaluation of patient records to the detailed application of measurements that apply scores to the various components of the malocclusion which either provide space or require space for their correction. This latter approach has the aim of justifying extraction and anchorage decisions and reducing variation in possible treatment plans.

Over recent years, several space analyses have been developed and advocated as tools that may be used in treatment planning (Herren et al., 1973Go; Rudge, 1982Go; Bhatia and Harrison, 1987Go; Harris et al., 1987Go; Richmond, 1987Go; Schirmer and Wiltshire, 1997Go). One of these is the Royal London Space Analysis (RLSA) (Kirschen et al., 2000aGo,bGo). This was one of the first analyses to include measurements for most of the components of a malocclusion including: crowding, spacing, space gained or required from arch width changes, antero-posterior change in incisor position, and angulation and inclination of the maxillary incisors. The first section of the analysis is followed by summing these measurements to produce a total space requirement for the upper and the lower arches. These measurements are then integrated with the space requirements of possible treatment mechanics such as tooth reduction or enlargement, extraction, space opening for prosthetic replacement, and mesiodistal molar movement (Figure 1). The benefits of this approach are to provide consistency in treatment planning decisions and act as an aid to trainee orthodontists so that they fully appreciate space requirements and their influence on treatment decisions.


Figure 1
View larger version (8K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 1 Royal London Space Proforma. Reprinted from the American Journal of Orthodontics and Dentofacial Orthopedics (Kirschen et al., 2000aGo), Copyright 2000, with permission from Elsevier.

 
While this analysis is gaining popularity, there have been no published studies that have evaluated its reliability and validity particularly with respect to any influence on orthodontic treatment decisions. Thus, this was the aim of the present study.

The null hypotheses tested were

  1. The RLSA does not have acceptable validity or reliability.
  2. There is no effect of RLSA on orthodontic treatment decisions.


   Materials and methods
Top
 Summary
 Introduction
 Materials and methods
Results
 Discussion
 Conclusions
 References
 

Sample

Using data from previous investigations (Han et al., 1991Go; Keeling et al., 1996Go; Ribarevski et al., 1996Go; Pair et al., 2001Go), it was calculated that 31 sets of patient records were required to be evaluated by 17 orthodontic examiners to conduct a study with an alpha level of 0.05 and beta value of 0.1. This would give sufficient power to detect a 3-mm change in the scores of RLSA and a 15 per cent change in the overall treatment decisions for each examiner. The 17 examiners included 10 orthodontic specialist and seven postgraduate students in the first and second years of their speciality programme.


Materials

Thirty-one sets of records comprising pre-treatment study models, coloured facial and intra-oral photographs, dental pantomograms, lateral cephalograms, and their tracings were made available. The cases were selected so that the sample was representative of different types of malocclusion (Table 1) and different levels of crowding severity according to Proffit (2000)Go.


View this table:
[in this window]
[in a new window]

  		Table 1 The characteristics of the records in the study.

 

Procedure

This study was carried out in three main stages as shown in Figure 2. The 17 participants were asked to examine the 31 sets of records and complete a data sheet that recorded information on their treatment decision, extraction pattern, and the anchorage method used (Figure 3). One month later, the examiners attended a course on the RLSA given by a clinician who routinely used the analysis (JS). The examiners then applied the space analysis to the 31 sets of study casts and recorded their findings. Finally, after a further one-month period, they reapplied the RLSA to the same study casts and repeated their treatment decision for each set of patient records.


Figure 2
View larger version (13K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 2 Flow chart illustrating the procedure and analysis used.

 

Figure 3
View larger version (10K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Figure 3 The data sheet used in the current investigation to record the examiners’ treatment decisions.

 

Statistical analysis

The data were checked for normality and simple summary statistics were produced using the Statistical Package for Social Sciences version 11.5 (SPSS Inc., Chicago, Illinois, USA) and Stata version 8 (Stata Corporation, College Station, Texas, USA) statistical software programs.

The agreement of each examiner’s scores with the gold standard scores of the expert was calculated using a paired sample t-test. Inter- and intra-rater reliability in scoring RLSA was evaluated with the intraclass correlation coefficient (ICC) and the paired Student’s t-test. The intra-reliability of the treatment decision was evaluated with kappa ({kappa}) statistics. An overall {kappa} value across all examiners was also computed.

Finally, so that any effect of the application of the analysis on the treatment decisions of the examiners could be measured, it was decided that the examiner’s decision was most likely to be influenced by the upper and lower arch space requirements. These requirements were divided into high (when the space required was 8 mm or more) and low (when it was less than 8 mm). The treatment decisions were then treated as a dependent variable and the effect of space requirements and possession of an orthodontic qualification were evaluated using a binary logistic regression. The analysis was run for each examiner.


   Results
Top
 Summary
 Introduction
 Materials and methods
 Results
Discussion
 Conclusions
 References
 
The agreement of the examiners with the gold standard is shown in Table 2. The correlation ranged between 0.96 (maximum 0.98, minimum 0.93) for lower arch crowding and 0.79 (maximum 0.90, minimum 0.57) for total upper space requirement.


View this table:
[in this window]
[in a new window]

  		Table 2 Agreement of the examiners with the ‘gold standard’ using paired samples t-test correlations.

 
The inter- and intra-examiner agreement when using the space analysis, represented by the ICC, is shown in Table 3. This reveals that inter-examiner agreement ranged from 0.93 for the assessment of lower arch crowding to 0.77 for total upper space requirement. When intra-examiner agreement was considered, this ranged from 0.93 for lower arch crowding to 0.68 for total upper space requirement.


View this table:
[in this window]
[in a new window]

  		Table 3 Inter- and intra-rater coefficients of agreement for scoring Royal London Space Analysis among the 17 examiners as represented by the intraclass correlation coefficient (ICC).

 
The data on intra-examiner reliability of the examiners’ treatment decisions before and after application of the space analysis are shown in Table 4. The average {kappa} value for all examiners was 0.52 (0.24–0.82) representing moderate agreement.


View this table:
[in this window]
[in a new window]

  		Table 4 Intra-rater reliability of treatment decisions made by the 17 examiners as represented by kappa.

 
The impact of the RLSA on the examiners’ treatment decisions was measured using regression analysis. This showed that the RLSA did not systematically influence their decisions in the use of headgear, anchorage reinforcement, extractions, or functional appliances. It did, however, indicate that some individual examiners were influenced for some decisions but this was not consistent, neither was it explained by whether the individual possessed an orthodontic qualification.


   Discussion
Top
 Summary
 Introduction
 Materials and methods
 Results
 Discussion
Conclusions
 References
 
The results of this study reveal that the RLSA had consistent reliability both between and within examiners. It was found that its use did not influence the relevant treatment decision of the orthodontists when applied as part of the treatment planning process. Nevertheless, the level of calibration against the gold standard was high and suggests that it is possible to easily apply and teach the RLSA.


Scoring and measurements

It is well established that there is an inherent error in the application of all orthodontic clinical measurements, and the error varies depending on the feature being measured. For example, it is not surprising that the assessment of crowding, which is carried out by direct measurement, is accurate (Schirmer and Wiltshire, 1997Go). However, other measurements such as estimating the curve of Spee, assessment of space gained by arch expansion, space required for incisors anterior/posterior position change, and angulation/inclination change were less accurate as they rely on the examiner’s judgement which is based on their own experience and knowledge.


Inter-examiner reliability in scoring the RLSA

When these data were evaluated in depth, it was found that scoring lower arch crowding had the highest inter-examiner agreement whereas the lowest agreement was for upper arch space requirement. These results are in agreement with those reported by Pair et al. (2001)Go where 30 orthodontists were asked to assess different aspects of malocclusion from the study casts of 10 patients. Among these variables, they assessed the reliability of scoring maxillary and mandibular arch crowding, which was 89 and 93 per cent, respectively.


Intra-examiner reliability in scoring the RLSA

In most orthodontic studies, intra-examiner agreement has been shown to be generally high (Keeling et al., 1996Go; Luke et al., 1998Go; Pair et al., 2001Go). This was the case in the present investigation, particularly for the assessment of lower arch crowding and lower space requirement. It also appears that reliability was lower for the total compared with crowding as a single variable, and this was due to accumulation of the error in scoring these variables when summing the total.


Intra-examiner reliability in treatment decisions

The main aim of the application of a space analysis is to provide information that may be used as a guide to treatment decisions. It was postulated that the application of the RLSA would lead to a change in the examiners’ decisions, following the addition of new information, and that this would be consistent. This was, however, not the case since their decisions taken without the analysis did not substantially change following its application.

Previous investigations have shown that orthodontic treatment decisions are somewhat unreliable and the results of the present study confirm this finding. An example is the research by Han et al. (1991)Go which looked at the consistency of orthodontic treatment decisions relative to diagnostic records. In that study, five orthodontists examined the records of 15 Class II division 1 patients twice with a 4 to 6 week interval. The intra-examiner reliability ranged from 53 to 73 per cent, with an average of 65 per cent.

In a similar study, Ribarevski et al. (1996)Go asked 10 orthodontists to evaluate the records of 60 Class II division 1 patients twice with a 1 month interval. The average {kappa} score for intra-examiner agreement was 0.69 (maximum 0.96, minimum 0.54). The two studies may not be strictly comparable because the investigation was confined to subjects with Class II division 1 malocclusions which may have resulted in less variation in treatment decisions.


The impact of RLSA on treatment decisions

The results of this study revealed that the use of the RLSA did not influence most of the orthodontists’ decisions. It appears that the assessment of space requirements alone does not influence the treatment decision in a simple linear relationship. This may be explained by considering that decision making and treatment planning are complex procedures involving several factors which are both external and personal to the clinician. This has been outlined by Kay and Nuttall (1995aGo,b)Go who stated that ‘sources of variation in clinical decisions include the clinician’s perception of the probability of success of treatment, patient–dentist relationship, patient attendance, risk–benefit ratio, dentist’s and patient’s values of dental health care, dentist’s personal treatment threshold, and finally patient’s financial ability’.

The process of treatment planning is also influenced by innate variability of the clinician that arises from two main sources. These are perceptual and judgemental variations. Perceptual variation is when people ‘see’ things differently. This is influenced by past experiences and factors such as poor lighting, whereas, judgemental variation is when a clinician decides on different treatment plans based on their personal opinion, experience, and the evidence base that underpin treatment options.

The results in this current investigation showed good reliability in scoring all components of the RLSA. Thus, the panel in this study agreed on what they saw and there was minimal perceptual variation. As a result, judgemental variation was likely to be the main source of the disagreement in their treatment decision.

In the present study, the patient was assumed to be ‘the ideal patient’ who is highly co-operative, with excellent oral hygiene and no relevant medical history. This assumption eliminated the judgemental variations produced from patient factors and left those related to the examiner and the surrounding environment as the main source of disagreement between the examiners. These factors may possibly include examiner knowledge and his/her basic training followed by personal experience.

The process of treatment planning is far from straightforward taking into consideration that orthodontic treatment is characterized by an almost complete lack of high-level evidence for the effectiveness of various treatment methods. In this respect, it is not surprising that there was such a marked variation between the orthodontists in the current study despite good agreement in the application of the RLSA. There is no doubt that attempts should be made to reduce this variation mainly by increasing the amount of research that is carried out into the effects of orthodontic treatment.


   Conclusions
Top
 Summary
 Introduction
 Materials and methods
 Results
 Discussion
 Conclusions
References
 
The following conclusions may be drawn from this study:

  1. The RLSA had good reliability.
  2. There was no impact of the use of the RLSA on the treatment decision-making process.
  3. The use of RLSA might have reduced the perceptual variability among orthodontists but had no effect on the judgemental variation, which was the main source of disagreement in the process of orthodontic treatment decision making.


   Acknowledgement
 
The authors thank the orthodontic consultants who participated in the study: D. Bearn, S. Chadwick, S. Caldwell, N. Mandall, M. Read, D. Lewis, P. Banks, and C. Low. In addition, we are grateful to the postgraduate orthodontic students who took part in the study. Finally, we would like to thank Professor H. Worthington for the statistical advice.


   References
Top
 Summary
 Introduction
 Materials and methods
 Results
 Discussion
 Conclusions
 References
 

    Bhatia SN, Harrison VE. Operational performance of the travelling microscope in the measurement of dental casts. British Journal of Orthodontics (1987) 14:147–153.[Abstract]

    Han UK, Vig KW, Weintraub JA, Vig PS, Kowalski CJ. Consistency of orthodontic treatment decisions relative to diagnostic records. American Journal of Orthodontics and Dentofacial Orthopedics (1991) 100:212–219.[Web of Science][Medline]

    Harris EF, Vaden JL, Williams RA. Lower incisor space analysis: a contrast of methods. American Journal of Orthodontics and Dentofacial Orthopedics (1987) 92:375–380.[CrossRef][Web of Science][Medline]

    Herren P, Schmoker R, Jordi T. Arch shape and space balance determined by arcogramme technique. Transactions of the European Orthodontic Society (1973) 61–73.

    Kay E, Nuttall N. Clinical decision making—an art or a science? Part II: making sense of treatment decisions. British Dental Journal (1995a) 178:113–116.[CrossRef][Web of Science][Medline]

    Kay E, Nuttall N. Clinical decision making—an art or a science? Part V: patient preferences and their influence on decision making. British Dental Journal (1995b) 178:229–233.[CrossRef][Web of Science][Medline]

    Keeling SD, McGorray S, Wheeler TT, King GJ. Imprecision in orthodontic diagnosis: reliability of clinical measures of malocclusion. Angle Orthodontist (1996) 66:381–391.[Web of Science][Medline]

    Kirschen RH, O’Higgins EA, Lee RT. The Royal London Space Planning: an integration of space analysis and treatment planning: part I: assessing the space required to meet treatment objectives. American Journal of Orthodontics and Dentofacial Orthopedics (2000a) 118:448–455.[CrossRef][Web of Science][Medline]

    Kirschen RH, O’Higgins EA, Lee RT. The Royal London Space Planning: an integration of space analysis and treatment planning: part II: the effect of other treatment procedures on space. American Journal of Orthodontics and Dentofacial Orthopedics (2000b) 118:456–461.[CrossRef][Web of Science][Medline]

    Luke LS, Atchison KA, White SC. Consistency of patient classification in orthodontic diagnosis and treatment planning. Angle Orthodontist (1998) 68:513–520.[Web of Science][Medline]

    Pair JW, Luke L, White S, Atchinson K, Englehart R, Brennan R. Variability of study cast assessment among orthodontists. American Journal of Orthodontics and Dentofacial Orthopedics (2001) 120:629–638.[CrossRef][Web of Science][Medline]

    Proffit WR. Contemporary orthodontics. (2000) St Louis: Mosby.

    Ribarevski R, Vig P, Vig KD, Weyant R, O’Brien K. Consistency of orthodontic extraction decisions. European Journal of Orthodontics (1996) 18:77–80.[Abstract/Free Full Text]

    Richmond S. Recording the dental cast in three dimensions. American Journal of Orthodontics and Dentofacial Orthopedics (1987) 92:199–206.[CrossRef][Web of Science][Medline]

    Rudge SJ. A computer program for the analysis of study models. European Journal of Orthodontics (1982) 4:269–273.[Abstract/Free Full Text]

    Schirmer UR, Wiltshire WA. Manual and computer-aided space analysis: a comparative study. American Journal of Orthodontics and Dentofacial Orthopedics (1997) 112:676–680.[CrossRef][Web of Science][Medline]


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
30/5/503    most recent
cjn007v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Al-Abdallah, M.
Right arrow Articles by O'Brien, K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Al-Abdallah, M.
Right arrow Articles by O'Brien, K.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?