Inter-rater Reliability of Diagnosing Thoracic Outlet Syndrome by Physiotherapists

NCT ID: NCT05252104

Last Updated: 2022-02-23

Basic Information

• Recruitment Status: UNKNOWN
• Total Enrollment: 30 participants
• Study Classification: OBSERVATIONAL
• Study Start Date: 2022-04-01
• Study Completion Date: 2022-08-20

Brief Summary

Thoracic outlet syndrome (TOS) denotes the collection of symptoms which may arise from compression of the neurovascular structures in the region of the brachial plexus. TOS has historically been a clinical enigma, with lack of consensus regarding its diagnosis limiting the validity of any research into it. Literature and local audit both note significant patient morbidity and redundant use of secondary care clinics and investigations in sufferers. The last decade has seen the creation of a consortium of leaders in the field and development of the CORE-TOS diagnostic tool. This tool has 5 subsets of clinical diagnostic criteria (CDC). Positivity in 4 or more suggests a diagnosis of TOS.

The current study seeks to specifically examine whether physiotherapy clinicians - both in primary and secondary care - can consistently identify cases of TOS using the CORE-TOS tool and refer them appropriately to an extended scope physiotherapist (ESP) specialising in the condition. Specific education will be provided to relevant physiotherapists who will be asked to note all relevant CDC in any suspected cases thereafter referred to the ESP in an out-patient physiotherapy department setting. These referrals will follow the standard local pathway to physiotherapy and no clinical testing manoeuvres out with the current scope of physiotherapy will be applied. The patients' case notes will thereafter be retrospectively examined, and the inter-rater reliability of the CDC recorded by the both the referring physiotherapist on their referral and the researcher at initial review. This will analysed using intraclass correlation coefficient, SEM and Bland and Altman's agreement tests, coupled with descriptive analysis.

Detailed Description

The principal method of the study will involve retrospective analysis of data prospectively recorded in patients' clinical notes. The data will pertain to clinical information recorded by both the physiotherapist referring the patient, and the principal investigator (P.I.) at their initial assessment of the patient. Patients will be selected from within the specific timeframe of the calendar year between April 2021 and April 2022.

At the first appointment with each rater patients will have undergone subjective and objective clinical examination within the current scope of physiotherapy practice. All relevant clinical CORE-TOS clinical diagnostic criteria (CDC) noted by each on examination will have been recorded, up to a maximum of 5.

Data Collection:

Pending ethical clearance from Queen Margaret University Edinburgh, collation and analysis of the data will take place in April & May 2022 (i.e., one calendar year following the final education session). Within the allocated resources of the study all data collection and analysis will be undertaken by the P.I. only and thus no blinding will be possible. A retrospective review of the P.I.s patient files will be undertaken to collate the relevant patient files. These are hand-written, paper files stored locally within the target physiotherapy department. An Excel® (Microsoft®) spreadsheet will be used to generate a database for the study. This will be stored on the local health service, password-protected server behind the researcher's unique login. Patients will be anonymised on the worklist to a random number using the random number generator function in Excel® and this number written into their case notes, along with the protocol identifier and the date the case note was reviewed. The purpose of this will be twofold. The first of these is for local data protection requirements. The second is to allow for any future consistency checking of the data, given that no blinding of the results will be possible within the study design.

Each CORE-TOS CDC noted by both the referrer (rater 1) and the principal investigator (rater 2) will be recorded on an Excel® spreadsheet. The worksheet will allow for both future quantification of CDC out of a possible 5 and to assist identification of any consistent errors or omissions. For the purposes of descriptive analysis, supplementary data will also be collected all of which will be personally non-identifiable in keeping with ethical and data protection agreements. These will include age, sex, initial quickDASH and SF-12 scores, the length of time symptoms were present prior to review by the researcher, the presence/absence of antecedent trauma, previous imaging and any other clinical sub-specialities seen for this presentation. Given the inability to blind the study, this Excel® file will be stored securely on the local NHS server, behind the researcher's password-protected login for 12 months to aid consistency checking, if required. In addition, it will also be used to inform an intended future study on the outcomes of local treatment for TOS and the optimal pathway for management of it.

Data analysis:

The patients used in this study will be extracted from the population of patients presenting to GP practices, secondary care orthopaedic clinics and outpatient physiotherapy departments with neck and arm discomfort and considered a representative sample of that cohort. The number of CORE-TOS clinical diagnostic criteria (CDC) recorded by both the referring physiotherapists and the researcher at initial assessment will be used for analysis. The possible number of relevant CDC will range from 0 to 5, assume an absolute zero and the absence of numbers of findings below zero (i.e. no negative values). Given this is a continuous scale with equal weight given to each score and equal distance between adjacent scores, these units of measurement will be considered as ratio data.

The primary purpose of the study is to assess the reliability of the CORE-TOS tool. All statistical analysis of the data will be carried out by the primary investigator using a combination of Microsoft® EXCEL® and the IBM® SPSS® software platforms. Analysis of correlation will be applied in the form of an intra class correlation co-efficient (ICC). A consideration in the use of ICCs lies in the fact that numerous models exist and application of the same data to each model can give different results. All ICCs use ANOVA in their calculations and the type of ANOVA model is one of three factors that must be considered in ensuring the appropriate ICC is chosen. The other two are the type of measurement and definition of the relationship considered to be important between measures.

The current study presents a model where the second rater is fixed for all patients, thought the first rater (the referrer) will be different each time. The presentation of a patient with TOS to any of these referring clinicians is random, as is whomever these patients first encounter (i.e., the referring clinician). The referring clinicians are not pre-chosen in this study and will thus be considered a random sample of the population of all potential referrers. In addition, each patient will have their own unique combination of CDC on the CORE-TOS tool. In this regard both the referring physiotherapist and the patient will be considered as sources of random error. Given this and the aim to generalise the results to all potential referrers a two-way random effects model of ANOVA will be used. In considering the type of measures, given that this study ultimately seeks to look at the use of the CORE-TOS in practice as a single measure to initiate a clinical action (i.e., referral to a specialist pathway) a single measure ICC will be used. For the purposes of the study the ability of each referring clinician to clinically assess a patient will be considered equal, irrespective of job title or level of experience. As the study is concerned with the extent to which the score of the referrer and researcher agrees, the definition of absolute agreement will be chosen. This study will use the ICC nomenclature convention of 2 numbers in parentheses and based on the above, the ICC model that will be used is ICC(2,1). This will be formally stated will along with the ANOVA model, type of measurement (in this case single) and definition of relationship in the analysis section of the study along with the ICC and its 95% confidence interval.

It is suggest that for an ICC to be effective, it must be estimated from data having at least 25 degrees of freedom (i.e. the number of measurements times the number of raters). In the case of the current study, the number of CDC per patient will be considered a single measure and thus a minimum of 25 patients will be required. As with all reliability co-efficients, the potential level of reliability will range from 0 to 1. An ICC result in this study of <0.75 will be considered fair to poor, >0.75 considered good, and >0.9 reasonable for clinical measurements .

It is accepted that using the result of an ICC in isolation is insufficient for clinical interpretation, as it gives no indication of the variance both between measurements on the same subject and measurements between subjects. It is universally recommended to supplement ICCs with supplementary tests. To estimate agreement between measurements a standard error of measurement (SEM) will be used. In addition, a Bland and Altman Agreement test (BAAT) will be plotted to graphically present the variance in the data. The raw data will be extracted from EXCEL, which will also be used for the analysis. The SEM will be calculated from the following formula:

SEM = SD (√1-ICC)

where SD equals the sample standard deviation and ICC is the calculated ICC score.

The BAAT will be plotted as follows. The differences between CORE-TOS scores will be calculated for each patient and plotted against the average score between raters per patient. As the BAAT seeks to identify how many of these plotted values fall within 2 standard deviations of the mean difference between scores, a line representing 2 SD above and below the mean will be added to the graph. This will be used to give a visual reference of agreement and to identify any outliers or specific areas of bias. The BAAT will be calculated on the entire sample of referrers irrespective of their type of job (i.e. ESP vs GP APP) or experience and this factor taken into consideration in the final discussion. In addition Bland and Altman plots will also be produced to subdivide the analysis into referring physiotherapy specialty to analyse for any specialty error or bias.

Descriptive analysis will be applied to the demographics of the patients in the study. These will be tabulated and include age, sex, presence/absence of trauma at onset, previous treatments and initial outcome measure scores of disability and quality of life (quickDASH and SF-12). The purpose of this is twofold. The first of these is to allow extrapolation of the study sample to the known literature regarding thoracic outlet syndrome (i.e., if the demographics significantly differ, the sample may not be representative). Secondly, the demographics and outcome measure scores will provide some measure of the local incidence and morbidity associated with TOS. These elements will inform both an intended future study on the outcomes of the researcher's physiotherapy intervention for this cohort and a business case to develop more optimal clinical pathways for TOS in the local health board.

Validity of the CORE-TOS Tool In addition to the testing of reliability and agreement of the number of recorded measures from the CORE-TOS tool, further descriptive statistics will be applied to evaluate the face validity of the tool itself in local practice. Within this a number of assumptions will be made. By the nature of the study design, all patients referred to the researcher will be considered by the referrer to have at least 4/5 positive CDC and thus be considered by them to be positive for TOS. For the purposes of the study, the researcher is deemed an expert in the condition, against whose opinion a direct analysis is made regarding diagnosis and (ultimately) appropriate treatment (i.e., they ultimately decide the initial course of treatment/management based on their diagnosis). The raw data will also therefore be analysed for the number of true positives versus false positives considered by the researcher. Analysis of false negatives is outwith the design of the study. Combined with analysis for any consistent errors, inconsistencies or omissions it is hoped this may provide a measure of the sensitivity of the tool in local practice.

The data analysis, will be formally recorded and written up in concise and journalistic style as part of the XMO29 module of the Master of Research degree at QMU in accordance with their local guidance.

Conditions

Thoracic Outlet Syndrome

Study Design

• Observational Model Type: COHORT
• Study Time Perspective: RETROSPECTIVE

Interventions

CORE-TOS diagnostic tool

Collection of clinical findings from clinical examination, subdivided into 5 sub-categories. Positivity in 4 or 5 of these subcategories is considered diagnostic for thoracic outlet syndrome (TOS).

Intervention Type: DIAGNOSTIC_TEST

Eligibility Criteria

Inclusion Criteria

• Individuals initially reviewed by a physiotherapist (rater 1).
• Individuals willing to attend for physiotherapy intervention.
• Minimum 12 weeks of symptoms of neck and upper limb pain and/or paraesthesia.
• Minimum 4/5 score on the CORE-TOS tool.

Exclusion Criteria

• Previous central neurologic injury with persisting neural deficit in the upper limb
• Known history of peripheral neuropathy
• Unstable angina
• Upper limb allodynia or persistent pain sensitization
• Systemic Inflammatory conditions
• Those unwilling to attend for face to face consultation

• Minimum Eligible Age: 16 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: Yes

Sponsors

Queen Margaret University

OTHER

Sponsor Role: lead

Responsible Party

John O'Toole

Extended Scope Physiotherapy Practitioner

Responsibility Role: PRINCIPAL_INVESTIGATOR

Principal Investigators

John A O'Toole, BSc (Hons)

Role: PRINCIPAL_INVESTIGATOR

Masters Student

Central Contacts

John A O'Toole, BSc (Hons)

Role: CONTACT

95278262@qmu.ac.uk

07958584097

References

Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979 Mar;86(2):420-8. doi: 10.1037//0033-2909.86.2.420.

Reference Type: RESULT

PMID: 18839484 (View on PubMed)

Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986 Feb 8;1(8476):307-10.

Reference Type: RESULT

PMID: 2868172 (View on PubMed)

Research methods for clinical therapists: applied project design and analysis Carolyn M Hicks Research methods for clinical therapists: applied project design and analysis Churchill LIvingstone 352 Fifth edition pound22.99 9780043074301 [Formula: see text]. Nurse Res. 2010 Oct 22;18(1):89. doi: 10.7748/nr.18.1.89.s3.

Reference Type: RESULT

PMID: 27707351 (View on PubMed)

Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016 Jun;15(2):155-63. doi: 10.1016/j.jcm.2016.02.012. Epub 2016 Mar 31.

Reference Type: RESULT

PMID: 27330520 (View on PubMed)

Hallgren KA. Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutor Quant Methods Psychol. 2012;8(1):23-34. doi: 10.20982/tqmp.08.1.p023.

Reference Type: RESULT

PMID: 22833776 (View on PubMed)

Bruton, A., Conway, J.H. and Holgate, S.T. (2000) Reliability: What Is It, and How Is It Measured? Physiotherapy, 86, 94-99.

Reference Type: RESULT

Nunnally, J.C. and Bernstein, I.H. (1994) The Assessment of Reliability. Psychometric Theory, 3, 248-292.

Reference Type: RESULT

Portney, L. G., & Watkins, M. P. (2009). Foundations of clinical research: applications to practice. Upper Saddle River: Pearson/Prentice Hall.

Reference Type: RESULT

Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clin Rehabil. 1998 Jun;12(3):187-99. doi: 10.1191/026921598672178340.

Reference Type: RESULT

PMID: 9688034 (View on PubMed)

Atkinson G, Nevill AM. Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med. 1998 Oct;26(4):217-38. doi: 10.2165/00007256-199826040-00002.

Reference Type: RESULT

PMID: 9820922 (View on PubMed)

Balderman J, Holzem K, Field BJ, Bottros MM, Abuirqeba AA, Vemuri C, Thompson RW. Associations between clinical diagnostic criteria and pretreatment patient-reported outcomes measures in a prospective observational cohort of patients with neurogenic thoracic outlet syndrome. J Vasc Surg. 2017 Aug;66(2):533-544.e2. doi: 10.1016/j.jvs.2017.03.419.

Reference Type: RESULT

PMID: 28735950 (View on PubMed)

Other Identifiers

Physio/IRR/TOS_20211210_v1.0

Identifier Type: -

Identifier Source: org_study_id