Do Motion Metrics Lead to Improved Skill Acquisition on Simulators?

NCT ID: NCT01052168

Last Updated: 2022-04-27

Basic Information

• Recruitment Status: COMPLETED
• Clinical Phase: NA
• Total Enrollment: 42 participants
• Study Classification: INTERVENTIONAL
• Study Start Date: 2009-11-30
• Study Completion Date: 2011-12-31

Brief Summary

Emphasizing the growing popularity of motion metrics are the majority of available virtual reality simulators and some newer hybrid models that offer motion tracking for performance assessment. A popular hybrid model (PROMIS) allows training with regular laparoscopic instruments in a box-trainer while automatically recording task duration and movement efficiency (pathlength and smoothness) that are immediately offered as feedback to trainees.

Despite the increasing availability of simulators that track motion, our knowledge of the impact those metrics have on trainee learning is severely limited. We do not know if it is more important to use speed, accuracy, motion efficiency or a combination thereof for performance assessment and how these metrics impact skill transfer to the OR.

Based on sound educational principles we have developed a proficiency-based laparoscopic suturing simulator curriculum. This curriculum focuses on deliberate and distributed practice, provides trainees with augmented feedback and sets expert-derived performance goals based on time and errors. We have previously demonstrated that this curriculum leads to improved operative performance of trainees compared to controls.

To measure operative performance and determine transferability, we will use a live porcine Nissen fundoplication model. Instead of placing actual patients at risk, the porcine model is preferable for this purpose as it offers objective metrics (targets are established, distances measured, knots are disrupted for slippage scoring), complete standardization, and allows multiple individuals to be tested on the same day.

We hypothesize that proficiency-based simulator training in laparoscopic suturing to expert-derived levels of speed and motion will result in better operative performance compared to participants training to levels of speed or motion alone. The study is powered to detect an at least 10% performance difference between the groups.

Specific Aims

1. Compare whether any performance differences between the groups persist long-term

2. Assess whether the groups demonstrate differences in safety in the operating room by comparing the inadvertent injuries in the animal OR between the groups

3. Identify the training duration required by novices to reach proficiency in laparoscopic suturing based on speed, motion efficiency, or a combination of these metrics

4. Identify any baseline participant characteristics that may predict individual metric-specific performance

Detailed Description

OBJECTIVE:: We hypothesized that training to expert-derived levels of speed and motion will lead to improved learning and will translate to better operating room (OR) performance of novices than training to goals of speed or motion alone. BACKGROUND:: Motion tracking has been suggested to be a more sensitive performance metric than time and errors for the assessment of surgical performance. METHODS:: An institutional review board-approved, single blinded, randomized controlled trial was conducted at our level-I American College of Surgeons accredited Education Institute. Forty-two novices trained to proficiency in laparoscopic suturing after being randomized into 3 groups: The speed group (n = 14) had to achieve expert levels of speed, the motion group (n = 15) expert levels of motion (path length and smoothness), and the speed and motion group (n = 13) both levels. To achieve proficiency, all groups also had to demonstrate error-free performance. The FLS suture module (task 5) was used for training inside the ProMIS simulator that tracks instrument motion. All groups participated in transfer and retention tests in the OR. OR performance was assessed by a blinded expert rater using Global Operative Assessment of Laparoscopic Skills, speed, accuracy, and inadvertent injuries.

Conditions

Performance Assessment; Motion Metrics

Study Design

• Allocation Method: RANDOMIZED
• Intervention Model: PARALLEL
• Primary Study Purpose: BASIC_SCIENCE
• Blinding Strategy: SINGLE

Study Groups

Speed Group

The Speed Group, (n=20) will train in laparoscopic suturing on the validated FLS suturing model until the expert level of speed (i.e. task duration \< 70 seconds) has been achieved on two consecutive attempts.

Group Type: ACTIVE_COMPARATOR

skills training

Intervention Type: OTHER

participants will train using different performance goals (based on different metrics)

Motion Group

The Motion Group, (n=20) will train in laparoscopic suturing until expert levels of motion (pathlength 6700 and smoothness 560) have been achieved.

Group Type: EXPERIMENTAL

skills training

Intervention Type: OTHER

participants will train using different performance goals (based on different metrics)

Speed and Motion Group

The Speed and Motion Group (n=20) will train in laparoscopic suturing until expert levels of speed AND motion have been achieved.

Group Type: EXPERIMENTAL

skills training

Intervention Type: OTHER

participants will train using different performance goals (based on different metrics)

Interventions

skills training

participants will train using different performance goals (based on different metrics)

Intervention Type: OTHER

Eligibility Criteria

Inclusion Criteria

• novices with no previous laparoscopic or simulation experience
• voluntary participation

Exclusion Criteria

• expert in or familiarity with laparoscopy or simulation
• physical condition that prevents the performance of laparoscopic suturing

• Eligible Sex: ALL
• Accepts Healthy Volunteers: Yes

Sponsors

Tulane University

OTHER

Sponsor Role: collaborator

Ethicon Endo-Surgery

INDUSTRY

Sponsor Role: collaborator

Wake Forest University Health Sciences

OTHER

Sponsor Role: lead

Responsible Party

Responsibility Role: SPONSOR

Principal Investigators

Dimitrios Stefanidis, MD, PhD

Role: PRINCIPAL_INVESTIGATOR

Carolinas Simulation Center

Locations

Carolinas Simulation Center

Charlotte, North Carolina, United States

Countries

United States

References

Southern Surgeons Club. A prospective analysis of 1518 laparoscopic cholecystectomies. N Engl J Med. 1991 Apr 18;324(16):1073-8. doi: 10.1056/NEJM199104183241601.

Reference Type: BACKGROUND

PMID: 1826143 (View on PubMed)

Moore MJ, Bennett CL. The learning curve for laparoscopic cholecystectomy. The Southern Surgeons Club. Am J Surg. 1995 Jul;170(1):55-9. doi: 10.1016/s0002-9610(99)80252-9.

Reference Type: BACKGROUND

PMID: 7793496 (View on PubMed)

Institute of Medicine (US) Committee on Quality of Health Care in America; Kohn LT, Corrigan JM, Donaldson MS, editors. To Err is Human: Building a Safer Health System. Washington (DC): National Academies Press (US); 2000. Available from http://www.ncbi.nlm.nih.gov/books/NBK225182/

Reference Type: BACKGROUND

PMID: 25077248 (View on PubMed)

Hasan A, Pozzi M, Hamilton JR. New surgical procedures: can we minimise the learning curve? BMJ. 2000 Jan 15;320(7228):171-3. doi: 10.1136/bmj.320.7228.171. No abstract available.

Reference Type: BACKGROUND

PMID: 10634741 (View on PubMed)

Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999 Jan;177(1):28-32. doi: 10.1016/s0002-9610(98)00289-x.

Reference Type: BACKGROUND

PMID: 10037304 (View on PubMed)

Callery MP. Expansion beyond compression. Surg Endosc. 2003 May;17(5):677-8. doi: 10.1007/s00464-003-0017-6. Epub 2003 Apr 3. No abstract available.

Reference Type: BACKGROUND

PMID: 12669229 (View on PubMed)

Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005 Jul;201(1):23-9. doi: 10.1016/j.jamcollsurg.2005.02.021.

Reference Type: BACKGROUND

PMID: 15978440 (View on PubMed)

Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus DM, Jeyarajah DR, Thompson WM, Jones DB. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg. 2000 Sep;191(3):272-83. doi: 10.1016/s1072-7515(00)00339-2.

Reference Type: BACKGROUND

PMID: 10989902 (View on PubMed)

Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, Tesfay ST, Jones DB. Improving operative performance using a laparoscopic hernia simulator. Am J Surg. 2001 Dec;182(6):725-8. doi: 10.1016/s0002-9610(01)00800-5.

Reference Type: BACKGROUND

PMID: 11839347 (View on PubMed)

Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002 Oct;236(4):458-63; discussion 463-4. doi: 10.1097/00000658-200210000-00008.

Reference Type: BACKGROUND

PMID: 12368674 (View on PubMed)

Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004 Feb;91(2):146-50. doi: 10.1002/bjs.4407.

Reference Type: BACKGROUND

PMID: 14760660 (View on PubMed)

Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K; SAGES FLS Committee. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004 Jan;135(1):21-7. doi: 10.1016/s0039-6060(03)00156-9. No abstract available.

Reference Type: BACKGROUND

PMID: 14694297 (View on PubMed)

Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004 Sep;240(3):518-25; discussion 525-8. doi: 10.1097/01.sla.0000136941.46529.56.

Reference Type: BACKGROUND

PMID: 15319723 (View on PubMed)

Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg. 2001 Nov;193(5):479-85. doi: 10.1016/s1072-7515(01)01041-9.

Reference Type: BACKGROUND

PMID: 11708503 (View on PubMed)

Van Sickle KR, McClusky DA 3rd, Gallagher AG, Smith CD. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc. 2005 Sep;19(9):1227-31. doi: 10.1007/s00464-004-8274-6. Epub 2005 Jul 21.

Reference Type: BACKGROUND

PMID: 16025195 (View on PubMed)

Smith WD, Berguer R. A simple virtual instrument to monitor surgeons' workload while they perform minimally invasive surgery tasks. Stud Health Technol Inform. 2004;98:363-9.

Reference Type: BACKGROUND

PMID: 15544306 (View on PubMed)

Sierra R, Korndorffer Jr.JR, Stefanidis D, Touchard CL, Dunne JB, Scott DJ. Proficiency-based training: a new standard for laparoscopic simulation. Presented at the 2005 annual SAGES meeting in Hollywood, Fl.

Reference Type: BACKGROUND

Stefanidis D, Korndorffer JR Jr, Sierra R, Touchard C, Dunne JB, Scott DJ. Skill retention following proficiency-based laparoscopic simulator training. Surgery. 2005 Aug;138(2):165-70. doi: 10.1016/j.surg.2005.06.002.

Reference Type: BACKGROUND

PMID: 16153423 (View on PubMed)

Stefanidis D, Korndorffer JR Jr, Markley S, Sierra R, Scott DJ. Proficiency maintenance: impact of ongoing simulator training on laparoscopic skill retention. J Am Coll Surg. 2006 Apr;202(4):599-603. doi: 10.1016/j.jamcollsurg.2005.12.018.

Reference Type: BACKGROUND

PMID: 16571429 (View on PubMed)

PROMIS surgical simulator.Web: http://www.haptica.com/. Accessed: 12/06/2007

Reference Type: BACKGROUND

Allen JW, Rivas H, Cocchione RN, Ferzli GS. Intracorporeal suturing and knot tying broadens the clinical applicability of laparoscopy. JSLS. 2003 Apr-Jun;7(2):137-40.

Reference Type: BACKGROUND

PMID: 12856844 (View on PubMed)

Ward M, MacRae H, Schlachta C, Mamazza J, Poulin E, Reznick R, Regehr G. Resident self-assessment of operative performance. Am J Surg. 2003 Jun;185(6):521-4. doi: 10.1016/s0002-9610(03)00069-2.

Reference Type: BACKGROUND

PMID: 12781878 (View on PubMed)

Wolfe BM, Szabo Z, Moran ME, Chan P, Hunter JG. Training for minimally invasive surgery. Need for surgical skills. Surg Endosc. 1993 Mar-Apr;7(2):93-5. doi: 10.1007/BF00704386.

Reference Type: BACKGROUND

PMID: 8456377 (View on PubMed)

Ericsson KA, Krampe RT, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993; 100:363-406.

Reference Type: BACKGROUND

Ericsson KA, Charness N. Expert performance: its structure and acquisition. Am Psychol 1994; 49:725-47.

Reference Type: BACKGROUND

Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004 Oct;79(10 Suppl):S70-81. doi: 10.1097/00001888-200410001-00022. No abstract available.

Reference Type: BACKGROUND

PMID: 15383395 (View on PubMed)

Schmidt RA, Lee TD. Motor control and learning: a behavioral emphasis. Champaign, IL. Human Kinetics Publishers; 2005.

Reference Type: BACKGROUND

Magill R. Motor Learning and Control: Concepts and Applications. New York, NY: Mc Graw Hill; 2004.

Reference Type: BACKGROUND

Risucci D, Cohen JA, Garbus JE, Goldstein M, Cohen MG. The effects of practice and instruction on speed and accuracy during resident acquisition of simulated laparoscopic skills. Curr Surg. 2001 Mar;58(2):230-235. doi: 10.1016/s0149-7944(00)00425-6.

Reference Type: BACKGROUND

PMID: 11275252 (View on PubMed)

Wulf G, McNevin N, Shea CH. The automaticity of complex motor skill learning as a function of attentional focus. Q J Exp Psychol A. 2001 Nov;54(4):1143-54. doi: 10.1080/713756012.

Reference Type: BACKGROUND

PMID: 11765737 (View on PubMed)

Fried GM, Derossis AM, Bothwell J, Sigman HH. Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator. Surg Endosc. 1999 Nov;13(11):1077-81; discussion 1082. doi: 10.1007/s004649901176.

Reference Type: BACKGROUND

PMID: 10556441 (View on PubMed)

Wickens CD, Hollands JG. Engineering psychology and human performance, 3rd Ed., Upper Saddle River, NJ: Prentice Hall; 2000.

Reference Type: BACKGROUND

Hart SG, Staveland L.E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock PA, Meshkati N (eds). Human Mental Workload. Amsterdam: Elsevier; 1987

Reference Type: BACKGROUND

Noether, Gottfried E., "Sample Size Determination for Some Common Nonparametric Tests". Journal of the American Statistical Association 1987; Vol. 82, No. 398, p. 647.

Reference Type: BACKGROUND

Stefanidis D, Yonce TC, Korndorffer JR Jr, Phillips R, Coker A. Does the incorporation of motion metrics into the existing FLS metrics lead to improved skill acquisition on simulators? A single blinded, randomized controlled trial. Ann Surg. 2013 Jul;258(1):46-52. doi: 10.1097/SLA.0b013e318285f531.

Reference Type: DERIVED

PMID: 23470570 (View on PubMed)

Other Identifiers

11-06-20E

Identifier Type: -

Identifier Source: org_study_id