Local Participatory Systems Dynamics to Increase Reach of Evidence Based Addiction and Mental Health Care

NCT ID: NCT04356274

Last Updated: 2025-06-03

Basic Information

• Recruitment Status: ENROLLING_BY_INVITATION
• Clinical Phase: NA
• Total Enrollment: 720 participants
• Study Classification: INTERVENTIONAL
• Study Start Date: 2019-02-01
• Study Completion Date: 2025-12-31

Brief Summary

The most common reasons Veterans seek VA addiction and mental health care is for help with opioid and alcohol misuse, depression and PTSD. Research evidence has established highly effective treatments that prevent relapse, overdose and suicide, but even with policy mandates, performance metrics, and electronic health records to fix the problem, these treatments may only reach 3-28% of patients. This study tests participatory business engineering methods (Participatory System Dynamics) that engage patients, providers and policy makers against the status quo approaches, such as data review, and will determine if participatory system dynamics works, why it works, and whether it can be applied in many health care settings to guarantee patient access to the highest quality care and better meet the addiction and mental health needs of Veterans and the U.S. population.

Detailed Description

The broad aim is to empower all healthcare stakeholders to provide the highest quality care to all patients. The specific aims address the complexities and tradeoffs associated with implementing evidence-based practices (EBPs) in outpatient addiction and mental health systems. There is scientific consensus about the best evidence-based psychotherapies and pharmacotherapies (EBPs) to meet the needs of patients with opioid and alcohol use disorder, PTSD and depression. However, EBP coordination over time, within and across multidisciplinary teams of providers, is complex and constantly changing. Veterans Health Administration (VA) policy mandates, national training programs, and incentivized quality measures, have been insufficient for reaching more than 3 to 28% of patients with the highest quality treatments. In fact, limited EBP reach is common in health systems and the field of implementation science seeks to address it. One routine strategy is data auditing with provider feedback (audit-and-feedback; AF), however, the impact is highly variable. As an alternative, participatory system dynamics (PSD) has been used to explain causes of complex problems in business management for 60 years. Partnering with frontline staff using PSD to determine how EBP reach emerges from local resources and constraints, and is determined by system dynamics, such as delays and feedback. The dynamics of EBP reach were formally specified in differential equation models, and tested against VA data drawn from a national VA SQL database. Using existing enterprise data to tailor model parameters to each care team. PSD models were made accessible via a 'Modeling to Learn' interface and training, during which teams safely evaluated local change scenarios via simulation to find the highest yield options for meeting Veterans' needs. PSD learning simulations produce immediate, real-time feedback to the teams who coordinate care, improving day-to-day decisions and long-term improvement plans. The study design is a two-arm, 24-site (12 sites/arm) cluster randomized trial to test the effectiveness of PSD simulation as compared to more standard team AF data review. The hypothesis is that PSD will be superior to AF for improving EBP initiation and dose (Aim 1). The investigators will test the PSD theory of change that the effect of PSD on improved EBP reach is explained by improvement in team systems thinking (Aim 2). To confirm the potential for widespread usefulness of PSD, by also testing the generalizability of PSD causal dynamics across PSD and AF arms (Aim 3). This study has the potential to inform a new paradigm, by determining what works to improve health system quality defined as EBP reach, why it works, and under what conditions. If PSD is effective, study activities will address a national priority to improve Veterans' addiction and mental health care to prevent chronic symptoms, relapse, suicide and overdose. Findings from the proposed tests of effectiveness, causality, and generality, could also catalyze future applications to make a significant public health impact across the continuum of healthcare.

Conditions

PTSD; Depression; Alcohol Use Disorder; Opioid Use Disorder

Study Design

• Allocation Method: RANDOMIZED
• Intervention Model: PARALLEL
• Primary Study Purpose: HEALTH_SERVICES_RESEARCH
• Blinding Strategy: NONE

Study Groups

Participatory System Dynamics (PSD)

12 clinics assigned to PSD

Group Type: EXPERIMENTAL

Participatory System Dynamics (PSD)

Intervention Type: OTHER

Participatory system dynamics is a facilitated health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff running simulations of clinic improvement strategies to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy.

Audit and Feedback (AF)

12 clinics assigned to AF

Group Type: EXPERIMENTAL

Audit and Feedback (AF)

Intervention Type: OTHER

Audit and feedback is a health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff reviewing clinical care team data to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy.

Interventions

Participatory System Dynamics (PSD)

Participatory system dynamics is a facilitated health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff running simulations of clinic improvement strategies to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy.

Intervention Type: OTHER

Audit and Feedback (AF)

Audit and feedback is a health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff reviewing clinical care team data to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy.

Intervention Type: OTHER

Eligibility Criteria

Inclusion Criteria

• VA divisions and community-based outpatient clinics (CBOCs) or 'clinics' from regional VA health systems
• Must be below the overall VA quality median (as assessed by the Strategic Analytics for Improvement and Learning or SAIL), which includes 3 of 8 SAIL measures associated with four evidence-based psychotherapies and three evidence-based pharmacotherapies for depression, PTSD, and opioid use disorder.

Exclusion Criteria

• clinics with less than 12 months of data in 2018
• clinics already involved in Office of Veterans Access to Care (OVACS) quality improvement program at baseline.
• clinics where the VA Cerner electronic health record (EHR) implementation rollout will occur during the project period (Veterans Integrated Services Networks (VISNs) 20, 21 ,22, and 7)
• clinics who serve less than 122 unique patients each month on average
• clinics without an onsite multidisciplinary team of mental health or addiction service providers (minimum required: 1 psychiatrist, 1 psychologist, 1 social worker onsite)

• Minimum Eligible Age: 18 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: Yes

Sponsors

Palo Alto Veterans Institute for Research

OTHER

Sponsor Role: lead

Responsible Party

Lindsey Zimmerman

Principal Investigator

Responsibility Role: PRINCIPAL_INVESTIGATOR

Principal Investigators

Lindsey E Zimmerman, PhD

Role: PRINCIPAL_INVESTIGATOR

National Center for PTSD, Dissemination & Training Division

Locations

VA Palo Alto Health Care System

Palo Alto, California, United States

Countries

United States

References

Elbogen EB, Wagner HR, Johnson SC, Kinneer P, Kang H, Vasterling JJ, Timko C, Beckham JC. Are Iraq and Afghanistan veterans using mental health services? New data from a national random-sample survey. Psychiatr Serv. 2013 Feb 1;64(2):134-41. doi: 10.1176/appi.ps.004792011.

Reference Type: BACKGROUND

PMID: 23475498 (View on PubMed)

Hermes ED, Hoff R, Rosenheck RA. Sources of the increasing number of Vietnam era veterans with a diagnosis of PTSD using VHA services. Psychiatr Serv. 2014 Jun 1;65(6):830-2. doi: 10.1176/appi.ps.201300232.

Reference Type: BACKGROUND

PMID: 24733559 (View on PubMed)

Mott JM, Grubbs KM, Sansgiry S, Fortney JC, Cully JA. Psychotherapy Utilization Among Rural and Urban Veterans From 2007 to 2010. J Rural Health. 2015 Summer;31(3):235-43. doi: 10.1111/jrh.12099. Epub 2014 Dec 3.

Reference Type: BACKGROUND

PMID: 25471067 (View on PubMed)

Hankin CS, Spiro A 3rd, Miller DR, Kazis L. Mental disorders and mental health treatment among U.S. Department of Veterans Affairs outpatients: the Veterans Health Study. Am J Psychiatry. 1999 Dec;156(12):1924-30. doi: 10.1176/ajp.156.12.1924.

Reference Type: BACKGROUND

PMID: 10588406 (View on PubMed)

Hoggatt KJ, Williams EC, Der-Martirosian C, Yano EM, Washington DL. National prevalence and correlates of alcohol misuse in women veterans. J Subst Abuse Treat. 2015 May;52:10-6. doi: 10.1016/j.jsat.2014.12.003. Epub 2014 Dec 19.

Reference Type: BACKGROUND

PMID: 25661517 (View on PubMed)

Fulton JJ, Calhoun PS, Wagner HR, Schry AR, Hair LP, Feeling N, Elbogen E, Beckham JC. The prevalence of posttraumatic stress disorder in Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) Veterans: a meta-analysis. J Anxiety Disord. 2015 Apr;31:98-107. doi: 10.1016/j.janxdis.2015.02.003. Epub 2015 Feb 19.

Reference Type: BACKGROUND

PMID: 25768399 (View on PubMed)

Karlin BE, Cross G. From the laboratory to the therapy room: National dissemination and implementation of evidence-based psychotherapies in the U.S. Department of Veterans Affairs Health Care System. Am Psychol. 2014 Jan;69(1):19-33. doi: 10.1037/a0033888. Epub 2013 Sep 2.

Reference Type: BACKGROUND

PMID: 24001035 (View on PubMed)

Karlin BE, Brown GK, Trockel M, Cunning D, Zeiss AM, Taylor CB. National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs health care system: therapist and patient-level outcomes. J Consult Clin Psychol. 2012 Oct;80(5):707-18. doi: 10.1037/a0029328. Epub 2012 Jul 23.

Reference Type: BACKGROUND

PMID: 22823859 (View on PubMed)

Ruzek JI, Karlin BE, & Zeiss AM. Implementation of Evidence-Based Psychological Treatments in the Veterans Health Administration. In: McHugh RK, Barlow DH, eds. Dissemination of evidence-based psychological treatments. New York, NY: Oxford University Press. , 2012.

Reference Type: BACKGROUND

Eftekhari A, Ruzek JI, Crowley JJ, Rosen CS, Greenbaum MA, Karlin BE. Effectiveness of national implementation of prolonged exposure therapy in Veterans Affairs care. JAMA Psychiatry. 2013 Sep;70(9):949-55. doi: 10.1001/jamapsychiatry.2013.36.

Reference Type: BACKGROUND

PMID: 23863892 (View on PubMed)

Watts BV, Shiner B, Zubkoff L, Carpenter-Song E, Ronconi JM, Coldwell CM. Implementation of evidence-based psychotherapies for posttraumatic stress disorder in VA specialty clinics. Psychiatr Serv. 2014 May 1;65(5):648-53. doi: 10.1176/appi.ps.201300176.

Reference Type: BACKGROUND

PMID: 24430622 (View on PubMed)

Department of Defense, & Department of Veterans Affairs. The management of MDD Working Group. VA/DOD clinical practice guideline for management of major depressive disorder (MDD). , 2009.

Reference Type: BACKGROUND

Department of Veterans Affairs, & Department of Defense. VA/DoD Clinical practice guideline for the management of post-traumatic stress. , 2010.

Reference Type: BACKGROUND

Department of Veterans Affairs, & Department of Defense. VA/DoD Clinical practice guideline for the management of substance use disorders. , 2009.

Reference Type: BACKGROUND

Lemke S, Boden MT, Kearney LK, Krahn DD, Neuman MJ, Schmidt EM, Trafton JA. Measurement-based management of mental health quality and access in VHA: SAIL mental health domain. Psychol Serv. 2017 Feb;14(1):1-12. doi: 10.1037/ser0000097.

Reference Type: BACKGROUND

PMID: 28134552 (View on PubMed)

Harris AH, Humphreys K, Bowe T, Kivlahan DR, Finney JW. Measuring the quality of substance use disorder treatment: evaluating the validity of the Department of Veterans Affairs continuity of care performance measure. J Subst Abuse Treat. 2009 Apr;36(3):294-305. doi: 10.1016/j.jsat.2008.05.011. Epub 2008 Oct 5.

Reference Type: BACKGROUND

PMID: 18835678 (View on PubMed)

Rosen CS, Matthieu MM, Wiltsey Stirman S, Cook JM, Landes S, Bernardy NC, Chard KM, Crowley J, Eftekhari A, Finley EP, Hamblen JL, Harik JM, Kehle-Forbes SM, Meis LA, Osei-Bonsu PE, Rodriguez AL, Ruggiero KJ, Ruzek JI, Smith BN, Trent L, Watts BV. A Review of Studies on the System-Wide Implementation of Evidence-Based Psychotherapies for Posttraumatic Stress Disorder in the Veterans Health Administration. Adm Policy Ment Health. 2016 Nov;43(6):957-977. doi: 10.1007/s10488-016-0755-0.

Reference Type: BACKGROUND

PMID: 27474040 (View on PubMed)

Seal KH, Maguen S, Cohen B, Gima KS, Metzler TJ, Ren L, Bertenthal D, Marmar CR. VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. J Trauma Stress. 2010 Feb;23(1):5-16. doi: 10.1002/jts.20493.

Reference Type: BACKGROUND

PMID: 20146392 (View on PubMed)

Mott JM, Mondragon S, Hundt NE, Beason-Smith M, Grady RH, Teng EJ. Characteristics of U.S. veterans who begin and complete prolonged exposure and cognitive processing therapy for PTSD. J Trauma Stress. 2014 Jun;27(3):265-73. doi: 10.1002/jts.21927.

Reference Type: BACKGROUND

PMID: 24948535 (View on PubMed)

Harpaz-Rotem I, Rosenheck RA. Serving those who served: retention of newly returning veterans from Iraq and Afghanistan in mental health treatment. Psychiatr Serv. 2011 Jan;62(1):22-7. doi: 10.1176/ps.62.1.pss6201_0022.

Reference Type: BACKGROUND

PMID: 21209295 (View on PubMed)

Lin LA, Bohnert AS, Ilgen MA, Pfeiffer PN, Ganoczy D, Blow FC. Outpatient Provider Contact Prior to Unintentional Opioid Overdose Among VHA Service Users. Psychiatr Serv. 2015 Nov;66(11):1149-54. doi: 10.1176/appi.ps.201400194. Epub 2015 Jul 1.

Reference Type: BACKGROUND

PMID: 26129993 (View on PubMed)

Harris AH, Bowe T, Del Re AC, Finlay AK, Oliva E, Myrick HL, Rubinsky AD. Extended release naltrexone for alcohol use disorders: quasi-experimental effects on mortality and subsequent detoxification episodes. Alcohol Clin Exp Res. 2015 Jan;39(1):79-83. doi: 10.1111/acer.12597.

Reference Type: BACKGROUND

PMID: 25623408 (View on PubMed)

Degenhardt L, Bucello C, Mathers B, Briegleb C, Ali H, Hickman M, McLaren J. Mortality among regular or dependent users of heroin and other opioids: a systematic review and meta-analysis of cohort studies. Addiction. 2011 Jan;106(1):32-51. doi: 10.1111/j.1360-0443.2010.03140.x. Epub 2010 Nov 4.

Reference Type: BACKGROUND

PMID: 21054613 (View on PubMed)

Kaplan MS, Huguet N, McFarland BH, Newsom JT. Suicide among male veterans: a prospective population-based study. J Epidemiol Community Health. 2007 Jul;61(7):619-24. doi: 10.1136/jech.2006.054346.

Reference Type: BACKGROUND

PMID: 17568055 (View on PubMed)

Desai RA, Dausey DJ, Rosenheck RA. Mental health service delivery and suicide risk: the role of individual patient and facility factors. Am J Psychiatry. 2005 Feb;162(2):311-8. doi: 10.1176/appi.ajp.162.2.311.

Reference Type: BACKGROUND

PMID: 15677596 (View on PubMed)

Gradus JL, Suvak MK, Wisco BE, Marx BP, Resick PA. Treatment of posttraumatic stress disorder reduces suicidal ideation. Depress Anxiety. 2013 Oct;30(10):1046-53. doi: 10.1002/da.22117. Epub 2013 May 1.

Reference Type: BACKGROUND

PMID: 23636925 (View on PubMed)

Hysong SJ. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care. 2009 Mar;47(3):356-63. doi: 10.1097/MLR.0b013e3181893f6b.

Reference Type: BACKGROUND

PMID: 19194332 (View on PubMed)

Foy R, Eccles MP, Jamtvedt G, Young J, Grimshaw JM, Baker R. What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Serv Res. 2005 Jul 13;5:50. doi: 10.1186/1472-6963-5-50.

Reference Type: BACKGROUND

PMID: 16011811 (View on PubMed)

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012 Jun 13;2012(6):CD000259. doi: 10.1002/14651858.CD000259.pub3.

Reference Type: BACKGROUND

PMID: 22696318 (View on PubMed)

Flottorp SA, Jamtvedt G, Gibis B, et al. Using audit and feedback to health professionals to improve the quality and safety of health care. World Health Organ. 2010.

Reference Type: BACKGROUND

Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM. No more 'business as usual' with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014 Jan 17;9:14. doi: 10.1186/1748-5908-9-14.

Reference Type: BACKGROUND

PMID: 24438584 (View on PubMed)

Colquhoun HL, Carroll K, Eva KW, Grimshaw JM, Ivers N, Michie S, Sales A, Brehaut JC. Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses. Implement Sci. 2017 Sep 29;12(1):117. doi: 10.1186/s13012-017-0646-0.

Reference Type: BACKGROUND

PMID: 28962632 (View on PubMed)

Colquhoun H, Michie S, Sales A, Ivers N, Grimshaw JM, Carroll K, Chalifoux M, Eva K, Brehaut J. Reporting and design elements of audit and feedback interventions: a secondary review. BMJ Qual Saf. 2017 Jan;26(1):54-60. doi: 10.1136/bmjqs-2015-005004. Epub 2016 Jan 25.

Reference Type: BACKGROUND

PMID: 26811541 (View on PubMed)

Landis-Lewis Z, Brehaut JC, Hochheiser H, Douglas GP, Jacobson RS. Computer-supported feedback message tailoring: theory-informed adaptation of clinical audit and feedback for learning and behavior change. Implement Sci. 2015 Jan 21;10:12. doi: 10.1186/s13012-014-0203-z.

Reference Type: BACKGROUND

PMID: 25603806 (View on PubMed)

Atkinson JA, Wells R, Page A, Dominello A, Haines M, Wilson A. Applications of system dynamics modelling to support health policy. Public Health Res Pract. 2015 Jul 9;25(3):e2531531. doi: 10.17061/phrp2531531.

Reference Type: BACKGROUND

PMID: 26243490 (View on PubMed)

Diehl E, & Sterman JD. Effects of feedback complexity on dynamic decision making. Organ Behav Hum Decis Process. 1995;62:198-215.

Reference Type: BACKGROUND

Sterman JD. Modeling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment. Manag Sci. 1989;35:321-339.

Reference Type: BACKGROUND

Cronin MA, Gonzalez C, & Sterman JD. Why don't well-educated adults understand accumulation? A challenge to researchers, educators, and citizens. Organ Behav Hum Decis Process. 2009;108:116-130.

Reference Type: BACKGROUND

Sterman JD. Learning from evidence in a complex world. Am J Public Health. 2006 Mar;96(3):505-14. doi: 10.2105/AJPH.2005.066043. Epub 2006 Jan 31.

Reference Type: BACKGROUND

PMID: 16449579 (View on PubMed)

Sterman JD. Learning in and about complex systems. Syst Dyn Rev. 1994;10:291-330.

Reference Type: BACKGROUND

Andersen DF, Vennix JA, Richardson GP, et al. Group model building: Problem structing, policy simulation and decision support. J Oper Res Soc. 2007;:691-694.

Reference Type: BACKGROUND

Vennix JAM. Group model building: facilitating team learning using system dynamics. Chichester ; New York: J. Wiley, 1996: 1-297.

Reference Type: BACKGROUND

Rouwette EAJA, Vennix JAM, & Mullekom T van. Group model building effectiveness: A review of assessment studies. Syst Dyn Rev. 2002;18:5-45.

Reference Type: BACKGROUND

Bendoly E. System dynamics understanding in projects: Information sharing, psychological safety, and performance effects. Prod Oper Manag. 2014;23:1352-1369.

Reference Type: BACKGROUND

Sterman JD. Does formal system dynamics training improve people's understanding of accumulation? Syst Dyn Rev. 2010;26:316-334.

Reference Type: BACKGROUND

Simon HA. Bounded rationality and organizational learning. Organ Sci. 1991;2:125-134.

Reference Type: BACKGROUND

Rahmandad H, Repenning N, & Sterman J. Effects of feedback delay on learning. Syst Dyn Rev. 2009;25:309-338.

Reference Type: BACKGROUND

Zimmerman L, Lounsbury DW, Rosen CS, Kimerling R, Trafton JA, Lindley SE. Participatory System Dynamics Modeling: Increasing Stakeholder Engagement and Precision to Improve Implementation Planning in Systems. Adm Policy Ment Health. 2016 Nov;43(6):834-849. doi: 10.1007/s10488-016-0754-1.

Reference Type: BACKGROUND

PMID: 27480546 (View on PubMed)

Oliva R. Model calibration as a testing strategy for system dynamics models. Eur J Oper Res. 2003;151:552-568.

Reference Type: BACKGROUND

Trafton JA, Greenberg G, Harris AH, Tavakoli S, Kearney L, McCarthy J, Blow F, Hoff R, Schohn M. VHA mental health information system: applying health information technology to monitor and facilitate implementation of VHA Uniform Mental Health Services Handbook requirements. Med Care. 2013 Mar;51(3 Suppl 1):S29-36. doi: 10.1097/MLR.0b013e31827da836.

Reference Type: BACKGROUND

PMID: 23407008 (View on PubMed)

Barlas Y. Formal aspects of model validity and validation in system dynamics. Syst Dyn Rev. 1996;12:183-210.

Reference Type: BACKGROUND

Hirsch G, Homer J, Trogdon J, Wile K, Orenstein D. Using simulation to compare 4 categories of intervention for reducing cardiovascular disease risks. Am J Public Health. 2014 Jul;104(7):1187-95. doi: 10.2105/AJPH.2013.301816. Epub 2014 May 15.

Reference Type: BACKGROUND

PMID: 24832142 (View on PubMed)

Lich KH, Tian Y, Beadles CA, Williams LS, Bravata DM, Cheng EM, Bosworth HB, Homer JB, Matchar DB. Strategic planning to reduce the burden of stroke among veterans: using simulation modeling to inform decision making. Stroke. 2014 Jul;45(7):2078-84. doi: 10.1161/STROKEAHA.114.004694. Epub 2014 Jun 12.

Reference Type: BACKGROUND

PMID: 24923722 (View on PubMed)

Forrester, J.W. The model versus a modeling process. Syst Dyn Rev. 1985;:133-134.

Reference Type: BACKGROUND

Derickson R, Fishman J, Osatuke K, Teclaw R, Ramsel D. Psychological safety and error reporting within Veterans Health Administration hospitals. J Patient Saf. 2015 Mar;11(1):60-6. doi: 10.1097/PTS.0000000000000082.

Reference Type: BACKGROUND

PMID: 24583957 (View on PubMed)

Edmondson A. Psychological safety and learning behavior in work teams. Adm Sci Q. 1999;44:350-383.

Reference Type: BACKGROUND

Campbell MJ, & Walters SJ. How to Design, Analyise and Report Cluster Randomised Trials in Medicine and Health Related Research. : Wiley, 2014: 1-264.

Reference Type: BACKGROUND

Forrester JW. System dynamics, systems thinking, and soft OR. Syst Dyn Rev. 1994;10:245-256.

Reference Type: BACKGROUND

Forrester JW. Some basic concepts in system dynamics. Sloan Sch Manag Mass Inst Technol Camb MA. 2009.

Reference Type: BACKGROUND

Hong G. Ratio of mediator probability weighting for estimating natural direct and indirect effects. In: Proceedings of the American Statistical Association, Biometrics Section. : American Statistical Association Alexandria, VA, 2010: 2401-2415.

Reference Type: BACKGROUND

Hong G, Deutsch J, & Hill HD. Ratio-of-Mediator-Probability Weighting for Causal Mediation Analysis in the Presence of Treatment-by-Mediator Interaction. J Educ Behav Stat. 2015;40:307-340.

Reference Type: BACKGROUND

Qin X, & Hong G. Causal mediation analysis in multi-site trials: An application of ratio-of-mediatorprobability weighting to the Head Start Impact Study. JSM Proc Soc Stat Sect. 2014;:912-926.

Reference Type: BACKGROUND

Homer JB. Partial-model testing as a validation tool for system dynamics (1983). Syst Dyn Rev. 2012;28:281-294.

Reference Type: BACKGROUND

Rahmandad H, & Sterman JD. Reporting guidelines for simulation-based research in social sciences. Syst Dyn Rev. 2012;28:396-411.

Reference Type: BACKGROUND

Singer SJ, Moore SC, Meterko M, Williams S. Development of a short-form Learning Organization Survey: the LOS-27. Med Care Res Rev. 2012 Aug;69(4):432-59. doi: 10.1177/1077558712448135. Epub 2012 May 25.

Reference Type: BACKGROUND

PMID: 22635274 (View on PubMed)

Batorowicz B, Shepherd TA. Measuring the quality of transdisciplinary teams. J Interprof Care. 2008 Dec;22(6):612-20. doi: 10.1080/13561820802303664.

Reference Type: BACKGROUND

PMID: 19012141 (View on PubMed)

Dolan ED, Mohr D, Lempa M, Joos S, Fihn SD, Nelson KM, Helfrich CD. Using a single item to measure burnout in primary care staff: a psychometric evaluation. J Gen Intern Med. 2015 May;30(5):582-7. doi: 10.1007/s11606-014-3112-6. Epub 2014 Dec 2.

Reference Type: BACKGROUND

PMID: 25451989 (View on PubMed)

Helfrich CD, Simonetti JA, Clinton WL, Wood GB, Taylor L, Schectman G, Stark R, Rubenstein LV, Fihn SD, Nelson KM. The Association of Team-Specific Workload and Staffing with Odds of Burnout Among VA Primary Care Team Members. J Gen Intern Med. 2017 Jul;32(7):760-766. doi: 10.1007/s11606-017-4011-4. Epub 2017 Feb 23.

Reference Type: BACKGROUND

PMID: 28233221 (View on PubMed)

Nelson KM, Helfrich C, Sun H, Hebert PL, Liu CF, Dolan E, Taylor L, Wong E, Maynard C, Hernandez SE, Sanders W, Randall I, Curtis I, Schectman G, Stark R, Fihn SD. Implementation of the patient-centered medical home in the Veterans Health Administration: associations with patient satisfaction, quality of care, staff burnout, and hospital and emergency department use. JAMA Intern Med. 2014 Aug;174(8):1350-8. doi: 10.1001/jamainternmed.2014.2488.

Reference Type: BACKGROUND

PMID: 25055197 (View on PubMed)

Helfrich CD, Dolan ED, Simonetti J, Reid RJ, Joos S, Wakefield BJ, Schectman G, Stark R, Fihn SD, Harvey HB, Nelson K. Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. J Gen Intern Med. 2014 Jul;29 Suppl 2(Suppl 2):S659-66. doi: 10.1007/s11606-013-2702-z.

Reference Type: BACKGROUND

PMID: 24715396 (View on PubMed)

Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017 Aug 29;12(1):108. doi: 10.1186/s13012-017-0635-3.

Reference Type: BACKGROUND

PMID: 28851459 (View on PubMed)

Sweeney LB, & Sterman JD. Bathtub dynamics: Initial results of a systems thinking inventory. Syst Dyn Rev. 2000;16:249-286.

Reference Type: BACKGROUND

Liberati EG, Ruggiero F, Galuppo L, Gorli M, Gonzalez-Lorenzo M, Maraldi M, Ruggieri P, Polo Friz H, Scaratti G, Kwag KH, Vespignani R, Moja L. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci. 2017 Sep 15;12(1):113. doi: 10.1186/s13012-017-0644-2.

Reference Type: BACKGROUND

PMID: 28915822 (View on PubMed)

Centers for Medicare & Medicaid Services Alliance to Modernize Healthcare (CAMH). Independent Assessment of the Health Care Delivery Systems and Management Processes of the Department of Veterans Affairs (Volume 1: Integrated Report).

Reference Type: BACKGROUND

Department of Veterans Affairs. Uniform mental health services in VA medical centers and clinics. Washington DC: Veterans Health Administration, 2008.

Reference Type: BACKGROUND

Powers MB, Halpern JM, Ferenschak MP, Gillihan SJ, Foa EB. A meta-analytic review of prolonged exposure for posttraumatic stress disorder. Clin Psychol Rev. 2010 Aug;30(6):635-41. doi: 10.1016/j.cpr.2010.04.007. Epub 2010 May 2.

Reference Type: BACKGROUND

PMID: 20546985 (View on PubMed)

Bradley R, Greene J, Russ E, Dutra L, Westen D. A multidimensional meta-analysis of psychotherapy for PTSD. Am J Psychiatry. 2005 Feb;162(2):214-27. doi: 10.1176/appi.ajp.162.2.214.

Reference Type: BACKGROUND

PMID: 15677582 (View on PubMed)

Powers MB, Zum Vorde Sive Vording MB, Emmelkamp PM. Acceptance and commitment therapy: a meta-analytic review. Psychother Psychosom. 2009;78(2):73-80. doi: 10.1159/000190790. Epub 2009 Jan 14.

Reference Type: BACKGROUND

PMID: 19142046 (View on PubMed)

Walser RD, Karlin BE, Trockel M, Mazina B, Barr Taylor C. Training in and implementation of Acceptance and Commitment Therapy for depression in the Veterans Health Administration: therapist and patient outcomes. Behav Res Ther. 2013 Sep;51(9):555-63. doi: 10.1016/j.brat.2013.05.009. Epub 2013 Jun 3.

Reference Type: BACKGROUND

PMID: 23851161 (View on PubMed)

Stewart MO, Raffa SD, Steele JL, Miller SA, Clougherty KF, Hinrichsen GA, Karlin BE. National dissemination of interpersonal psychotherapy for depression in veterans: therapist and patient-level outcomes. J Consult Clin Psychol. 2014 Dec;82(6):1201-6. doi: 10.1037/a0037410. Epub 2014 Jul 21.

Reference Type: BACKGROUND

PMID: 25045906 (View on PubMed)

Cuijpers P, Geraedts AS, van Oppen P, Andersson G, Markowitz JC, van Straten A. Interpersonal psychotherapy for depression: a meta-analysis. Am J Psychiatry. 2011 Jun;168(6):581-92. doi: 10.1176/appi.ajp.2010.10101411. Epub 2011 Mar 1. Erratum In: Am J Psychiatry. 2011 Jun;168(6):652.

Reference Type: BACKGROUND

PMID: 21362740 (View on PubMed)

Fournier JC, DeRubeis RJ, Hollon SD, Dimidjian S, Amsterdam JD, Shelton RC, Fawcett J. Antidepressant drug effects and depression severity: a patient-level meta-analysis. JAMA. 2010 Jan 6;303(1):47-53. doi: 10.1001/jama.2009.1943.

Reference Type: BACKGROUND

PMID: 20051569 (View on PubMed)

Knudsen HK, Roman PM. The Diffusion of Acamprosate for the Treatment of Alcohol Use Disorder: Results From a National Longitudinal Study. J Subst Abuse Treat. 2016 Mar;62:62-7. doi: 10.1016/j.jsat.2015.10.005. Epub 2015 Dec 1.

Reference Type: BACKGROUND

PMID: 26689318 (View on PubMed)

Knudsen HK, Roman PM. Service delivery and pharmacotherapy for alcohol use disorder in the era of health reform: Data from a national sample of treatment organizations. Subst Abus. 2016;37(1):230-7. doi: 10.1080/08897077.2015.1028699. Epub 2015 Apr 20.

Reference Type: BACKGROUND

PMID: 25893539 (View on PubMed)

Pettinati HM, O'Brien CP, Rabinowitz AR, Wortman SP, Oslin DW, Kampman KM, Dackis CA. The status of naltrexone in the treatment of alcohol dependence: specific effects on heavy drinking. J Clin Psychopharmacol. 2006 Dec;26(6):610-25. doi: 10.1097/01.jcp.0000245566.52401.20.

Reference Type: BACKGROUND

PMID: 17110818 (View on PubMed)

Anton RF, O'Malley SS, Ciraulo DA, Cisler RA, Couper D, Donovan DM, Gastfriend DR, Hosking JD, Johnson BA, LoCastro JS, Longabaugh R, Mason BJ, Mattson ME, Miller WR, Pettinati HM, Randall CL, Swift R, Weiss RD, Williams LD, Zweben A; COMBINE Study Research Group. Combined pharmacotherapies and behavioral interventions for alcohol dependence: the COMBINE study: a randomized controlled trial. JAMA. 2006 May 3;295(17):2003-17. doi: 10.1001/jama.295.17.2003.

Reference Type: BACKGROUND

PMID: 16670409 (View on PubMed)

Barnett PG, Rodgers JH, Bloch DA. A meta-analysis comparing buprenorphine to methadone for treatment of opiate dependence. Addiction. 2001 May;96(5):683-90. doi: 10.1046/j.1360-0443.2001.9656834.x.

Reference Type: BACKGROUND

PMID: 11331027 (View on PubMed)

West SL, O'Neal KK, Graham CW. A meta-analysis comparing the effectiveness of buprenorphine and methadone. J Subst Abuse. 2000;12(4):405-14. doi: 10.1016/s0899-3289(01)00054-2.

Reference Type: BACKGROUND

PMID: 11452842 (View on PubMed)

Hysong SJ, Kell HJ, Petersen LA, Campbell BA, Trautner BW. Theory-based and evidence-based design of audit and feedback programmes: examples from two clinical intervention studies. BMJ Qual Saf. 2017 Apr;26(4):323-334. doi: 10.1136/bmjqs-2015-004796. Epub 2016 Jun 10.

Reference Type: BACKGROUND

PMID: 27288054 (View on PubMed)

Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci. 2006 Apr 28;1:9. doi: 10.1186/1748-5908-1-9.

Reference Type: BACKGROUND

PMID: 16722539 (View on PubMed)

Hysong SJ, Teal CR, Khan MJ, Haidet P. Improving quality of care through improved audit and feedback. Implement Sci. 2012 May 18;7:45. doi: 10.1186/1748-5908-7-45.

Reference Type: BACKGROUND

PMID: 22607640 (View on PubMed)

Shiner B, D'Avolio LW, Nguyen TM, Zayed MH, Young-Xu Y, Desai RA, Schnurr PP, Fiore LD, Watts BV. Measuring use of evidence based psychotherapy for posttraumatic stress disorder. Adm Policy Ment Health. 2013 Jul;40(4):311-8. doi: 10.1007/s10488-012-0421-0.

Reference Type: BACKGROUND

PMID: 22535469 (View on PubMed)

Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999 Sep;89(9):1322-7. doi: 10.2105/ajph.89.9.1322.

Reference Type: BACKGROUND

PMID: 10474547 (View on PubMed)

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011 Mar;38(2):65-76. doi: 10.1007/s10488-010-0319-7.

Reference Type: BACKGROUND

PMID: 20957426 (View on PubMed)

Brownson RC, Colditz GA, & Proctor EK. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford, UK: Oxford, 2012.

Reference Type: BACKGROUND

Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012 Sep;43(3):337-50. doi: 10.1016/j.amepre.2012.05.024.

Reference Type: BACKGROUND

PMID: 22898128 (View on PubMed)

Tabak RG, Khoong EC, Chambers D, et al. Models in dissemination and implementation research: useful tools in public health services and systems research. Front Public Health Serv Syst Res. 2013;2:8.

Reference Type: BACKGROUND

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009 Aug 7;4:50. doi: 10.1186/1748-5908-4-50.

Reference Type: BACKGROUND

PMID: 19664226 (View on PubMed)

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015 Feb 12;10:21. doi: 10.1186/s13012-015-0209-1.

Reference Type: BACKGROUND

PMID: 25889199 (View on PubMed)

Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013 Oct 2;8:117. doi: 10.1186/1748-5908-8-117.

Reference Type: BACKGROUND

PMID: 24088228 (View on PubMed)

Breuer E, Lee L, De Silva M, Lund C. Using theory of change to design and evaluate public health interventions: a systematic review. Implement Sci. 2016 May 6;11:63. doi: 10.1186/s13012-016-0422-6.

Reference Type: BACKGROUND

PMID: 27153985 (View on PubMed)

Schierhout G, Hains J, Si D, Kennedy C, Cox R, Kwedza R, O'Donoghue L, Fittock M, Brands J, Lonergan K, Dowden M, Bailie R. Evaluating the effectiveness of a multifaceted, multilevel continuous quality improvement program in primary health care: developing a realist theory of change. Implement Sci. 2013 Oct 8;8:119. doi: 10.1186/1748-5908-8-119.

Reference Type: BACKGROUND

PMID: 24098940 (View on PubMed)

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015 Apr 21;10:53. doi: 10.1186/s13012-015-0242-0.

Reference Type: BACKGROUND

PMID: 25895742 (View on PubMed)

Sales A, Smith J, Curran G, Kochevar L. Models, strategies, and tools. Theory in implementing evidence-based findings into health care practice. J Gen Intern Med. 2006 Feb;21 Suppl 2(Suppl 2):S43-9. doi: 10.1111/j.1525-1497.2006.00362.x.

Reference Type: BACKGROUND

PMID: 16637960 (View on PubMed)

Damschroder LJ, & Lowery JC. Efficient Synthesis: Using Qualitative Comparative Analysis (QCA) and the CFIR across Diverse Studies. 2015.

Reference Type: BACKGROUND

Baumgartner M, & Thiem A. Often Trusted but Never (Properly) Tested: Evaluating Qualitative Comparative Analysis. Sociol Methods Res. 2017;:4912411770148.

Reference Type: BACKGROUND

Hug S. Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference. Polit Anal. 2013;21:252-265.

Reference Type: BACKGROUND

Krogslund C, Choi DD, & Poertner M. Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA. Polit Anal. 2015;23:21-41.

Reference Type: BACKGROUND

Braumoeller B. QCAfalsePositive: Tests for Type I Error in Qualitative Comparative Analysis (QCA). 2015.

Reference Type: BACKGROUND

Powell BJ, Proctor EK, Glass JE. A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions. Res Soc Work Pract. 2014 Mar 1;24(2):192-212. doi: 10.1177/1049731513505778.

Reference Type: BACKGROUND

PMID: 24791131 (View on PubMed)

Williams NJ. Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice. Adm Policy Ment Health. 2016 Sep;43(5):783-798. doi: 10.1007/s10488-015-0693-2.

Reference Type: BACKGROUND

PMID: 26474761 (View on PubMed)

Senge P. The Fifth Discipline. : Currency/Doubleday, 1990.

Reference Type: BACKGROUND

Sweeney LB, & Sterman JD. Bathtub dynamics: initial results of a systems thinking inventory. Syst Dyn Rev. 2000;16:249-286.

Reference Type: BACKGROUND

Hysong SJ, Knox MK, Haidet P. Examining clinical performance feedback in Patient-Aligned Care Teams. J Gen Intern Med. 2014 Jul;29 Suppl 2(Suppl 2):S667-74. doi: 10.1007/s11606-013-2707-7.

Reference Type: BACKGROUND

PMID: 24715398 (View on PubMed)

Hysong SJ, Thomas CL, Spitzmuller C, Amspoker AB, Woodard L, Modi V, Naik AD. Linking clinician interaction and coordination to clinical performance in Patient-Aligned Care Teams. Implement Sci. 2016 Jan 15;11:7. doi: 10.1186/s13012-015-0368-0.

Reference Type: BACKGROUND

PMID: 26772972 (View on PubMed)

Richardson GP. Concept models in group model building: G. P. Richardson: Concept Models in Group Model Building. Syst Dyn Rev. 2013;29:42-55.

Reference Type: BACKGROUND

Richardson GP. Reflections on the foundations of system dynamics: Foundations of System Dynamics. Syst Dyn Rev. 2011;27:219-243.

Reference Type: BACKGROUND

Garcia HA, McGeary CA, Finley EP, et al. Burnout among psychiatrists in the Veterans Health Administration. Burn Res. 2015;2:108-114.

Reference Type: BACKGROUND

Garcia HA, Finley EP, Ketchum N, Jakupcak M, Dassori A, Reyes SC. A survey of perceived barriers and attitudes toward mental health care among OEF/OIF veterans at VA outpatient mental health clinics. Mil Med. 2014 Mar;179(3):273-8. doi: 10.7205/MILMED-D-13-00076.

Reference Type: BACKGROUND

PMID: 24594461 (View on PubMed)

Garcia HA, Kelley LP, Rentz TO, et al. Pretreatment predictors of dropout from cognitive behavioral therapy for PTSD in Iraq and Afghanistan war veterans. Psychol Serv. 2011;8:1-11.

Reference Type: BACKGROUND

Senge, P.M. The fifth discipline: The art and practice of the learning organization. : Broadway Business, 2006.

Reference Type: BACKGROUND

Meadows DH. Thinking in systems: A primer. : Chelsea Green Publishing, 2012.

Reference Type: BACKGROUND

Senge PM, ed. The dance of change: the challenges of sustaining momentum in learning organizations, 1st ed. New York: Currency/Doubleday, 1999: 1-596.

Reference Type: BACKGROUND

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. : McGraw-Hill Education, 2000: 1-1008.

Reference Type: BACKGROUND

Rouwette EA, & Vennix JA. System dynamics and organizational interventions. Syst Res Behav Sci. 2006;23:451-466.

Reference Type: BACKGROUND

Akhnif E, Macq J, Idrissi Fakhreddine MO, Meessen B. Scoping literature review on the Learning Organisation concept as applied to the health system. Health Res Policy Syst. 2017 Mar 1;15(1):16. doi: 10.1186/s12961-017-0176-x.

Reference Type: BACKGROUND

PMID: 28249608 (View on PubMed)

Morecroft J, & Sherman J. Modeling for learning organizations. Portland OR: Productivity Press, 1994.

Reference Type: BACKGROUND

Argyris C, & Schön DA. Organizational learning: A theory of action perspective. Reis. 1997;:345-348.

Reference Type: BACKGROUND

Forrester JW. Industrial Dynamics. Cambridge, MA: MIT Press, 1961.

Reference Type: BACKGROUND

Simon HA. Models of Man: Social and Rational-Mathematical Essays on Rational Human Behavior in a Social Setting. Oxford, UK: Wiley, 1957.

Reference Type: BACKGROUND

Huz S, Andersen DF, Richardson GP, et al. A framework for evaluating systems thinking interventions: an experimental approach to mental health system change. Syst Dyn Rev. 1997;13:149-169.

Reference Type: BACKGROUND

Evans JS. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol. 2008;59:255-78. doi: 10.1146/annurev.psych.59.103006.093629.

Reference Type: BACKGROUND

PMID: 18154502 (View on PubMed)

Evans JS. In two minds: dual-process accounts of reasoning. Trends Cogn Sci. 2003 Oct;7(10):454-9. doi: 10.1016/j.tics.2003.08.012.

Reference Type: BACKGROUND

PMID: 14550493 (View on PubMed)

Evans JS, Stanovich KE. Dual-Process Theories of Higher Cognition: Advancing the Debate. Perspect Psychol Sci. 2013 May;8(3):223-41. doi: 10.1177/1745691612460685.

Reference Type: BACKGROUND

PMID: 26172965 (View on PubMed)

Dolansky MA, Moore SM, Palmieri PA, Singh MK. Development and Validation of the Systems Thinking Scale. J Gen Intern Med. 2020 Aug;35(8):2314-2320. doi: 10.1007/s11606-020-05830-1. Epub 2020 Apr 27.

Reference Type: BACKGROUND

PMID: 32342481 (View on PubMed)

Hovmand PS. Community Based System Dynamics. New York, NY: Springer New York, 2014.

Reference Type: BACKGROUND

Minkler M, & Wallerstein N. Community-Based Participatory Research for Health: From Process to Outcomes. : John Wiley & Sons, 2011: 1-758.

Reference Type: BACKGROUND

Case AD, Byrd R, Claggett E, DeVeaux S, Perkins R, Huang C, Sernyak MJ, Steiner JL, Cole R, LaPaglia DM, Bailey M, Buchanan C, Johnson A, Kaufman JS. Stakeholders' perspectives on community-based participatory research to enhance mental health services. Am J Community Psychol. 2014 Dec;54(3-4):397-408. doi: 10.1007/s10464-014-9677-8.

Reference Type: BACKGROUND

PMID: 25245601 (View on PubMed)

Israel BA, Coombe CM, Cheezum RR, Schulz AJ, McGranaghan RJ, Lichtenstein R, Reyes AG, Clement J, Burris A. Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities. Am J Public Health. 2010 Nov;100(11):2094-102. doi: 10.2105/AJPH.2009.170506. Epub 2010 Sep 23.

Reference Type: BACKGROUND

PMID: 20864728 (View on PubMed)

Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promot Pract. 2006 Jul;7(3):312-23. doi: 10.1177/1524839906289376. Epub 2006 Jun 7.

Reference Type: BACKGROUND

PMID: 16760238 (View on PubMed)

Aarons GA. Measuring provider attitudes toward evidence-based practice: consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005 Apr;14(2):255-71, viii. doi: 10.1016/j.chc.2004.04.008.

Reference Type: BACKGROUND

PMID: 15694785 (View on PubMed)

Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011 Jan;38(1):4-23. doi: 10.1007/s10488-010-0327-7.

Reference Type: BACKGROUND

PMID: 21197565 (View on PubMed)

Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255-74. doi: 10.1146/annurev-publhealth-032013-182447.

Reference Type: BACKGROUND

PMID: 24641560 (View on PubMed)

Aarons GA, Green AE, Trott E, Willging CE, Torres EM, Ehrhart MG, Roesch SC. The Roles of System and Organizational Leadership in System-Wide Evidence-Based Intervention Sustainment: A Mixed-Method Study. Adm Policy Ment Health. 2016 Nov;43(6):991-1008. doi: 10.1007/s10488-016-0751-4.

Reference Type: BACKGROUND

PMID: 27439504 (View on PubMed)

Brimhall KC, Fenwick K, Farahnak LR, Hurlburt MS, Roesch SC, Aarons GA. Leadership, Organizational Climate, and Perceived Burden of Evidence-Based Practice in Mental Health Services. Adm Policy Ment Health. 2016 Sep;43(5):629-639. doi: 10.1007/s10488-015-0670-9.

Reference Type: BACKGROUND

PMID: 26152770 (View on PubMed)

Senge P, & Forrester JW. Tests for building confidence in system dynamics models. 1980;14:209-228.

Reference Type: BACKGROUND

Rahmandad H, & Sterman JD. Reporting guidelines for simulation-based research in social sciences: Reporting Guidelines for Simulation-Based Research. Syst Dyn Rev. 2012;28:396-411.

Reference Type: BACKGROUND

Hovmand PS, Andersen DF, Rouwette E, et al. Group model-building "scripts" as a collaborative planning tool: Scripts as a collaborative planning tool. Syst Res Behav Sci. 2012;29:179-193.

Reference Type: BACKGROUND

Andersen D, & Richardson G. Scripts for group model building. Syst Dyn Rev. 1997;13:107-129.

Reference Type: BACKGROUND

Hovmand, Peter, S., et al. Scriptapedia 4.0.6. 2013.

Reference Type: BACKGROUND

McCarthy JT, Hocum CL, Albright RC, Rogers J, Gallaher EJ, Steensma DP, Gudgell SF, Bergstralh EJ, Dillon JC, Hickson LJ, Williams AW, Dingli D. Biomedical system dynamics to improve anemia control with darbepoetin alfa in long-term hemodialysis patients. Mayo Clin Proc. 2014 Jan;89(1):87-94. doi: 10.1016/j.mayocp.2013.10.022.

Reference Type: BACKGROUND

PMID: 24388026 (View on PubMed)

Gallaher E, Steensma DP, Chrisope TR, et al. Individualized Medicine and Biophysical System Dynamics: An Example from Clinical Practice in End Stage Renal Disease. 2011.

Reference Type: BACKGROUND

Homer J. Levels of evidence in system dynamics modeling. Syst Dyn Rev. 2014;30:75-80.

Reference Type: BACKGROUND

Oliva R, & Sterman JD. Death spirals and virtuous cycles. In: Handbook of Service Science. : Springer, 2010: 321-358.

Reference Type: BACKGROUND

Snow G. Package "blockrand." 2015.

Reference Type: BACKGROUND

Holowka DW, Marx BP, Gates MA, Litman HJ, Ranganathan G, Rosen RC, Keane TM. PTSD diagnostic validity in Veterans Affairs electronic records of Iraq and Afghanistan veterans. J Consult Clin Psychol. 2014 Aug;82(4):569-579. doi: 10.1037/a0036347. Epub 2014 Apr 14.

Reference Type: BACKGROUND

PMID: 24731235 (View on PubMed)

Emmons KM, Weiner B, Fernandez ME, Tu SP. Systems antecedents for dissemination and implementation: a review and analysis of measures. Health Educ Behav. 2012 Feb;39(1):87-105. doi: 10.1177/1090198111409748. Epub 2011 Jul 1.

Reference Type: BACKGROUND

PMID: 21724933 (View on PubMed)

Rabin BA, Lewis CC, Norton WE, Neta G, Chambers D, Tobin JN, Brownson RC, Glasgow RE. Measurement resources for dissemination and implementation research in health. Implement Sci. 2016 Mar 22;11:42. doi: 10.1186/s13012-016-0401-y.

Reference Type: BACKGROUND

PMID: 27000147 (View on PubMed)

Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015 Nov 4;10:155. doi: 10.1186/s13012-015-0342-x.

Reference Type: BACKGROUND

PMID: 26537706 (View on PubMed)

Glasgow RE. What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation. Health Educ Behav. 2013 Jun;40(3):257-65. doi: 10.1177/1090198113486805.

Reference Type: BACKGROUND

PMID: 23709579 (View on PubMed)

Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med. 2013 Aug;45(2):237-43. doi: 10.1016/j.amepre.2013.03.010.

Reference Type: BACKGROUND

PMID: 23867032 (View on PubMed)

Singh MK, Ogrinc G, Cox KR, Dolansky M, Brandt J, Morrison LJ, Harwood B, Petroski G, West A, Headrick LA. The Quality Improvement Knowledge Application Tool Revised (QIKAT-R). Acad Med. 2014 Oct;89(10):1386-91. doi: 10.1097/ACM.0000000000000456.

Reference Type: BACKGROUND

PMID: 25119555 (View on PubMed)

Rosseel Y, Oberski D, Byrnes J, et al. Package "lavaan" Version 0.5-23.1097. 2017.

Reference Type: BACKGROUND

Shavelson RJ, & Webb NM. Generalizability theory: A primer. Newbury Park, CA: Sage., 1991.

Reference Type: BACKGROUND

Millsap RE, Kwok OM. Evaluating the impact of partial factorial invariance on selection in two populations. Psychol Methods. 2004 Mar;9(1):93-115. doi: 10.1037/1082-989X.9.1.93.

Reference Type: BACKGROUND

PMID: 15053721 (View on PubMed)

Wolak M. Package "ICC" Version 2.3.0. 2015.

Reference Type: BACKGROUND

Bates D, Maechler M, Bolker B, et al. Package "lme4" Version 1.1-13. 2017.

Reference Type: BACKGROUND

Rotondi MA. Package "CRTSize" Version 1.0. 2015.

Reference Type: BACKGROUND

Rotondi MA, & Donner A. Sample Size Estimation in Cluster Randomized Educational Trials: An Empirical Bayes Approach. J Educ Behav Stat. 2009;34:229-237.

Reference Type: BACKGROUND

Ventana Systems Inc. Vensim@ Version 6.3. 2014.

Reference Type: BACKGROUND

Hyndman, R.J., & Khandakar, Y. Automatic time series forecasting: The forecast package for R. J Stat Softw. 2008;26.

Reference Type: BACKGROUND

Hyndman R, O'Hara-Wild Mi, Bergmeir C, et al. Package "forecast" Version 8.0. 2017.

Reference Type: BACKGROUND

Mott JM, Hundt NE, Sansgiry S, Mignogna J, Cully JA. Changes in psychotherapy utilization among veterans with depression, anxiety, and PTSD. Psychiatr Serv. 2014 Jan 1;65(1):106-12. doi: 10.1176/appi.ps.201300056.

Reference Type: BACKGROUND

PMID: 24081213 (View on PubMed)

Oliva EM, Trafton JA, Harris AH, Gordon AJ. Trends in opioid agonist therapy in the Veterans Health Administration: is supply keeping up with demand? Am J Drug Alcohol Abuse. 2013 Mar;39(2):103-7. doi: 10.3109/00952990.2012.741167.

Reference Type: BACKGROUND

PMID: 23421571 (View on PubMed)

Diaz M, Neuhauser D. Pasteur and parachutes: when statistical process control is better than a randomized controlled trial. Qual Saf Health Care. 2005 Apr;14(2):140-3. doi: 10.1136/qshc.2005.013763. No abstract available.

Reference Type: BACKGROUND

PMID: 15805461 (View on PubMed)

Duclos A, Voirin N. The p-control chart: a tool for care improvement. Int J Qual Health Care. 2010 Oct;22(5):402-7. doi: 10.1093/intqhc/mzq037. Epub 2010 Jul 30.

Reference Type: BACKGROUND

PMID: 20675711 (View on PubMed)

Ogrinc GS, & Headrick L. Fundamentals of health care improvement: A guide to improving your patients' care. : Joint Commission Resources, 2008.

Reference Type: BACKGROUND

Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, Huizinga MM, Liu SK, Mills P, Neily J, Nelson W, Pronovost PJ, Provost L, Rubenstein LV, Speroff T, Splaine M, Thomson R, Tomolo AM, Watts B. The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008 Oct;17 Suppl 1(Suppl_1):i13-32. doi: 10.1136/qshc.2008.029058.

Reference Type: BACKGROUND

PMID: 18836062 (View on PubMed)

Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood). 2011 Mar;30(3):464-71. doi: 10.1377/hlthaff.2011.0178.

Reference Type: BACKGROUND

PMID: 21383365 (View on PubMed)

Raffa SD, Maciejewski ML, Zimmerman LE, Damschroder LJ, Estabrooks PA, Ackermann RT, Tsai AG, Histon T, Goldstein MG. A System-Level Approach to Overweight and Obesity in the Veterans Health Administration. J Gen Intern Med. 2017 Apr;32(Suppl 1):79-82. doi: 10.1007/s11606-016-3948-z.

Reference Type: BACKGROUND

PMID: 28271428 (View on PubMed)

Kilbourne AM, Elwy AR, Sales AE, Atkins D. Accelerating Research Impact in a Learning Health Care System: VA's Quality Enhancement Research Initiative in the Choice Act Era. Med Care. 2017 Jul;55 Suppl 7 Suppl 1(7 Suppl 1):S4-S12. doi: 10.1097/MLR.0000000000000683.

Reference Type: BACKGROUND

PMID: 27997456 (View on PubMed)

Mabry PL, Olster DH, Morgan GD, Abrams DB. Interdisciplinarity and systems science to improve population health: a view from the NIH Office of Behavioral and Social Sciences Research. Am J Prev Med. 2008 Aug;35(2 Suppl):S211-24. doi: 10.1016/j.amepre.2008.05.018.

Reference Type: BACKGROUND

PMID: 18619402 (View on PubMed)

Mabry PL, Milstein B, Abraido-Lanza AF, Livingood WC, Allegrante JP. Opening a window on systems science research in health promotion and public health. Health Educ Behav. 2013 Oct;40(1 Suppl):5S-8S. doi: 10.1177/1090198113503343. No abstract available.

Reference Type: BACKGROUND

PMID: 24084401 (View on PubMed)

Luke DA, Stamatakis KA. Systems science methods in public health: dynamics, networks, and agents. Annu Rev Public Health. 2012 Apr;33:357-76. doi: 10.1146/annurev-publhealth-031210-101222. Epub 2012 Jan 3.

Reference Type: BACKGROUND

PMID: 22224885 (View on PubMed)

Moore GF, Evans RE. What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM Popul Health. 2017 Dec;3:132-135. doi: 10.1016/j.ssmph.2016.12.005.

Reference Type: BACKGROUND

PMID: 29302610 (View on PubMed)

Monks T. Operational research as implementation science: definitions, challenges and research priorities. Implement Sci. 2016 Jun 6;11(1):81. doi: 10.1186/s13012-016-0444-0.

Reference Type: BACKGROUND

PMID: 27268021 (View on PubMed)

May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016 Oct 19;11(1):141. doi: 10.1186/s13012-016-0506-3.

Reference Type: BACKGROUND

PMID: 27756414 (View on PubMed)

Brainard J, Hunter PR. Do complexity-informed health interventions work? A scoping review. Implement Sci. 2016 Sep 20;11(1):127. doi: 10.1186/s13012-016-0492-5.

Reference Type: BACKGROUND

PMID: 27647152 (View on PubMed)

Provided Documents

Document Type: Study Protocol and Statistical Analysis Plan

View Document

Other Identifiers

ZIM_0002

Identifier Type: -

Identifier Source: org_study_id