Report Cards and Feedback for PCCE

NCT ID: NCT06155292

Last Updated: 2025-01-30

Basic Information

• Recruitment Status: RECRUITING
• Clinical Phase: NA
• Total Enrollment: 330 participants
• Study Classification: INTERVENTIONAL
• Study Start Date: 2023-10-03
• Study Completion Date: 2025-10-31

Brief Summary

As part of UCLA Health's commitment to developing an integrated health system built on a foundation of physician-led, team-based primary care, the Department of Medicine (DOM) implemented a performance-based incentive plan called the Primary Care Clinical Excellence (PCCE) Incentive Plan.

The UCLA Health DOM Quality team is leading the implementation and evaluation of this incentive plan across the UCLA Health primary care network, with the primary goal to immediately produce improvements in the quality of primary care. In order to rigorously measure the most efficacious ways to frame and communicate information about the quality improvement (QI) program, the DOM Quality team has partnered with the UCLA Anderson School of Management.

Understanding the factors that motivate physicians to deliver high quality primary care will provide pivotal insights into the successful implementation of performance based programs nationwide.

Detailed Description

The investigators will use a communication strategy that leverages behavioral principles to motivate providers to improve in all four of the evaluated domains of the PCCE program: clinical quality, professional participation, patient experience, and risk coding.

The investigators will implement a three-arm experimental communication campaign that includes quarterly emails and quarterly survey messages. The communication strategies will utilize motivation and behavior change theories to improve physician performance in the program and attitudes towards the program. In particular, the investigators will test the independent and joint effects of communicating with physicians (a) personalized performance feedback and (b) the "co-creation" of the program (i.e., sharing how physician feedback informed the program design).

The investigators will randomly assign eligible physicians to one of the three experimental arms, stratified by overall baseline performance (the total percent allocated in the PCCE program for the April, May, June 2023 quarter), specialty (based on classification as Adult or Adult/Peds), and contract (based on classification as DOM or PCN (CPN/EIMG)).

The investigators will evaluate whether arm 3 differs from arm 1 in terms of the measures listed in the Outcome Measures section. If this comparison is statistically significant, the investigators will next compare arms 3 vs. 2 and arms 2 vs. 1.

Analysis plan

• Physician-quarter-level linear regression models with heteroskedastic-consistent robust standard errors, clustered at the physician level.
• The primary model term will be indicator variables for arms that patients are assigned to.
• Control variables:

• Baseline values of the outcome measure: the investigators will control for the baseline measures of the outcomes, based on performance from July through September 2023 and survey responses in April and July 2023, if available.
• Physician baseline subjective understanding of the PCCE program from the July 2023 survey, with four items which asked the extent to which physicians understood how their score in each domain of the program was calculated.
• Physician characteristics: self-reported gender, race, and age, plus years at UCLA Health and clinic from administrative data
• Missing covariate values will be handled by including 'unknown' indicator variables, along with mean imputation for continuous covariates.
• Exploratory analyses will investigate heterogeneous treatment effects by the following characteristics:

• Physician baseline performance in the PCCE program (evaluated based on performance in July, August, and September 2023)
• Physician baseline perceived leadership support reported in the July 2023 survey
• Physician baseline program-related attitudes (agency, value in PCCE program) reported in the April 2023 survey
• Physician baseline subjective understanding of the PCCE program from the July 2023 survey
• Objective valence of the feedback delivered in the first email report card, measured by 1) the number of metrics within each domain for which the physician improved relative to the prior quarter, 2) the number of metrics within each domain for which the physician met or exceeded the benchmark, and 3) average distance from the benchmark across the metrics within each domain
• Random assignment to providing feedback on the report card design in a survey conducted before the intervention launch, in July 2023
• Physician gender
• Physician years in practice
• The investigators will investigate physicians' program-related attitudes and perceptions as proposed mechanisms of the interventions. These will be measured with 11 items that form three subscales about perceived justice, antecedents to one's intentions to succeed, and perceived value of the feedback.

• Perceived justice in the PCCE program structure and implementation: Physicians will be surveyed regarding the structural justice (2 items), distributive justice (1 item), and informational justice of the program (2 items).
• Antecedents of intentions to succeed in the program: Physicians will be surveyed regarding their perceived agency (behavioral control) over their performance in the PCCE program (1 item), their perceived value of the PCCE program (2 items), and their subjective norms about the endorsement and success of their colleagues in the program (2 items).
• Perceived value of the performance feedback: Physicians will be surveyed about the utility of the quarterly performance feedback in contextualizing their goal pursuit efforts (1 item).
• The investigators will investigate physicians' overall workplace attitudes as additional outcomes. These will be measured as a 4-item questionnaire about trust in UCLA Health leadership (1 item), perceived leadership support (1 item), job satisfaction (1 item), and burnout (1 item).
• Robustness checks will be performed without covariates, and using logistic regression models in place of linear regression models for dependent variables that are measured as binary indicators.

Conditions

Behavioral Economics; Primary Health Care; Health Maintenance

Study Design

• Allocation Method: RANDOMIZED
• Intervention Model: PARALLEL
• Primary Study Purpose: HEALTH_SERVICES_RESEARCH
• Blinding Strategy: SINGLE

Study Groups

Arm 1: Standard Communication Arm

Quarterly email communication:

Quarterly standard communication via email providing a link to physicians to check their PCCE program performance over the prior quarter, and a link to access the PCCE dashboard. The email will also include a link to resources. Starting with the email communication in February 2024, there will be a reminder email sent two weeks after the first email with the same content.

Quarterly survey:

Quarterly standard communication via survey with questions about physician attitudes and beliefs.

Group Type: PLACEBO_COMPARATOR

Standard Communication Email

Intervention Type: BEHAVIORAL

This is a standard quarterly email communication without personalized performance metrics.

Arm 2: Personalized Report Card

Quarterly email communication:

Quarterly personalized communication via email providing individualized performance metrics to physicians for the PCCE program from the prior quarter. All the links in the Arm 1 emails will be included in Arm 2 emails. Starting with the email communication in February 2024, there will be a reminder email sent two weeks after the first email with the same content.

Quarterly survey:

Quarterly standard communication via survey with the same questions about physician attitudes and beliefs as in Arm 1.

Group Type: EXPERIMENTAL

Personalized Report Card Email

Intervention Type: BEHAVIORAL

This personalized information about physician performance replaces the standard communication email to provide personalized feedback to physicians.

Arm 3: Personalized Report Card + Bottom-Up Framing

Quarterly email communication as in Arm 2. Bottom-up intervention: The quarterly email communication will also describe the ways in which the PCCE program and its features were informed by physician feedback and recommendations.

Quarterly survey: The quarterly survey will include information about the ways in which the PCCE program and its features were informed by physician feedback and recommendations. Physicians will respond to the same questions about physician attitudes and beliefs as in Arms 1 and 2.

Group Type: EXPERIMENTAL

Personalized Report Card Email

Intervention Type: BEHAVIORAL

This personalized information about physician performance replaces the standard communication email to provide personalized feedback to physicians.

Bottom-Up Framing

Intervention Type: BEHAVIORAL

The bottom-up framing is added onto the personalized communication email to provide insight on how the PCCE program was informed by physician feedback.

The bottom-up framing is added onto the survey communication to provide insight on how the PCCE program was informed by physician feedback.

Interventions

Standard Communication Email

This is a standard quarterly email communication without personalized performance metrics.

Intervention Type: BEHAVIORAL

Personalized Report Card Email

This personalized information about physician performance replaces the standard communication email to provide personalized feedback to physicians.

Intervention Type: BEHAVIORAL

Bottom-Up Framing

The bottom-up framing is added onto the personalized communication email to provide insight on how the PCCE program was informed by physician feedback.

The bottom-up framing is added onto the survey communication to provide insight on how the PCCE program was informed by physician feedback.

Intervention Type: BEHAVIORAL

Eligibility Criteria

Inclusion Criteria

• Primary care physicians within the UCLA Health Department of Medicine Primary Care Network that are eligible for the PCCE Incentive program as of October 1, 2023.
• Physicians with the clinical full-time employee level (FTE) of ≥ 40% as of October 1, 2023
• Physicians with panel size >50 patients as of October 1, 2023

Exclusion Criteria

• Physicians classified as Pediatrics will be excluded from data analysis given the structural differences in health maintenance guidelines for children.
• Physicians classified as Urgent Care will be excluded from data analysis given the structural differences in their performance evaluation. They are all in Arm 1.
• Physicians who participate in the design of this experiment will be excluded from analysis.

• Minimum Eligible Age: 18 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: No

Sponsors

University of California, Los Angeles

OTHER

Sponsor Role: lead

Responsible Party

Richard K. Leuchter, MD

Clinical Instructor

Responsibility Role: PRINCIPAL_INVESTIGATOR

Principal Investigators

Richard Leuchter, MD

Role: PRINCIPAL_INVESTIGATOR

UCLA Health

Locations

UCLA Health Department of Medicine, Quality Office

Los Angeles, California, United States

Site Status: RECRUITING

Countries

United States

Central Contacts

Ilana Brody

Role: CONTACT

ilana.brody.phd@anderson.ucla.edu

7032445470

Facility Contacts

Richard Leuchter, MD

Role: primary

rleuchter@mednet.ucla.edu

References

Colquitt JA. On the dimensionality of organizational justice: a construct validation of a measure. J Appl Psychol. 2001 Jun;86(3):386-400. doi: 10.1037/0021-9010.86.3.386.

Reference Type: BACKGROUND

PMID: 11419799 (View on PubMed)

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211.

Reference Type: BACKGROUND

Konovsky, M. A. (2000). Understanding procedural justice and its impact on business organizations. Journal of Management, 26(3), 489-511. https://doi.org/10.1016/S0149-2063(00)00042-8

Reference Type: BACKGROUND

Leventhal, H. (1980). Toward a comprehensive theory of emotion. In Advances in experimental social psychology (Vol. 13, pp. 139-207). Elsevier.

Reference Type: BACKGROUND

Shapiro, D. L., Buttner, E. H., & Barry, B. (1994). Explanations: What factors enhance their perceived adequacy? Organizational Behavior and Human Decision Processes, 58(3), 346-368.

Reference Type: BACKGROUND

Thibault, J., & Walker, L. (1975). Procedural justice: A social psychological analysis. Hillsdale, NJ: Lawrence Elbaum Associates.

Reference Type: BACKGROUND

Bies, R. J., & Moag, J. S. 1986. Interactional justice; Communication criteria of fairness. In R. J. Lewicki, B. H. Sheppard, & B. H. Bazerman (Eds.), Research on negotiation in organizations, Vol. 1: 43-55. Greenwich, CT: JAI Press.

Reference Type: BACKGROUND

Other Identifiers

PCCE202324

Identifier Type: -

Identifier Source: org_study_id