Evaluation of the Research to Policy Collaboration Model
NCT ID: NCT03671434
Last Updated: 2021-10-28
Study Results
The study team has not published outcome measurements, participant flow, or safety data for this trial yet. Check back later for updates.
Basic Information
Get a concise snapshot of the trial, including recruitment status, study phase, enrollment targets, and key timeline milestones.
COMPLETED
NA
322 participants
INTERVENTIONAL
2019-01-08
2021-02-17
Brief Summary
Review the sponsor-provided synopsis that highlights what the study is about and why it is being conducted.
This study assesses both processes for collaboration and policymakers' use of research within a randomized controlled trial (RCT) employing a mixed methods approach-including quantitative and qualitative evaluation of impact. The proposed project will be guided by three overarching questions:
1. How does the RPC impact researchers and legislative staff?
2. How does the RPC impact legislative activity?
3. How might perceptions and experiences of collaboration through the RPC relate to different forms of evidence use among researchers and policymakers?
The RPC's effectiveness will be tested through experimental design (randomization) using qualitative and quantitative assessments of researcher-policymaker interactions and impact. This includes surveying congressional staff and researchers, reviewing records of policymaker's public statements and introduced legislation, and conducting qualitative interviews around researchers' and legislative staffs' experiences with researcher-policymaker collaboration prior to and during the RPC.
Detailed Description
Dive into the extended narrative that explains the scientific background, objectives, and procedures in greater depth.
Intervention: The Research-to-Policy Collaboration (RPC) builds upon theoretical and empirical literature that suggests the need to (a) translate research relevant to current policy priorities, and (b) facilitate productive interactions between policymakers and researchers. The RPC is a nonpartisan, manualized model implemented by an intermediary organization that cultivates relationships between researchers and legislative offices. Implementation occurs in two phases involving a series of seven interrelated activities. During the capacity-building phase, the RPC simultaneously aims to support researcher's development of honest knowledge broker skills with training and coaching, while also conducting iterative needs assessments with congressional offices regarding their current priorities and desire for research evidence. In the collaboration phase, researchers with relevant expertise are coalesced into rapid response teams that are matched with legislative offices. Through a scaffolded series of interactions, the RPC model is used to cultivate productive researcher-policymaker relationships and support responses to legislative requests. Evaluating the effectiveness of the RPC is the focus of the current study.
Study Design:
This study employs both qualitative and quantitative analyses of survey, interview, and observational data within the context of an RCT. Three overarching research questions guide this work and align with the study's three different components - self-report via a quantitative survey, observation of research use via coded legislative activity, and ethnography via in-depth interviews and participant observations.
1. How does the RPC impact researchers and legislative staff? (Self-Report)
1. Do researchers report improved policy competencies and motivation for conducting policy-relevant research?
2. Will legislative staff report increases in positive attitudes toward, knowledge of, intended application, and actual use of research evidence?
3. Might the impact on legislative and research participants will vary as a function of aspects of their collaborative relationship (e.g., enhanced by mutual trust)?
2. How does the RPC impact legislative activity? (Observation)
a) Will congressional offices increase their use of research evidence in legislative activities (i.e., bills, official statements)?
3. How might perceptions and experiences of collaboration through the RPC relate to different forms of evidence use among researchers and policymakers? (Ethnography)
1. What are the barriers and facilitators of participation in the RPC?
2. What types of interactions between researchers and policymakers does the RPC influence?
3. How does the RPC affect participants' assumptions about each other, interactions with each other, and their intentions?
4. What types of evidence use are supported by RPC participation?
Sampling Framework All study components draw from an initial sampling framework through which researchers and congressional staff who opt into participating in the RPC will be asked to participate in the study; those who agree to participate in the study will be randomized for either the full RPC intervention or a light-touch control group that receives little direct engagement. This study will oversample to combat anticipated attrition. The final sample includes 96 congressional offices (48 in RPC, 48 in control group) and 226 researchers (151 in RPC, 75 in control group). These samples were derived from an identified population of eligible intervention participants.
* A population of relevant legislative offices (Senate and House) will be identified based on the number of child/family bills legislator have (co)sponsored. The random selection process will be stratified to ensure proportional partisan representation and representation of legislators who range in activity related to child welfare. Prior to randomization, the population of relevant legislators will be assigned an Activity Ranking based on the number of times a legislator has sponsored a child/family policy bill. Activity Rankings will range from 1-10 such that 10% of legislators who (co)sponsored the fewest bills will be ranked 1, and 10% of legislators who co-sponsored the most bills will be ranked 10. Recruitment and stratified sampling across a continuum of activity on child/family policy will improve bipartisan representation of the sample. Also aiding with bipartisan representation are targeted outreach efforts that intensify along party lines if participation becomes skewed.
* Researchers who work on relevant child/family research will be identified via research institutions, professional networks and listservs, literature searches, and referrals from other researchers. Researchers include research-oriented practitioners and program evaluators from both academic and non-academic settings who are interested in translating their expertise with policymakers. Subsequent to opting-in to participate in a policy engagement intervention, researchers are asked to participate in the study. Study participants will be randomized to receive the RPC or the control condition.
Study Component 1: Quantitative Evaluation of Self-Reported RPC Impact A structured survey will be used to assess researchers' capacity for engaging in public policy processes. A corresponding survey protocol will assess legislative staff's attitudes, knowledge, intent, and reported behavior regarding the use of research evidence. Both researchers and legislative staff will be asked about the nature of their collaborative interactions with one another. Surveys will be conducted approximately every three months. Survey constructs are described in outcome measures. Missing data will be handled with multiple imputation because this is best practice when data are not missing at random. Multilevel modelling will be used to examine individual trajectories in outcome variables. Multiple regression will examine changes between select time points; for instance, whether researchers' reported policy skills improve between baseline and subsequent to trainings. These analyses will allow us to examine whether RPC participation is associated with increases in (1) researchers' reported efficacy to engage with policymakers, (2) legislative staff's interest and willingness to use research evidence, and (3) both researchers' and policymakers' positive sentiment toward working together. Longitudinal analyses will further allow us to explore the trajectory of outcome variables over time (e.g., improved researcher competencies, legislators' use of research)-specifically modelling change in evidence use and other outcomes of interest. Analyses of researcher and legislative staff outcomes are all powered at least at a 90% level to detect effect sizes of 0.2 or greater (power analysis conducted with Optimal Design plus Empirical Evidence).
Study Component 2: Quantitative Evaluation of Observed RPC Impact on Legislative Activity To complement the self-reported quantitative assessment of change, use of research evidence in legislative activities will be observed before and following RPC participation. Data will be selected, extracted, and coded prior to quantitative analyses with indicators derived from deductive codes. Legislative data will be selected based upon the use of keywords indicating relevance to child/family. Since each bill will comprise more text than is feasible to code with traditional qualitative methods, relevant provisions will be extracted prior to coding (those containing keywords). Further, all subsection titles for each sampled bill will be reviewed for potential relevance and extracted if research evidence is used but not captured otherwise by keywords. Data on legislative activity will be coded based on a detailed codebook informed by prior findings of how research has been used by policymakers and in legislation. This coding scheme will be applied by two coders and the codebook will be revised and further specified until the intercoder reliability reaches at least .70 and 30% of the data for the first time period has been coded. Subsequently, trained coders will code individually and a lead investigator will randomly sample 10% of these to ensure consistency in coding quality. These codes will be converted into quantitative indicators of research use, which are described in outcome measures. Legislative activity will be assessed annually. Multivariate Poisson regression analyses will assess change in research use in thematic codes of (1) bills and (2) official statements between the RPC intervention and control conditions. Moderation analyses will further explore the extent to which changes in legislative activity depend on the nature of collaboration experiences, other survey data (e.g., policymakers' perceptions of the value of research), and legislator characteristics (e.g., party affiliation, years of age).
Study Component 3: Ethnographic Evaluation of the RPC model The qualitative component of this study is closely linked with the quantitative work and seeks to better understand the experiences of RPC participants and processes underlying the use of research evidence in the RPC context. Using participatory and ethnographic methods, data will be collected from two sources: (1) semi-structured interviews of trial participants and RPC staff, and (2) observations of researcher trainings and researcher-staff meetings. Interview participants will be sampled from the pool of study participants randomly assigned to receive the RPC, including 11 RPC researchers and 11 RPC congressional office staff that will be interviewed prior to their involvement in the RPC, and 11 RPC researchers and 11 congressional office staff that will be interviewed following collaboration activities via the RPC. Additionally, 3 RPC staff will be interviewed prior to and following the RPC. To gain additional insight into the RPC operations and impact, observations will be conducted of trainings for researchers, formal meetings and informal interactions between researchers and congressional staff, and meetings that occur as part of any RPC collaboration. Following ethnographic best practices, "open-observations" of key meetings and training events will be conducted to enrich the interviews with more information about how attendees interact and how they respond to the RPC. Field notes and transcripts will be analysed with thematic analysis and an interpretive policy analysis that explores different rationalities and rhetoric when discussing evidence used in the collaboration.
Conditions
See the medical conditions and disease areas that this research is targeting or investigating.
Keywords
Explore important study keywords that can help with search, categorization, and topic discovery.
Study Design
Understand how the trial is structured, including allocation methods, masking strategies, primary purpose, and other design elements.
RANDOMIZED
PARALLEL
OTHER
DOUBLE
* Provider: No masking
* Investigator: No masking
* Outcome Assessor: Survey data will indicate intervention assignment; observational data will be masked such that document coders will not be aware of the condition associated with coded documents.
Study Groups
Review each arm or cohort in the study, along with the interventions and objectives associated with them.
RPC Researchers
Researchers who are assigned to the experimental group that is eligible to receive the full RPC intervention
Research-to-Policy Collaboration
The RPC is a behavioral intervention through which RPC Researchers and RPC Congressional Offices are prepared and matched for collaboration. Specifically, congressional offices are asked to identify opportunities for researcher engagement in policy efforts, researchers with expertise related to policy opportunities are identified and prepared to collaborate with congressional offices, researchers and congressional staff are matched for ongoing collaborative partnerships, and both researchers and congressional staff receive ongoing support to facilitate research translation.
RPC Congressional Offices
Congressional offices that are assigned to the experimental group that is eligible to receive the full RPC intervention
Research-to-Policy Collaboration
The RPC is a behavioral intervention through which RPC Researchers and RPC Congressional Offices are prepared and matched for collaboration. Specifically, congressional offices are asked to identify opportunities for researcher engagement in policy efforts, researchers with expertise related to policy opportunities are identified and prepared to collaborate with congressional offices, researchers and congressional staff are matched for ongoing collaborative partnerships, and both researchers and congressional staff receive ongoing support to facilitate research translation.
Control Researchers
Researchers who are assigned to an Active Comparator control group that is enrolled in a light-touch intervention
Light Touch Policy Training
Control Researchers are provided information on policy engagement via email.
Control Congressional Offices
Congressional offices that are assigned to the control group that receives no intervention
No interventions assigned to this group
Interventions
Learn about the drugs, procedures, or behavioral strategies being tested and how they are applied within this trial.
Research-to-Policy Collaboration
The RPC is a behavioral intervention through which RPC Researchers and RPC Congressional Offices are prepared and matched for collaboration. Specifically, congressional offices are asked to identify opportunities for researcher engagement in policy efforts, researchers with expertise related to policy opportunities are identified and prepared to collaborate with congressional offices, researchers and congressional staff are matched for ongoing collaborative partnerships, and both researchers and congressional staff receive ongoing support to facilitate research translation.
Light Touch Policy Training
Control Researchers are provided information on policy engagement via email.
Eligibility Criteria
Check the participation requirements, including inclusion and exclusion rules, age limits, and whether healthy volunteers are accepted.
Inclusion Criteria
Exclusion Criteria
18 Years
ALL
Yes
Sponsors
Meet the organizations funding or collaborating on the study and learn about their roles.
William T. Grant Foundation
OTHER
Penn State Social Science Research Insititute
UNKNOWN
London School of Hygiene and Tropical Medicine
OTHER
Child Trends
OTHER
Penn State University
OTHER
Responsible Party
Identify the individual or organization who holds primary responsibility for the study information submitted to regulators.
Daniel Max Crowley
Assistant Professor of Human Development and Family Studies
Principal Investigators
Learn about the lead researchers overseeing the trial and their institutional affiliations.
D. Max Crowley, Ph.D.
Role: PRINCIPAL_INVESTIGATOR
Penn State University
J. Taylor Scott, Ph.D.
Role: STUDY_DIRECTOR
Penn State University
Locations
Explore where the study is taking place and check the recruitment status at each participating site.
Child Trends
Bethesda, Maryland, United States
Penn State University
University Park, Pennsylvania, United States
London School of Hygiene & Tropical Medicine
London, , United Kingdom
Countries
Review the countries where the study has at least one active or historical site.
References
Explore related publications, articles, or registry entries linked to this study.
Crowley M, Scott JTB, Fishbein D. Translating Prevention Research for Evidence-Based Policymaking: Results from the Research-to-Policy Collaboration Pilot. Prev Sci. 2018 Feb;19(2):260-270. doi: 10.1007/s11121-017-0833-x.
Scott JT, Larson JC, Buckingham SL, Maton KI, Crowley DM. Bridging the research-policy divide: Pathways to engagement and skill development. Am J Orthopsychiatry. 2019;89(4):434-441. doi: 10.1037/ort0000389.
Provided Documents
Download supplemental materials such as informed consent forms, study protocols, or participant manuals.
Document Type: Study Protocol
Document Type: Statistical Analysis Plan
Related Links
Access external resources that provide additional context or updates about the study.
Describes the intervention
Other Identifiers
Review additional registry numbers or institutional identifiers associated with this trial.
00010061
Identifier Type: -
Identifier Source: org_study_id