Study Process

For 30 years, the California Council on Science and Technology (CCST) has been advising California on issues of science and technology by leveraging exceptional talent and expertise. CCST studies are viewed as valuable and credible because of the organization’s reputation for providing independent, objective, and nonpartisan advice with high standards of scientiﬁc and technical quality. Checks and balances are applied at every step in the study process to protect the integrity of the studies and to maintain public conﬁdence in them.

CCST Entities Involved in the Study Process

The study process, including accepting and defining projects and building the teams to carry them out, involves a number of entities that are a part of CCST.

CCST Leadership – Consisting of the CCST Executive Director and the CCST Deputy Director, these positions are generally involved in interfacing with the sponsor and working through the initial ideation of the project and securing the contract. They work with the Board on all steps after ideation.
CCST Board of Directors (“Board”) – Consisting of directors from CCST’s academic and research partner institutions as well as independent directors often from industry, philanthropy or with a policy background. The Board gives final approval to take on a peer-reviewed report.
Program Committee – A subcommittee of the CCST Board, the Program Committee oversees and advises the programs by which CCST fulfills its mission to provide science advice to inform decision-making in the State of California. The Program Committee provides oversight throughout the study process.

Study Process Overview

Ensuring Independent, Objective Advice

CCST enlists the state’s foremost scientists, engineers, health professionals, and other experts to address the scientiﬁc and technical aspects of society's most pressing problems.

CCST studies are funded by state agencies, foundations, and other private sponsors. CCST provides independent advice; external sponsors have no control over the conduct of a study once the statement of task and budget are ﬁnalized. Authors and the Steering Committee gather information from many sources in public and private meetings, but they carry out their deliberations in private in order to avoid political, special interest, and sponsor inﬂuence. After the report has been drafted, it undergoes a rigorous peer review process, overseen by an independent Report Monitor who ensures all Peer Reviewer comments are sufficiently considered..

Stage 1:

Deﬁning the Study

Before the author(s) and Steering Committee selection process begins, CCST staff, and other CCST experts as needed and informed by the CCST Program Committee work with the study sponsors to determine the speciﬁc set of questions to be addressed by the study in a formal "statement of task," as well as the duration and cost of the study. In line with CCST’s dedication to supporting diversity, equity, and inclusion (DEI) through its work, CCST intentionally integrates the social sciences and questions of equity. The statement of task deﬁnes and bounds the scope of the study, and it serves as the basis for determining the expertise and the balance of perspectives needed for the study authors, Steering Committee members, and peer reviewers.

The statement of task, work plan, and budget must be approved by CCST leadership in consultation with CCST’s Project Director. This review sometimes results in changes to the proposed task and work plan. On occasion, it results in turning down studies that CCST believes are inappropriately framed or not within its purview.

Stage 2:

Study Authors and Steering Committee (SC) Selection and Approval

Selection of appropriate authors and SC members, individually and collectively, is essential for the success of a study. CCST intentionally recruits a diverse team of experts. All authors and SC members serve as individual experts, not as representatives of organizations or interest groups. Each expert is expected to contribute to the project on the basis of his or her own expertise and good judgment.

To build the SC and Author teams, CCST staff solicit an extensive number of suggestions for potential SC members and authors from a wide range of sources, then recommend a slate of nominees, and send invitations to each provisional SC member and author to complete a non-disclosure agreement (NDA), a conflict of interest (COI) form and submit their current Curriculum Vitaes (CVs). The NDA is essential for ensuring an environment which supports frank and open discussion among study participants, both in establishing the team and as the study is ongoing. CCST staff send the COIs and current CVs to outside counsel for a thorough COI review and then organize all results and recommendations from the outside counsel. CCST organizes an in-person meeting for the provisional SC and lead authors to discuss the balance of the committee and evaluate each person for any potential COIs based on the outside counsel feedback. Any issues raised in this discussion are investigated and addressed. CCST sends the proposed study participant list and associated COI information, including any recommendations or concerns noted at the in-person meeting, to the Program Committee of the CCST Board for final approval. In some cases, the Program Committee is asked to review potential COIs ahead of the in-person SC meeting at the discretion of CCST Leadership. While the lead authors attend the in-person meeting for the discussion of their own potential COIs, they do not contribute to the discussion of the provisional SC Members’ COIs. Members of a SC and the lead author(s) are anonymous until this process is completed.

Careful steps are taken to convene SCs that meet the following criteria:

An appropriate range of expertise for the task.The SC must include experts with the speciﬁc expertise and experience needed to address the study's statement of task. A major strength of CCST is the ability to bring together recognized experts from diverse disciplines and backgrounds who might not otherwise collaborate. These diverse groups are encouraged to conceive new ways of thinking about a problem.

A balance of perspectives. Having content expertise is not sufﬁcient for success. It is also essential to evaluate the overall composition of the SC in terms of different experiences and perspectives. The goal is to ensure that the relevant points of view are, in CCST’s and the Program Committee’s judgment, reasonably balanced so that the SC can carry out its charge objectively and credibly.

Screened for conﬂicts of interest. All provisional SC members are screened in writing and in a conﬁdential group discussion about possible conﬂicts of interest. For this purpose, a "conﬂict of interest" means any ﬁnancial or other interest which conﬂicts with the individual’s service because it could signiﬁcantly impair the individual's objectivity or could create an unfair competitive advantage for any person or organization. The term "conﬂict of interest" is beyond individual bias. There must be an interest, ordinarily ﬁnancial, that could influence the work of the SC or that could be directly affected by the work of the SC, for an individual to be disqualified from serving. Except for a rare situation in which CCST and the Program Committee determine that a conﬂict of interest is unavoidable and promptly and publicly disclose the conﬂict of interest, no individual will be appointed to serve (or continue to serve) on a SC used in the development of studies while having a conﬂict of interest relevant to the required functions.

SC members and authors continue to be screened for conﬂict of interest at regular intervals throughout the life of the committee. (In addition to the SC and Authors, co-authors, peer reviewers and CCST staff working on each project are also screened for COI.)

Point of View is different from Conﬂict of Interest. A point of view or bias is not necessarily a conﬂict of interest. SC members are expected to have points of view, and CCST attempts to balance these points of view in a way deemed appropriate for the task. SC members are asked to consider respectfully the viewpoints of other members, to reﬂect their own views rather than be a representative of any organization, and to base their scientiﬁc ﬁndings and conclusions on the evidence. Each SC member has the right to issue a dissenting opinion to the study if he or she disagrees with the consensus of the other members. COIs are updated throughout the study process to capture any new or updated information and to ensure a continued lack of conflicts.

Diversity. CCST members are often asked to serve on an SC, though membership in CCST is not a requirement SC selection. CCST seeks a diverse SC in all dimensions, including women, individuals from underrepresented groups, and professionals in varying career stages where available.

Stage 3:

Author and Steering Committee Meetings, Information Gathering, Deliberations, and Drafting the Study

Authors and the Steering Committee typically gather information through:

meetings;
submission of information by outside parties;
reviews of the scientiﬁc literature; and
investigations by the study authors and/or SC members and CCST staff.

In all cases, efforts are made to solicit input from individuals who have been directly involved in, or who have special knowledge of, the problem under consideration.

The lead author(s) maintain continued communication with the SC as the study progresses through frequent updates and background meetings.

For larger reports, lead authors may request additional authors to ensure the appropriate expertise is included. Every author must be approved by the SC Chair(s) and CCST staff. Some of the additional authors may become section leads. The lead author reviews and approves the work of all other chapter authors, including section leads.

During the course of a report, authors’ duties may shift which may change the lead author or section lead designations. Any such changes must be made in conjunction with CCST staff and the SC Chair(s). If the reorganization of author responsibilities or the addition of a new author raises conflict of interest concerns, they are presented to and resolved by the Program Committee.

The authors shall draft the study and the SC shall draft the Executive Summary which includes findings, conclusions, and recommendations (FCRs). The SC deliberates in meetings closed to the public in order to develop FCRs free from outside inﬂuences. All interim analyses and drafts of the study remain conﬁdential.

Stage 4:

Report Review

As a ﬁnal check on the quality and objectivity of the study, all CCST full commissioned reports must undergo a rigorous, independent external peer review by experts whose comments are provided anonymously to the authors and SC members. CCST recruits independent experts with a range of views and perspectives to review and comment on the draft report prepared by the authors and the SC. The proposed list of peer reviewers is approved by the Program Committee to ensure all report sections are adequately reviewed.

The review process is structured to ensure that each report addresses its approved study charge, that the ﬁndings are supported by the scientiﬁc evidence and arguments presented, that the exposition and organization are effective, and that the report is impartial and objective. Peer Reviewers will be made aware of any COIs that have been disclosed on the website by CCST.

The authors and the SC must respond to, but need not agree with, reviewer comments in a detailed "response to review" that is examined by one or more independent “report monitor(s)" responsible for ensuring that the report review criteria have been satisﬁed. After all SC members and appropriate CCST ofﬁcials have signed off on the ﬁnal report, it is transmitted to the sponsor of the study and the sponsor or CCST can release it to the public. Sponsors are not given an opportunity to suggest changes to the content of the reports though may ask clarifying questions about findings, conclusions, and recommendations. All reviewer comments and SC deliberations remain confidential. The names and afﬁliations of the report reviewers are made public when the report is released.

About the Peer Review Process

Also, view our:

Peer Review Process

See All Publications

Our Mission

To engage leading experts in science and technology to advise State policymakers - ensuring that California policy is strengthened and informed by scientific knowledge, research, and innovation.