California Council on Science and Technology (CCST) studies are viewed as valuable and credible because of the organization’s reputation for providing independent, objective, and nonpartisan advice with high standards of scientiﬁc and technical quality. Checks and balances are applied at every step in the study process to protect the integrity of the studies and to maintain public conﬁdence in them.
Study Process Overview
Ensuring Independent, Objective Advice
For 30 years, CCST has been advising California on issues of science and technology by leveraging exceptional talent and expertise.
CCST enlists the state’s foremost scientists, engineers, health professionals, and other experts to address the scientiﬁc and technical aspects of society's most pressing problems.
CCST studies are primarily funded by state agencies and foundations. CCST provides independent advice; external sponsors have no control over the conduct of a study once the statement of task and budget are ﬁnalized. Authors and the Steering Committee gather information from many sources in public and private meetings but they carry out their deliberations in private in order to avoid political, special interest, and sponsor inﬂuence.
Deﬁning the Study
Before the author and Steering Committee selection process begins, CCST staff and members work with sponsors to determine the speciﬁc set of questions to be addressed by the study in a formal "statement of task," as well as the duration and cost of the study. The statement of task deﬁnes and bounds the scope of the study, and it serves as the basis for determining the expertise and the balance of perspectives needed for the study authors, Steering Committee members, and peer reviewers.
The statement of task, work plan, and budget must be approved by CCST’s Project Director in consultation with CCST leadership. This review sometimes results in changes to the proposed task and work plan. On occasion, it results in turning down studies that CCST believes are inappropriately framed or not within its purview.
Study Authors and Steering Committee (SC) Selection and Approval
Selection of appropriate authors and SC members, individually and collectively, is essential for the success of a study. All authors and SC members serve as individual experts, not as representatives of organizations or interest groups. Each expert is expected to contribute to the project on the basis of his or her own expertise and good judgment. CCST sends nominations of experts to the Oversight Committee (made up of two CCST Board Members and an outside expert) for final approval after conducting a thorough balance and conflict of interest (COI) evaluation including an in-person discussion. Any issues raised in that discussion are investigated and addressed. Members of a SC are anonymous until this process is completed.
Careful steps are taken to convene SCs that meet the following criteria:
- An appropriate range of expertise for the task. The SC must include experts with the speciﬁc expertise and experience needed to address the study's statement of task. A major strength of CCST is the ability to bring together recognized experts from diverse disciplines and backgrounds who might not otherwise collaborate. These diverse groups are encouraged to conceive new ways of thinking about a problem.
- A balance of perspectives. Having the right expertise is not sufﬁcient for success. It is also essential to evaluate the overall composition of the SC in terms of different experiences and perspectives. The goal is to ensure that the relevant points of view are, in CCST's judgment, reasonably balanced so that the SC can carry out its charge objectively and credibly.
- Screened for conﬂicts of interest. All provisional SC members are screened in writing and in a conﬁdential group discussion about possible conﬂicts of interest. For this purpose, a "conﬂict of interest" means any ﬁnancial or other interest which conﬂicts with the service of the individual because it could signiﬁcantly impair the individual's objectivity or could create an unfair competitive advantage for any person or organization. The term "conﬂict of interest" means something more than individual bias. There must be an interest, ordinarily ﬁnancial, that could be directly affected by the work of the SC. Except for those rare situations in which CCST determines that a conﬂict of interest is unavoidable and promptly and publicly discloses the conﬂict of interest, no individual can be appointed to serve (or continue to serve) on a SC used in the development of studies if the individual has a conﬂict of interest that is relevant to the functions to be performed.
- Point of View is different from Conﬂict of Interest. A point of view or bias is not necessarily a conﬂict of interest. SC members are expected to have points of view, and CCST attempts to balance these points of view in a way deemed appropriate for the task. SC members are asked to consider respectfully the viewpoints of other members, to reﬂect their own views rather than be a representative of any organization, and to base their scientiﬁc ﬁndings and conclusions on the evidence. Each SC member has the right to issue a dissenting opinion to the study if he or she disagrees with the consensus of the other members.
- Other considerations. Membership in CCST and previous involvement in CCST studies are taken into account in SC selection. The inclusion of women, minorities, and young professionals are additional considerations.
Speciﬁc steps in the SC selection and approval process are as follows:
CCST staff solicit an extensive number of suggestions for potential SC members from a wide range of sources, then recommend a slate of nominees. Nominees are reviewed and approved at several levels within CCST. A provisional slate is then approved by the Oversight Committee. Prior to approval, the provisional SC members complete background information and conﬂict-of-interest disclosure forms. The SC balance and conﬂict-of-interest discussion is held at the ﬁrst SC meeting. Any conﬂicts of interest or issues of SC balance and expertise are investigated; changes to the SC are proposed and ﬁnalized. The Oversight Committee formally approves the SC. SC members continue to be screened for conﬂict of interest throughout the life of the committee.
CCST uses a similar approach as described above for SC development to identify study authors who have the appropriate expertise and availability to conduct the work necessary to complete the study. In addition to the SC, all authors, peer reviewers, and CCST staff are screened for COI.
Author and Steering Committee Meetings, Information Gathering, Deliberations, and Drafting the Study
Authors and the Steering Committee typically gather information through:
- submission of information by outside parties;
- reviews of the scientiﬁc literature; and
- investigations by the study authors and/or SC members and CCST staff.
In all cases, efforts are made to solicit input from individuals who have been directly involved in, or who have special knowledge of, the problem under consideration.
The authors shall draft the study and the SC shall draft findings and recommendations. The SC deliberates in meetings closed to the public in order to develop draft ﬁndings and recommendations free from outside inﬂuences. All analyses and drafts of the study remain conﬁdential.
As a ﬁnal check on the quality and objectivity of the study, all full commissioned reports must undergo a rigorous, independent external peer review by experts whose comments are provided anonymously to the authors and SC members. CCST recruits independent experts with a range of views and perspectives to review and comment on the draft report prepared by the authors and the SC.
The review process is structured to ensure that each report addresses its approved study charge, that the ﬁndings are supported by the scientiﬁc evidence and arguments presented, that the exposition and organization are effective, and that the report is impartial and objective.
The authors and the SC must respond to, but need not agree with, reviewer comments in a detailed "response to review" that is examined by one or more independent “report monitor(s)" responsible for ensuring that the report review criteria have been satisﬁed. After all SC members and appropriate CCST ofﬁcials have signed off on the ﬁnal report, it is transmitted to the sponsor of the study and the sponsor can release it to the public. Sponsors are not given an opportunity to suggest changes in reports. All reviewer comments and SC deliberations remain confidential. The names and afﬁliations of the report reviewers are made public when the report is released.