The California Council on Science and Technology (CCST) regularly issues important, peer-reviewed
reports authored by the State's foremost technical experts to address some of society's toughest
challenges. Often, CCST is requested to provide advice on specific topics such as waste to energy,
smart meters, telemedicine, and nanotechnology. CCST members also bring issues that are important to
the state policy makers, including innovation capacity, transportation, energy research, science and
mathematics education and economic development. CCST staff convene experts from the CCST Board and
Council as well as other scientific experts to meet with legislators, state agency officials, the
Governor and Lieutenant Governor and their staffs, as well as federal and private sector
representatives to consult on issues that have scientific or technological components. CCST
organizes briefings, convenes meetings, and reviews more than 100 bills introduced in the California
legislature each year that involve science and technology. It has become a valued resource within
California and a model of involvement for scientists, engineers, health professionals and policy
makers in other states.
CCST has the highest principles in providing independent, objective and respected high-quality
work. All work that bears CCST's name is reviewed by Board Members, Council Members and selected
Senior Fellows. In addition, outside reviewers and experts in the report's field are asked to edit
and review final reports. All peer reviewers are kept confidential until the study is published to
ensure the integrity of the process by preventing outside influence and allowing reviewers to
provide candid anonymous feedback to the study authors. This rigorous peer review results in a
protocol that ensures the specific issue being addressed is done so in a targeted way with results
that are clear and sound. Reviewers can include experts from academia, the private sector, state
agencies, national laboratories, and non-profit organizations.
The process of peer review is the cornerstone of the research evaluation process in the physical
sciences, life sciences and engineering. The great success of the American research enterprise of
the past 40 years has been attributed in large measure to the unbiased and objective evaluation of
proposed research projects through the peer review process. As the name implies, peer review is the
process of the evaluation of the scientific and technical merit (and likelihood of success) of the
proposed research project/program by a panel of reviewers with direct expertise in the area of
research to be evaluated who have no personal stake or interest in the outcome of the evaluation
process. The salient features of the peer review process are the evaluation of the research program
by "peer" experts in relevant fields who are deemed qualified to evaluate the product based solely
on the scientific and technical merit of the content. It is standard practice to keep the identity
of peer reviewers confidential as well as all of the comments and deliberations.
All CCST reports are peer reviewed using guidelines and processes established by CCST to assure
the highest scientific and technical standards. Guidelines are similar to those of the National
Academy of Sciences, adapted to be appropriate for California. Reviewers are not asked to approve
the report or to replace the scientific judgment of the committee with their own, but rather asked
to indicate whether:
- the report is clear and concise,
- its arguments and conclusions appear to rest on adequate data, properly represented,
- uncertainties in the data are recognized,
- policy matters are handled objectively,
- the report reveals or suggests bias, and
- the report seems to be complete, fair, and responsive to the committee's charge.
This process ensures the credibility and authority of every CCST report by subjecting it to
critical review by a body of peers highly knowledgeable in the subject matter. Adherence to the
review process protects against the report taking a narrow or parochial view of a problem, or
failing to consider fully or properly document data or information pertinent to the issue under
review. The process is particularly aggressive in differentiating opinions and judgment findings of
fact well grounded in research.
The timeline for the review process depends on the scope and size of the report and takes
approximately three to six months.
Phase 1 - Identify Appropriate Expert Reviewers
- The number of reviewers depends on size and scope of the report
CCST will select independent scientific expert reviewers for review of products and reports based upon expert recommendations.
- Review Oversight - CCST selects one or more individuals who will oversee the review
process. Persons serving in this capacity are termed Review Monitors. If more than one Review
Monitor is appointed, the report under review can be divided into sections to distribute review
load. A member of the Board of Directors will serve as liaison to the Board.
- CCST evaluates the product to determine the range of expertise needed. CCST
generates an evaluation request in order to compile internal recommendations for expert reviewers.
- Peer Reviewers - CCST sends the expert review request to recognized experts. The
expert may suggest him/herself. CCST compiles the final recommendation list, and further vets it
through members of CCST's Board, Council and other channels, and makes a final determination for
reviewers using the criteria below:
- Expertise: The reviewer should have demonstrated knowledge, experience, and skills in
an area relevant to the product or report.
- Objectivity: The reviewer should be able to provide an objective, open minded, and
thoughtful review, in the best interest of a thorough and objective review of the product. In
addition, the reviewer should be comfortable sharing his or her knowledge and perspectives and
openly identifying his or her knowledge gaps.
- Conflict of Interest: The reviewer should not have any financial or other interest that
conflicts or that could impair his or her objectivity or create an unfair competitive advantage (see
section on conflict of interest below).
- CCST contacts and engages expert reviewers. The names and affiliations of the expert
reviewers who accept the request will be provided to the client.
Phase 2 - The Evaluation Process
- CCST prepares detailed review instructions and questions based on the report and the
needs of the client. This is presented to the client for review. CCST provides these instructions
and a copy of the report to the reviewers.
- Reviewers are asked to prepare written responses according to the instructions. CCST receives reviewer comments and compiles and organizes reviewer comments into a summary document.
Phase 3 - Comments Summary and Response to Review
- CCST will present all the written comments received to the project committee.
- The project committee will address each of the comments and amend the report as deemed appropriate.
- The revised report will be reviewed by the Review Monitor to determine if the comments have been adequately addressed.
- Finally, CCST prepares and provides to the client, the final report.
A reviewer cannot review a product if any of the parties below have a specific financial interest in the outcome of the review:
- The reviewer, the reviewer's spouse, minor child, or business partner;
- The organization where the reviewer is employed, has an arrangement for future employment or is negotiating for employment; or
- The organization where the reviewer is an officer, director, trustee, or partner.
Furthermore, a potential reviewer is asked to disclose other potential conflicts of interest, including having a personal relationship with an author, such as a close relative, current or former collaborator, or former thesis student/advisor. Potential conflicts will be considered and may exclude a potential reviewers' participation.
A potential reviewer is instructed to disclose if the report involves an institution or other entity with which the potential reviewer has an association. Conflicts of association include:
- A reviewer's recent former employer;
- An organization in which the reviewer is an active participant;
- An institution at which the reviewer is currently enrolled as a student, or at which he/she serves as a visiting committee member; or
- An entity with which the reviewer had, has or seeks some other business or financial relationship (including receipt of an honorarium).
Conflicts of association, such as those above, may preclude a reviewer's participation.
1This process can be changed if deemed necessary by the CCST Board.