Facebook Twitter LinkedIn
Board of Directors
Council
Board of Directors
CalTAC
Policy Fellows
Institutes & Affiliates
Staff
LOGIN
Board/Council
Staff
1130 K Street, Suite 280, Sacramento, CA, 95814-3965, (916) 492-0996
Support CCST
About CCST

OUR MISSION

To engage leading experts in science and technology to advise State policymakers - ensuring that California policy is strengthened and informed by scientific knowledge, research, and innovation.

PEER REVIEW PROCESS

The California Council on Science and Technology (CCST) regularly issues important, peer-reviewed reports authored by the State's foremost technical experts to address some of society's toughest challenges. Often, CCST is requested to provide advice on specific topics such as waste to energy, smart meters, telemedicine, and nanotechnology. CCST members also bring issues that are important to the state policy makers, including innovation capacity, transportation, energy research, science and mathematics education and economic development. CCST staff convene experts from the CCST Board and Council as well as other scientific experts to meet with legislators, state agency officials, the Governor and Lieutenant Governor and their staffs, as well as federal and private sector representatives to consult on issues that have scientific or technological components. CCST organizes briefings, convenes meetings, and reviews more than 100 bills introduced in the California legislature each year that involve science and technology. It has become a valued resource within California and a model of involvement for scientists, engineers, health professionals and policy makers in other states.

CCST has the highest principles in providing independent, objective and respected high-quality work. All work that bears CCST's name is reviewed by Board Members, Council Members and selected Senior Fellows. In addition, outside reviewers and experts in the report's field are asked to edit and review final reports. All peer reviewers are kept confidential until the study is published to ensure the integrity of the process by preventing outside influence and allowing reviewers to provide candid anonymous feedback to the study authors. This rigorous peer review results in a protocol that ensures the specific issue being addressed is done so in a targeted way with results that are clear and sound. Reviewers can include experts from academia, the private sector, state agencies, national laboratories, and non-profit organizations.

The process of peer review is the cornerstone of the research evaluation process in the physical sciences, life sciences and engineering. The great success of the American research enterprise of the past 40 years has been attributed in large measure to the unbiased and objective evaluation of proposed research projects through the peer review process. As the name implies, peer review is the process of the evaluation of the scientific and technical merit (and likelihood of success) of the proposed research project/program by a panel of reviewers with direct expertise in the area of research to be evaluated who have no personal stake or interest in the outcome of the evaluation process. The salient features of the peer review process are the evaluation of the research program by "peer" experts in relevant fields who are deemed qualified to evaluate the product based solely on the scientific and technical merit of the content. It is standard practice to keep the identity of peer reviewers confidential as well as all of the comments and deliberations.

All CCST reports are peer reviewed using guidelines and processes established by CCST to assure the highest scientific and technical standards. Guidelines are similar to those of the National Academy of Sciences, adapted to be appropriate for California. Reviewers are not asked to approve the report or to replace the scientific judgment of the committee with their own, but rather asked to indicate whether:

  1. the report is clear and concise,
  2. its arguments and conclusions appear to rest on adequate data, properly represented,
  3. uncertainties in the data are recognized,
  4. policy matters are handled objectively,
  5. the report reveals or suggests bias, and
  6. the report seems to be complete, fair, and responsive to the committee's charge.

This process ensures the credibility and authority of every CCST report by subjecting it to critical review by a body of peers highly knowledgeable in the subject matter. Adherence to the review process protects against the report taking a narrow or parochial view of a problem, or failing to consider fully or properly document data or information pertinent to the issue under review. The process is particularly aggressive in differentiating opinions and judgment findings of fact well grounded in research.

CCST Report Review Process1

The timeline for the review process depends on the scope and size of the report and takes approximately three to six months.

Phase 1 - Identify Appropriate Expert Reviewers

  • The number of reviewers depends on size and scope of the report
CCST will select independent scientific expert reviewers for review of products and reports based upon expert recommendations.
  1. Review Oversight - CCST selects one or more individuals who will oversee the review process. Persons serving in this capacity are termed Review Monitors. If more than one Review Monitor is appointed, the report under review can be divided into sections to distribute review load. A member of the Board of Directors will serve as liaison to the Board.

  2. CCST evaluates the product to determine the range of expertise needed. CCST generates an evaluation request in order to compile internal recommendations for expert reviewers.

  3. Peer Reviewers - CCST sends the expert review request to recognized experts. The expert may suggest him/herself. CCST compiles the final recommendation list, and further vets it through members of CCST's Board, Council and other channels, and makes a final determination for reviewers using the criteria below:
    • Expertise: The reviewer should have demonstrated knowledge, experience, and skills in an area relevant to the product or report.
    • Objectivity: The reviewer should be able to provide an objective, open minded, and thoughtful review, in the best interest of a thorough and objective review of the product. In addition, the reviewer should be comfortable sharing his or her knowledge and perspectives and openly identifying his or her knowledge gaps.
    • Conflict of Interest: The reviewer should not have any financial or other interest that conflicts or that could impair his or her objectivity or create an unfair competitive advantage (see section on conflict of interest below).


  4. CCST contacts and engages expert reviewers. The names and affiliations of the expert reviewers who accept the request will be provided to the client.

Phase 2 - The Evaluation Process

  1. CCST prepares detailed review instructions and questions based on the report and the needs of the client. This is presented to the client for review. CCST provides these instructions and a copy of the report to the reviewers.
  2. Reviewers are asked to prepare written responses according to the instructions. CCST receives reviewer comments and compiles and organizes reviewer comments into a summary document.

Phase 3 - Comments Summary and Response to Review

  1. CCST will present all the written comments received to the project committee.
  2. The project committee will address each of the comments and amend the report as deemed appropriate.
  3. The revised report will be reviewed by the Review Monitor to determine if the comments have been adequately addressed.
  4. Finally, CCST prepares and provides to the client, the final report.

Conflicts of Interest

A reviewer cannot review a product if any of the parties below have a specific financial interest in the outcome of the review:

  • The reviewer, the reviewer's spouse, minor child, or business partner;
  • The organization where the reviewer is employed, has an arrangement for future employment or is negotiating for employment; or
  • The organization where the reviewer is an officer, director, trustee, or partner.
Furthermore, a potential reviewer is asked to disclose other potential conflicts of interest, including having a personal relationship with an author, such as a close relative, current or former collaborator, or former thesis student/advisor. Potential conflicts will be considered and may exclude a potential reviewers' participation.

A potential reviewer is instructed to disclose if the report involves an institution or other entity with which the potential reviewer has an association. Conflicts of association include:

  • A reviewer's recent former employer;
  • An organization in which the reviewer is an active participant;
  • An institution at which the reviewer is currently enrolled as a student, or at which he/she serves as a visiting committee member; or
  • An entity with which the reviewer had, has or seeks some other business or financial relationship (including receipt of an honorarium).
Conflicts of association, such as those above, may preclude a reviewer's participation.


1This process can be changed if deemed necessary by the CCST Board.

Updated 9/16/17
COPYRIGHT © 2003-2017, CALIFORNIA COUNCIL ON SCIENCE AND TECHNOLOGY. ALL RIGHTS RESERVED.