Peer Review
CCST Report Review Process
The process of peer review is the cornerstone of the research evaluation process in the physical sciences, life sciences and engineering.
All CCST reports are peer reviewed using guidelines and processes established by CCST to assure the highest scientific and technical standards. Guidelines are similar to those of the National Academy of Sciences, adapted to be appropriate for California. Reviewers are not asked to approve the report or to replace the scientific judgment of the committee with their own, but rather asked to indicate whether:
- The report is clear and concise,
- Its arguments and conclusions appear to rest on adequate data, properly represented,
- Uncertainties in the data are recognized,
- Policy matters are handled objectively,
- The report reveals or suggests bias, and
- The report seems to be complete, fair, and responsive to the committee's charge.
This process ensures the credibility and authority of every CCST report by subjecting it to critical review by a body of peers highly knowledgeable in the subject matter. Adherence to the review process ensures the report does not: (1) take a narrow or parochial view of a problem, (2) fail to fully consider the breadth of relevant information, or (3) properly document data or information pertinent to the issue under review. The process is particularly aggressive in differentiating opinions from conclusions, and ensuring findings of fact are well-grounded in research.
Also, view our:
The timeline for the review process depends on the scope and size of the report and takes approximately one to three months.
Phase 1
Identify Appropriate Expert Reviewers
The number of reviewers depends on size and scope of the report.
CCST will select independent scientific expert reviewers for review of products and reports based upon expert recommendations. The selection of reviewers is approved by the Program Committee of the CCST Board prior to the start of the review period.
- Review Oversight: CCST recommends one or more individuals who will oversee the review process. Persons serving in this capacity are termed Review Monitors and their selection is reviewed and approved by the Program Committee. In the case of a very large report, more than one Review Monitor may be appointed and the report under review divided into sections to distribute review load.
- CCST evaluates the scope of the report to determine the range of expertise needed to provide a robust review. CCST then generates an evaluation request specific to the suite of needed expertise.
- Peer Reviewers: CCST sends the expert review request to the CCST Board, Advisory Panel and other recognized experts. Experts may suggest themselves as peer reviewers. CCST compiles the final recommendation list, further vets it through legal counsel and members of the Program Committee, and makes a final determination for reviewers using the criteria below:
- Expertise: The reviewer should have demonstrated knowledge, experience, and skills in an area relevant to the product or report.
- Objectivity: The reviewer should be able to provide an objective, open-minded, and thoughtful review, in the best interest of a thorough review of the study. In addition, the reviewer should be comfortable sharing his or her knowledge and perspectives and openly identifying his or her knowledge gaps.
- Conflict of Interest: The reviewer should not have any financial or other interest that conflicts or that could impair his or her objectivity or create an unfair competitive advantage (see section on conflict of interest below).
- CCST contacts and engages expert reviewers. The names and affiliations of the expert reviewers who accept the request are made public only when the report is released.
Phase 2
The Evaluation Process
- CCST prepares detailed review instructions and questions based on the report and the needs of the client. CCST provides these instructions and a copy of the report to the reviewers.
- Reviewers are asked to prepare written responses according to the instructions. CCST receives, compiles, and organizes reviewer comments into a summary document.
Phase 3
Comments Summary and Response to Review
- CCST presents the summary document with all the written comments received to the project team.
- The project team addresses each of the comments and amends the report as deemed appropriate, documenting changes made in response to comments.
- The revised report and the author response to comments is reviewed by the Review Monitor to determine if the comments have been adequately addressed.
- Finally, CCST prepares and provides to the client the final report.
Conflicts of Interest
A reviewer cannot review a product if any of the parties below have a specific financial interest in the outcome of the review:
- The reviewer, the reviewer's spouse, minor child, or business partner;
- The organization where the reviewer is employed, has an arrangement for future employment or is negotiating for employment; or
- The organization where the reviewer is an officer, director, trustee, or partner.
Furthermore, a potential reviewer is asked to disclose other potential conflicts of interest, including having a personal relationship with an author, such as a close relative, current or former collaborator, or former thesis student/advisor. Potential conflicts will be considered and may exclude a potential reviewers' participation.
A potential reviewer is instructed to disclose whether the report involves an institution or other entity with which the potential reviewer has an association. Conflicts of association include:
- A reviewer's recent former employer;
- An organization in which the reviewer is an active participant;
- An institution at which the reviewer is currently enrolled as a student, or at which he/she serves as a visiting committee member; or
- An entity with which the reviewer had, has, or seeks some other business or financial relationship (including receipt of an honorarium).
Conflicts of association, such as those above, may preclude a reviewer's participation.
All potential COIs disclosed to CCST are reviewed by CCST’s lawyers. If a COI is unavoidable, it is promptly and publicly disclosed on the CCST project website.
Our Mission
To engage leading experts in science and technology to advise State policymakers - ensuring that California policy is strengthened and informed by scientific knowledge, research, and innovation.
