The California Council on Science and Technology (CCST) regularly issues important, peer-reviewed
reports authored by the State's foremost technical experts to address some of society's toughest
challenges. CCST is committed to the highest principles in providing independent, objective and
respected high-quality work. (Click here for an overview of CCST's study process.)
All work that bears CCST's name is reviewed by Board Members, Council
Members and selected Senior Fellows. In addition, outside reviewers and experts in the report's
field are asked to edit and review final reports. This rigorous peer review ensures the specific
issue being addressed is done so in a targeted way with results that are clear and sound. Reviewers
can include experts from academia, the private sector, state agencies, national laboratories, and
The process of peer review is the cornerstone of the research evaluation process in the physical
sciences, life sciences, social sciences and engineering. The great success of the American
research enterprise of the past 40 years has been attributed in large measure to the evaluation of
scientific publications through the peer review process. As the name implies, peer review is the
process of the evaluation of the scientific and technical merit of the proposed publication by a
panel of reviewers with direct expertise in the area of research who have no personal stake or
interest in the outcome of the evaluation process. The salient features of the peer review process
are the evaluation of the research program by "peer" experts in relevant fields who are deemed
qualified to evaluate the product based solely on the scientific and technical merit of the
All CCST reports are peer reviewed using guidelines and processes established by CCST to assure the highest scientific and technical standards. Guidelines are similar to those of the National Academy of Sciences, adapted to be appropriate for California. Reviewers are not asked to approve the report or to replace the scientific judgment of the committee with their own, but rather asked to indicate whether:
- the report is clear and concise,
- its arguments and conclusions appear to rest on adequate data, properly represented,
- uncertainties in the data are recognized,
- policy matters are handled objectively,
- the report is unbiased, and
- the report seems to be complete, fair, and responsive to the committee's charge.
This process ensures the credibility and authority of every CCST report by subjecting it to critical review by a body of peers highly knowledgeable in the subject matter. Adherence to the review process protects against the report taking a narrow or parochial view of a problem, or failing to consider fully or properly document data or information pertinent to the issue under review. The process is particularly aggressive in differentiating opinions and judgment from findings based on data.
The timeline for the review process depends on the scope and size of the report and takes approximately three to six months.
Phase 1 - Identify Appropriate Expert Reviewers
- The number of reviewers depends on size and scope of the report
- All of Phase 1 can be completed concurrently with development of report
CCST will select independent scientific expert reviewers for review of products and reports based upon expert recommendations.
- Review Oversight - CCST selects one or more individuals who will oversee the review process. Persons serving in this capacity are termed Review Monitors. If more than one Review Monitor is appointed, the report under review can be divided into sections to distribute review load. A Review Monitor will serve as liaison to CCST staff and Board.
- CCST evaluates the product to determine the range of expertise needed. CCST generates an evaluation request in order to compile internal recommendations for expert reviewers.
- Peer Reviewers - CCST sends the expert review request to recognized experts. The expert may suggest him/herself. CCST compiles the final recommendation list, and further vets it through members of CCST's Board, Council and other channels, and makes a final determination for reviewers using the criteria below:
- Expertise: The reviewer should have demonstrated knowledge, experience, and skills in an area relevant to the product or report.
- Objectivity: The reviewer should be able to provide an objective, open minded, and thoughtful review, in the best interest of a thorough and objective review of the product. In addition, the reviewer should be comfortable sharing his or her knowledge and perspectives and openly identifying his or her knowledge gaps.
- Conflict of Interest: The reviewer should not have any financial or other interest that conflicts or that could impair his or her objectivity or create an unfair competitive advantage (see section on conflict of interest below).
- CCST contacts and engages expert reviewers. The names and affiliations of the expert reviewers who will be published in the final report.
- CCST prepares detailed review instructions and questions based on the report and the needs of the client. This is presented to the client for review. CCST provides these instructions and a copy of the report to the reviewers.
Phase 2 - The Evaluation Process
- Reviewers are asked to prepare written responses according to the instructions for the specific product. (3-5 weeks)
- CCST receives reviewer comments and compiles and organizes reviewer comments into a summary document, which is sent to the Project Lead. (1 week)
- Project Lead reviews comments and assigns comments to authors and project steering committee members. (2 weeks)
Phase 3 - Comments Summary and Response to Review2
- The authors and steering committee members will respond to each of the comments and amend the report as deemed appropriate. (3 weeks)
- Project Lead compiles the responses to comments and consolidates the revised draft. (1 week)
- Review Monitors review the revised report and response to comments to determine if the comments have been adequately addressed. (3 weeks)
- Project Lead addresses any needs or concerns raised by Review Monitors (1-3 weeks)
- CCST sends to editor for technical edits. (2 weeks)
- Project Lead conducts final review before report goes to layout. (1 week)
- Finally, CCST prepares and provides to the client, the final report. (2-4 weeks)
CCST conducts a rigorous conflict of interest evaluation that is summarized below.
A reviewer cannot review a product if any of the parties below have a specific financial interest in the outcome of the review:
- The reviewer, the reviewer's spouse, minor child, or business partner;
- The organization where the reviewer is employed, has an arrangement for future employment or is negotiating for employment; or
- The organization where the reviewer is an officer, director, trustee, or partner.
Furthermore, a potential reviewer is asked to disclose other potential conflicts of interest, including having a personal relationship with an author, such as a close relative, current or former collaborator, or former thesis student/advisor. Potential conflicts will be considered and may exclude a potential reviewers' participation.
A potential reviewer is instructed to disclose if the report involves an institution or other entity with which the potential reviewer has an association. Conflicts of association include:
- A reviewer's recent former employer;
- An organization in which the reviewer is an active participant;
- An institution at which the reviewer is currently enrolled as a student, or at which he/she serves as a visiting committee member; or
- An entity with which the reviewer had, has or seeks some other business or financial relationship (including receipt of an honorarium).
Conflicts of association, such as those above, may preclude a reviewer's participation.
1This process can be changed if deemed necessary by the CCST Board.
2This phase may require additional iterations of review depending on comments received and feedback from Review Monitors.