RAG minutes, 10 September 2003

Minutes approved: 15 January 2004

Present: Phil Alderson, Jacqueline Birks, Chris Cates, Mike Clarke, Katharine Deane, Sonja Henderson, Dymphna Hermans, Julian Higgins, Monica Kjeldstrøm, Valeria Marinho, Heather Maxwell, Tess Moore, Rasmus Moustgaard, Simon Newton, Elizabeth Pienaar, and Lindsay Stead.

Apologies: Gerd Antes, Bill Macguire.

In attendance: Claire Allen (minutes), and Laura Mellor (The Cochrane Library Desk Editor).

Contents:
1. Welcome and apologies
2. Approval of the agenda
3. Approval of minutes and matters arising not on this agenda, from RAG meeting on 28 April 2003
4. Matters arising
5. RevMan 5 - Structure and specification
6. Dates relevant to Cochrane reviews: in RevMan and CDSR
7. Additional figures
8. Search strategies in additional tables
9. Style guidance for levels of headings in Cochrane reviews
10. Remit of RevMan Advisory Group
11. Future plans for the Information Management System
12. Any other business
13. Date of next meeting

Welcome and apologies

Mike welcomed everyone to the meeting and everyone introduced themselves to Simon Newton and Laura Pritchard from John Wiley and Sons. Mike advised that the main item for discussion was the specification for RevMan 5.

Approval of the agenda

The agenda was approved with no further additions.

Approval of minutes and matters arising not on this agenda, from RAG meeting on 28 April 2003

The minutes were approved without amendment. Rasmus agreed to revise their status on the IMSG website to reflect this.

Matters arising

Item 6 - Citations for other Cochrane reviews within a Cochrane review

Simon reported that Wiley’s processing had shown that there were some mistakes in how reviewers were citing references for all types of articles, not just other Cochrane reviews. The input of incorrect citations will have implications when linking to outside sources (e.g. through CrossRef). It was agreed that reviewers should be encouraged to enter accurate references, with particular emphasis on author, journal and year.
Action: Everyone

Mike pointed out that data from The Cochrane Library showed that citations of Cochrane reviews within Cochrane reviews might not always be accurate and this is particularly important because of the need to obtain an accurate citation rate when reviews are included in ISI (enabling an impact factor to be calculated). It was suggested that there should be guidance for reviewers in the form of a pop up box with an example reference, and that it should be possible to import the review citation into RevMan directly from The Cochrane Library. Rasmus agreed to add this to the wish list.
Action: Rasmus

Simon pointed out that Wiley plans to assign a Digital Object Identifier (DOI) for each Cochrane review. When a DOI is clicked on from another source, this will lead to the latest version of the Cochrane review. Rasmus advised that RevMan 5 would contain a field for a DOI.

Item 9 - Wishlist for RevMan

Julian advised that the Statistical Methods Group Convenors would discuss the options for statistical analyses in RevMan 5 (item 170 on the Wishlist), at their meeting during the Cochrane Colloquium in October 2003.
Action: Julian

With reference to making a .pdf builder available to reviewers (item 166 on the Wishlist), so that they can send a .pdf version of their review to, for example, co-reviewers or advisers, it was noted that the software pdfFactory has been distributed to all CRGs that expressed an interest in this. However it is not possible to distribute this software to all reviewers or include it in RevMan 4.3. Rasmus advised that the ability to produce a .pdf version of a review would eventually be included in the new IMS.

Item 14b - List of journals

Rasmus advised that he had not yet investigated the possibility of including a standard pick list of journals. Sonja pointed out the need to be able to create a master list of accurate journal names in RevMan, rather than a list that was created from all reviews in a copy of RevMan 4, because the latter would contain the inaccurate journal names. She suggested including a comprehensive list that could be edited by both reviewers and RGCs. Monica agreed to raise this issue at the IMSG meeting so that it could be discussed with Kay Dickersin (CENTRAL/CCTR Advisory Group representative on the IMSG).
Action: Monica

RevMan 5 - Structure and specification

Mike explained that by moving to a centralised information management system, structural changes should not have the same implications because some reformatting of reviews might be able to take place centrally. However, he noted that there are more than 3000 reviews and protocols in The Cochrane Database of Systematic Reviews at the moment and that structural changes would have implications for these and additional reviews prepared between now and the release of RevMan 5. Rasmus explained that this meeting needed to concentrate on structural rather than functionality changes for RevMan 5. The decisions made at this meeting are required so that the development of the new IMS can progress. Many of the items have been discussed previously but decisions on their implementation were needed now.

Cover sheet

Dates for 'What's New'

Mike explained that currently the dates displayed in the What's New' section of a Cochrane review do not capture the 'up-to-datedness' of the review. He, Frances Fairman, and Mandy Collingwood, had circulated a discussion paper explaining the background of the current 'dates fields', and their suggestions for the future. Rasmus explained that discussion needs to take place about what should be included in RevMan 5, but that discussion of which of those items should be published could take place later. Consideration also needs to be given to what belongs in RevMan and what belongs in ModMan (for example in relation to the tracking system).

Sonja explained that the dates currently displayed in RevMan had been introduced both to help inform users of The Cochrane Library of the history and current status of each Cochrane review and also to provide information for The Cochrane Collaboration’s Monitoring and Registration Group on the activity of CRGs (e.g. by tracking how many reviews are up-to-, or out-of-, date).

There was extensive discussion of this aspect of Cochrane reviews, which highlighted the different ways in which reviewers, editorial teams and users interpret the current dates. It was noted that the ‘Date of substantive update’ could be used as an alert, so it is important that it is consistently used as a prompt to re-read the review. The ‘Date of substantive update’ field also triggers a new MEDLINE citation, treating the review as a new publication. The following was agreed:
It was agreed that the ‘Date of last edit’ is vital to reviewers, and Rasmus confirmed that RevMan 5 would automatically record it.

1. Issue protocol first published – It was agreed that this should remain (it will probably be entered automatically in the new IMS). It will be recorded as issue and year (and the IMS will add the calendar date as an additional piece of information).
2. Issue review first published – It was agreed that this should remain. It was noted that the ‘Published notes’ section could be used to explain what has happened to the review since it was first published and that the ‘What’s New’ section should describe changes made since the last version of the review. The date will be recorded as issue and year (and the IMS will add the calendar date as an additional piece of information).
3. Date of last substantive update – Currently, if the ‘Date of substantive update’ has been edited since the last submission of the review, it indicates to the publisher that the review is updated and it automatically flagged. It was suggested that it would be better to have a check box and a date, and that this would be unchecked automatically once the review has been submitted to the publisher. If another substantive update is made, the reviewer should check the box again. It was suggested that RevMan should not ask the reviewer what kind of edit they have made until they have finished editing a review during a session. Simon suggested more options when checking which status applied to the edit, e.g. new, updated, unchanged, deleted, withdrawn etc. But it was noted that some of these options (such as deleted and withdrawn) were likely to be decisions for the editorial bases rather than the reviewers and, as such, are currently dealt with in ModMan. Mike agreed to circulate Simon’s suggestion after the meeting.
   Action: Mike
4. Issue and year of last substantive update – It was agreed that this should be added (and the IMS will add the calendar date for the publication, as an additional piece of information)
5. Date of last minor update - It was agreed to remove this date in the future version of RevMan
6. Date the review was most recently assessed as up-to-date – It was agreed that this addition should be forwarded to the Cochrane Library Users’ Group and Co-ordinating Editors for comment. There was concern about the need for good guidance on the use and interpretation of this date. It was agreed that a different name might be preferable.
   Action: Mike
7. Date review re-formatted – It was agreed that this date field is no longer required.
8. Date new studies sought but none found – It was agreed that a new ‘Date of last search’ would replace the current field. Reviewers should describe the outcome and impact of this search in the ‘What’s new’ section.
9. Date new studies found but not yet included/excluded – It was agreed that this date field is no longer needed. See 5.1.1vii above.
10. Date new studies found and included or excluded – It was agreed that this date field is no longer needed. See 5.1.1vii above.
11. Date reviewers’ conclusions section amended – It was agreed that this date field is no longer required.
12. Date comment/criticism added – It was agreed that this date field is no longer required, but that the relevant date should be written within each comment.
13. Date response to comment/criticism added – It was agreed that this date field is no longer required, but that the relevant date should be written within each response.

MeSH terms in review

Mike explained that the suggestion was that a field be included in the review for MeSH terms. It was agreed that reviewers should not be involved in assigning MeSH terms, but that the field might be completed by approved indexers before publication. Wiley and the RevMan development team would discuss this outside the meeting.
Action: Monica, Rasmus, Simon, Laura

Section to describe the difference between protocol and review

This section would be added to describe the difference between what the reviewers did in the review to what they said they had intended to do in the protocol. It was agreed that this section would be added to RevMan 5, but that if it were not completed, it would not be published. Tess pointed out that it was important and, so, it should be included in the text of the review with a fixed heading. It was agreed that the section should be added, and that it should be able to contain links, but where and how it will appear in RevMan will be a discussion for a future RAG meeting.

Text of review

Tables in the text

This item relates to creating tables within the main body text of a review. It has been discussed previously and agreed that, if possible, it should be included in RevMan 5. Reviewers can already add tables to reviews by using the additional tables function, but these are viewed in a separate window in RevMan. Mike suggested that where users will view the information stored in the additional tables is a publishing, not a RevMan, issue. Simon confirmed that Wiley are able to publish tables within the body text if they are reasonably straightforward.

Katharine pointed out that reviewers would like to be able to see the tables in RevMan at the same time as they are writing the body of the text. It was agreed that RevMan 5 should have functionality to insert and edit tables in the text of reviews.

More links

The facility for the reviewer to insert links to individual analysis and ‘Characteristics of ... studies’ tables was approved for implementation in RevMan 5.

Hyperlinks to web sites

A formal system for inserting hyperlinks into the main text by marking the relevant URL was approved for implementation in RevMan 5. It was agreed that it would be useful to be able to insert hyperlinks in tables as well.

Styles for headings

It was agreed that a pick list of heading styles (once text is highlighted) should be implemented in RevMan 5, in accordance with the Cochrane Style Guide.

Track changes

It had been previously agreed that a simple track changes feature within the text of the review should be implemented in RevMan 5. It was noted that that, although the current ability to compare documents in RevMan is helpful when only a few changes had been made, it was difficult (or impossible) to use productively if there are a lot of differences. Chris asked about better ways to compare documents in the future, as it would be a very useful tool for editors. Rasmus confirmed that this would be available in the new IMS. Dymphna pointed out that currently reviewers are able to type in one additional colour to black. It was agreed to add a wish to turn the pen colour permanently on and off, and Rasmus agreed to implement this as soon as possible. He explained that a simple system would show additions in red and deletions in strikethrough. He also confirmed that it would probably not be possible to build up a complete document with many different colours for many different people, as in Word.

Simon suggested that the IMS programmers meet with the authors of the ‘Hungry Minds’ series of books to discuss Word templates. Monica advised that she had met with other publishers using Word templates and been advised that their templates are proving problematic.

Appendices

This wish concerned having an additional section for appendices. This was agreed and it was also agreed that it was acceptable to have more than one appendix, but that reviewers and others should be given the ability to select which appendices to print.

Numbered and nested lists

It was agreed that RevMan 5 should include the ability to create numbered and nested lists. The style to be used for these should be in accordance with the Cochrane Style Guide.

Nested lists

This had been discussed in item 5.b.7 above.

Studies and references

Notes for ‘Other published versions’

An extra field in references to ‘Other published versions of this review’ had been requested for additional information about each of these other versions of the review. It was agreed to reject this wish for RevMan 5 because the information could be stored in ‘What’s new’ or ‘Published notes’.

More than 99 references per study

This wish was to increase the maximum number of references for a study beyond 99. It was agreed to set the level at 999. There are limits of 99 for various other features within RevMan, and it was agreed to change the maximum number of rows in a table to 999.

ISRCTN (International Standard Randomised Controlled Trial Number)

It was agreed that a field for the ISRCTN number should be created alongside the study identifier. Simon agreed that if the field for this number was left blank, the heading would not be shown in the published version.
Action: Simon

DOI field for references

It was agreed to add a dedicated DOI field for each reference. The DOI will be more inclusive than MEDLINE identifiers. The theory is that the DOI will always exist (making them more stable than URLs). If the match is in CrossRef (provided by Wiley), the DOI will take you directly to the document. If this is not possible, the search tool will search other databases. Lindsay agreed to advise TSCs of the decision to include a DOI field in RevMan so that they can bear this in mind when developing their specialised register.
Action: Lindsay

Longer IDs

Katharine raised the issue of needing more than 20 characters for reference and study IDs. Rasmus agreed to identify the number of extra characters, which could be added to the study ID without widening the column on the graphs. It was agreed that, if possible, 30 characters should be made available.

Reference identifiers should ideally be increased to a maximum length of 50 characters. It was agreed that the study and reference ID did not need to have the same limits and that reference IDs would be truncated if references were moved to the study section. Rasmus agreed to report to a future meeting on the relevant technical limitations, as the RAG were not able to decide how many extra characters should be added.
Action: Rasmus

Limit date of publication field

It had been suggested that the date of the publication field in references should be changed from a free text field to a field that can only contain four digits to contain the year only. It was not possible to reach an agreement on this field, and it was agreed that the structure used for references as a whole should be referred to the IMSG for discussion.
Action: Monica

Tables

Methods for ‘Ongoing studies’

It was agreed to add a ‘methods’ column to the ongoing studies table in RevMan 5 so that it has a similar structure to the ‘Included studies’ table. The new column would be empty for all existing reviews.

Footnotes for ‘Additional tables’

It was agreed that footnotes for additional tables should be included in RevMan 5. It was noted that the footnotes field for the ‘Characteristics of included studies’ table is not currently labelled ‘Footnotes’ in The Cochrane Library. It was agreed that it should be so labelled.
Action: Simon

Sort options for ‘Characteristics of included studies’ table

It was agreed that the ability to re-sort the characteristics of included studies table within RevMan on the basis of a pick list, or for the reviewers to move the rows around, should be included in RevMan 5. The preferred order should be passed to the publisher and used to publish this table. It was agreed that the ability to print each included study on a single page should be added to the Wishlist. Studies should continue to be listed alphabetically in the references section. The sort options should also be available for the tables of excluded and ongoing studies.
Action: Rasmus

Longer column headings for ‘Additional tables’

It was agreed that longer column headings for ‘Additional tables’ should be allowed, ensuring compatibility with other data. The current maximum is 20 characters, and the new maximum would be 30 characters.

More columns in ‘Additional tables’

It was agreed that the maximum of columns should be increased from 10 to 15 in ‘Additional tables’.

Economics’ tables

The addition of a field for complex Economics tables was rejected for RevMan 5.

Studies awaiting assessment

A suggestion had been made to add a table for ‘Studies awaiting assessment’ with the same structure as the ‘Included studies’ table to enable reviewers to move the data to the ‘Included studies’ table at the appropriate time. (Sonja advocated that the ‘Characteristics of studies awaiting assessment’ should be a simple text box instead, whilst Lindsay advocated that the table should be included in the structure of the review, but not published.) After discussion, it was agreed to include a free text section, but that the reviewer should be able to include a table in this section, if this would help her/him to present the information in a structured way.

More advanced tables

The table functionality for ‘Additional tables’ should be improved, including merging and splitting cells. The information would be retained in RevMan and passed to the publisher for publication. This was approved and Rasmus agreed to investigate it further.
Action: Rasmus

Data and analysis

Calculated data

It had been agreed previously that users of Cochrane reviews should have the ability to use raw data and make their own calculations. Simon noted that no products are currently available in Wiley’s product line to allow users to recalculate the meta-analyses, and they would need to use software such as MetaView. It was agreed that the RAG would discuss this further at their next meeting after the Statistical Methods Group had discussed it during the Cochrane Colloquium in October 2003.
Action: Mike, Julian

Add studies to data tables more than once

It was agreed to add this functionality to RevMan 5.

Longer graph labels

The addition of longer graph labels was discussed. It was agreed that the current 20 characters is too limiting and should be increased. Rasmus agreed to investigate the maximum that could be displayed, and to increase the labels to that length.
Action: Rasmus

Additional items for the Wishlist

QUORUM flowchart

Tess requested that reviewers should be able to create and edit flow charts in RevMan, such as the QUORUM flowchart. This allows a reviewer to describe how many hits they find with their search strategy, and as the flow continues, the reviewer would detail the number of studies rejected, or accepted etc. Rasmus advised that this could probably be implemented in RevMan 5.

Importing data

Rasmus advised that the ability to import, for example, ASCII files or data sets is a functionality issue and could be discussed at a later date.

Scales for graphs

Julian agreed to revisit the scales for graphs that are currently available, during the Statistical Methods Group meeting at the Cochrane Colloquium in October 2003.
Action: Julian

Date edited

It was previously agreed that the date edited would be added to all sections of RevMan from version 5.

Sources of support - additional information

This issue relates to the inclusion of a descriptive field for sources of support. It was agreed that each source of support would have an associated free text box.

Version numbers for reviews

Monica reported that a version number for reviews would be implemented in RevMan 5 and the new IMS.

Which version of RevMan had been used to edit a review

It was approved that a new field should be created detailing which version of RevMan had been used to edit the review.

Comments and criticisms

The addition of a date field for each comment in a review was rejected.

Additional headings

Julian suggested that RevMan should have a pick-list of optional standard headings that could be inserted. Phil reported that the Handbook Advisory Group would discuss this during its meeting at the Cochrane Colloquium in October 2003.
Action: Phil

Functions in each section of a review

It was agreed that a working group would be established to decide what functionality and formatting should be available in each section. Lindsay, Rasmus and Mike agreed to form this group and would make a recommendation to the next meeting of the RAG.
Action: Lindsay, Rasmus, Mike

Implementation

Rasmus agreed to prepare a document describing what will be new in RevMan 5 and to implement the decisions taken during the meeting as soon as possible. He pointed out that the final structure for RevMan 5 should be finalised at the next RAG meeting.

Dates relevant to Cochrane reviews: in RevMan and CDSR

This was discussed in item 5.a.1 above. Mike advised that he would submit the RAG’s recommendation to the Publishing Policy Group and ModMan Advisory Group. Sonja proposed that completion of the ‘Date next stage of review expected’ should be encouraged for full reviews and that this date should be published. She suggested that this would serve two purposes, (i) as a prompt for reviewers to update their reviews; and (ii) as an indication to users of when the review might change. It was agreed that the date to be published should take account of the time expected for editorial and publication processes. Tess suggested that when the new IMS was in place, a central e-mailing system could be used to remind reviewers to update their review. Mike agreed to defer this item to a future RAG meeting.
Action: Mike

Additional figures

Rasmus advised that better guidance is needed for reviewers and CRGs about the way in which to include additional figures. Monica agreed to raise this issue at the IMSG meeting.
Action: Monica

Search strategies in additional tables

Heather advised that she had canvassed RGCs and TSCs to collect views on whether to place search strategies in additional tables. Mike had also canvassed The Cochrane Library Users’ Group. There had been general agreement with the suggestion, and Phil agreed to take it to the Handbook Advisory Group. Sonja agreed to send her further comments to Phil.
Action: Phil, Sonja

Style guidance for levels of headings in Cochrane reviews

Sonja reported that the Style Guide Working Group had revised the Style Guide in relation to levels of headings within Cochrane reviews. There were no additional comments from the RAG and Sonja agreed to report back to the Style Guide Working Group. Mike advised that the Publishing Policy Group would also discuss this.
Action: Mike, Sonja

Remit of RevMan Advisory Group

Mike advised that the ModMan Advisory Group remit had been agreed. Mike agreed to use this to revise the RAG remit. He will circulate it in advance of the next RAG meeting.
Action: Mike

Future plans for the Information Management System

Monica reported that because a substantial number of structural as well as functional changes had been agreed for RevMan, RevMan 4.3 (rather than RevMan 5) would be issued by the end of 2004 as part of the new IMS. This will ensure that the IMS continues to be developed to schedule. She and Rasmus would present the current status of the IMS during the Cochrane Colloquium in October 2003. Monica encouraged RAG members to attend the workshop about the IMS during the Cochrane Colloquium, and agreed to e-mail the relevant details to the RAG and Review Group Co-ordinators.
Action: Monica

Any other business

RevMan 4.2.3

Rasmus advised that a patch for updating RevMan to version 4.2.3 would be available in the near future.

Date of next meeting

Mike agreed to canvass members for the date of the next RAG meeting, after discussions at the IMSG meeting on 12 September 2003 about the timescale for future developments.
Action: Mike