RAG minutes, 14 November 2002

Minutes approved on 28 April 2003

Present: Phil Alderson, Jacqueline Birks, Chris Cates, Mike Clarke (Convenor), Monica Kjeldstrøm, Sonja Henderson, Kate Henshaw, Julian Higgins, Bill McGuire, Rasmus Moustgaard, Elizabeth Pienaar, Mark Starr

In attendance: Claire Allen (minutes), Dave Booker

Apologies: Gerd Antes, Katherine Deane, Dymphna Hermans, Valeria Marinho, Heather Maxwell, Tess Moore, Carol Preston, Phil Wiffen

Contents:
1. Welcome
2. Approval of agenda
3. Approval of minutes and matters arising not on this agenda from the RevMan Advisory Group meeting on 20 March 2002
4. Publication of co-reviewers' contact details in The Cochrane Database of Systematic Reviews
5. Use of additional tables for search strategies
6. Tables in RevMan
7. Guidance on types of additional figures
8. Provision of relevant information when the methods used for a Cochrane review change between the protocol and the review
9. Statistical issues
10. Method for identifying which versions of RevMan have been used for editing Cochrane reviews
11. Links to other Cochrane reviews from within a Cochrane review
12. Dates in Cochrane reviews
13. Inclusion of 'studies' without references
14. RevMan 4.2 beta testing
15. RevMan 4.2
16. Needs Assessment Survey
17. Review Manager wish list
18. Chinese version of RevMan
19. Future plans for the Information Management System
20. Any other business
21. Date of next meeting

Welcome

Mike welcomed everyone to the meeting. Apologies for absence had been received from the people listed above.

Julian declared his new potential conflict of interest. He is now a consultant to the developer of the commercial software, Comprehensive Meta-analysis.

Approval of agenda

The agenda was approved.

Approval of minutes and matters arising not on this agenda from the RevMan Advisory Group meeting on 20 March 2002

The minutes were approved without amendment. Rasmus agreed to change their status on the IMSG web site.
Action: Rasmus

Matters arising:

Multi-user version of RevMan (item 3.e)

Sonja reported that she was no longer able to use the multi-user version of RevMan because of local network problems, which might take some time to resolve. Rasmus had not received any feedback from other Collaborative Review Groups who had expressed an interest in it. He advised that a multi-user version would probably not be available until the new Information Management System (IMS) was ready.

Withdrawn/suspended Cochrane protocols and reviews

Mike reported that advice relating to withdrawn/suspended Cochrane protocols and reviews would be incorporated in the version of the Cochrane Reviewers' Handbook being released in January 2003.
Action: Mike

Input to the RevMan Advisory Group from outside the UK

Mike reported that he had received expressions of interest from users in Canada, but the Canadian Cochrane Centre is unable to co-ordinate a user group at present.

Publication of co-reviewers' contact details in The Cochrane Database of Systematic Reviews

Mike advised that the RAG was being asked to recommend whether or not the contact details of co-reviewers should be published in the Cochrane Database of Systematic Reviews. After much discussion it was agreed that full names (not initial and surname), institutional base or organisation, and country, should be published for all co-reviewers. It was agreed that before implementation, there is a need to assess the completeness of the existing data and of how well the co-reviewers stored in Cochrane reviews match up to the 'byline' for the review. Rasmus agreed to extract the existing information and forward it to Mike who would assess it. Mike also agreed to raise this issue with the Publishing Policy Group at its next teleconference. It was noted that the Publisher will need to be asked formally to implement change such as this to Cochrane reviews.
Action: Rasmus, Mike

Guidelines also need to be developed for a situation whereby a reviewer has no institutional affiliation. Rasmus agreed to investigate whether it would be possible to highlight the fields that would be published, in RevMan 4.2. This will also help the copy editing process. He also agreed to add validation checks in RevMan and ModMan.
Action: Rasmus

Use of additional tables for search strategies

Mike had received a request from Steve McDonald relating to reporting search strategies as 'Additional tables' in Cochrane reviews. Mark advised that these Tables appear differently on the various publishing platforms. Chris outlined the advantage of being able to link to the search strategy as an Additional table. It was agreed to recommend that reviewers could use Additional tables for search strategies but that they should use one row for each search line. Mike will discuss this with the Handbook Advisory Group. From this discussion, it was agreed that future versions of RevMan should allow for the creation of Appendices for Cochrane reviews and that it should be possible to link to these from within the text.
Action: Mike, Rasmus

Tables in RevMan

Included studies and studies awaiting assessment

The suggestion that the 'Table for included studies' should be divided into three (Trial characteristics, Patient characteristics and Types of outcome) was made a Wish list item but, following discussion, it was agreed that it should be marked as 'Rejected'. It was agreed that it was preferable to keep these items of information for a study in a single table.
Action: Rasmus

Need for table for studies awaiting assessment

Mike reported that the guidelines for labelling a study as "awaiting assessment" needed to be improved. Two options were suggested for a new Table to store and present information on these: (1) To use the table in the way an excluded studies table is used; and (2) To use the table in the way an included studies table is used. It was agreed that, because either option would require a change to the structure of a Cochrane review, the discussion of this should be deferred to a future meeting at which major changes to the structure would be discussed. Monica agreed to report back to the RAG with the timeline for this discussion, bearing in mind the need to implement changes in RevMan 5.
Action: Monica

Need for an improved table function

This is currently a Wish List item. Rasmus reported that the ability to use images would be included in the RevMan 4.2 and that pictures of tables could be added to reviews in this way.
Action: Rasmus

Guidance on types of additional figures

Mike reported that there was currently guidance on the inclusion of statistical figures but not images. He reported that it would be the responsibility of the Handbook Advisory Group to draft guidance and that the views of the RAG were being sought to help their discussions.

It was agreed that reviewers should be able to include any image that makes their review easier to understand. However, after discussion, it was agreed that this should not actively encouraged until there is experience of the consequences of including images for, for example, the size of RevMan files and the speed with which Cochrane reviews can be viewed online. It was agreed that editors would have final editorial control of which images are included. It was noted that reviewers who include images must obtain permission from the original source before doing so.

Mike agreed to ask the PPG and the Cochrane Libraries Users' Group for comments on whether there should be guidelines on what types of image could be included.
Action: Mike

Provision of relevant information when the methods used for a Cochrane review change between the protocol and the review

This issue was discussed in response to a request from Peter Sandercock. Mike had obtained advice from Sally Hopewell who had been involved in a study of changes made to the methodology of Cochrane reviews between the protocol and the full review. Sally had provided information to show that 43 of the 47 Cochrane reviews assessed had changed in a major way between the protocol and the review. Her suggestions for this were discussed and it was agreed that it is important that the reviewers are explicit if the methods used for their review are different to those set out in their protocol.

Generally it was thought that this issue was primarily one on which the Handbook Advisory Group should make recommendations but it was noted that it would be relevant to RevMan if structural changes were necessary (e.g. to add a new section for changes made between the protocol and the to review). It was agreed that the primary consideration is awareness raising for reviewers. It was agreed to encourage reviewers to include an explanation of any changes in the appropriate section of the review, with a reference to any past publication. Mike will ask the Handbook Advisory Group to discuss adding this guidance to the Cochrane Reviewers' Handbook and agreed to thank Sally for her advice
Action: Mike

Statistical issues

New heterogeneity statistic

It had been proposed that a new heterogeneity statistic (I2) be included in RevMan and this proposal had received a favourable response from the Statistical Methods Group. Julian explained that the new statistic does not change the way in which heterogeneity is calculated, but is simply an additional calculation. He confirmed that the I2 statistic might lead to changes to the interpretation of existing reviews, but he did not expect that any of these changes would be major. It was agreed that adding this statistic had implications for the Cochrane Reviewers' Handbook, as documentation will be required to explain the statistic and what it means for the reader. Julian agreed that such documentation would be part of the revision to the statistical section of the Handbook, which is currently being finalised by the Statistical Methods Group. It was agreed to add the new statistic to RevMan 4.2 and Rasmus will do so. If the new statistic is to be shown in Cochrane reviews when they are published, the Publisher will need to be asked formally to implement this.
Action: Julian, Rasmus

Display of totals in meta-analyses

Julian explained the background to this item. There had been confusion about how to use the totals at the bottom of forest plots, which are a simple addition of data from the individual trials with binary data in RevMan. There was no clear consensus within the RAG or the Statistical Methods Group on changes to the display of these totals. It was agreed that all the information should continue to be published but that the format should be changed so that the number of events and participants do not appear as a fraction. Rasmus advised that it was easy to make this change within RevMan and Mark did not anticipate any difficulties in doing so when the reviews are published but the Publisher will need to be asked formally to implement this. Julian agreed to let Rasmus have some suggested wording for the new labels for the totals.
Action: Rasmus, Mark, Julian

Automatic linking and display of data within and between reviews

Text links to data

Sonja explained that information had to be entered manually when putting the results of the analysis of an outcome into the text of a review. On behalf of the editorial team of the Pregnancy and Childbirth Group, she suggested that it would be useful to insert a link to the data that would appear in the text, rather than having to type in the results. It was agreed that it is potentially dangerous to link in this way as errors of interpretation may be generated if the results change. However, it was also agreed that it would be useful to have the facility to cut and paste standardised results into the text. Rasmus agreed to investigate cutting and pasting a text string from the summary screen and would add this to the Wish List.
Action: Rasmus

Linking of data between secondary and primary reviews of induction of labour

Sonja explained that the Pregnancy and Childbirth Group were now preparing 'umbrella' reviews of 'primary' reviews, which had standardised patient sub-groups and outcomes. On behalf of her editorial team, she proposed linking data in comparisons and outcomes from the primary to the umbrella ('secondary') reviews. It was agreed that this is potentially dangerous because there is no way of ensuring that the text of the review being linked from had been updated to match updated results in the review being linked to. However, Rasmus advised that data could easily be copied and pasted between reviews in RevMan 4.2. Further discussion of umbrella reviews was deferred to a future meeting of the RAG.
Action: Mike

Method for identifying which versions of RevMan have been used for editing Cochrane reviews

Mark suggested that, starting with RevMan 4.2, a code should be added to Cochrane reviews to show which version of RevMan had been used to edit the review. Following discussion, it was agreed that each time the reviewer edits the review in a different version of the software, a note would be added to the unpublished notes field. Rasmus advised that this would be included in RevMan 4.2. beta and confirmed that methodology reviews will follow the same format. Mike agreed to advise RGCs and reviewers that the statements about the version used would be generated in the unpublished notes field.
Action: Mike

Links to other Cochrane reviews from within a Cochrane review

There was no strong desire by RAG members to be able to link between reviews within RevMan. Inactive links should be inserted as reference to a published protocol or review. Rasmus agreed to add this wish to the Wish list with the status 'rejected'. Mike advised that linking between Cochrane reviews when these are published was a publishing, not a RevMan, issue.
Action: Rasmus

Dates in Cochrane reviews

General discussion

It was agreed to defer this discussion, pending a paper that Frances Fairman had prepared.

Omission of 'Substantive update' date for new reviews

Toby Lasserson had proposed that the substantive update date should not be shown when a Cochrane review is published for the first time. Currently, the New Generation of The Cochrane Library shows this date near the top of all Cochrane reviews. After discussion, it was agreed that it should remain there but renamed 'Substantive amendment'. Mike agreed to consult the PPG about this.
Action: Mike

Inclusion of 'studies' without references

Where these studies are not real studies (for example, 'studies' added to produce analyses of patient-level sub-groups

Mark advised that there are a number of Cochrane reviews, which have included studies with no reference information and very little information in the Characteristics of included studies table. This may lead to problems with the data validation routines used by publishers of Cochrane reviews. Mike noted that one reason for including these 'dummy' studies was to allow forest plot analyses for patient-level subgroup analyses in reviews based on individual patient data. Julian pointed out that the ability to include additional figures in Cochrane reviews would allow such analyses to be shown in this way, rather than having them generated from within RevMan.

Julian also noted that 'phantom' studies are sometimes used to allow a study to be used more than once within the same comparison. It is possible to include the same study more than once by using additional figures. Rasmus agreed to add the ability to include a study more than once within RevMan to the Wish list.
Action: Rasmus

Ability to turn off warnings for empty references

Libby Weir had suggested being able to turn off the warning/validations for the empty references to studies contained in individual patient data reviews. Rasmus agreed to add this item to the Wish list and, following discussion, it was 'Rejected'.
Action: Rasmus

RevMan 4.2 beta testing

Schedule

Rasmus advised that beta testing of RevMan 4.2 would begin on 2 December 2002. Ten dedicated beta testers had agreed to test all the new features. There will also a second round of statistical beta testing. Subject to the satisfactory completion of this testing, it was hoped that RevMan 4. 2 would be released as a non-mandatory version shortly after the module submission deadline in February 2003. Once the beta version is released, reviewers will be encouraged to comment.
Action: Rasmus

Use with 'live' data

It was agreed to discourage strongly reviewers from using RevMan 4.2 beta for 'live' data, because the publisher will reject reviews submitted using that version.

RevMan 4.2

Likely release and first submission date

See 14a above.

Use by co-reviewers

It was agreed that on its non-mandatory release, it should be recommended that all co-reviewers for a review should switch to version 4.2 if one of them will use this version. This is because if some of the new features in version 4.2 are used, those edits cannot be seen in version 4.1 (although the data are not lost). Sonja reported that although RevMan 4.2 is not a mandatory release for reviewers, it will be mandatory for editorial bases so all reviews being distributed from editorial bases would be in version 4.2. It is anticipated that most reviewers will want to use version 4.2 because of its improved functionality. Rasmus advised that on general release, version 4.2 would be available to download from the Internet, and that the CD-ROM release will follow about six weeks later (to allow time for any minor fixes to be added to the CD-ROM version).

Needs Assessment Survey

Reporting of initial survey

Mike advised that a summary of findings had been published in Cochrane News and that Monica had agreed to upload the full reports to the IMSG website, along with copies of the instruments used. Mike agreed to send Monica the full reports for uploading.
Action: Monica, Mike

Developing country survey

Elizabeth reported that the original survey had been amended for use for people in developing countries. She noted that the survey was much shorter than the original but that additional questions would be sent to people who were willing to answer these. She hoped to distribute the survey in January 2003 and to upload it on the South African Cochrane Centre web site. Her initial report will be presented at the next RAG meeting. A statistician will be employed for six months at the SACC commencing January 2003, to undertake the analyses.
Action: Elizabeth

Review Manager wish list

ID Number	Priority / Notes
117	Major structural change for version 5. Should be in description of studies section.
118	Agreed to prioritise after version 4.2.
119	No change.
120	Agreed to prioritise after version 4.2.
121	Validation checks to be added giving the number of words in the abstract. Mike will check whether MEDLINE truncates at 400 words.
122	Part of the discussion of the new IMS. Currently no desire to implement.
123	Rejected.
124	Approved after version 4.2. In the short term, use additional figures.
125	High priority after version 4.2.
126	High priority after version 4.2
127 - 143	Done
144	Agreed to prioritise after version 4.2.
145	Agreed to prioritise after version 4.2.
146	Low priority after version 4.2
147	Low priority after version 4.2
148	High priority after version 4.2
149	High priority after version 4.2.
150	Low priority after version 4.2.
151	Rejected. Handbook Advisory Group to improve guidance.
152	Rejected.
153 - 170	Leave as unassigned.

Action: Rasmus, Mike

Chinese version of RevMan

Monica reported that she had contacted the Chinese Cochrane Centre who had confirmed that they had translated the interface for RevMan 4.1. They had offered to send Monica a version for testing and Monica had pointed out that RevMan 4.1 is copyrighted freeware. It was agreed to allow the Chinese Cochrane Centre to release the translated interface when they have explained what method they have used for translating. Monica agreed to ask the RAG by e-mail if they had any concerns about this. A recommendation will then be sent to the Steering Group for discussion.
Action: Monica

Future plans for the Information Management System

Monica explained that the proposal for the new Information Management System would be discussed fully at the IMSG meeting on 15 November 2002. If the proposal is approved, the Nordic Cochrane Centre will start to design the new system. Mike agreed to provide feedback to the IMSG on behalf of the RAG.
Action: Mike

Monica reported that assuming agreement of the draft document, a proposal for implementation would be submitted to the Steering Group for its meeting in April 2003. She confirmed that advice was required from the RAG on the concepts being proposed. Concerns were raised about how the proposal would meet with reviewers' requests to write reviews in other software (e.g. Word) and whether the new system might prove too restrictive.

Monica advised that the new IMS would provide a tracking system for reviews, especially from the time the review is submitted to the editorial base to publication of the review. There are currently two major tracking issues: (1) a track changes facility in RevMan (similar to the track changes facility in Word); and (2) a system to track Cochrane reviews from titles, through protocols, to full reviews and subsequent updates. The tracking system will have the ability to generate automated reminders for editorial bases and Mike asked that the same functionality be made available to reviewers so that they could use it when working with co-reviewers. Monica agreed add this to the wish list for the new IMS.
Action: Monica

Reviewers will be able to 'check out' their reviews from the central system and circulate this to their co-reviewers. For example, this will allow a reviewer to check out a HTML version of a review, to add comments to it and then to pass it to other people to comment on. However, only the person who checked out the review would be able to check it in. Making this more flexible will be investigated.

It was agreed that reviewers should be able to ask their editorial base to check out a review for them and send it to them on floppy disk.

Monica agreed to make the document more explicit about the potential problems of hackers, and hardware and software problems. She confirmed that the system would not be implemented unless it is judged to be sufficiently secure and its ability to withstand an attack by hackers has been tested. Timelines for implementation will be proposed to the Steering Group in time for its meeting in April 2003 and will be brought to the next RAG meeting, shortly thereafter.
Action: Monica

Any other business

There were no other items of business.

Date of next meeting

Mike agreed to circulate dates between 22 April and 2 May 2003. [Post hoc note: the meeting will be on 28 April 2003.]
Action: Mike