Minutes of the RevMan Advisory Group meeting held at the UK Cochrane Centre on 13 December 2001

Provisional minutes: 18 February 2002

Present: Gerd Antes, Phil Alderson, Chris Cates, Mike Clarke (Convenor), Monica Fischer, Sonja Henderson, Julian Higgins, Valeria Marinho, Heather Maxwell, Tess Moore, Rasmus Moustgaard, Elizabeth Pienaar, Mark Starr, Phil Wiffen.

Apologies: Clive Adams, Katherine Deane, Kate Henshaw, Dymphna Hermans, Carrol Preston, Lindsay Stead, Paul Whatling.

In attendance: Jacob Riis (minutes).

Contents:
Welcome
Approval of the agenda
Approval of minutes and matters arising from the previous meeting (28 March 2001)
Distribution of future agendas, minutes and background papers
RevMan 4.1 - RTF difficulties
Feedback to Cochrane entities on pilot projects
RevMan 4.2
Metalyser 1.3
Needs assessment survey
Validation of references in Cochrane reviews
Inclusion of MEDLINE numbers in references
Identification of primary references
Additional figures in Cochrane reviews
Design of the RevMan wishlist
Content of the RevMan wishlist
Membership of the RevMan Advisory Group
Budget for the RevMan Advisory Group
Any other business
Date of next meeting
Appendix A: wishlist items

Welcome

Mike began the meeting with the very sad news that had arrived overnight from Australia that Chris Silagy, the second chair of the Cochrane Collaboration Steering Group and many more things besides inside and outside the Collaboration (not at least an inspiration to many people as the Collaboration has developed) had died. The RevMan Advisory Group expressed its deepest sympathy to all who knew Chris.

Mike then welcomed everyone to the meeting, in particular the new members, Elizabeth and Valeria. Apologies for absence had been received from the people listed above. Mike advised that Lindsay Stead had stepped down from the Group although she would continue through the email discussion list. Kate Henshaw and Dymphna Hermans had both volunteered to replace Lyndsay on the Group, but unfortunately neither was able to attend this meeting.

It was noted with approval that the group now had a strong representation of RGCs and TSCs.

Approval of the agenda

The agenda was approved.

Approval of minutes and matters arising from the previous meeting (28 March 2001)

The minutes were approved with no amendments.

Guidance on good outcomes (item 3.b)

Chris had no news to report. Julian informed the group that the revised statistical section of the Handbook will cover some of these issues.

Web-registering for RevMan 4.1 on CD-ROM (item 3.c)

Rasmus reported that there were about 50 RevMan 4.1 CD-ROMs left, and that there were no plans to produce more for version 4.1. When RevMan 4.2 is ready, consideration will be given to releasing it on CD-ROMs as well as by download from the Internet.

MetaView bugs (item 3.f)

Mark reminded the Group that only statistical errors in MetaView 4.1 will be fixed and that there are no plans to develop it further. No statistical errors have been found to date. In the event of a new version being produced, it will be released through the Nordic Cochrane Centre.

Archiving of The Cochrane Library (item 3.h)

Mark noted that there are two separate issues related to this. One concerns archiving past issues of the complete library, the other concerns the archiving of previous versions of individual reviews. He advised that Update Software is working on both. Currently, people can purchase copies of previous versions of reviews from Update Software. All questions about the content of Cochrane Reviews are referred to the relevant Collaborative Review Group - as are requests for withdrawn reviews where the reason for withdrawal is not readily available. Mark agreed to clarify this policy in a message to Collaborative Review Groups.
Action: Mark

The policy for withdrawn reviews is currently being discussed by the Handbook Advisory Group.

Multi-user version of RevMan (item 6)

Rasmus reported that it was currently too risky to build RevMan 4.2 using the multi-user code because the latest version of RevMan 4.1 multi-user has not yet been tested by any Collaborative Review Groups. Sonja, Tess and Heather agreed to test it and to report back at the next meeting of the RevMan Advisory Group.
Action: Rasmus, Sonja, Tess, Heather

Validation rules for Cochrane reviews before publication in The Cochrane Library (item 7)

ModMan 4.1 notifies the Review Group Coordinator if mandatory fields in Cochrane Reviews are empty when they attempt to submit their module.

'Meta-analysis' of single trials - should it be avoided? (item 9)

Mike reported that advice on this is not yet included in the Handbook. It is one of the things to be discussed by the Handbook Advisory Group as part of a substantial revision of the Handbook for release within RevMan 4.2.

Distribution of future agendas, minutes and background papers

Agendas for the RevMan Advisory Group can now be made available on the web site of the Information Management System Group - along with background papers. It was agreed that in future all background papers should be combined together and made available as a single PDF file.

Because some background materials might be considered confidential, the public availability of the background papers was discussed and was referred to the Information Management System Group for discussion (see IMSG Minutes, 14 December 2001, item 8.a).

It was suggested that summaries of the minutes for specific issues might be prepared and published on the web site. For example, there could be a summary of the recommendations concerning additional figures.

RevMan 4.1 - RTF difficulties

The main text window of RevMan is programmed using a "RTF control" and the text is stored in RTF format. Mark reported that the reviews submitted to Update Software contain variations in the RTF codes that, to date, have been fixed by Update Software's parsing routines. However, these problems are becoming more common, and the possibility that the fixes will cause new problems is increasing. Mark, therefore, would like to encourage all editorial teams to check their reviews in each issue of The Cochrane Library. He will circulate a message about this.
Action: Mark

The possible causes of the problems were discussed. It was suggested that they may be related to users pasting text into RevMan from other word processing software. Mark agreed that Update Software would inform the relevant Collaborative Review Groups whenever problems arise, so that the causes can be investigated. Sonja agreed to investigate what the reviewers in her Group, whose review had been particularly problematic, had done. Gerd suggested that Update Software should create a log of the problems, and that this could be discussed by the RevMan Advisory Group, as appropriate.
Action: Mark

Rasmus proposed two possible solutions to these problems: using a new RTF control (to be investigated) or modifying the parsing routines.

Feedback to Cochrane entities on pilot projects

Mark apologised to the Heart Group for the missed opportunity of them not knowing that the additional figures they had piloted were to be demonstrated at the Cochrane Colloquium in October 2001. He advised that figures that had been submitted as part of this pilot would not be published until the next generation of the The Cochrane Library is available.

RevMan 4.2

Rasmus had precirculated the latest version of RevMan 4.2 and the Group agreed with his suggestion that only major items were discussed during this meeting. He would be would be happy to be sent information about any other issues by email.

The inclusion of additional figures was discussed separately (item 13).

Special text formatting

Mark advised that Update Software would need six months before they were able to publish the text formatting in The Cochrane Library.

Rasmus clarified that text pasted into RevMan from other word processing software would not retain any special formatting, such as bold. This formatting would have to be reapplied within RevMan.

It was agreed that users should be made aware (through documentation and tool tips) of the features which will and will not be published (e.g. yellow highlighting will not be published).

It was agreed that all the formatting features should be included in the beta version. If Update Software are unable to publish some of the features, these could simply be disabled before release - without requiring a new round of beta testing.

Inverse Variance Method - now 'Generic Method'

The Statistical Methods Group proposed that the new statistical method which had previously been called the inverse variance method should be renamed the generic method. This was approved. The new section of the Handbook on statistics will include a discussion of the generic method. Julian agreed (on behalf of the Statistical Methods Group) to also supply text for the RevMan User Guide, if requested.

Julian will send Rasmus any feedback from the Statistical methods Group on the implementation of the Generic Method.
Action: Julian

Mark advised that Update Software would prefer to generate the graphs for the generic method graphs from data. However, if this is not feasible, the graphs would need to be included in Cochrane reviews as additional figures.

Exporting reviews

It was agreed that the method of selecting reviews for export should be changed to a window with checkboxes, with versions and labels clearly identifiable. Rasmus will design a draft window and circulate it to the RevMan Advisory Group.
Action: Rasmus

Internal/External vs Intramural/Extramural

For the sake of consistency with ModMan, it was decided to rename the headings for sources of support in RevMan to 'Internal' and 'External'. Rasmus will implement this in RevMan 4.2. Mike will formally notify Update Software that it needs changing in The Cochrane Library and make the necessary change to the Handbook.
Action: Rasmus, Mike

Metalyser 1.3 - now 'RevMan Analyses'

Name of software

The name of the software was discussed. It was agreed that it should be "RevMan Analyses".
Action: Rasmus

Edit functionality

Rasmus demonstrated the functionality currently available in the software. It was agreed that the option of editing the graphs should be removed.
Action: Rasmus

Graphs not automatically published

It was agreed that it should be made clearer to the user of RevMan that some graphs generated within RevMan Analyses will not be published automatically with the review in The Cochrane Library (e.g. funnel plots). This might be done by including a note on the title bar.
Action: Rasmus

Distribution with RevMan 4.2

It was agreed that RevMan Analyses should be installed as the default analysis program with RevMan 4.2 - but that Metaview 4.1 should also be installed as a hidden backup. Users will only be informed of this if necessary. This will be recommended to the next meeting of the Information Management System Group.

Totals of events

There was discussion of whether the total number of events should be presented in the graphs because some users make the mistake of assuming that this total is used to generate the summary analyses. Julian will arrange for this to be discussed by the Statistical Methods Group.
Action: Julian

RevMan 4.2 testing

The preliminary schedule was outlined as follows:
• 1 week for updating the test dataset
• 1 week for a statistician to test analyses
• final beta version ready for the next RevMan Advisory Group meeting (March 2002)
• 6 weeks of beta testing

Julian agreed to oversee the statistician testing the analyses.

The beta version will be tested by both a dedicated team of testers and by anyone else who may be interested.

There was discussion of whether Collaborative Review Groups should be allowed or even encouraged to submit their modules using the beta version. This would make the testing more thorough, but would also risk the loss of data and would require extra safeguards in place. To date, it has been policy that beta versions should not be used for "live data". A decision will be made on this when the beta version is ready.

Mark noted that he expected that Update Software would be able to accept RevMan 4.2 submissions by late August 2002.

Needs Assessment Survey

A preliminary analysis of the results for section 3 of the Needs Assessment Survey: 'Experience of Cochrane Reviewers' had been circulated as a background paper. This was discussed.

Mark suggested that these results supported letting reviewers use other review authoring tools than RevMan.

It was noted that Reference Manager was the most widely used reference management software. It was therefore deemed important to ensure that RevMan had good support for Reference Manager files, for example in relation to wish no. 77: Import whole studies. Currently, RevMan is supplied with import and export definitions for Reference Manager, but these may not be up to date. The Nordic Cochrane Centre would prefer that someone with experience of using Reference Manager should assume responsibility for these files, and Monica will try to identify such a person.
Action: Monica

Items 3-10 and 3-11 in the survey relating to means of sharing reviews with co-reviewers and editorial bases were discussed. There was a high number of responses for "Telephone or personal contact" and it was agreed that there should be a more comprehensive analysis of the survey in regard to these questions.

Items 3-18 and 3-19 (relating to the support of different pieces of software) were discussed, and it was noted that although there was a majority in favour of providing support for other software, the number of different packages suggested would make this very difficult.

There was discussion of how to collect more information from people in developing countries. Valeria and Elizabeth agreed to follow up on this. They will consider conducting interviews based on the survey results, so as to obtain a developing countries perspective.
Action: Valeria, Elizabeth

Validation of references in Cochrane reviews

The background paper from Tess on errors in references was discussed.

Sonja reported that the ModMan Advisory Group had agreed at its meeting on 12 December 2001 that validation checks should also be in RevMan; that it should be remembered that warnings are only warnings; and that the reference formats should be revisited. Heather agreed to prepare a suggestion for how some of the reference formats, about which concerns had been raised during the Cochrane Colloquium in October 2001, could be revised.
Action: Heather

It was agreed that the fields to be used for the different types of citation should be marked in bold to make them more prominent.
Action: Rasmus

It was agreed that a validation check should be added to warn users if an Additional reference was not linked to in the text.
Action: Rasmus

The following items will be added to the wish list:

• Links in text should also be underlined, this will make them clearer when printing. RAG priority: high.
• Global edit for references. RAG priority: low.
• Pulldown list of journals already included in a reference to be available in a reference window. RAG priority: high.

Action: Jacob

It was agreed that it should be easier to find the section of the help file that provides information on Reference format.
Action: Jacob

Inclusion of MEDLINE numbers in references

The background paper from Tess asking about the use of MEDLINE numbers in references was discussed.

Mark advised that the MEDLINE numbers are not currently used for linking from published Cochrane reviews to any external databases. It was agreed that this feature was a good idea but it was problematic. It was suggested that it might be better to use CENTRAL identifiers in the future.

Identification of primary references

The background paper from by Steve McDonald and Philippa Middleton of the Quality Advisory Group was discussed. The following recommendation was approved:

A reference should only be starred when multiple publications are listed for a study. This guidance should be included in the Handbook, Style Guide and RevMan User Guide. A tool tip should will be added to the Add reference screen in RevMan to note something like "Only tick this box if there are two or more publications for this study".
Action: Mike, Jacob, Rasmus

Additional figures in Cochrane reviews

Displaying figures on The Cochrane Library

Mark confirmed that The Cochrane Library will be able to display additional figures in Cochrane reviews by the time that RevMan 4.2 is released.

Generic Method as additional figure

Mark advised that, at the moment, he assumed that Update Software will be able to generate Generic Method graphs from raw data and these would not need to be included as additional figures.

Caption field

Julian advised that the Statistical Methods Group's recommendation will be that all figures have a caption field. Rasmus will investigate the feasibility of adding this to RevMan 4.2.
Action: Rasmus

Figure format

The Technical Implementation Advisory Group and Update Software will collaborate on deciding the technical constraints on figure files, and prepare a guidance document with examples. This will be circulated to the RevMan Advisory Group before the next meeting.
Action: Rasmus

Mark noted that good publishing practice dictates archiving images with the highest resolution possible for future presentation.

Data inclusion

There was discussion of whether the data used for generating additional figures should also be stored in the Cochrane review. It was agreed that, if possible, this should follow the policy for all graphs in Cochrane reviews.

References to figures

In the current alpha version of RevMan 4.2, "Figure 1", etc. has to be entered manually as the identifier. It was agreed that Rasmus should investigate the automatic numbering of the figures.
Action: Rasmus

Design of the RevMan wishlist

It was agreed that a Search function should be added to the wishlist web site so that users could more easily find items related to their wish.
Action: Rasmus

Content of the RevMan wishlist

A new wish was added in accordance with the recommendation from the RevMan Advisory Group: The installation program should draw more attention to the presence of the User Guide and help file, for example by including a shortcut to the User Guide on the desktop.

The New items in the wishlist were evaluated and the decisions are shown in appendix A.

Membership of the RevMan Advisory Group

It was agreed that there should be an attempt to recruit more Cochrane reviewers. This might be done by sending out a message to all contact reviewers using the email list that is generated from the Parent database. Mike will ask Ruth Jepson (the representative of reviewers on the Steering Group) about doing this.
Action: Mike

Budget for the RevMan Advisory Group

Monica informed the group about its expenditure to date during 2001. She noted that expenses for the present meeting still need to be included. All members of the group were asked to submit their expense claim forms as soon as possible.
Action: Everyone

Any other business

There was no other business.

Date of next meeting

Mike will canvass for availability for the next meeting, which will probably be held around the time of the UK Cochrane Contributors' Meeting in March 2002.
Action: Mike

Appendix A