Minutes of the RevMan Advisory Group Meeting held at the UK Cochrane Centre on Wednesday 8 January 1997

Present: Clive Adams, Mike Clarke (minutes), Inez Cooke, Carl Counsell (chair for most of meeting), Jon Deeks, Lelia Duley (chair for part of meeting), Monica Fischer, Sonja Henderson, Jini Hetherington, Reive Robb, Elizabeth Royle, Mark Starr, Lindsay Stead.

Apologies: Christopher Cates, Bill Gillespie, Andy Oxman, Patrick Vanderkerckhove.

In attendance: Rasmus Moustgaard (Nordic Cochrane Centre), Veronica Yank (San Francisco Cochrane Center).

Contents:
1. Welcome
2. Approval of minutes and matters arising
3. Status of software
4. Status of software manual, Handbook and help screens
5. Bugs and problems
6. Wishlist for RevMan and MetaView
7. RevMan training and support
8. Targets for 1997
9. Next meeting
APPENDIX

The new members of the RevMan Advisory Group were welcomed to their first meeting.

Approval of minutes and matters arising

1. The minutes of the previous meeting (12 June 1996) were approved, subject to a change in section 2/3 whereby it needed to be made clear that the flagging of a primary reference meant the primary citation for each included study rather than the primary reference for the whole systematic review.

2. Because the software has now been finalised, there is no need for a checklist of useful tips to be included in the RevMan manual.

3. 61 of the 68 Pregnancy and Childbirth reviews now have abstracts.

4. The printing options have been increased in RevMan 3.0. Work is continuing on printing and searching options.
   Action: Mark Starr, Rasmus Moustgaard.

5. A form was produced (but no one used it) for the beta testing of MetaView.

6. 95% confidence intervals is to be the default for both trials and reviews. A decision is needed on whether the settings chosen for, for example, confidence intervals and statistical methods by a reviewer should be automatically displayed for that review in CDSR. This should be discussed by the group responsible for the structure of a Cochrane review. Mike Clarke will write to Andy Oxman requesting that a meeting be convened to discuss this - Jon Deeks would like to join that group.
   Action: Mike Clarke

7. "Section II" no longer exists; "section VI" is now the Handbook and needs to contain advice that each CRG should recommend to their reviewers which default statistics to apply to reviews in their Group.
   Action: Andy Oxman

8. Most of the changes suggested for RevMan have been implemented. Those that have not been implemented follow along with additional comments:

   1. It would be useful if the Editorial Notes field could go from ModMan to Parent and then to CDSR. This should be decided by the group responsible for the structure of a Cochrane review. If they agree, guidance would be needed on the content of this field and CRGs would need to be informed that the field would be made public. Among the items that might be included in such a Field are comments from the CRG editors about a review not being as up-to-date as it could be. Mike will include a request for a decision on this in his letter to Andy.
      Action: Mike Clarke

   2. Reviewers could mark the major publication for an included study with a *. A standard footnote should be added to the Cochrane Library to explain this mark.
      Action: Mark Starr

   3. Text should be added to the summary graph in CDSR to warn that the effects might not all be in the same direction.
      Action: Mark Starr

   4. The RevMan Manual suggests that reviewers could add "a", "b", "c", etc to the short titles of their reviews to distinguish between various versions of the same review.

9. A system of rotation needs to be established for membership of the RevMan Advisory Group. In addition, it would be helpful if members of the Group could nominate alternates who could attend in their place. However, in order that this would work satisfactorily, it needed to be clear who each member of the Group was representing. For example, some members might be representatives of a specific CRG or of a particular type of participant in the Collaboration, such as CRG coordinators, reviewers, editors. (A system of alternation is already in place for representation from reviewers using individual patient data, with Mike Clarke and Derek Harvey taking it in turns to attend RAG meetings.)
   Action: Andy Oxman

10. Minutes of this and future meetings should be circulated as ASCII files by E-Mail rather than as file attachments of word processing documents.
    Action: Mike Clarke, future minute takers

Status of software

1. RevMan 3.0

   1. The windows version has been released and is in use. The intention is that the only amendments that will be made to this over the coming year will be minor maintenance changes and there will be no structural changes for at least a year.

   2. The DOS version (which is the equivalent of the windows version) is finished and needs testing. It should run on most DOS based computers regardless of their specification and anyone who expects to use it on a computer that might not be able to cope will be encouraged to be part of the beta-testing. Anyone using the older DOS versions will be encouraged to change to the new version, especially since the statistics will be different. A message will be sent to the ADMINORS electronic discussion list to try to identify users of the DOS versions.
      Action: Rasmus Moustgaard

   3. The MAC version (which will be the equivalent of the windows version) should be ready for beta-testing in February. A message will be sent to ADMINORS to try to identify MAC users who would be willing to do this testing.
      Action: Rasmus Moustgaard

   4. The statistics in the DOS and MAC version still need to be tested. Jon Deeks will provide the data sets that were used to test the windows version.
      Action: Jon Deeks, Rasmus Moustgaard

   5. Jon requested that the DOS and MAC versions should not be released formally without adequate testing of the statistics. If these versions are released in advance of this, an appropriate warning must be added.
      Action: Jon Deeks, Rasmus Moustgaard

2. MetaView

   1. This has now been tested within the Centre for Statistics in Medicine in Oxford by Jon and Mike Bradburn and amended as necessary. There were very few other pieces of software against which MetaView could be tested (particularly for continuous data) but they are now reasonably content that the software should give correct answers, although there is a very slight possibility that errors may have "crept in behind" the testers. The new version of MetaView is available on the FTP server and a new packaged version of it and RevMan will be placed on the server next week with a message to be sent to CRGs and Centres by Update Software.
      Action: Rasmus Moustgaard, Mark Starr

   2. Seven statistical methods are now available for dichotomous data and four for continuous data, and a technical document has been produced which will be sent to all CRGs. Jon and Mark will liaise about whether it will be possible to include this document in MetaView and Jon will discuss the formal publication of the document with other members of the Statistics Methods Working Group.
      Action: Jon Deeks, Mark Starr

   3. It was agreed that the combination of RevMan and MetaView made the software a very powerful tool for the preparation of systematic reviews and that the formal definition of a Cochrane review (as printed in Cochrane News) should be added to RevMan. In addition, Mark will ask the Publications Policy Group to decide on how available the software should be to people outside of the Collaboration.
      Action: Rasmus Moustgaard, Mark Starr

3. CritMan: alternative names would be welcomed.

   1. This is a component of the comments and criticisms management system that is being developed by the San Francisco Cochrane Center, the Nordic Cochrane Centre and Update Software so as to lessen the workload that might fall on CRGs if comments and criticisms were to be handled in an unstructured way. The intention is that criticisms on Cochrane reviews will be sent to the San Francisco Cochrane Centre, from there they will go to the criticisms editor for a CRG who will pass these on, with appropriate comments, to the relevant reviewers. The responses to the criticisms will be returned to the editor for transfer to Parent and CDSR, and the San Francisco Cochrane Center.
   2. CRGs have been written to, asking that they identify a criticisms editor who will be responsible for the comments and criticisms within that CRG. Training will be available through the San Francisco Cochrane Center.

   3. "View criticisms" will be available on the April issue of the Cochrane Library, including some examples of criticisms that have already been received.

4. It was agreed that the RevMan Advisory Group did not need to be given reports on the status of ModMan, Parent, RefTrack and the Software Development Group. These items were removed from the agenda.

Status of software manual, Handbook and help screens

1. The new versions of the RevMan manual and the Handbook are now packaged with RevMan. The help screens are not yet contact-sensitive but this should be possible with the next major release. In the interim, new help files could be made available on the FTP server.
   Action: Jon Deeks, Lelia Duley, Andy Oxman, Rasmus Moustgaard, Mark Starr

2. A manual might need to be prepared for MetaView, depending on the content of the revised Handbook.
   Action: Jon Deeks, Lelia Duley, Andy Oxman

3. There should be closer liaison, with sufficient advance warning, between the software developers and the writers of the Manual and Handbook, before any future major releases of RevMan.
   Action: Lelia Duley, Andy Oxman, Rasmus Moustgaard, Mark Starr

4. The new version of the Handbook is on the January 1997 issue of the Cochrane Library.

5. Inconsistencies have been found in the summary screens for reviews in CDSR. Some of these are blank despite the presence of data in the review, because a reviewer has input outcomes at the sub-category level and no overall total is given for all sub-categories combined. The group responsible for the structure of a Cochrane review should prepare guidance on this and also consider whether or not all reviews should contain a summary graph. Mike will add this to his letter to Andy.
   Action: Mike Clarke

Bugs and problems

1. Case sensitivity needs to be removed from the authors list so that surnames such as van Tinteran will appear in the appropriate alphabetic position.
   Action: Mark Starr, Rasmus Moustgaard

2. Some CRGs have sent problems directly to Lois Sims but, in future, these should all be sent to the Nordic Cochrane Centre for distribution as appropriate. A message about this will be posted to ADMINORS.
   Action: Rasmus Moustgaard

3. Some CRGs have found that, occasionally, another reviewer's contact details are incorrectly added to a review when it is imported. A warning will be sent to ADMINORS along with a query to see if other CRGs have encountered this problem.
   Action: Sonja Henderson

Wish list for RevMan and MetaView

1. These will be maintained by the Nordic Cochrane Centre and will become a supplement to the minutes of the meeting of the RevMan Advisory Group. The RevMan Advisory Group will give a priority (high, medium, low) to each item on the wish list.
   Action: Rasmus Moustgaard

2. Suggestions for the wish list were noted by Rasmus until he had to leave the meeting. Mike then noted these to pass them to Rasmus. There was insufficient time to assign a priority to each item.
   Action: Mike Clarke, Rasmus Moustgaard

RevMan training and support

1. . The next training day is in Edinburgh in February. There are still spaces available.

Targets for 1997

1. Avoid any major new release of RevMan for windows during the year.

2. Release the DOS and MAC version of RevMan 3.0.
   Action: Rasmus Moustgaard

3. Consider how to provide training to existing users of RevMan when a new version is released.
   Action: Lelia Duley

4. Continue to work on CritMan.
   Action: Rasmus Moustgaard, Mark Starr, Veronica Yank

5. Meet the highest priority items on the wish list if these are consistent with not needing a major new release of RevMan.
   Action: as on wish list

Next meeting

1. Wednesday 25 June, 11am-4pm at the UK Cochrane Centre.

APPENDIX: Suggestions for the development of Review Manager

Rasmus Moustgaard

6 January 1997 (amended 21 January 1997)

Suggestions for RevMan collected through the RevMan discussion list and other channels. Each item is given from one to three stars by the person responsible for implementing the changes depending on the estimated time of implementation (3 stars means that implementing the request takes more time than a request with 1 star assigned). Implications for changes to documentation, such as the RevMan manual, are not included.

Data import features (***)
RAG priority: Low

It should be possible to import numeric data and characteristics of studies, for example, from spreadsheets to RevMan. The problem is to decide a format for the data and which spreadsheets to support. A solution is that RevMan can import data from the clipboard in a given format. The user will need to determine how to get the data to the clipboard, but most times marking and copying the data from the spreadsheet should work.

Data export features (**)
RAG priority: Moderate

It is difficult to export data from RevMan to other programs. It should be possible to print a review to an ASCII file, at the least. This is very easy to do in Visual FoxPro but harder in FoxPro 2.6. If users need to export numeric data a special export feature for this kind of data or an option to print them to an ASCII file must be implemented. This could also be done in MetaView.

Study name on 'References to studies' screen (*)
RAG priority: High

When you edit citations on the 'References to studies' screen the name of the study should be displayed as a heading to the citation. As it is now you can not see to which study the reference belongs.

Automatic creation of a text subdirectory (*)
RAG priority: No desire to have this implemented

A directory should be created automatically for storage of text files before they are imported to RevMan .

Telephone numbers in secondary address (**)
RAG priority: MAG will prioritize this (Monica will follow up with HIREx developments)

There is no telephone number in the secondary address. It is fairly easy to add one but it will change the format of a review, and needs to be co-ordinated with the HIREx development team.

Remove exit message (*)
RAG priority: No desire to have this implemented

The 'Are you sure you want to exit?' message displayed when you leave RevMan is not necessary since the data are saved anyway.

Size of 'Sources of support screen' (*)
RAG priority: High

The size of the 'Sources of support' screen is unfortunate since the save and close buttons from the previous screen are visible at the bottom of the screen and cause confusion.

International characters (***)
RAG priority: High

If international characters are to be used in RevMan each review must have a code page assigned. The code page and character information must be preserved through ModMan and Parent Database so it can be used for displaying reviews on The Cochrane Library. The implementation time for RevMan may be small but all other parts of the information management system must be amended too.

Formatting codes (**)
RAG priority: Moderate priority (for implementation of a viewer that can display the output) (This needs to be agreed upon by ALL registered review groups to ensure consistency)

Use of formatting codes for bold, italics etc. could improve the visual appearance of a review. Unfortunately it is not possible to view the formatted text in RevMan if the editor is to be programmed in FoxPro. A simple solution is to allow certain codes in reviews (like /I for italics start and /i for italics stop) which will be translated when displayed on The Cochrane Library.

Use of tables (**)
RAG priority: Moderate

Some parts of a review (e.g. numeric data and characteristics of studies) would be easier to view and work on if entered into a real table. Visual FoxPro allows the use of such tables.

Context sensitive help (**)
RAG priority: High

Relevant parts of the manual are displayed when the user press F1. With Visual FoxPro it is also possible to have ToolTip text (an explaining text appearing when the user pauses the mouse pointer over a control for a moment).

Redesign of the menu system (*)
RAG priority: Low

The menu system in RevMan could be redesigned for greater clarity, e.g.:
File:
Export review
Import review
Print
Error log
Exit

Edit:
Undo
Redo
Cut
Copy
Paste
Edit default values

Review:
Copy review
Renumber review
Delete review

Help:
RevMan help
FAQ
Handbook
About RevMan

Conversion program from RevMan 1.04b to 3.0 (*)
RAG priority: No priority assigned yet

Some reviewers have asked for a conversion program from RevMan 1.04b to 3.0. This program already exists in a DOS version and could easily be translated to Windows.

Default statistical method (**)
RAG priority: No priority assigned yet (The group dealing with the structure of a review should address this)

It should be possible to select a statistical method for each outcome, which should be passed onto MetaView. (As it is now, MetaView always uses the same default.)

RevMan in a window (***)
RAG priority: High

The RevMan windows takes up the entire screen and can not be moved. Moving RevMan to Visual FoxPro would solve this problem.

Review version control
RAG priority: High

Highlighting differences between different version of the same review (to allow editors and CRG coordinators, as well as reviewers, to see quickly how a review has changed).

Internal notes field
RAG priority: High

Field to hold messages between the CRG coordinator and editors about a review within RevMan (especially if the editorial notes field will be part of the CDSR version of a review since these coordinator/editor message would not be for publication).

Amendment screen
RAG priority: High

Screen to show the location of amendments. This would be a form that could be ticked to show which section of the review had been amended. It would be part of the review in CDSR. It might be possible to add it to RevMan at the same time as CritMan is added.

Check for missing data
RAG priority: Low

Ability to check which trials and outcomes have data added.

Egg timer
RAG priority: No priority assigned yet

Mouse pointer should change to an egg timer when the computer is working.

IPD and dichotomous data
RAG priority: No priority assigned yet

Should be possible to combine IPD and dichotomous data under one outcome.

Printing protocol
RAG priority: No priority assigned yet

Should be possible to print text related to the protocol only.

Tabular text
RAG priority: High

Ability to present text in a tabular form.

Data extraction forms
RAG priority: High

Ability to print out data extraction forms.

Alphabetically ordering
RAG priority: High

RevMan should sort study IDs by lower case.

Labels for treatment and control groups
RAG priority: No priority assigned yet

Labels for the treatment and control groups are not transferred to MetaView.

MetaView 3 issues:

Export graphs
RAG priority: Moderate

It should be possible to save a graph in a windows meta file, for example, so it easily can be used in other programs.

Save settings
RAG priority: High (settings should however revert to default values on start-up until something else is decided)

Each time you open a new window the statistics and display options changes back to default values. This can be very annoying. (see Default statistical method above.)

Another way to view an outcome
RAG priority: No priority assigned yet

In MetaView 3 the only way to view an outcome is by double-clicking on the summary graph. Maybe there should be an icon or a menu item to use too.

Display comparison and outcome numbers on graphs
RAG priority: High

The figure number should match the number in the comparisons table.

Calculation of standard deviations
RAG priority: No priority assigned yet

A technique for calculating standard deviations from other measures of variance such as p-values.

Quality score analysis
RAG priority: No priority assigned yet

Investigations of the relationship between the quality scores given to each trial and the effect sizes they report.

Option to print next graph
RAG priority: No priority assigned yet

Option to print next graph

Calculation of p-values
RAG priority: No priority assigned yet

Calculation of p-values from chisquare and z statistics.

Funnel plots
RAG priority: No priority assigned yet

Implementation of funnel plots.

Sensitivity analysis
RAG priority: No priority assigned yet

Ability to do sensitivity analyses by excluding trials from particular analyses.

Display of number of trials and patients
RAG priority: No priority assigned yet

Indicate the number of trials and patients for each outcome on the summary graph and possibly the heterogeneity within each outcome: this should be considered by the Structure of a Cochrane review group.

Warning on summary graph
RAG priority: High

A warning should be displayed on the summary screen saying that the graph does not always display a positive effect on the left side.

RAG minutes | MinMan home