Minutes of the RevMan Advisory Group Meeting held at the UK Cochrane Centre on Wednesday 12 June 1996

Present: Clive Adams, Mike Clarke, Carl Counsell, Jon Deeks, Lelia Duley, Monica Fischer, Derek Harvey, Jini Hetherington (minutes), Andy Oxman (convenor), Lois Sims, Mark Starr, Lesley Stewart.

Apologies: Zarko Alfirevic, Nick Freemantle, Bill Gillespie, Patrick Vandekerckhove.

Contents:
1. Approval of minutes and review of action items from the last meeting
2. Status reports - RevMan, ModMan, Parent, Manuals and Handbook Section VI
-
4. Review of specifications for RevMan and development plans for 1996
5. Review and summary of bugs, problems, suggestions and requests
6. Manuals, help screens, training and support
7. Mandate, membership and selection of convenor of R.A.G
8. Next meeting
9. Any other business
10. Wish lists

1. Approval of minutes and review of action items from the last meeting
1. The minutes from the last meeting, held on 19 January 1996, were approved.
2. Mark and Monica are unable to compile a checklist of useful tips for inclusion in the RevMan manual until the software has been finalised, but have this in hand to do.

Action: Mark, Monica
3. Andy and Monica have checked the headings in RevMan as compared to those referred to in Section VI of the Handbook. This task will need to be done again when the improved software has been thoroughly beta-tested. The RevMan manual now contains advice on a simple system for numbering reviews and keeping track of different versions.
4. Andy will work with Cindy Mulrow and check the headings for structured abstracts, as included in Section VI of the Handbook, ensure that they match the headings required within the text of reviews, and issue a reminder to CRG Administrators of what these headings should be, bearing in mind the reviews on CDSR that already have abstracts. The heading Data synthesis' should be changed to Results', and Data extraction' should not be a required heading. Guidance should be given to keep abstracts to one screen-full of text.

Action: Andy
5. Jini agreed to check with Sonja Henderson that existing Pregnancy and Childbirth reviews contained on CDSR will have abstracts added to them by 2nd September, in time for the October issue.

Action: Jini
6. Mark will ensure that one can print out from RevMan those pages relevant to titles and protocols only; he is also looking at increased options for printing (such as all graphs from a review in one go), and displaying OR (CI) on the graphs. He is also looking at better facilities for searching titles in the Topics List provided by CRGs.

Action: Mark
7. Software testing
1. Jon will produce a form for documenting problems experienced when testing MetaView. Members of the RevMan Advisory Group and the person designated to deal with RevMan queries within each Cochrane Centre should be testers of this software. Jon will also contact the members of the Statistical Methods Working Group to let them know that testing needs to take place during the month of August.

Action: Jon
2. Jon also agreed to check with Doug Altman regarding the wisdom of implementing the facility to toggle between 95% and 99% confidence intervals.

Action: Jon
8. Jon is preparing both a lay and a technical document about the statistical methods available in RevMan. Final drafts should be available in August, and these documents should be ready to be widely circulated at the next Colloquium. A help screen in MetaView should tell the user how to access the documentation via the FTP Server.

Action: Jon, Gordon
9. CRGs should recommend to their reviewers which default statistics to apply to reviews within their individual Group, and state what these defaults are in the Information section of their module. Sections II and VI of the Handbook should contain this recommendation in due course; in the meantime, the CRG Administrators could pass this advice on within their editorial teams. Jini to convey this message to CRG administrators.

Action: Jini
2. Status reports - RevMan, ModMan, Parent Manuals and Handbook Section VI
1. RevMan: Lois demonstrated version 3.0 (beta) of RevMan, screen by screen, and various changes were suggested. She will try to implement as many of these as possible, within the constraints of the time available to do this before the software needs to be released more widely for beta-testing. Details are as follows:
1. When a protocol changes status to a review, there should be a prompt to remind the reviewer (or CRG Administrator) to delete the redundant Date next stage expected'.
2. It should be made impossible to export a protocol from RevMan without an expected date', because this should not be exported in a Group's module to the Parent Database, since it must not appear on CDSR without one.
3. The notes field in RevMan should be exported to ModMan, but not to the Parent Database.
4. There should be added an 'Editorial Notes' field in ModMan, which is attached to the cover sheet of a review. This field should be exported to the Parent Database.
5. Notes made on a particular review should remain with that review, and not be overwritten when the review is updated.
6. There is a bug such that when one has moved from the long title to the short title, one cannot go back up to the long title.
7. On the cover sheet, it should say "Standard citation format" rather than "Enter reviewers as", and there should be a prompt if this is not adhered to.
8. It is not necessary to display the Group ID on the RevMan cover sheet.
9. There should be a master pick list of the names of all countries in RevMan itself, from which to make the appropriate selection, including options Other' and Transnational'. Guidance can be given in the manual on this.
10. It should be made possible to move a study from one section to another (e.g. from Included to Excluded), with a warning that any attached data will be deleted if this is done.
11. The sort order needs to be investigated, or it should be possible to flag the primary reference in a list of Included Studies.
12. There should be a pop-up or help screen providing an explanation of how to assess the quality of allocation concealment. Standard definitions should be used (A, B and C from the Handbook, plus D=not used in this review), and there should be no default.

Action: Lois Sims, via Monica
13. For Other references' and Previously published versions of a review', a unique identifier should be assigned, so that this can be cited within the text of the review. Jini should let CRG Administrators know this, so that they can tell reviewers.

Action: Jini
14. Lesley will pass her requirements for dealing with individual patient data in RevMan directly to Lois.

Action: Lesley
15. Labels stating Treatment' and Control' should be added, so that one sees what these labels are before editing them.
16. There should be a warning in RevMan and the Cochrane Library not to assume that the effects are all displayed in the same direction (i.e. representing a reduction or increase in 'undesirable outcomes' with results indicating treatment better than control on the left side) in summary graphs.
17. Comments and Criticisms: Andy said that funding was being sought by the San Francisco Cochrane Centre to implement this.
18. There is a bug such that one cannot assign different contact details for different reviews within the same version of RevMan. If this is not correctable, instructions need to be provided for getting around this.

Action: Mark, Lois, Monica
2. ModMan: It had been decided at the March 1996 meeting of the ModMan Advisory Group that there is still work for the CRG Administrators to do in providing feedback and suggestions for improving this software. Its basic composition should therefore remain much the same, Hazel Fraser resigning and being replaced by Sonja Henderson, and Monica leaving the Group because of her return to Denmark.
3. Parent: Monica advised that there is currently a document in preparation to propose possible future management arrangements for the Parent Database.
4. Manuals: Andy will co-ordinate taking these forward with Cindy Mulrow, Jon, Lelia, and Monica, possibly by arranging an August meeting in Oxford just before the Steering Group meeting in Oslo. Rasmus Moustgaard and Mike Clarke should also consider attending this meeting. As part of her continuing role in software development after her move to Denmark, Monica would continue overseeing production of the RevMan manual, with Lelia providing input on all aspects of doing reviews, and Jini ensuring that common problem areas brought to her attention by the CRG Administrators are adequately addressed.

Action: Andy
1. The manual should include the suggestion that the short title of a protocol/review includes the current date, to distinguish between various versions of the same one. Also, reviewers should be warned not to have too many copies of the same review, and to pay attention to which version they are currently working on.

Action: Lelia, Monica, Jini
2. There should be guidance given in the manual as to the correct body or institution to cite as a source of support to a review. This differs from intramural sources of support cited within a module, which are not necessarily always sources of financial support.
3. CRG administrators should warn reviewers to look out for, and delete, headings in the text of reviews which duplicate what appears in the software (such as Background, Objectives, etc.). Jini to send a reminder to CRG administrators.

Action: Jini
5. Handbook Section VI: Andy outlined the revised plans for co-ordinating this Section with the training manual being put together by himself and Cindy Mulrow.
3. was merged with 2)
4. Review of specifications for RevMan and development plans for 1996
1. Most of the items in Appendix C have now been implemented in RevMan 3.0 (beta). Those items remaining have been moved to a Wish' List (attached), and will be put in order of priority by Andy and Monica and circulated prior to the next meeting.

Action: Andy, Monica
2. Jini will prepare a statement to CRG Administrators and Cochrane Centres, with Mark's help, explaining that it will not be possible in the foreseeable future to use unusual' characters in the free text fields within RevMan, since they will have been typed on non-standard (i.e. non-English) keyboard character sets, which are lost when imported into the Parent Database. Solving this problem will require a substantial amount of programming. Funding for this might be available through BIOMED II.

Action: Jini, Mark
5. Review and summary of bugs, problems, suggestions and requests
1. See RevMan and MetaView Wish Lists.
6. Manuals, help screens, training and support
1. The contents of Section VI should be integrated within Revman, Metaview and The Cochrane Library as pop-up and help screens wherever appropriate/feasible.
2. The provision of help screens for the statistics in MetaView cannot be implemented until after MetaView has been adequately beta-tested this summer. The text of these screens should be taken from the Handbook and statistical appendices being prepared by Jon.
3. Once the manual has been finalised, Rasmus Moustgaard at the Nordic Cochrane Centre has volunteered to set up a help system in RevMan which can be used in both the DOS and Windows versions.
4. Jini described the proposed pilot RevMan training workshops for the next Colloquium, targetted to CRG Administrators, to equip them to provide better support to reviewers. A better understanding of the point and process of doing a systematic review might also perhaps interest some of them in participating in doing a review themselves.
5. It was agreed that all reviewers' queries regarding RevMan should be put initially to the Administrator of their CRG. This strategy for supporting reviewers should be explained to Centre Directors and CRG Administrators, giving the rationale for it.

Action: Monica, Andy
7. Mandate, membership and selection of convenor of R.A.G.
1. Jini had prepared a method of drawing lots for rotating membership from other CRGs on the R.A.G. She will implement this with Monica's help and let the results be known as soon as possible, so that Andy can invite participation from CRGs not currently represented. Paul Garner had already resigned and Patrick Vandekerckhove is unlikely to be able to continue to participate. It was agreed at her suggestion that Lesley be replaced by Derek Harvey. It was agreed that Andy should continue as Convenor, and would also represent CCEPP, so he would suggest to Nick Freemantle that his place be filled by someone from a CRG not currently represented.

Action: Jini, Andy
2. Minute-taking at these meetings should rotate, and Andy would ask someone other than Monica or Jini to take the minutes of the next meeting. It was agreed that minutes of this meeting be circulated to CRG Administrators, as they contain very useful information as to things that are in the pipeline to do with the software.

Action: Jini
3. It was thought impractical for a North American RevMan Advisory Group to be established, as the discussions it would have about the software would reinvent the wheel'; Joe Lau had therefore been invited to be a member of the UK-based R.A.G.
8. Next meeting
1. This will be held in the Meeting Room at the UK Cochrane Centre on Wednesday 8th January 1997, from 11.00 a.m. to 4.00 p.m.
9. Any other business
1. Testing and release of the software:

The time-scale for the release of RevMan 3.0 (beta) for beta-testing is such that it would not be practical to encourage all Cochrane Centres to run RevMan beta-testing workshops such as that held at St George's Hospital Medical School at the beginning of the year. It was agreed that the new MetaView should be released at the same time as RevMan 3.0 (beta), by the end of July, for the statistics to be thoroughly tested by members of the R.A.G. during August. The Software Development Group should agree when everything is ready to be released, and aim for this to happen in September. Jini will contact the CRG Administrators to nominate someone within their Group to test RevMan 3.0 (beta), and compile a list of these people. Mark agreed to check off on this list those that have provided feedback as a result of beta-testing the software in August.
Action: Jini, Mark
2. RevMan at the Colloquium:

Mark agreed to organise with David Badger of the ACC a two-hour slot sometime during the Colloquium when people could play with RevMan on computer, and members of the R.A.G. would take it in turns to be present to answer questions. He would also arrange for there to be a suggestion box on hand.
Action: Mark
10. Wish lists
1. RevMan Wish List
1. The possibility to print a review from RevMan to a file.
2. The possibility to export analysis table/graphics as a figure that can be imported into a word processing package.
3. On installation of RevMan, a text' subdirectory should be automatically created (automatic creation of a working' subdirectory has already been implemented).
4. Inclusion of a pick list of sources of support. Jini would ask Carol Lefebvre to obtain the Wellcome Trust's list.

Action: Jini
5. Inclusion of a pick list of countries (displayed in upper case letters).
6. The ability to refer to specific comparisons that one wants to print out.
7. The comparisons should be labelled to match the comparison numbers. Carl will draft instructions to reviewers, check them with Mark, and pass them to Jini to forward to CRG Administrators, to advise quoting the comparison in the text by its number (e.g. see Comparison 01.01.01).

Action: Carl, Jini
8. There should be links between RevMan and a CRG's module so that one can easily get to the text provided in that module.
2. MetaView Wish List
1. The figure number should match the number in the Comparisons Table
2. A technique for calculating standard deviations from other measures of variance such as p-values.
3. Investigation of the relationship between the quality scores given to each trial and the effect sizes they report.