Minutes of the RevMan Advisory Group Meeting held at the UK Cochrane Centre on Thursday, 27 November 1997

Present: Clive Adams, Christopher Cates, Mike Clarke (minutes), Inez Cooke, Jon Deeks, Lelia Duley, Monica Fischer, Bill Gillespie, Sonja Henderson, Kate Oldfield, Andy Oxman (chair), Elizabeth Royle, Mark Starr, Lindsay Stead.

Apologies: Carl Counsell, Reive Robb, Patrick Vanderkerckhove.

In attendance: Rasmus Moustgaard (Nordic Cochrane Centre), Veronica Yank (San Francisco Cochrane Center). Contents:
1. Welcome
2. Approval of minutes and matters arising that are not on the agenda
3. Wish list for RevMan 4.0
4. Criticism manager
5. RevMan training and support
6. RevMan Advisory Group membership and convenor
7. Next meeting
APPENDIX

Welcome

Kate Oldfield was welcomed to her first meeting of the RevMan Advisory Group (RAG). She has replaced Jini Hetherington on the RAG, having taken over from Jini as convenor of the ModMan Advisory Group. Jini was thanked, in her absence, for her very valuable work within both the RevMan and the ModMan Advisory Groups.

Approval of minutes and matters arising that are not on the agenda

1. The minutes of the previous meeting (8 January 1996) were approved.

2. Item 2.h.ii: a footnote will be added to The Cochrane Library to indicate that the major publication for an included study is marked with a *. Recommendations that reviewers make use of this symbol will be added to the RevMan manual and the Handbook and a message will be sent to CRGs.
   Action: Andy Oxman, Mark Starr

3. The Cochrane Library will also be changed so that the "data source" of each study will be shown. Suitable wording will be devised for the four classifications. The need for this classification to be input for each outcome for each study, rather than for a study as a whole should be added to the RevMan wish list. Reviewers should be advised to mark the relevant trials where unpublished data have been used for particular outcomes.
   Action: Rasmus Moustgaard, Andy Oxman, Mark Starr

4. Item 3.b.iii: wording to allow for the formal definition of a Cochrane review to be added to RevMan will be devised. It was suggested that this should come on screen when "Create new review" is selected.
   Action: Andy Oxman

5. Item 4: Mike Clarke has taken over responsibility from Lelia Duley for oversight of the production of the RevMan manual.
   Action: Mike Clarke

6. No discussion was required for any of the other action items in the previous minutes that were not already on the agenda for this meeting.

Wish list for RevMan 4.0

1. The proposed timetable was that this meeting would review the results of the survey about the wish list for RevMan 4.0 and make recommendations to the Software Development Group (SDG) which will meet on 28 November (ie, the following day). The SDG will make decisions on issues related solely to software and will prepare recommendations on issues that will affect the structure of reviews or have major resource implications. These will be circulated for comment and will be considered by the Steering Group at their next meeting, in February 1998. A specification will then be prepared for the software so that it can be constructed and tested. The intention is that RevMan 4.0 will be released in mid-1999. It is expected that any subsequent major release (RevMan 5.0) will not be necessary before 2002.

2. It was noted that the survey whose results were to be discussed was primarily of those involved in the production of reviews. There will also need to be consultation with users of reviews if a proposed change to RevMan will have consequences for them.

3. The survey report was discussed in great detail. Each item on the wish list was considered and a recommendation was made about whether or not it should be adopted for RevMan 4.0. It was recognised that whether, and how, these recommendations could be implemented should be left to those involved in the production of the software. The wish list, annotated to show the Advisory Group's recommendations, is appended to these minutes.

4. The Handbook needs to be changed to clarify that the notes field in the Table of Included Studies is included in CDSR.
   Action: Andy Oxman

Criticism manager

1. If approved by the Software Development Group, CritMan will be incorporated into RevMan 4.0 and will not be added to RevMan before then.

2. Veronica demonstrated the current prototype for CritMan and discussed the consultation process that has taken place so far. It was agreed that an Advisory Group should be established for CritMan and that this should contain adequate representation of criticism editors, contact editors, Review Group coordinators and reviewers. One way to identify the appropriate representative would be to look at which CRGs are receiving most comments. This Advisory Group should be distinct from the Evaluation Group which was to be convened by Mildred Cho. A discussion list should also be established. CritMan issues that relate to RevMan should be discussed by the RevMan Advisory Group.
   Action: Veronica Yank

3. All mentions of "author(s)" in CritMan should be replaced with "Reviewer(s)" for consistency throughout the Collaboration.
   Action: Veronica Yank

4. Consideration needs to be given of how interactions with, for example, contact editors and the commentator could be incorporated into CritMan.

RevMan training and support

This item was passed over due to time constraints.

RevMan Advisory Group membership and convenor

1. Carl Counsell, Bill Gillespie, Elizabeth Royle and Patrick Vanderkerckhove will be stepping down from the Group. Much appreciation was expressed for their help with the development of RevMan. It was agreed that they should be replaced by experienced reviewers and that these should come from CRGs with a relatively large number of protocols and reviews. Stroke, Subfertility, Musculoskeletal injuries and Neonatal were suggested.
   Action: Andy Oxman

2. Andy was unanimously supported as the continuing convenor of the RevMan Advisory Group.

Next meeting

1. At the UK Cochrane Centre in April or May 1998 (date to be determined by email).
   Action: Andy Oxman

APPENDIX: RAG RECOMMENDATIONS FOR REVMAN 4.0

1. Suggestions that should be implemented in RevMan 4.0 and do not need further discussion by the RAG or SDG

A.9 DEFAULT STATISTICAL METHOD
It should be possible to select a statistical method for each outcome, which should be passed on to MetaView and CDSR and be the default when the review is displayed. (As it is now, MetaView always uses the same default within and across reviews.)
Agreed.

A.23 DISPLAY USER DEFINED ORDER
It should be possible to display the values of the user defined order. It is useful to be able to sort trials by a user defined order, and it would be even more helpful to know why the trials are ordered as they are. For example, if a user defined order where A, B, C, there is no way for the end user to know if all the trials were A, or most were A and some B, etc.
Agreed.

A.24 OPTIONS FOR GRAPH PRINTING
There should be a number of options to decide the size and format of the graphs to be printed.
Agreed. The options will need to be determined. A draft list would be compiled for consultation.
Action: Rasmus Moustgaard

A.25 PRINT ALL FACILITY
There should be a menu option to print all the detail graphs.
Agreed.

B.3 CHOICE OF STATISTICS FOR SUMMARY SCREEN
It should be possible to select any of the statistical methods for displaying the summary screen. Currently only the Peto odds ratio and the weighted mean difference are available.
Agreed. Each CRG should decide on the default for their group, but reviewers would be able to change this for their reviews and the defaults will be set at the review level. The Statistics Methods Working Group will be asked to provide written guidance to help CRGs and reviewers select the appropriate default. The RevMan manual should stress that the summary screen be kept as simple as possible.

B.12 SENSITIVITY ANALYSIS
It should be possible to do sensitivity analyses by excluding trials from particular analyses. (Note: This could encourage post-hoc subgroup analysis.)
Agreed, but appropriate guidance will be needed.

C.2 EXPORT OF REVIEWS TO OTHER PROGRAMS
It should be possible to print a review to an ASCII file, at the least.
Agreed.

C.3 DATA EXPORT
There should be a special export feature for numeric data or an option to print them to an ASCII file.
Agreed.

C.8 REDESIGN OF THE MENU SYSTEM
The menu system in RevMan should be redesigned for greater clarity.
Agreed.

C.15 TOOLTIP TEXTS
Explanatory text should appear when the user pauses the mouse pointer over a button or menu option for a moment.
Agreed.

C.18 DATE ON PRINTED REPORTS
There should be an option for adding the date (or the date last amended) when printing from RevMan.
Agreed.

C.20 EXPORT GRAPHS
It should be possible to save a graph in, for example, a Windows meta file so it can easily be used in other programs.
Agreed.

C.21 TAGGING OF REVIEWS
Reviews should be tagged with RevMan version number, platform, date and perhaps code page so that future versions of RevMan will be able to auto convert reviews from previous versions.
Agreed.

2. Suggestions that should be implemented in RevMan 4.0, if possible and do not need further discussion by the RAG, but should be discussed by the SDG

A.26 PRINTING SEVERAL GRAPHS ON SAME PAGE
One ought to be able to print several graphs on a page. At present, each graph from MetaView is printed on a separate page.
Agreed.

B.11 HAZARD RATIOS
It should be possible to calculate hazard ratios from survival curves. (Note: See Stewart LA, "How to make the best use of published summary statistics in systematic reviews of survival data" presented at the Cochrane Colloquium in Amsterdam.)
Agreed.

B.16 DOCUMENTATION OF STATISTICAL METHODS
The statistical methods and formulas used in MetaView should be documented.
Agreed.

C.1 COMPLETELY WINDOWS BASED REVMAN
Use of the DOS programs pkzip and pkunzip prohibits RevMan 3.0 for Windows from being completely Window based. If the DOS programs were replaced by Windows programs or a compression library, RevMan would be able to run on Windows emulators, and perhaps other problems concerning import/export would be eliminated as well.
Agreed.

C.5 DATA IMPORT FEATURES
It should be possible to import numeric data and characteristics of studies, for example, from spreadsheets to RevMan.
Agreed.

D.19 USE OF TEMPLATES?
"Having spent some time working with RevMan I wonder if things could be done much more quickly with a series of templates based on Microsoft Office. This would reduce the tremendous workload at the programming end and solve many of the issues raised in these questionnaires."
Not recommended.

3. Suggestions for making RevMan easier to use that should be discussed and decided by the RAG

3.1 General features

C.6 EGG TIMER
Mouse pointer should change to an egg timer when the computer is working.
Agreed, but low priority.

C.11 RESIZEABILITY OF SCREENS
It should be possible to resize the screens.
Agreed.

C.13 SPANISH VERSION OF REVMAN
All text within and associated with RevMan should be translated into Spanish.
Not recommended.

C.16 WISH LIST/BUG REPORTING FORM IN REVMAN
A pre-formatted form for reporting problems and suggestions for change should be included in RevMan.
Agreed.

D.25 BIGGER NOTES WINDOW
Notes on the cover sheet: It should be clear that these can be used for sending notes to the CRG (which will not be published). There should be a bigger window for editing the notes. Notes that are not to be published should be moved to a separate section within RevMan.

D.44 TRANSFER BETWEEN MAC AND PCS
Transfers between Mac and PCs might be eased.
Agreed.

3.2 Printing

C.7 PRINTING OF PROTOCOL
It should be possible to print text related to the protocol only.
Agreed.

C.4 DATA EXTRACTION FORMS
It should be possible to print out data extraction forms.
Agreed, but these would not come directly from RevMan. Instead, examples should be included in the Handbook and, therefore, packaged with RevMan.

D.43 PRINTING OF FOOTNOTES
Footnotes: when you print out say, the table of included studies, the footnotes come just as a bit of text with no heading (e.g. "Footnotes"). Agreed to consider changing this.

3.3 Main screen

C.10 REORDERING OF BUTTONS ON MAIN SCREEN
The 'Table of comparisons' button should be moved below 'Table of excluded studies' so the buttons are ordered in accordance with the procedure followed when entering a review.
Not recommended.

C.9 REMOVE INCLUDED AND EXCLUDED TABLE SCREENS
The 'Table of included studies' and 'Table of excluded studies' screen do nothing you cannot do under 'References to studies', so they should be removed (for clarity).
Not recommended.

D.24 PULL DOWN SORT MENU
The sort buttons on the main screen should be moved to a pull down menu. Options to sort on 'Date review last updated' and 'reviewer ID/review number' (like in ModMan) should be added. Useful if/when RevMan is separated from ModMan.
Agreed.

D.37 REMOVE THE EXPORT SCREEN
There should be an option to mark reviews as selected on the main screen. The export function should not open a new screen but simply ask for a filename into which to export the selected reviews.
Agreed.

D.38 GLOBAL FIND FUNCTION
Global find function which marks all reviews containing a certain string as selected.
Agreed.

D.39 WORKING DIRECTORY FOR IMPORTING REVIEWS
The import function should not ask for details about 'working directory' because it's confusing and you almost never need to change it. 'Working directory' should only appear at 'edit default values' screen. The import function should just ask for a file name for the import file.
Agreed.

D.40 CHANGE ZIP EXTENSION
RevMan files should use another extension than '.zip' (e.g. '.rm') to distinguish them from other zip files. You should be able to override this, of course.
Agreed.

3.4 Cover sheet, text, tables

C.12 REVIEW VERSION CONTROL
It should be possible to highlight differences electronically between different versions of the same review (to allow editors and CRG co-ordinators, as well as reviewers, to see quickly how a review has changed). (Note: Implementation of this suggestion must be co-ordinated with the Criticism Management software that is being developed.) Not recommended as cannot be implemented, but the desirability of an option to show that two versions were identical was
Agreed. Guidance is needed to ensure that the top version of a review is the one being edited, submitted, etc. Mark Starr will convene a subgroup (comprising Sonja Henderson, Rasmus Moustgaard and Veronica Yank) to make recommendations on this.
Action: Mark Starr, Sonja Henderson, Rasmus Moustgaard, Veronica Yank

C.14 SPELL CHECKER
There should be an option for spell checking text.
Agreed.

C.17 WORD COUNT
An option for counting the number of words in the abstract or text of a review should be added.
Agreed.

D.29 FIND AND REPLACE
A find and replace facility which works on all the text sections of a review.
Agreed, and that it should be possible to do this globally for all reviews. Confirmation of each replacement should be an option.

D.16 MULTIPLE WINDOWS
To be able to open more than one description of studies window at a time - it would make cutting and pasting much easier.
Agreed.

D.27 DISPLAY OF SHORT TITLE
The short title of the review displayed at the top of many screens could be moved to the window title. This would give more room on otherwise crowded screens.
Agreed.

D.28 REMOVE EXPLAINING TEXT
The explaining text on many screens should be removed. Instead the user should be encouraged to press F1 for context sensitive help (found in RevMan 3.1 and above).
Agreed. It might be possible to replace this with tooltips.

D.26 AUTHOR CITATION
Could the 'standard citation format' labels on the cover sheet be changed to simply saying 'citation' or 'author citation . Agreed that this should be changed and that a tooltip might be useful. A new phrase and suitable wording would be drafted.
Action: Andy Oxman

D. 41 COLOUR CODED TABLE OF COMPARISONS
The table of comparisons could be colour coded (red, blue, light blue) for comparisons, outcomes and sub-categories like on the edit data screens.
Agreed.

D.34 TREE STRUCTURE FOR COMPARISONS AND DATA TABLES
The 'table of comparisons' screen and the 'data tables' screen could be combined. It could be a tree structure (e.g. known from 'relationships' in HIREx) where branches could be expanded or collapsed. Under each outcome or subgroup the studies, for which data were entered, could be listed and simply double-clicking on one would take you to the data edit screen for this study. Likewise highlighting a study and pressing the delete button would delete the data for that study. The table could be restructured by 'dragging and dropping'.
Agreed.

D.35 TREE STRUCTURE FOR EVERYTHING
To take D.34 further the entire structure of RevMan could be built as a single tree including cover sheet, text, references, studies and the table from D.34.
Not recommended.

D.32 SKIP IMPORT REFERENCES SCREEN
Skip the 'import references' screen and jump directly to an 'open file' dialogue box.
Agreed.

D.33 SHORTER STUDY YEAR FIELD
The study year should be reduced to four digits. What is the fifth for?
Not recommended. A tooltip explaining the use of the fifth character (to allow year sorting in MetaView) might be helpful.
Action: Mike Clarke

D.42 DISPLAY ALL OTHER REFERENCES AT ONE TIME
It would be nice to view all of the "other references" at one time rather than having to scroll through one by one, always starting at the beginning.
Agreed.

D.31 SMALLER FOOTNOTES SCREEN
The footnotes screen should be smaller.
Agreed.

D.36 GRID FOR DATA ENTRY
All entered (numeric) data for an outcome or subcategory could be displayed and edited in a grid on the same screen (i.e. the study IDs in a column to the left followed by columns for treatment and control groups etc.). It should be possible to copy and paste data from a spreadsheet into this grid.
Agreed. The checking for illegal values would still work in such a grid.

C.19 DISPLAY COMPARISON AND OUTCOME NUMBERS ON GRAPHS
The comparison, outcome and sub-category numbers should be displayed on the detail screens.
Agreed. This change will also be needed in CDSR.

4. Changes in the format of Cochrane Reviews or in existing reviews or of potential interest to both reviewers and users of reviews

4.1 Cover sheet

A.4 DATE FIRST PUBLISHED
There should be a field in a review for the date when it was first published.
Agreed.

A.7 TELEPHONE NUMBERS IN SECONDARY ADDRESS
It should be possible to include a telephone number in the secondary address.
Agreed.

A.5 LONGER 'SOURCES OF SUPPORT' FIELD
The 'sources of support' field should be made longer to accommodate the full name of sources of support that are longer than the current limit (80 characters).
Agreed. This would be increased to 150 characters and a picklist could be added.

A.8 USER DEFINED ORDER FOR 'SOURCES OF SUPPORT'
The reviewer should be able to determine in RevMan the display order of sources of support that will appear in CDSR.
Agreed. The intramural/extramural distinction should be shown in CDSR.

D.9 FIELD TO NOTE UPDATE WITHOUT NEW DATA
Option to indicate that, on a certain date, a review had been thoroughly checked and was felt to be up to date, but that a substantial revision had not been made (e.g. because no new data were available).
Agreed. This should be noted in "What's new?".

4.2 Text

A.1 WHAT'S NEW
A 'what's new?' section should be added to the text for updated reviews. (Note: Implementation of this suggestion must be co-ordinated with the Criticism Management software that is being developed.)
Agreed. Guidance will be needed for the contents of this and consultation will be sought for a draft. It will include a tick-box to show whether a review has been substantially updated (whether by editing or for another reason), whether a minor change has been made or whether a response to criticism is included. Reviewers will be asked to indicate one of these four options when they export a review.
Action: Mike Clarke

A.2 AMENDMENTS SCREEN
There should be a screen to show the location of amendments; i.e. a form that could be ticked to show which sections of the review had been amended. This could be part of the review in CDSR. (Note: Implementation of this suggestion must be co-ordinated with the Criticism Management software that is being developed.)
Agreed. This would distinguish between substantive and minor changes. It would be for internal use only and should not be included in CDSR.

A.3 BACKGROUND SECTION IN ABSTRACT
A background section should be added to the abstract.
Not recommended, to avoid requiring reviewers to revise their abstracts.

A.19 ABSTRACTS FOR PROTOCOLS
Abstracts should be mandatory for protocols.
Not recommended.

D.3 ADVERSE EFFECTS SECTION IN THE TEXT
RevMan should have a heading and a space for adverse effects, both in the text, and a table showing the sources of the reports of the adverse effects. If no reports of adverse effects have been found by the reviewers, then this should be explicitly stated. However, since the Cochrane Collaboration has not yet discussed let alone agreed on appropriate search strategies for finding reports of adverse effects, we can do no more than encourage reviewers to describe what they did to find such reports.
Not recommended. These should be dealt with as outcomes'. There was no consensus on having a fixed heading but guidance would be added to the Handbook about the importance of considering harmful effects of interventions.
Action: Andy Oxman

D.14 SECTION FOR CHANGES TO A PROTOCOL
It would be useful to have a section to explain why a reviewer may have made changes to a protocol.
Not recommended. This information should be included in "What's new?".

D.7 LONGER FIELD FOR STUDY LABELS
Greater flexibility in labelling of studies (more space for names, particularly in MetaView).
Agreed.

A.15 HYPERTEXT LINKS IN REVIEWS
It should be possible for reviewers to insert hypertext links in reviews (to sections [fixed headings] of the CRG module, Handbook headings and possibly other parts of the Library).
Not recommended that hypertext links should connect parts of a review to other sources of information, but the facility for hypertext links within a review was agreed.

A.16 FORMATTING
It should be possible to use bold, italics etc. (Note: This would introduce the possibility of undesirable variations in style.)
Agreed, but clear guidance on style will be needed.

A.17 INTERNATIONAL CHARACTERS
It should be possible to use international characters in reviews. (Note: Although there is unlikely to be much disagreement about Cochrane Reviews including non-English characters, there are problems with implementing this.)
Agreed.

D.17 SUBHEADINGS FOR ONGOING STUDIES
There should be specified sections to fill in for ongoing studies e.g. interventions, planned size, method of randomisation, outcomes, expected finish date etc. This has been discussed and the need agreed at previous RAG meetings.
Agreed. These need to be decided and should reflect those in CENTRAL. Because of their involvement in the latter, the Advisory Group asked Mike Clarke and Mark Starr to compile a list.
Action: Mike Clarke, Mark Starr

4.3 Tables

A.6 FORMAT OF TABLES FOR OTHER DATA AND CHARACTERISTICS OF INCLUDED STUDIES
It should be possible to customise the format of these tables, including the number of columns and formatting within columns.
Agreed that this should be more flexible.

A.18 OUTCOMES SHOULD BE ORDERED BY IMPORTANCE
Reviewers should be instructed to place most important outcomes at the top. This is not an issue for the RevMan software. The RevMan Manual and the Handbook should recommend that users do this.
Action: Mike Clarke, Andy Oxman

A.20 TABLES IN TEXT OF REVIEW
It should be possible to present tables in the text of a review.
Agreed.

D.3 ADVERSE EFFECTS TABLE
RevMan should have a heading and a space for adverse effects, both in the text, and a table showing the sources of the reports of the adverse effects. If no reports of adverse effects have been found by the reviewers, then this should be explicitly stated. However, since the Cochrane Collaboration has not yet discussed let alone agreed on appropriate search strategies for finding reports of adverse effects, we can do no more than encourage reviewers to report what they did to find such reports.
Not recommended (see D.3 above).

D.22 LONGER FIELD FOR USER DEFINED ORDER
Please allocate at least three digits for the user defined order in the data entry table. When the user defined order is based on something such as entry blood pressure, it makes it awkward to be limited to just two digits.
Agreed. This should increase to four digits.

4.4 MetaView

A.10 CUSTOMISED GRAPH LABELS
The user should be allowed to make a saved change in the graph labels used for each outcome under each comparison. Labels for the treatment and control groups should be transferred to and used in MetaView and in CDSR. For outcomes measured on a continuous scale, it should be possible to change the signs on the scale (to negative numbers on the right and positive on the left, if desired, to retain the convention of plotting benefits to the left).
Agreed. Reviewers will need to decide if an outcome is favourable, unfavourable or could be considered in either way. Various options will be piloted to determine how to display this information within MetaView.

A.11 DEFAULT GRAPH LABELS
The default labels on meta analysis graphs (i.e. the default labels in RevMan) should be decided by CRGs. Currently the default labels are hard coded into RevMan. To avoid having different versions of RevMan for each CRG, reviewers would need to change the labels themselves, possibly using the 'edit default values' screen.
Not recommended. The defaults will be removed and reviewers will need to enter them.

A.12 WARNING ON SUMMARY GRAPH
A warning should be displayed on the summary screen saying that the graph does not always display a positive effect on the left side.
Not recommended for RevMan.

A.13 NOTES FIELD ON SUMMARY GRAPH
It should be possible for reviewers to add a note to the summary graph. This could replace the need for a warning on the summary graph saying that beneficial effects can be displayed on both sides of the graph. However, if some reviewers use the field and others do not, it might add more confusion.
Not recommended.

A.14 OUTCOMES DIVIDED INTO GROUPS ON SUMMARY GRAPH
Outcomes should be divided, under each comparison, into desirable, undesirable and other outcomes on the summary graph. The reviewer would have to mark each outcome as desirable, undesirable or other. (Note: This would group studies according to whether beneficial or harmful effects are displayed on the left.)
This should be one of the options to be piloted for MetaView (see A.10 above).

A.21 ANOTHER WAY TO VIEW A DETAIL SCREEN
There should be a icons or a menu item to select graphs to view. The only way to go to a detail screen currently is by double-clicking on the summary graph.
Agreed.

A.22 DISPLAY OF NUMBER OF TRIALS AND PATIENTS
The number of trials and patients should be shown for each outcome on the summary graph and, possibly, the heterogeneity within each outcome should also be shown.
Agreed, but might not be possible.

A.27 VIEW CITATIONS ON DETAIL GRAPHS
By clicking on a study ID, its citations should be displayed.
Agreed, but might not be possible.

D.12 HYPERTEXT LINKS BETWEEN TEXT AND GRAPHS
It might be good if there could be hypertext links to appropriate graphs in the results.
Agreed.

5. New analytic features

5.1 Data entry

B.2 DOUBLE DATA ENTRY
It should be possible to use double data entry to help prevent errors in data entry.
Agreed.

D.6 DATA CHECKS
"Although it is important to keep a balance between annoying messages and useful checks, I wonder if some checks for data inconsistencies could be implemented. A statistician would help here - but extreme differences could be queried."
Agreed. The Statistics Methods Working Group will be asked for advice on whether any simple checks could be included.
Action: Jon Deeks

5.2 Analyses

B.10 REMOVAL OF PETO ODDS RATIO
Peto odds ratio should be removed from the options altogether and only remain in RevMan for the analysis of reviews based on individual patient data.
Not recommended. It was agreed that the Peto odds ratio should be moved in the options list, The Statistics Methods Working Group need to write brief recommendations on which methods should but, perhaps more importantly, should not be used in certain situations.
Action: Jon Deeks

B.4 CALCULATION OF P-VALUES
P-values should be calculated (and presented) from (and instead of) chi square and z statistics.
Agreed. These should be shown wherever the chi square or the z statistic is shown.

B.6 CALCULATION OF STANDARD DEVIATIONS
It should be possible to calculate standard deviations from other measures of variation or uncertainty, such as range or p-values.
Not recommended.

B.15 COMBINATION OF IPD AND AGGREGATE DATA
It should be possible to combine individual patient data (which is now entered as O-E and variance) and aggregate data (2x2 tables).
Agreed.

D.13 NNT CALCULATOR
Add a facility to calculate NNT.
Agreed, if the Statistics Methods Working Group can overcome the methodological problems - particularly in regard to when there are differences in follow-up between studies.

5.3 Subgroup, sensitivity and exploratory analyses

B.14 DIFFERENCES BETWEEN SUBGROUPS OF STUDIES
It should be possible to do a chi square test for differences between subgroups of studies.
Agreed. Guidance will be needed for continuous data.

B.1 COPYING OUTCOME/SUBGROUPS WITH DATA
In order to facilitate outcome/subgroup restructuring, there should be an option to include copies of the data tables when editing the comparison table. Note: This could encourage post-hoc subgroup analysis.
Agreed, but appropriate guidance will be needed.

B.7 FUNNEL (SCATTER) PLOTS
It should be possible to create funnel plots (scatter plots), i.e. a graphical display of sample size plotted against effect size, that can be used to investigate publication bias. Scatter plots could also be used to graphically explore the relationship between effect size and other study characteristics. (Note: It is possible now to sort studies by study size or other study characteristics in the graphs.)
Agreed.

B.8 QUALITY SCORE ANALYSIS
It should be possible to analyse the relationship between quality scores and effect sizes more easily; e.g. using a chi square test between subgroups of studies grouped by quality, or cumulative meta-analysis by study quality.
Not recommended. If the reviewers want to do this, they should use the sensitivity analyses option which will be introduced (see B.12 above).

B.13 CUMULATIVE META-ANALYSIS
It should be possible to do cumulative meta-analyses. (Note: This could encourage post-hoc subgroup analysis.)
Not recommended. If the reviewers want to do this, they should use the sensitivity analyses option which will be introduced (see B.12 above).

5.4 Presentation of results

B.9 PETO ODDS RATIO ON SUMMARY SCREEN
The Peto odds ratio should not be the default and not a display choice for the summary screen, except for individual patient data meta-analyses. It should be replaced by a Mantel-Haenszel method - either the odds ratio or relative risk.
Agreed.

B.5 Z VALUE IN TABLES AND GRAPHS
The chi square and Z values (or P-values, if these are used instead) in tables and graphs should be explained.
Agreed.

D.15 PERCENTAGES IN GRAPHS
An option to show percentages on graphs as well as raw numbers e.g. 8/20 (40%).
Agreed.

D.18 NOTE ON SUMMARY GRAPH
There should be a note on the summary graph telling readers that they can see detail graphs by clicking on the outcome.
Not recommended, to avoid making the appearance of this graph more complicated.

D.45 REPRESENTATION OF MISSING STUDIES
Users of CDSR should be made aware that all of the trials in a review might not contribute data to a particular outcome analysis.
Agreed. The number of trials and patients eligible for a particular outcome analysis should be shown on the relevant figure.

ADDITIONAL ITEM

CONFLICT OF INTEREST
The "Conflict of interest" statement should be re-titled "Potential conflict of interest".
Agreed.

RAG minutes | MinMan home