Personal tools
You are here: Home → IMSG → MAG → Minutes → MAG minutes, 14 January 2004
 

MAG minutes, 14 January 2004

Minutes approved: 1 October 2004

Present: Sonja Henderson (Convenor), Dymphna Hermans (from item 5), Kate Hey, Pauline Howarth, Kate Jewitt, Monica Kjeldstrøm, Rasmus Moustgaard, and Lois Sims.

In attendance: Claire Allen (minutes), Peter Friis, Nancy Owens (for items 5b and 6), and Jacob Riis.

Apologies: Clive Adams, Jane Dennis, Mandy Collingwood, and Reive Robb.

Contents:
1. Welcome
2. Approval of agenda
3. Potential conflicts of interest of members present at this meeting
4. Approval of minutes and matters arising from the previous meeting
5. Update on the development of the new IMS
6. Standardised workflows
7. Wish list (originally ModMan Wish list)
8. Additional figures
9. Dates to be stored in Cochrane reviews
10. Any other business
11. Date of next meeting

  1. Welcome

    2. Sonja welcomed everyone to the meeting, especially Dymphna, for whom this was her first meeting. Apologies had been received from Clive, Jane, Mandy and Reive. Sonja reported that Mandy had resigned with immediate effect from the MAG.

    Sonja advised that she had asked Nancy Owens to attend for item 5.b because of her previous work as the Collaboration’s Quality Improvement Manager.

  2. Approval of agenda

    3. The agenda was approved with the addition of ‘Systems for merging/splitting of reviews’, and ‘Name change and remit of MAG’.

  3. Potential conflicts of interest of members present at this meeting

    4. Name Group represented Other Group membership roles Other potential conflicts of interest
    Sonja Henderson Review Group Co-ordinators (Convenor of ModMan Advisory Group). Review Group Co-ordinator, Pregnancy and Childbirth Group; member of RevMan Advisory Group; represents ModMan Advisory Group on the Information Management System Group (IMS); co-convenor of the Cochrane Style Guide Working Group. The Pregnancy and Childbirth Group is experiencing a severe funding cut, so the CRG is unstable, possibly having an impact on Sonja’s time available for MAG issues.
    Dymphna Hermans Trials Search Co-ordinators. Review Group Co-ordinator and Trials Search Co-ordinator of the Cochrane Dementia and Cognitive Improvement Group; member of the RevMan Advisory Group. Member of a European Union Group creating a Mental Health Library.
    Kate Hey Review Group Co-ordinators. Review Group Co-ordinator, Tobacco Addiction Group.  
    Pauline Howarth Publisher of The Cochrane Library.   Employed by John Wiley & Sons Limited, publishers of The Cochrane Library.
    Kate Jewitt Review Group Co-ordinators. Review Group Co-ordinator, Neuromuscular Diseases Group.  
    Monica Kjeldstrøm Information Management System team at the Nordic Cochrane Centre (non-voting member). Director of the Cochrane Collaboration IMS; employed at the Nordic Cochrane Centre; member of IMS Group; member of RevMan Advisory group; member of the Publishing Policy Group. Job dependent on in-house software development. Receives funding from the Cochrane Collaboration to co-ordinate the development of the Information Management System.
    Rasmus Moustgaard Technical Advisor, Programmer of RevMan and ModMan. Employed to work on software development at the Nordic Cochrane Centre; member of IMS Group; member of RevMan Advisory group; Convenor of Technical Implementation Advisory Group. Job dependent on in-house software development.
    Lois Sims Non-voting   Employed by Update Software. Update Software Ltd provides software solutions for electronic publishing and reporting and is the publisher of La Cochrane Library plus en español, the WHO Reproductive Health Library and La Biblioteca de Salud Reproductiva de la OMS. Update Software also have a contract with John Wiley & Sons Ltd to assemble all the data for publication in The Cochrane Library.

     

    Dymphna to inform Sonja of her potential conflict of interests, if any, to be included in the above table.
    Action: Dymphna, Sonja

  4. Approval of minutes and matters arising from the previous meeting

    5. Sonja explained that it is important that members of the MAG respond to the draft minutes as they are circulated widely as ‘provisional’. The minutes were approved and Jacob agreed to change their status on the web site.
    Action: Jacob

    Matters arising from the previous meeting:

    1. Item 4 - Role of the ModMan Advisory Group (MAG) in the development of the new Information Management System (IMS)

      2. Sonja advised that she had called for volunteers from TSCs, Co-ordinating Editors, and Criticism Editors. Dymphna is the new TSC representative. She had not received any volunteers from Co-ordinating or Criticism Editors.

      Monica explained that she had attended the last Quality Advisory Group (QAG) meeting in Barcelona and reported to the QAG that the MAG would be discussing workflows and subsequently making a recommendation to the IMSG.

    2. Item 5.a - ModMan Wish List

      3. Sonja reported that Mandy had been unable to establish who was responsible for producing and editing the Module Guidelines document, but it seemed that the contact for the original document had been Monica. MAG agreed that the decision about who is responsible for updating the document needs to be made.
      Action: Sonja

    3. Item 6 - RevMan Advisory Group

      4. Sonja explained that because of Mandy’s resignation from MAG, she will no longer join the small working group being convened by Mike Clarke to consider formatting and functionality in each section of RevMan. Sonja agreed to contact Mike to establish whether another member from MAG was required.
      Action: Sonja

  5. Update on the development of the new IMS

    6. Monica gave an update on the development of the new IMS, advising that the timelines for milestones were on schedule, and most of the document management module had been developed.

    Monica explained that documents can be checked in and out using RevMan 4.3, as it will have the ability to communicate with the central server. Once the document management module is complete, the IMS team will move on to work on access control over the next four months in parallel with developing the Contact Database further. Work will also be done on improving the quality of the contact details in the Cochrane Contact Database, to make the data as up-to-date as possible for when the Contact Database becomes mandatory for CRGs in July 2004. Monica had sent an e-mail to CRGs in this regard. Jacob confirmed that he would be working with RGCs to help them to move their data into the Contact Database and this will ensure that the data set will be as up-to-date as possible.

    Monica reported that the new system will be piloted by two or three Danish Groups and the IMS team will meet with them on a monthly basis in the time leading up to the pilot to show progress on the system and test it. Once the pilot is running (from September 2004) the IMS team will be working closely with the pilot groups to solve any potential problems as soon as they arise. Monica had also set up IMS training days for all RGCs and TSCs, primarily in connection with regional meetings, to demonstrate the new system and to enable participants to return to their editorial base and plan for the future.

    Concern was raised about the Contact IDs for reviewers because the new system will only allow one Contact ID per person, and currently a reviewer who is a member of several groups may have several IDs. In order to make the new ID unique, it will be based on a combination of the current Contact ID in RevMan and the CRG ID (which can be seen in ModMan). The IDs will be taken from the CRG with the most up-to-date contact record (based on date edited), so for some persons, the ID in RevMan may have to be modified to match the new one in the Contact Database. The ID is not essential for importing contact details into RevMan, though, because the import function also tries to match on name and email address.

    Concern was also raised about duplicate records within the Contact Database, but Jacob confirmed that if there are duplicate records (after automatic checking), the appropriate groups involved will be contacted, and asked to make a joint decision about which record to keep.

    Jacob agreed to send the list of available roles within the Contact Database to members of the MAG, and MAG members were asked to identify any missing roles. Monica explained that it was for the CRG to decide which role applied to which person and that multiple roles could be applied to one person. Sonja requested that the role ‘possible member’ be added to the list.
    Action: Jacob, everybody

    Peter demonstrated what has already been developed of the new IMS, and explained that when the system goes live it can be accessed from any Internet point around the world. The IMS was split into three sections, document management, contact management (the Contact Database), and workflow management. Peter confirmed that unlocking and locking reviews would be controlled by the editorial base. The main focus between January and April will be further development of the Contact Database so that person data will be available and integrated into the new IMS. Although there would be default access rights applied, MAG felt that it was important to stress to CRGs that they would be able to override those defaults. Monica agreed to ensure that the control of access rights would be explained in detail at the IMS training days for editorial base staff.
    Action: Monica

    1. Training and support for CRGs

      2. Monica explained that the proposal for IMS support staff had been circulated to RGCs TSCs, the IMSG, Co-ordinating Editors, the Quality Advisory Group and Centre Directors. The document circulated to the MAG included the more substantial feedback she had received about the proposal. A large number of people had expressed the opinion that it would be useful to have a support person in their own time zone that could provide support by telephone, as well as by e-mail. Concern was raised that the people eligible to apply for the positions were small in number. Rasmus also raised concerns about the support people not being able to devote enough time to the post if the post were split between three or four people. It would be the IMS Director’s responsibility to ensure that continuous support was available.
      Action: Monica

      It was agreed that policy and user guide instructions should be separated, with the policy being contained in The Cochrane Manual, and the user guide instructions elsewhere. Sonja suggested that Centre staff might be involved in the subgroup responsible for writing training materials for the IMS. Dymphna suggested that IMS training workshops could be added to the Centre training programmes.

      Monica agreed to contact Centres in different time zones to invite their training officers to help prepare any training materials needed. She also agreed to develop a system to monitor the time spent on support by the people appointed to the post. Kate Hey and Pauline continued to express reservations.

      Monica agreed to delete any reference to the amount of time an applicant might want to consider giving to this role to allow them flexibility. It was agreed that the proposal should be specific about the need for people in different time zones to provide telephone and e-mail support.
      Action: Monica

      It was agreed that the MAG would support the post as a pilot in the first instance with the following provisos.

      1. It should be made clear that this person/people would be required to provide telephone support.
      2. There should be no mention of the number of possible support people required to allow candidates to suggest the amount of time they would be able to give to this role, and allow them flexibility in the way they might want to suggest they manage it.
      3. One of the questions to be asked by the interview panel would be the effect this role would have on their existing job, and how they would be organizing their current workload to accommodate this new role.
      4. Nobody should be appointed to these roles unless the panel felt confident that the applicant(s) had the necessary skills and experience, i.e. if all the applicants interviewed were considered not to have the necessary skills and experience, then no appointment would be made.
      5. Appointments would be made on the clear understanding that the role(s) would be part of a pilot and would be evaluated after an agreed period of time, i.e. short-term contracts would be issued.
      6. Flexibility in discussions with applicants.
      7. How these posts should be administered and monitored should be clearly documented, e.g. timesheets. This would be the responsibility of the Director of the IMS.

      It was also agreed that if there were no applicants by the end of February 2004, Monica would reassess the strategy being used.
      Action: Monica

    2. Quality measures in the new IMS

      3. Monica circulated a document authored by Andy Oxman, as member of the Quality Advisory Group, about how to include quality assurance into the new IMS. The issues he raised were as follows:

      i. Prompts (reminders), for example when:

      • Information is missing from a protocol or review (A validation check is already implemented that disallows submission of a review when a whole section is missing. This might potentially be expanded to include other key information.)
        – Monica confirmed that if a specific section of a review is missing, it is not published. It was agreed to expand this to other sections of reviews.

      • Continuous outcomes are reported (to ensure appropriate reporting)
        – Monica agreed to ask Andy Oxman to clarify what is meant by this suggestion.

      • An update is due
        – Monica confirmed that this has already been planned.

      • A comment or criticism is submitted
        – Monica confirmed that this has already been planned.
        Action: Monica

      ii. Checklists:

      These can be mandatory or optional and could include, for example, whether:

      • A fixed or random effects model was pre-specified in a protocol
        – Monica explained that this is to check what is stated in the protocol.
      • There are inconsistencies between abstract, text and tables
        – Monica suggested that a checklist/workflow could be developed to assist the RGC and editors to check such inconsistencies.
      • Additional figures have been controlled for quality
        – A checklist for reminding CRGs what to check when accepting additional figures within a review is currently being developed. This could be included in the IMS to help CRGs to ensure the quality of their reviews.
      • Lack of evidence reported as evidence of no effect
        – Monica explained that this relates to evidence reported in misleading ways.

      It was agreed that checklists should be standardized and incorporated into the new system, but that the Collaboration needs to decide whether the checklists should be mandatory, or stated guidelines. Sonja pointed out that validation checks would be useful for reviewers.

      iii. Data collection:

      • It is likely that there are many errors in data collection and entry. Electronic collection of data extraction forms or spreadsheets could facilitate checks of accuracy by editorial bases. It is possible that reviewers would improve care with which they collect data simply because they know they need to submit data extraction forms
        – Dymphna explained that not all CRGs use data extraction forms, and it was a policy decision for each CRG about whether reviewers should submit data extraction forms. It was suggested that reviewers could use software that is already developed elsewhere rather than making this part of the IMS.
      • There already is an option for double data entry. The IMS could be used to find out if this is used and when it is used, the frequency of data entry errors.

      iv. Reference management:

      • Linking CRG trial registers and the IMS could reduce errors in references and help ensure the comprehensiveness of CENTRAL, CRG trial registers and searches for reviews. This might require the use of a standard reference management program by CRGs. Ideally, this should also include direct linkage with CENTRAL
        – MAG agreed that this would be useful, as there is a need to move towards standard reference software, ensuring standard inputting, and quality assurance. Dymphna agreed to survey the opinions of TSCs.
      • It could also provide a mechanism for supporting updates through automatically running searches and notifying reviewers when new references are entered.
        Action: Dymphna

      v. Online help:

      • The IMS could be used to improve access to help through expanded, context sensitive help, more user-friendly access to resources such as the Cochrane Reviewers’ Handbook, open learning material, material prepared by CRGs, and support from editorial bases, Centres, and methods groups.

      vi. Facilitating refereeing, scientific and copyediting, criticism management and updating:

      • The IMS can be used to make it easier for CRGs to manage all of these processes and, potentially to improve them
        – Monica explained that the IMS team was planning to incorporate this into the new IMS.

      vii. Summary findings (balance sheets):

      • A proposal to include a summary of findings will be presented to the RevMan Advisory Group on 15 January.

      viii. Randomised trials of quality interventions when there is important uncertainty:

      • The IMS can be used to conduct trials of interventions such as using prompts, mandatory checklists or double data entry
        – MAG supported this.

      Pauline reported that Wiley would like to be involved in any work on quality assurance and whether data in specific fields have valid data. Monica agreed to work with Simon Newton on this process. She asked members that any other suggestions be e-mailed to her over the next two weeks, and she would share those suggestions with the QAG.
      Action: Monica, Everyone

  6. Standardised workflows.

    7. Rasmus explained what a workflow entails and how it could be used to track the progress of the editorial process. It also means that you always know the next step in your system. The flow can be linked to documents, people or other resources. Each workflow consists of a number of tasks and sub-tasks that together support a particular part of the editorial process, e.g., approving a review title. The IMS team will initially focus on the development of standardized workflows, but CRGs should also be able to customize their own workflows. MAG’s role is to help identify and design the standard workflows. Rasmus explained that the IMS team would start working on the workflow module from May 2004, but it is anticipated that a prototype can be demonstrated at the IMS training days, the first of which takes place in March 2004.

    The group discussed the workflow examples presented by Rasmus.

    1. Approve title:

      2. Rasmus explained that in the new system, the ‘approve title workflow’ starts at the point at which a reviewer proposes a title. At the ‘internal duplicate check’ the RGC would check to see if the title already exists. If it does, the workflow will end. If not, when it gets to the point of ‘external duplicate check’, the RGC would check to identify if there is overlap with another group. This step is equivalent to the current step whereby CRGs register titles on the titles web site. If the title is not approved (e.g. if it is already registered in another group), it was agreed that it would be useful for CRGs to be able to keep a record of why it was not approved. The final approval status is where the RGC can assign the contact editor for example, and then the workflow stops.

      After discussion, Rasmus agreed to change ‘internal duplicate check’ to ‘internal checks’, ‘external duplicate check’ to ‘external checks’, ‘duplicate found’ to ‘failed’ or ‘rejected’, and to remove the ‘discussion and comments’ box, since this is integral to both the internal and external check. Monica reported that the system would alert the editorial base if the reviewer had already registered numerous titles.
      Action: Rasmus

      Nancy agreed to share with Monica and Rasmus the data she had collected on title workflows in her work as Quality Improvement Manager. Monica agreed to contact the QAG to let them know about the work of MAG in helping to identify editorial workflows and to establish whom MAG should be liaising with in relation to this work. Nancy informed MAG that she is a member of QAG and would therefore be willing to act as the liaison person until such time as the QAG had agreed the liaison person. Monica and Rasmus agreed to revise the workflows in response to the agreed changes and circulate them to the MAG for comment.
      Action: Nancy, Monica, Rasmus

    2. Other phases

      3. Rasmus explained that the term ‘Write draft’ applies to the reviewer, and the system will allow automatic time reminders to be built in between this stage and the formal submission of the draft to the editorial base. He agreed to add a box about the search, remove ‘copy edit’ after ‘validate’ since this is part of the validation process, add ‘approve’ after peer review, change ‘peer review’ to ‘external peer review’, include ‘approve protocol/approve review’ and change ‘publish’ to ‘submit for publication’.
      Action: Rasmus

      Rasmus agreed to develop three new workflows: for protocols, reviews, and updates. Monica and Rasmus agreed to draft the three new workflows adding in more description, liaise with Nancy, and circulate to the MAG in the first instance.
      Action: Rasmus, Monica, Nancy

    3. Cochrane terminology

      4. It was agreed that there is a potential problem with the term ‘reviewer’ in the Collaboration, for instance because the UK interpretation of this word is different to the USA interpretation. The introduction of the new IMS offers a good opportunity to reassess Cochrane terminology. Sonja agreed to recommend to the IMSG that discussion about the terms used in the new IMS, in particular, the word ‘reviewer’ should be re-opened within the Collaboration to try to use terms that will ensure a universal understanding, if possible.
      Action: Sonja

  7. Wish list (originally ModMan Wish list).

    1. SECTION C (Discussed by the MAG/IMSG but require further investigation)

      2. ID Title Description Priority by MAG/IMSG Notes
      1 ModMan should not be connected to RevMan ModMan should not be connected to RevMan. Most RGCs preferred approved protocols/reviews to be stored separately in ModMan for publication. Approved For the new IMS.
      22 Internet links from modules To be able to place Internet links in the module text that will be active when published. Approved This is a non-technical issue, for which guidelines are needed.
      29 Editorials The possibility to include editorials with reviews. Not assigned yet Monica has put this on the agenda of the next meeting of the PPG (12 February 2004)    
      34 Date updated used in tracking report The ‘date updated’ is the last substantive update. I would find it more helpful if the program could pick the latest recorded date from any of the following three fields: last substantive update; studies sought but not found; studies found and included/excluded. This would be far more useful as an internal housekeeping tool, and would enhance ModMan as a tracking function. Approved The new IMS should generally be able to produce reports on any recorded date
      35 A system for merging /splitting reviews We should have a system for merging or splitting reviews that will make it possible to flag this in The Cochrane Library, and to link between the old versions and the merged/split reviews. Approved – high priority This is being discussed by the IMS team and John Wiley & Sons in relation to DOIs.

    2. SECTION E (New items)

      3. ID Title Description Priority by MAG/IMSG Notes
      37 Alert system to warn about a reviewer who registers many titles across several CRGs A facility to search by reviewer name and ‘alert’ if a single reviewer has registered lots of titles with different CRGs and are trying to register a new title with another CRG. Approved This will be part of the "approve title" wizard in the new IMS.
      38 Post copy edit RGCs should have the capability to place revision under ‘track changes only’ mode, and tracked changes should be in neon highlights. Not yet assigned Sonja to get clarification of this wish.
      39 Expand sort options in contact database It would be good to be able to sort people/ entities by country. Please add more sort options to the contact database. Approved This should be implemented before the CD becomes mandatory.
      40 Post publication warning Reviewers trying to gain access should receive a warning, i.e. ‘contact RGC’. Approved The new IMS will lock the version marked for publication, and work can continue on a new version.
      41 CRG specific glossary There is a need for CRG specific consumer glossaries that is accessible from each published review through hyperlinks. Approved Monica has tried to find out from Deborah Dixon if this is possible. Pauline agreed to ask Deborah again. No technical implementation problems.
      42 Double line spacing The ability to generate a printed version of a review with double line spacing for editors. Approved  
      43 Shared pool of referees

      a) Sharing referees between groups.
      b) That the workflow system should offer refereeing jobs to a pool of referees, allowing one of them to volunteer

      		 a) Rejected
      b) Approved but optional      		 (The rejected item 43a has been moved to section B on the IMS wishlist.)
      44 Wysiwyg or ‘preview in browser’ like function Could reviewers and RGCs have the opportunity to see the review as it would be when published on CDSR and/or have proofs to correct? Approved  
      45 Reviews shared between review groups There needs to be a way to ‘recognise’ the involvement of more than one review group with individual reviews. Approved It will be possible to link to other CRGs’ reviews from the topics list. A tick box could be provided in the topics list to indicate whether the other group contributed to the review. Text can also be added to the Acknowledgements section.
      46 Should we have an errata section Would this be useful if it were possible to have a date when review edited for purpose of sorting out errors? We could then avoid using the ‘date’ functions assigned to other aspects of the review, eg updating the review search etc. Not yet assigned An errata section is needed in The Cochrane Library, because it is the only way to trigger a new MEDLINE record for reviews for which it is important to correct a serious error. John Wiley and Sons has agreed to investigate the best way forward.
      47 Export of topic list at different levels Although it is possible to export an .html version of the topics list, would like to be able to export an .html version of the topics list at different levels. Approved Perhaps include links to DOIs for reviews?

  8. Additional figures

    9. Rasmus explained that there are already guidelines for reviewers about preparing images. An updated checklist, with both technical and copy editing checks, had been prepared and circulated by Simon Newton of John Wiley reflecting the problems that Wiley have had with images not being of high enough quality or format. Pauline advised that there had been 27 images submitted for Issue 4, 2003, of which six were problematic. MAG agreed that the new checklist would be useful as a validation check in RevMan. There were concerns that some RGCs might not have the knowledge to resolve the more technical checks. It was agreed that Rasmus should remind RGCs about the pilot he was running whereby they could send additional figures to him for checking. It was also agreed that if the new checklist is approved, that RGCs should be asked to send this to Rasmus to indicate what they have checked and to identify what the RGC could not check.
    Action: Rasmus

  9. Dates to be stored in Cochrane reviews

    10. There was not enough time to discuss this item. Sonja agreed to report back to the MAG on the outcome of such discussions at the IMSG meeting later in the week. It was agreed that when a review is published for the first time, the date of last substantive update should be the date on which it changed from a protocol to a review. The MAG also agreed that the ‘date of last search’ is important.
    Action: Sonja

  10. Any other business

    1. Name change and remit of MAG

      2. Sonja suggested that the MAG name may be a barrier to the recruitment of additional members and could also be a barrier to change. She suggested that a new name be considered although the remit of the new group that will take over from MAG should be drawn up first. The following names were suggested:
      Editorial Workflow Advisory Group
      Workflow Advisory Group
      Editorial Systems Advisory Group

      Members felt it important to have the word ‘Editorial’ within the name. Rasmus felt it was important to include the word ‘system’. Sonja agreed to raise this issue at the IMSG meeting, and discuss further with MAG members by e-mail.
      Action: Sonja

    2. System for merging/splitting reviews

      3. There was not enough time to discuss this item in detail at the MAG meeting, but it was agreed that there should be a system in place to deal with merging and splitting reviews. Sonja agreed to raise this issue with the IMSG.
      Action: Sonja

  11. Date of next meeting

    12. Sonja agreed to advise MAG members of the date of the next meeting, which is likely to be during the Ottawa Colloquium.
    Action: Sonja

Document Actions