First review of data usage surveys

An initial meeting has been held to review initial returns from the data usage survey from the Silk and Development research groups.

Meeting notes are at
http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_20100219_meeting_Data_Surveys_Review.

The main outcome is that our current development priorities appear to be about what the researchers would like to see us provide.

Sprint 4 plan

The ADMIRAL sprint 4 plan has been posted at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_SprintPlan_4.

Notes from the planning meeting are at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_PlanMeeting_4.

Notes from sprint 3 retrospective meeting are at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_SprintReview_3. The summary position for this sprint is:

The general feeling is that forward progress is being made, but slower than hoped due mainly to unanticipated technical problems with integration of SSO authentication with file sharing services. We plan to push ahead with these issues only partially resolved in the hope that solutions will be found later. Meanwhile, some features will not benefit fully from SSO integration.

ADMIRAL public code repository established

We have finally created a public code repository for the ADMIRAL project. It is at http://code.google.com/p/admiral-jiscmrd/.

We had put off establishing a code repository for ADMIRAL until we had a clearer view of the requirements:

• Should it be public, or private?  If we are using the repository for user-related information, then a public repository is not appropriate.  In any case, changes to Shuffl performed as part of the ADMIRAL project will be maintained within the Shuffl project (http://code.google.com/p/shuffl/).
• What kind of version management is required? We had in mind to use Mercurial, for reasons mentioned elsewhere (http://imageweb.zoo.ox.ac.uk/wiki/index.php/Mercurial_repository_publication), but did not want to commit in case any specific reasons to use a different versioning system were discovered.

Over the past couple of weeks, we have been investing some effort into configuring Samba, Apache and WebDAV to work with Kerberos authentication (http://imageweb.zoo.ox.ac.uk/wiki/index.php/Zakynthos_Configuration).  It seems that what we are doing with Kerberos is (a) pushing at the boundaries of what is commonly deployed and documented, and (b) something that a number of people have asked about doing, so we felt it was time to put some of the things we are learning into public view.

We've chosen a Google Code project ("admiral-jiscmrd", as "admiral" was already taken) and have decided to go with the original plan of using Mercurial version management.

The initial content of the public code repository is a set of scripts and configuration files we are developing to automate the assembly of a virtual machine image for file sharing and web access with access control linked to a Kerberos SSO authentication infrastructure (in our case, Oxford University's).

Research data management support issues

We had a useful meeting today with our departmental IT support group to discuss ADMIRAL and its ongoing support beyond the life of the ADMIRAL project.  I believe this meeting brought into focus some policy management issues that will apply across a range of systems that attempt to support data management for individuals and small research groups that don't have resources to do their own IT support.  Notes from the meeting are at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_20100204_meeting_-_IT_Support.

The issue raised is that the IT support team are (understandably) hesitant to take on support for any particular system - even when these are just assemblies of off-the-shelf components, such as the base ADMIRAL data sharing platform.  Yet, if we are to succeed in providing data management services to researchers, some framework for ongoing support must be worked out. The range of options is additionally constrained by the fact that commercial cloud-hosted services bring problems of responsibility for and jurisdiction over the data. In Oxford, the legal team have expressed concern about having University-owned content hosted on systems that may be anywhere in the world.

In our department, IT support do provide some help and support for staff desktop and personal machines, though not (generally) for particular applications running on them.

Assuming we have effective systems for research data management, that researchers are actually using, then what options do we have, and what resourcing is needed, to ensure that such systems are adequately supported.  One key element of support will be application of security fixes.

These considerations give rise to some additional requirements:

• The basic research data management system should be simple, generic and built from standard software elements which are individually supported.
• The basic system, incorporating essential data storage and backup features, should be generic and usable by a sufficient number of researchers that a standard base configuration can be deployed and supported for diverse research groups.
• Essential security fixes should generally be automatically applied, with a minimum of user intervention.  This means, for example, that product version that are part of a standard operating system distribution should be used in preference to newer versions.  (Occasional manual restarts will inevitably be required.)
• The integration of basic system components should, as far as possible, be a loose coupling by lightweight and standard protocols, so that that system as a whole is not unduly dependent on a particular version of any software component.
• Any specially-written software should be incorporated in such a way that its failure does not jeopardize essential data security or accessibility.  The outputs of any such software (e.g. annotation tools) should be simple, standard formats that can easily be recovered using widely available off-the-shelf applications.

I currently perceive three possible (and not mutually exclusive) modes of support:

1. Universities and departments recognize the value to their research goals of taking on the burden of supporting some additional systems (e.g. as Southampton University have done).
2. If a basic system can serve the needs of researchers across several institutions, then maybe an open-source style of support model can be used, with users as a community, backed up by a handful of technical staff, can provide mutual support.
3. If a sufficiently large community use the system, maybe there is an opportunity for a small business to provide support on a commercial basis, e.g. funded by a fixed fee paid from research grants where data preservation and publication is a key requirement.

These are just some initial thoughts - I think there is plenty of scope for further discussion, to involve researchers, developers, funding agencies and policy makers.

Sprint 3 plan

The ADMIRAL sprint 3 plan has been posted at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_SprintPlan_3.

Notes from the planning meeting are at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_PlanMeeting_3.

Notes from sprint 2 retrospective meeting are at http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_SprintReview_2.

Now that we have a larger team, the style of reporting is being changed to reflect that much of the material is initially being drawn up in face-to-face meetings on index cards: the meeting notes summarize the discussions, and the plan focuses more on the selected schedule of activities and tasks.