Monday, 24 January 2011

Data Storage Costs

The ADMIRAL project is creating a local data management facility and testing its use with researchers in the Zoology Department at Oxford University. The facility is built using a standard Linux operating system running in a VMWare hosting environment installed within the department.  The environment has a limited amount of local fast SCSI disk storage, but is also capable of using network storage via the iSCSI protocol.

One of the research groups for whom we deployed an ADMIRAL system started using it so enthusiastically that they rapidly filled up their allocated space, at which point they then stopped using the system without telling us.

With opportune timing, the Zoology Department has recently installed an iSCSI network storage array facility, to which disk storage capacity can be added on an as-needed basis. This facility is to be paid for on a cost recovery basis by projects that use it. We have performed tests to prove that we can use this facility within the ADMIRAL data management framework, and are now waiting for the Silk Group and IT Support to order and install the additional storage capacity so we can deploy an enlarged ADMIRAL server to meet the Silk Group's storage requirements.

We have also been in discussion with other research groups not currently involved with ADMIRAL about storage provision to meet funding body requirements for data sharing. Recently released BBSRC data sharing requirements (http://www.bbsrc.ac.uk/web/FILES/Policies/data-sharing-policy.pdf) stipulate that research data should be made available for sharing for at least 10 years beyond the life of a research project (i.e. a total of 12-14 years for a typical BSRC-funded project), and claims to allow the cost of this to be added to a research grant proposal to cover the costs thus incurred. See section "BBSRC Data Sharing Policy Statement " in their data sharing policy document. This does not require that data be kept online for this period, but considering cost, attrition rate and time to obsolescence of alternative solutions, some kind of online facility would appear to be as cost effective as any.

The cost of providing the departmental network storage facility has been estimated at about £400 per Terabyte over a 5 year period.  This is estimated on a hardware cost recovery basis, allowing for a normal rate of disk failures over the 5 year period, but not including departmental IT support personnel costs.  In discussions with our IT support team, we estimate that the project+10 year duration of required availability will approximately double the hardware-only cost of delivering on such a requirement. Based on these discussions, my current recommendation to Zoology department researchers bidding to meet these requirements would be to cost data preservation and sharing to meet BSRC requirements at £1000/Tb, assuming that departmental IT support remains committed to operational management of the storage server. I would also recommend that they allow 1 person day for each month of the project at appropriate FEC cost to cover data management support, especially if the research team itself does not have IT server systems expertise: I estimate this would reasonably cover ongoing support of an ADMIRAL-like system for a project.

For comparison, Oxford University Computing Service offers a 5 year data archive facility with multiple offsite tape backups for a Full Economic Cost of about £4000/Terabyte. They do not offer an archive service of longer duration. (It is important to distinguish here between a backup service and an archive service: the same OUCS facility provides a daily backup service, but said backups are destroyed after just a few months of no update or access.)

The university library service has tentatively indicated a slightly higher cost per Terabyte for perpetual storage.  The meaning of "perpetual" here is open to debate, but the intent is to maintain copies of the data in storage for at least several decades, much as historical books and papers are help by the Bodleian Library for the long term.

10 comments:

  1. Excellent pieces. Keep posting such kind of information on your blog. I really impressed by your blog.
    Vee Eee Technologies

    ReplyDelete
  2. Wow! That’s very expensive! Well, though the cost are really high, I guess it’ll be all worth it since the data stored and managed by an administrator will be in safe hands. The services of data storage providers can be of great help in storing a lot of files that needs to be kept for future usage.

    Manda Maldanado

    ReplyDelete
  3. Investing in data storage facilities can be expensive, but it will be your guarantee that your important data will be in safe hands. With the amount you stated for the period of five years, I can say that it's already very economical. This investment will not only benefit you for the period that you paid for, but for the whole lifetime of the company, since the important documents of your business is in the safe hands of your data administrator.

    WilliamsDataManagement.com

    ReplyDelete
  4. Money is no object in keeping your data are kept secured 24/7. In case you're on a tight budget, you can get cheaper deals without risking your identity. Make sure that your investment won't be put to waste and your documents in the wrong hands. Look for trusted IT service partners and cloud storage facilitators.

    WayPointSG.com

    ReplyDelete
  5. Articles and content in this section of the website are really amazing. Great ideas indeed! I will surely keep these in my mind!

    Agenzia di marchi sportivi & Cos’è il licensing di una marca

    ReplyDelete
  6. Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon.

    Licence de marque & Extension de marque

    ReplyDelete
  7. Articles and content in this section of the website are really amazing. Great ideas indeed! I will surely keep these in my mind!

    Lizenzen & Was sind Lizenzen

    ReplyDelete
  8. I thought it was going to be some boring old post, but it really compensated for my time. I will post a link to this page on my blog. I am sure my visitors will find that very useful.


    Licence de marque (e) & Extension de marque

    ReplyDelete
  9. Please keep sharing more and more information about this.This is a good information of the fashion licensing articles and really like your site.
    Licence de marque (e) & Extension de marque

    ReplyDelete