Wednesday, 16 December 2009

Datasets to the UK

The UK data placement group is beginning to receive requests for datasets to be copied to the Tier 2s UK, sometimes to specific sites. It is perhaps timely to remind people of the computing model constraints on what we do and the ATLAS rules for transfers.

The Tier 2 disk is for Grid access only, and is in principle for all of ATLAS. We consulted the ATLAS community in the UK via the institute physics contacts about specific interests, and we managed to have all of those requests met by having a physics and/or performance group in the UK cloud. ATLAS now believes we have enough groups assigned in the UK. It should not be your expectation that everything you want to access will be in the UK (and certainly not at a geographically local site, if you have one), and there are multiple copies of group areas worldwide. That said, we try to have as much as we can in the UK.

There is also some space for longer-term (but not indefinite) storage of UK user data in a special space token. Again, this would not normally be at a ‘local’ site – this space is for all ATLAS UK users, and so could and should be at any of our sites, which are required to meet minimum standards for data access. This space will be managed by the UK, and unreasonable usage will not be welcomed, just as with maxi disks in the past. ‘The management’ reserve the right to remove data from this space. This space has to hold user-derived datasets, so open-ended subscriptions to a stream or data type cannot go here.

All access to the Tier 2 and collocated UK Grid space must be by the Grid mechanisms. If you have files that you want to access locally, these are by ATLAS policy copied into the scratch area, and you should copy them to some (Tier 3) storage private to your institute/HEPGroup, or for a while and only in modest volume to the UK shared space.

In general people requesting transfers to the UK should be telling us
  1. What need for the data is (activity, which physics and performance group)
  2. Why it is not already covered by a physics or performance group area
  3. How big it will be *at a maximum*
  4. How the data will be used (what sort of job to be run, database access etc)