Data deletion — Rucio 0.3.8 documentation

# Data deletion

Replica data model:

- rse_id: The rse identifier
- scope: The scope name of the file
- name: Filename
- state: Replica state (AVAILABLE, UNAVAILABLE, COPYING, BEING_DELETED, BAD)
- lock_cnt: Counter of the nr of locks
- created_at: Creation date of the replica1
- accessed_at: Last access time of the replica (null for the moment)
- tombstone: Date of the last lock removal (null otherwise) (Maintained in the rule core part.)

## Reaper daemon

Parameters:

- MinFreeSpace: Minimun free space(in bytes) which should be available at a RSE.
- MaxBeingDeletedFiles: Maximum number of files beeing deleted for a RSE.

For each RSE, The reaper checks the available free space. If the available free space is less than a limit defined per RSE(MinFreeSpace), a cleanup cycle is triggered. The reaper selects (MinFreeSpace - ActualFreeSpace) bytes of files and then deletes them. Replicas of any file which is declared obsolete are immediately deleted. Those files are marked with a tombstone set to the epoch value (January 1, 1970).

Pseudo-code:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    For each RSE in RSEs:
        Check rse free space # Explanation below
        replicas = list_unlocked_and_obsolete_replicas (MinFreeSpace - ActualFreeSpace)
        # Gives n replicas corresponding to the amount of bytes to free up or all obsolete ones
        # The list is ordered by created_at and tombstone.
        for each replica in replicas:
            status = Mark the replica state as BEING_DELETED
                # Update conditions: where lock_cnt=0 and tombstone is not null
                # Rules should not select replicas in the state BEING_DELETED
            if status:
                    status = Delete file from RSE
                    if status:
                Remove file from replica table
    freed_space += filesize