Manual¶

Workflows¶

Workflows are designed by defining the states and the allowed transitions. Using these states and transitions a graph is build. The study governor software will keep track of the states each experiment is in and has passed. It will also enforce valid transistions. At each state/node a Callback can be executed. These callbacks can be used to start custom script/commands, run pipelines, or send emails.

Workflows are defined using a json file, which has two main fields “states” and “transistions”. There’s two additinal options “external_systems” and “scantypes” which will be discussed later on.

States¶

States are defined in the states field of the workflow json. Each workflow needs an “untracked” state. This is the state each of the experiments start in. This state is limited to only one ouput transition and is tranistioned by the study governor. This state is also the most basic state we can define. Each state needs at least the following fields:

fields	Description
“label”	The name of this state, must be unique.
“lifespan”	Triggers something
“callback”	The Callback function.
“freetext”	A description of this state.

Example of a state:

{
    "label": "untracked",
    "lifespan": 0,
    "callback": {
        "function": ""
    },
    "freetext": null
}

Transistions¶

Besides states we need to define the possible tranistions between those states. We define a source a destination and a contition:

fields	Description
“source”	Label of the source state.
“destination”	Label of the destination state.
“condtion”	Extra condition for the transition.

Example of a transition:

{
    "source": "untracked",
    "destination": "Manual_QA_task",
    "condition": null
}

Permissions¶

The permissions are defined as follows:

Permission	Description
`sample_state_update`	Allows updating a sample. (super-user)
`sample_add`	Allows adding a sample. (super-user)
`sample_update`	Allows updating a sample. (admin)
`sample_delete`	Allows deleting a sample. (admin)
`roles_manage`	Allows adding and removing roles from users
`user_read`	Allows seeing your user information.
`user_read_all`	Allows seeing all users.
`user_add`	Allows adding users.
`user_update_all`	Allows updating all users.
`user_delete`	Allows deleting users.

Database Models¶

Database model for the Study Governor tables

Database model of the Study Governor tables.¶

Database model for the User and Role tables

Database model of the User and Role tables.¶

Callbacks¶

studygovernor.callbacks.external_program.external_program(experiment, action_url, binary, args=None, kwargs=None, xnat_external_system_name='XNAT')¶

Calls an external command. The binary gets the command in the form:

binary $ARGS $KWARGS

Parameters

experiment (str) – experiment uri
action_url (str) – action url
binary (str) – binary that gets executed
args (Optional[Sequence[str]]) – list of args [val1 val2 …]
kwargs (Optional[Mapping[str, str]]) – list of [key1 val1 key2 val2 …]
xnat_external_system_name (str) – name of the external xnat [XNAT]

The items in args and values in kwargs that contain certain VARS will be replaced. Accepted VARS:

$EXPERIMENT - will be substituted with the experiment URL.
$XNAT - will be substituted with the xnat URL.

Example:

{
  "function": "external_program",
  "binary": "check.py",
  "args": ["$EXPERIMENT"],
  "kwargs": {
    "-x": "$XNAT"
  }
}

studygovernor.callbacks.create_task.create_task(experiment_url, action_url, task_base, task_info, progress_state=None, done_state=None, xnat_external_system='XNAT', taskmanager_external_system_name='TASKMANAGER')¶

Create taskmanager task

Parameters

experiment_url (str) – experiment url
action_url (str) – action url
task_base – task_base is a Template that contains info for the task
task_info – Additional info for the task as a list of [key1 val1 key2 val2 …]
progress_state – State while queued
done_state – State when done
xnat_external_system_name – name of the external xnat [XNAT]
taskmanager_external_system_name (str) – Taskmanager external ID

Example:

{
  "function": "create_task",
  "task_base": "manual_qa.json",
  "task_info": {
      "project": "sandbox",
      "application_name": "ViewR",
      "application_version": "5.1.4",
      "template": "manual_qa",
      "tags": ["QA", "Quality Assurance"],
      "distribute_in_group": "quality_assurance"
  },
  "done_state": "/api/v1/states/Automated_processing",
  "progress_state": "/api/v1/states/Queued_for_manual_qa"
}

studygovernor.callbacks.fastr.fastr(experiment_uri, action_uri, network_id, source_mapping, sink_mapping, log_dir, process_state, done_state, failed_state, xnat_external_system_name='XNAT')¶

Execute Fastr pipeline

Parameters

experiment_uri (str) – experiment uri
action_uri (str) – action uri
network_id (str) – network that gets executed
source_mapping (Mapping[str, str]) – Mapping[str, str],
sink_mapping (Mapping[str, str]) – Mapping[str, str],
log_dir (str) – str,
process_state (str) – str,
done_state (str) – str,
failed_state (str) – str,
xnat_external_system_name (str) – name of the external xnat [XNAT]

Example:

{
  "function": "fastr",
  "network_id": "preprocessing",
  "source_mapping": {
    "t1": "/scans/T1W*/resources/DICOM",
    "flair": "/scans/*FLAIR*/resources/DICOM",
  },
  "sink_mapping": {
    "t1_nii": "/scans/T1W*/resources/NIFTI/files/image{ext}",
    "flair_nii": "/scans/*FLAIR*/resources/NIFTI/files/image{ext}",
    "flair_coregistered": "/scans/T1W*/resources/NIFTI/files/flair_to_t1{ext}",
  },
  "log_dir": "/home/logs/",
  "process_state": "preprocessing",
  "done_state": "preprocessing_finished",
  "failed_state": "preprocessing_failed"
}

studygovernor.callbacks.ifdb.ifdb(experiment_url, action_url, fields_uri, templates, done_state, failed_state, ifdb_external_system_name='IFDB', xnat_external_system_name='XNAT', taskmanager_external_system_name='TASKMANAGER')¶

Add experiment to IFDB

Parameters

experiment_url (str) – str,
action_url (str) – str,
fields_uri (Union[str, List[str]]) – Union[str, List[str]],
templates (Union[str, List[str]]) – Union[str, List[str]],
done_state (str) – str,
failed_state (str) – str,
ifdb_external_system_name (str) – str=’IFDB’,
xnat_external_system_name (str) – str=’XNAT’,
taskmanager_external_system_name (str) – str=’TASKMANAGER’

Example:

{
   "function": "ifdb",
   "fields_uri": [
     "resources/FIELDS/files/mask_{timestamp}.json",
     "resources/FIELDS/files/QA_{timestamp}.json"
   ],
   "templates": [
     "mask",
     "manual_qa"
   ],
   "done_state": "/data/states/done",
   "failed_state": "/data/states/write_inspect_data_failed"
}

studygovernor.callbacks.send_mail.send_mail(experiment_url, action_url, subject, body)¶

Send an email with SUBJECT and BODY.

Parameters

experiment_url (str) – str,
action_url (str) – str,
subject (str) – str,
body (str) – str):

Substitution fields for SUBJECT and BODY are:

{experiment}: experiment id

{experiment_url}: full url for an experiment

{action_url}: full url for an action.