Acceleration annotation
Annotation of acceleration data is increasingly important for a wide range of projects. Firetail provides you with a set of tools for manual and machine learning assisted (FireSOM) annotation. You can rapidly annotate
- Movebank individuals
- Movebank tags
- Movebank deployments
Core concepts
Firetail features layers
as highest level of organisation. A layer
can hold arbitrary many categories
.
Each category
resides on one specific layer.
A burst
(see also Burst vs. continuous recording) can be assigned to
one or more categories.
Add new categories
To add a new category press Add Category
above the acceleration window
Assign a name to the category
The category will show up above the acceleration window
Save and Load Annotation Categories
For a proper annotation workflow a common ground set of categories is crucial. With Firetail you can save the current set of categories and layers using File > Annotations > Export Annotation Categories
.
To add the categories to your project, use File > Annotations > Import Annotation Categories
.
Assign a category to a region
Once you have defined categories, you can use the selection mechanism described here to define regions that you’d like to annotate with one of your categories.
- Select a region within the bursts or the map
- Press on the category you want to assign this region to
The newly assigned category will be shown in a row/layer above the acceleration data.
Each category can be assigned to a single layer (visual group shown as a single row of data).
Note that overlapping categories shown in a single layer are possible, yet may not be visually sensible.
Right-clicking a category will yield a popup-menu for:
- category modification
- annotation deletion
Saving Annotations and Removing a category
You can remove existing categories via right-clicking on the respective category and select
delete category
. You are prompted that all annotations contained will be lost. There is currently
no undo for this, so make sure to use
File > Annotations > Save Annotations
to save the current state.
Expert hint: the annotation resides in the respective
$USER/movebank_downloads/study_<num>
folder asannotation2.csv
(to ensure compatiblity with AccelerationViewer, this will likely change in future versions). Make sure to include this folder in you regular backup procedure
FireSOM: Machine-learning assisted acceleration segmentation
IMPORTANT: Firetail’s annotation features are activated when loading data via
Download Movebank by ID/Tag/Deployment
. Your annotation is saved locally (on your machine) along with the downloaded data.
When working with acceleration data manual annotation of large-scale data may be time-consuming or near-impossible.
Via Data > Acc: FireSOM Segmentation
you can trigger Firetail’s automated segmentation algorithm FireSOM.
Below, we provide a rough overview of the FireSOM procedure:
Overview of FireSOM
- calculate a set of user-selected features for each burst
- normalise (standardization) extracted features
- train the FireSOM (self-organizing map) using the bursts mapped into feature space
- assign each burst to a category
The model dimension determines the number of predicted categories. For a 4x5 sized map you’ll retrieve 20 categories.
Video Tutorial
Here you can find a video tutorial on FireSOM, see below for more details
Parameters
Model Constraints
Width: the number of horizontal nodes, we suggest to use at least 2 depending on the complexity of data patterns
Height: the number of vertical nodes, we suggest to use at least 3 depending on the complexity of data patterns
Annotation Prefix: Each predicted category will be prefixed with this string
Clear existing Annotations with the same Prefix: Any previous annotation with this prefix will be removed. Consider renaming categories to avoid data loss
Feature Selection
Use Axis Statistics: add mean, median, and quantile statistics to the set of features, each axis Use ODBA: add overall dynamic body acceleration as a feature, all axes Use axis fold-changes: add statistics over log-2 fold-changes for selected pairs of axes mean-corrected norm: add the mean-corrected norm value for each burst continuity: add a measure of sum of changes
Click OK
to train a model and see your results as new layers in the acceleration window.
Inspecting the model
While it is easy to train some model in Firetail you should not expect that your initial model will explain your data perfectly. Inspecting a predicted model is crucial. Luckily, Firetail provides you with a lot of context that makes it easier to interpret the predictions.
Overall, the model is based on the similarity of two bursts in feature space, so similar patterns should be assigned to identical or nearby categories. Neighborhood is defined in a tabular row-column sense, i.e node (1,4) is a direct neighbor of (1,3), (1,5), (0,4) and so on.
Think of patterns in terms of activities like rest, running or feeding. The more categories you predict the more behavioural complexity can be detected. For increasingly many categories it may become necessary to join predictions via layers to group similar categories.
Start simple
Therefore, start by training a simple model with dimension 3 by 3 and few features. You can easily train a quick model as a preview or even experiment with how well a single feature works for discrimination of visual patterns or regions of known behaviour.
Increase complexity if required
An increasing number of features means there is more freedom to discriminate bursts. The result model will be harder to interpret though. Incrementally increase model complexity and add features when two regions you deem dissimilar are placed in the same category.
Refine using local models
If the data features local specificities it makes sense to run the segmentation on a selection.
Make sure that at least a few hundred bursts are selected to avoid a poorly fitted model.
In this context, the option Restrict to Selection must be enabled. The algorithm will predict categories only for selected bursts. Choose a distinct model prefix to make sure to maintain both global and local predictions.
Refine specific categories
Right clicking on an annotation layer allows you to select annotatated regions by choosing “Annotations to Selections”.
This leads to a powerful workflow:
- select regions that could not be segmented properly by the initial, coarse model
- make sure that the regions to be refined are selected
- re-run FireSOM using
Restrict to Selection
and a new model prefix - evaluate the local model via merge/delete semantics as discussed below
Assign, Rename and group
The machine-assigned group names cannot magically predict “running”, “feeding” or “flying”. Screen the predicted categories visually and in their best possible context to assign a category to a possibly preliminary behaviour. Repeated
- application of the segmentation
- changes in parameters and
- successive joining of categories into layers
- refinement of coarse categories
- deletion of superfluous categories
will lead to sensible annotations for a complete dataset.
Saving a model
You can save a trained model for later use on another dataset or selection. This provides a very powerful way to avoid the inclusion artifacts or to build more specific models. The core idea is to sort bursts into buckets defined in another context. Therefore, this strategy should work best on similar input data, although there is no technical constraint keeping the user from applying models cross-species or across tag-types.
A common use case is to save global and refined models for one dataset (see refinement) and then re-apply the models on another dataset.
An appropriate setup could also help to enforce consistency checks and enable in-depth validation.
The key steps are:
- check
Save Model after Training
- set the required parameters and features
- press
OK
to train the model - select a file to save the training state
A model is a json file that includes required map-weights, parameter settings and the selection of features. The application of this model on different data requires the same parameters and features.
Using a previously trained model
To apply a pre-trained model:
- load the model via
Load Trained Model
- optional: choose a new prefix to avoid losing existing annotations
- note that the feature selection and most parameters will be deactivated
- press
OK
The (selected) bursts are then classified using the pre-defined model.
General
Combine the concepts of
- selecting regions
- selecting by layer content
- merge categories into layers
- delete categories
- train local models
- train global models
- vary parameters
- vary features
- predict locally
to build models specific to your use-case.