Extracting labelled images

class simba.labelling.extract_labelled_frames.AnnotationFrameExtractor(config_path: Union[str, PathLike], data_paths: List[Union[str, PathLike]], clfs: List[str], img_downsample_factor: Optional[Union[float, int]] = None, img_format: typing_extensions.Literal['png', 'webp', 'jpg'] = 'png', img_greyscale: Optional[bool] = False)[source]

Bases: ConfigReader

Extract frames annotated as behavior-present and save them as image files.

Parameters

  • config_path (Union[str, os.PathLike]) – Path to the SimBA project_config.ini file.

  • data_paths (List[Union[str, os.PathLike]]) – Annotation file paths to read labels from.

  • clfs (List[str]) – Names of classifiers to extract behavior-present images for.

  • img_downsample_factor (Optional[Union[float, int]]) – Optional image downsampling factor. If None or 1, no downsampling is applied.

  • img_format (Optional[Literal['png', 'webp', 'jpg']]) – Output image format.

  • img_greyscale (Optional[bool]) – If True, save images in grayscale.

Example

>>> extractor = AnnotationFrameExtractor(
...     config_path='project_folder/project_config.ini',
...     data_paths=['project_folder/csv/targets_inserted/video_1.csv'],
...     clfs=['Sniffing', 'Attack'],
...     img_downsample_factor=2,
...     img_format='png',
...     img_greyscale=False
... )
>>> extractor.run()

Calculate annotation statistics in SimBA project

class simba.labelling.extract_labelling_meta.AnnotationMetaDataExtractor(config_path: Union[str, PathLike], split_by_video: Optional[bool] = True, annotated_bouts: Optional[bool] = True)[source]

Bases: ConfigReader

Extract annotation statistics (number of annotated frames, seconds, bouts etc.) for all classifiers in a SimBA project to MS Excel format.

Note

Example expected output.

Parameters

  • config_path (Union[str, os.PathLike]) – path to SimBA configparser.ConfigParser project_config.ini

  • annotated_bouts (Optional[bool]) – If True, includes information on annotated bouts (start and stop time and bout length). Default True.

  • split_by_video (Optional[bool]) – If True, includes a worksheet where the annotation counts are split by video. Default True.

Example

>>> annotation_meta_extractor = AnnotationMetaDataExtractor(config_path='/Users/simon/Desktop/envs/simba/troubleshooting/two_black_animals_14bp/project_folder/project_config.ini')
>>> annotation_meta_extractor.run()
>>> annotation_meta_extractor.save()

SimBA advanced GUI labelling interface

class simba.labelling.labelling_advanced_interface.AdvancedLabellingInterface(config_path: str, file_path: str, continuing: bool)[source]

Bases: ConfigReader

Launch advanced labelling (annotation) interface in SimBA.

Note

Advanced annotation tutorial.

_images/adv_annotator.png
Parameters

  • config_path (str) – path to SimBA project config file in Configparser format

  • file_path (str) – Path to video that is to be annotated

  • continuing (bool) – If True, user is continuing the annotations of a video with started but incomplete annotations.

Examples

>>> select_labelling_video_advanced(config_path='MyProjectConfig', file_path='MyVideoFilePath', continuing=True)

SimBA GUI labelling interface

Playing videos in the annotation interface

simba.labelling.play_annotation_video.annotation_video_player()[source]

Private methods for playing the video that is being annotated in SimBA GUI

Annotation GUI mixins

class simba.mixins.annotator_mixin.AnnotatorMixin(config_path: Union[str, PathLike], video_path: Union[str, PathLike], data_path: Union[str, PathLike], frame_size: Optional[Tuple[int]] = (1280, 650), title: Optional[str] = None)[source]

Bases: ConfigReader

Methods for creating tkinter GUI frames and functions associated with annotating videos.

Currently under development (starting 01/24). As the number of different annotation methods and interfaces increases, this class will contain common methods for all annotation interfaces to decrease code duplication.

Parameters

  • config_path (Union[str, os.PathLike]) – Path to SimBA project config ini file.

  • video_path (Union[str, os.PathLike]) – Path to video file to-be annotated.

  • data_path (Union[str, os.PathLike]) – Path to featurized pose-estimation data associated with the video.

  • frame_size (Optional[Tuple[int]]) – The size of the subframe displaying the video frame in the GUI.

change_frame(new_frm_id: int, min_frm: Optional[int] = None, max_frm: Optional[int] = None, update_funcs: Optional[List[Callable[int, None]]] = None, store_funcs: Optional[List[Callable[None]]] = None, keep_radio_btn_choices: Optional[bool] = False) → None[source]

Change the frame displayed in annotator GUI.

Note

store_funcs will be executed before update_funcs.

Parameters

  • new_frm_id (int) – The frame number of the new frame.

  • min_frm (Optional[int]) – If the minimum frame number is not the first frame of the video, pass the minimum frame number here.

  • max_frm (Optional[int]) – If the maximum frame number is not the last frame of the video, pass the max frame number here.

  • max_frm – If the maximum frame number is not the last frame of the video, pass the max frame number here.

  • update_funcs (Optional[List[Callable[[int], None]]]) – Optional functions that takes accepts the new frame numers that should be called. E.g., if updating the frame number should cause the display of the new frame numbers in any other Frame.

  • store_funcs (Optional[List[Callable[[], None]]]) – Optional functions that saves user frame selections in memory.

  • keep_radio_btn_choices (Optional[bool]) – If True, then any update_funcs that causes the update of radio button choices in the newly displayed frame will be supressed. Thus, the choices in the prior frame is maintained.

find_proximal_annotated_frm(forwards: bool, present: bool)[source]

Helper to find the most proximal preceding or proceeding frame where any one behavior are annotated as present or absent.

get_annotation_of_frame(frm_num: int, clf: str, allowed_vals: Optional[List[Optional[int]]] = None) → Optional[int][source]

Helper to retrieve the stored annotation of a specific classifier at a specific frame index

h_nav_bar(parent: Frame, update_funcs: Optional[List[Callable[int, None]]] = None, store_funcs: Optional[List[Callable[None]]] = None, size: Optional[Tuple[int, int]] = (300, 700), loc: Optional[Tuple[int, int]] = (1, 0), previous_next_clf: Optional[bool] = False) → None[source]

Creates a horizontal frame navigation bar where the buttons are tied to callbacks for changing and displaying video frames.

_images/h_nav_bar.png
Parameters

  • parent (Frame) – The tkinter Frame to place navigation bar within.

  • update_funcs (Optional[List[Callable[[int], None]]]) – Optional list of callables that accepts a single integer inputs. Can be methods that updates part of the interface as the frame number changes.

  • store_funcs (Optional[List[Callable[[], None]]]) – Optional list of callables without arguments. Can be methods that stores the selections in memory as users proceeds through the frames.

  • size (Optional[Tuple[int, int]]) – The size of the navigation bar in h x w. Default 300 x 700.

  • loc (Optional[Tuple[int, int]]) – The grid location (row, column) within the parent frame at which the navigation bar should be displayed. Defualt: (1, 0).

  • previous_next_clf (Optional[bool]) – If True, then include four buttons allowing users to navigate to the most proximal preceding or proceeding frame where behaviors are annotated as present or absent.

store_annotation() → None[source]

Method to store annotations in memory

store_targeted_annotations_frames()[source]

Method to store annotations in memory while annotating targeted bouts frame-wise

targeted_annotations_frames_save()[source]

Method to save annotations to disk when using targeted bout frame-wise annotations

targeted_bouts_pane(parent: Frame) → Frame[source]

Create a pane for choosing bouts start and end and a radiobutton truth table for targeted bouts annotations. Used by simba.labelling.targeted_annotations_bouts.TargetedAnnotatorBouts

targeted_frames_selection_pane(parent: Frame, loc: Optional[Tuple[int, int]] = (0, 1)) → None[source]

Creates a vertical pane that includes tkinter frames for selecting bouts and annotating behaviours in those bouts frame-wise.

_images/targeted_frames_selection_pane.png
Parameters

  • parent (Frame) – The tkinter Frame to place the vertical annotation frame within.

  • loc (Optional[Tuple[int, int]]) – The grid location (row, column) within the parent frame at which the pane should be displayed. Default: (0, 1).

update_clf_radiobtns(frm_num: int)[source]

Update helper to set the radio button to annotated values

update_current_selected_frm_lbl(new_frm: Union[int, str])[source]

Helper to update label showing current frame text shown when annotating bouts frame-wise.

v_navigation_pane_targeted(parent: Frame, save_func: Callable[None], update_funcs: Optional[List[Callable[int, None]]] = None, store_funcs: Optional[List[Callable[None]]] = None, loc: Optional[Tuple[int, int]] = (0, 2)) → None[source]

Create a vertical navigation pane for playing a video and displaying and activating keyboard shortcuts when annotating bouts.

_images/h_nav_bar.png
Parameters

  • parent (Frame) – The tkinter Frame to place the vertical navigation bar within.

  • save_func (Callable[[], None]) – The save-data-to-disk function that should be called when using the save data shortcut.

  • update_funcs (Optional[List[Callable[[int], None]]]) – Optional list of callables that accepts a single integer inputs. Can be methods that updates part of the interface as the frame number changes.

  • store_funcs (Optional[List[Callable[[], None]]]) – Optional list of callables without arguments. Can be methods that stores the selections in memory as users proceeds through the frames.

  • loc (Optional[Tuple[int, int]]) – The grid location (row, column) within the parent frame at which the navigation bar should be displayed. Default: (1, 0).

video_frm_label(frm_number: int, max_size: Optional[Tuple[int]] = None, loc: Tuple[int] = (0, 0)) → None[source]

Inserts a video frame as a tkinter label at a specified maximum size at specified grid location.

_images/video_frm_label.png
Parameters

  • frm_number (int) – The frame number if the video that should be displayed as a tkinter label.

  • max_size (Optional[Tuple[int, int]]) – The maximum size of the image when displayed. If None, then frame_size defined at instance init.

  • loc (Tuple[int, int]) – The grid location (row, column) within the main frame at which the video frame should be displayed.