Config reader

class simba.mixins.config_reader.ConfigReader(config_path: str, read_video_info: bool = True, create_logger: bool = True)

Methods for reading SimBA configparser.Configparser project config and associated project data.

Parameters

• config_path (configparser.Configparser) – path to SimBA project_config.ini

• read_video_info (bool) – if true, read the project_folder/logs/video_info.csv file.

add_missing_ROI_cols(shape_df: DataFrame) → DataFrame

Helper to add missing ROI definitions (Color BGR, Thickness, Color name) in ROI info dataframes created by the first version of the SimBA ROI user-interface but analyzed using newer versions of SimBA.

Parameters

shape_df (pd.DataFrame) – Dataframe holding ROI definitions.

:return pd.DataFrame with Color BGR, Thickness, Color name fields

add_video_to_video_info(video_name: str, fps: float, width: int, height: int, pixels_per_mm: int, distance_mm: float = 0.0) → None

check_multi_animal_status() → None

Helper to check if the project is a multi-animal SimBA project.

create_body_part_dictionary(multi_animal_status: bool, animal_id_lst: list, animal_cnt: int, x_cols: List[str], y_cols: List[str], p_cols: Optional[List[str]] = None, colors: Optional[List[List[Tuple[int, int, int]]]] = None) → Dict[str, Union[List[str], List[Tuple]]]

Helper to create dict of dict lookup of body-parts where the keys are animal names, and values are the body-part names.

Parameters

• multi_animal_status (bool) – If True, it is a multi-animal SimBA project.

• multi_animal_id_lst (List[str]) – Animal names. Eg., [‘Simon, ‘JJ’]. Note: If a single animal project, this will be overridden and set to Animal_1.

• animal_cnt (int) – Number of animals in the SimBA project.

• x_cols (List[str]) – column names for body-part coordinates on x-axis. Returned by simba.mixins.config_reader.ConfigReader.get_body_part_names()

• y_cols (List[str]) – column names for body-part coordinates on y-axis. Returned by simba.mixins.config_reader.ConfigReader.get_body_part_names()

• p_cols (List[str]) – column names for body-part pose-estimation probability values. Returned by simba.mixins.config_reader.ConfigReader.get_body_part_names()

• colors (Optional[List[List[Tuple[int, int, int]]]]) – Optional bgr colors to associate with the body-parts. Returned by simba.utils.data.create_color_palettes().

:returns dict

Example

>>> ConfigReader.create_body_part_dictionary(multi_animal_status=True, animal_id_lst=['simon',])
>>> {'simon': {'X_bps': ['Nose_1_x', 'Ear_left_1_x', 'Ear_right_1_x', 'Center_1_x', 'Lat_left_1_x', 'Lat_right_1_x', 'Tail_base_1_x', 'Tail_end_1_x'], 'Y_bps': ['Nose_1_y', 'Ear_left_1_y', 'Ear_right_1_y', 'Center_1_y', 'Lat_left_1_y', 'Lat_right_1_y', 'Tail_base_1_y', 'Tail_end_1_y'], 'colors': [[255.0, 0.0, 255.0], [223.125, 31.875, 255.0], [191.25, 63.75, 255.0], [159.375, 95.625, 255.0], [127.5, 127.5, 255.0], [95.625, 159.375, 255.0], [63.75, 191.25, 255.0], [31.875, 223.125, 255.0], [0.0, 255.0, 255.0]], 'P_bps': ['Nose_1_p', 'Ear_left_1_p', 'Ear_right_1_p', 'Center_1_p', 'Lat_left_1_p', 'Lat_right_1_p', 'Tail_base_1_p', 'Tail_end_1_p']}, 'jj': {'X_bps': ['Nose_2_x', 'Ear_left_2_x', 'Ear_right_2_x', 'Center_2_x', 'Lat_left_2_x', 'Lat_right_2_x', 'Tail_base_2_x', 'Tail_end_2_x'], 'Y_bps': ['Nose_2_y', 'Ear_left_2_y', 'Ear_right_2_y', 'Center_2_y', 'Lat_left_2_y', 'Lat_right_2_y', 'Tail_base_2_y', 'Tail_end_2_y'], 'colors': [[102.0, 127.5, 0.0], [102.0, 143.4375, 31.875], [102.0, 159.375, 63.75], [102.0, 175.3125, 95.625], [102.0, 191.25, 127.5], [102.0, 207.1875, 159.375], [102.0, 223.125, 191.25], [102.0, 239.0625, 223.125], [102.0, 255.0, 255.0]], 'P_bps': ['Nose_2_p', 'Ear_left_2_p', 'Ear_right_2_p', 'Center_2_p', 'Lat_left_2_p', 'Lat_right_2_p', 'Tail_base_2_p', 'Tail_end_2_p']}}

create_log_msg_from_init_args(locals: dict)

create_logger() → None

create_third_part_append_logger(path: str) → None

drop_bp_cords(df: DataFrame, raise_error: bool = False) → DataFrame

Helper to remove pose-estimation fields from dataframe.

Parameters

• df (pd.DataFrame) – pandas dataframe containing pose-estimation fields (body-part x, y, p fields)

• raise_error (bool) – If True, raise error if body-parts cant be found. Else, print warning

Return pd.DataFrame

df without pose-estimation fields

Example

>>> config_reader = ConfigReader(config_path='test/project_folder/project_config.csv')
>>> df = read_df(config_reader.machine_results_paths[0], file_type='csv')
>>> df = config_reader.drop_bp_cords(df=df)

find_animal_name_from_body_part_name(bp_name: str, bp_dict: dict) → str

Given body-part name and animal body-part dict, returns the animal name

Parameters

• bp_name (str) – Name of the body-part. E.g., Ear_1.

• bp_dict (dict) – Nested dict holding animal names as keys and body-part names as and coordinates as values. Created by simba.mixins.config_reader.ConfigReader.create_body_part_dictionary()

:returns str

Example

>>> config_reader = ConfigReader(config_path='tests/data/test_projects/two_c57/project_folder/project_config.ini')
>>> ConfigReader.find_animal_name_from_body_part_name(bp_name='Ear_1', bp_dict=config_reader.animal_bp_dict)
>>> 'simon'

find_video_of_file(video_dir: Union[str, PathLike], filename: str, raise_error: bool = False) → Union[str, PathLike]

Helper to find the video file representing a known data file basename.

Parameters

• video_dir (Union[str, os.PathLike]) – Directory holding putative video file.

• filename (str) – Data file name, e.g., Video_1.

• raise_error (bool) – If True, raise error if no video can be found.

Return Union[str, os.PathLike]

Path to video file.

Example

>>> config_reader = ConfigReader(config_path='My_SimBA_Config')
>>> config_reader.find_video_of_file(video_dir=config_reader.video_dir, filename='Video1')
>>> '/project_folder/videos/Video1.mp4'

get_all_clf_names() → List[str]

Helper to return all classifier names in SimBA project

Return List[str]

get_body_part_names()

Helper to extract pose-estimation data field names (x, y, p)

Example

>>> config_reader = ConfigReader(config_path='test/project_config.csv')
>>> config_reader.get_body_part_names()

get_bp_headers() → None

Helper to create ordered list of all column header fields for SimBA project dataframes.

>>> config_reader = ConfigReader(config_path='test/project_folder/project_config.ini')
>>> config_reader.get_bp_headers()

get_number_of_header_columns_in_df(df: DataFrame) → int

Helper to find the count of non-numerical rows at the top of a dataframe.

Parameters

data_df (pd.DataFrame) – Dataframe to find the count non-numerical header rows in.

:returns int :raises DataHeaderError: All rows are non-numerical.

Example

>>> ConfigReader.get_number_of_header_columns_in_df(df=pd.DataFrame(data=[[1, 2, 3], [1, 2, 3]]))
>>> 0
>>> ConfigReader.get_number_of_header_columns_in_df(df=pd.DataFrame(data=[['Head_1', 'Body_2', 'Tail_3'], ['Some_nonsense', 'A_mistake', 'Maybe_multi_headers?'], [11, 99, 109], [122, 43, 2091]]))
>>> 2

insert_column_headers_for_outlier_correction(data_df: DataFrame, new_headers: List[str], filepath: str) → DataFrame

Helper to insert new column headers onto a dataframe.

Parameters

• data_df (pd.DataFrame) – Dataframe where headers to to-bo replaced.

• new_headers (List[str]) – Names of new headers.

• filepath (str) – Path to where data_df is stored on disk

Returns pd.DataFrame

Dataframe with new headers

Raises

DataHeaderWarning – If new headers are fewer/more than columns in dataframe

Example

>>> df = pd.DataFrame(data=[[1, 2, 3], [1, 2, 3]], columns=['Feature_1', 'Feature_2', 'Feature_3'])
>>> ConfigReader.insert_column_headers_for_outlier_correction(data_df=df, new_headers=['Feature_4', 'Feature_5', 'Feature_6'], filepath='test/my_test_file.csv')

read_config_entry(config: ConfigParser, section: str, option: str, data_type: typing_extensions.Literal['str', 'int', 'float', 'folder_path'], default_value: Optional[Any] = None, options: Optional[List[Any]] = None) → Union[str, int, float]

Helper to read entry from a configparser.ConfigParser object

Parameters

• config (ConfigParser) – Parsed SimBA project config

• section (str) – Project config section name

• option (str) – Project config option name

• data_type (str) – Type of data. E.g., str, int, float, folder_path.

• default_value (Optional[Any]) – If entry cannot be found, then default to this value.

• options (Optional[List[Any]]) – If not None, then list of viable entries.

Raises

• InvalidInputError – If returned value is not in options.

• MissingProjectConfigEntryError – If no entry is found and no default_value is provided.

:return Union[str, float, int, os.Pathlike]

Example

>>> config = ConfigReader(config_path='tests/data/test_projects/two_c57/project_folder/project_config.ini')
>>> config.read_config_entry(config=self.config, section='Multi animal IDs', option='id_list', data_type='str')
>>> 'simon,jj'

read_roi_data() → None

Method to read in ROI definitions from SimBA project

read_video_info(video_name: str, video_info_df_path: Optional[Union[str, PathLike]] = None, raise_error: Optional[bool] = True) → Union[Tuple[DataFrame, float, float], Tuple[None, None, None]]

Helper to read the meta-data (pixels per mm, resolution, fps) from the video_info.csv for a single input file.

Parameters

• video_name (str) – The name of the video without extension to get the metadata for

• raise_error (Optional[bool]) – If True, raise error if video info for the video name cannot be found. Default: True.

• video_info_df_path (Optional[Union[str, os.PathLike]]) – Optional path to the video_info.csv on disk. If None, uses path from self.

Returns

Tuple representing all video info, pixels per mm, and fps. If raise_error is False, returns a tuple of Nones

Return type

Union[Tuple[pd.DataFrame, float, float], Tuple[None, None, None]]

read_video_info_csv(file_path: Union[str, PathLike]) → DataFrame

Helper to read the project_folder/logs/video_info.csv of the SimBA project in as a pd.DataFrame

Parameters

file_path (Union[str, os.PathLike]) – Path to the project_folder/logs/video_info.csv file.

Returns

Dataframe representation of the file.

Return type

pd.DataFrame

remove_a_folder(folder_dir: str, raise_error: Optional[bool] = False) → None

Helper to remove single directory.

Parameters

• folder_dir – Directory to remove.

• raise_error (bool) – If True, raise NotDirectoryError error of folder does not exist.

Raises

NotDirectoryError – If raise_error and directory does not exist.

Example

>>> self.remove_a_folder(folder_dir'gerbil/gerbil_data/featurized_data/temp')

remove_multiple_folders(folders: List[PathLike], raise_error: Optional[bool] = False) → None

Helper to remove multiple directories.

Parameters

• List[os.PathLike] (folders) – List of directory paths.

• raise_error (bool) – If True, raise NotDirectoryError error of folder does not exist. if False, then pass. Default False.

Raises

NotDirectoryError – If raise_error and directory does not exist.

Example

>>> self.remove_multiple_folders(folders= ['gerbil/gerbil_data/featurized_data/temp'])

remove_roi_features(data_dir: Union[str, PathLike]) → None

Helper to remove ROI-based features from datasets within a directory. The identified ROI-based fields are move to the project_folder/logs/ROI_data_{datetime} directory.

Note

ROI-based features are identified based on the combined criteria of (i) The prefix of the field is a named ROI in the project_folder/logs/ROI_definitions.h5 file, and (ii) the suffix of the field is contained in the [‘in zone’, ‘n zone_cumulative_time’, ‘in zone_cumulative_percent’, ‘distance’, ‘facing’]

Parameters

data_dir (Union[str, os.PathLike]) – directory with data to remove ROi features from.

Example

>>> self.remove_roi_features('/project_folder/csv/features_extracted')