Investigating Deep Learning for Identification of Crabs and Lobsters on Fishing Boats

This paper describes a collaboration between marine and computer scientists to improve fisheries data collection. We evaluate deep learning (DL)-based solutions for identifying crabs and lobsters onboard fishing boats. A custom made electronic camera systems onboard the fishing boats captures the video clips. An automated process of frame extraction is adopted to collect images of crabs and lobsters for training and evaluating DL networks. We train Faster R-CNN, Single Shot Detector (SSD), and You Only Look Once (YOLO) with multiple backbones and input sizes. We also evaluate the efficiency of lightweight models for low-power devices equipped on fishing boats and compare the results of MobileNet-based SSD and YOLO-tiny versions. The models trained with higher input sizes result in lower frames per second (FPS) and vice versa. Base models are more accurate but compromise computational and run time cost. Lighter versions are flexible to install with lower mAP than full models. The pre-trained weights for training models on new datasets have a negligible impact on the results. YOLOv4-tiny is a balanced trade-off between accuracy and speed for object detection for low power devices that is the main step of our proposed pipeline for automated recognition and measurement of crabs and lobsters on fishing boats.