SHF: A DNNs accelerator with software/hardware fusion mechanism

Deploying complex Deep Neural Networks (DNNs) on resource-constrained edge devices demands efficient acceleration but often faces the challenge of software–hardware mismatch. This paper proposes a Software/Hardware Fusion (SHF) mechanism to address these challenges. Guided by a performance evaluation model, the software component (SHF-S) utilizes the particle swarm optimization (PSO) algorithm to generate an optimal task partitioning scheme, assigning computation-intensive tasks to an artificial intelligence (AI) accelerator and control-intensive ones to the general–purpose processor (GPP). The hardware component (SHF-H) efficiently executes these tasks, forming a collaborative closed-loop system via a status feedback mechanism. We validated the mechanism on a heterogeneous system comprising an OK3588 and a reconfigurable AI chip, using four networks with distinct characteristics: ResNet-18, Inception-v3, LSTM, and Capsule Network. The results demonstrate that SHF exhibits excellent adaptability. Compared to a GPP-only baseline, it achieves up to 2.35× acceleration and outperforms state-of-the-art frameworks, providing an efficient software–hardware co-design solution for edge AI inference.