Feature extraction and matching has been widely used for the registration of overlapping partial shapes. However, the established tentative point matches (TPMs) usually include a large proportion of false ones due to a number of factors: imaging noise, occlusion, repetitive structures, and cluttered background. To apply the rigidity constraint to refine these TPMs in three main steps is proposed. First, a set of one-to-many point matches is established, then use them to vote against each other, and finally select such TPMs with the same surface type, the minimum error, and the most votes signifi- cantly more than the second most ones as the more reliable ones. A comparative study using real data captured by a Microsoft Kinect sensor shows that the refinement is successful.