Reconstruction of Banknote Fragments Based on Keypoint Matching Method

Solving Double-Sided Puzzles: Automated Assembly of Torn-up Banknotes Evidence

Reconstruction of 2D object is a problem concerning many different fields such as forensics science, archiving, and banking. In the literature, it is considered as one-sided puzzle problem. But this study handles torn banknotes as a double-sided puzzle problem for the first time. In addition to that, a new dataset (ToB) is created for solving this problem. A selection approach based on the Borda count method is adopted in order to make the right decision as to which keypoint-based method is to be used in the proposed reconstruction system. The selection approach was determined the Accelerated-KAZE (AKAZE) as the most successful keypoint-based method. This study also proposes new measures determining the success ratio of the reconstructed banknotes and calculating their loss ratio. When the torn banknotes were reconstructed with the AKAZE-based reconstruction system, the average success rate was calculated as 95.55% by the proposed metric.

Shredded banknotes reconstruction using AKAZE points

Shredded banknote reconstruction is a recent topic and can be viewed as solving large-scale jigsaw puzzles. Also, problems such as reconstruction of fragmented documents, photographs and historical artefacts are closely related with this topic. The high computational complexity of these problems increases the need for the development of new methods Reconstruction of shredded banknotes consists of three main stages. (1) Matching fragments with a reference banknote. (2) Aligning the fragments by rotating at certain angles. (3) Assembling the fragments. The existing methods can successfully applied to synthetic banknote fragments which are created in computer environment. But when real banknote reconstruction problem is considered, different sub problems arise and make the existing methods inadequate. In this study, a keypoint based method, named AKAZE, was used to make the matching process effective. This is the first study that uses the AKAZE method in the reconstruction of shredded banknotes. A new method for fragment alignment has also been proposed. In this method, the convex hulls that contain all true matched AKAZE keypoints were found on reference banknote and fragments. The orientations of fragments were estimated accurately by comparing these convex polygons. Also, a new criterion was developed to reveal the success rates of reconstructed banknotes. In addition, two different data sets including real and synthetic banknote fragments of different countries were created to test the success of proposed method.