This paper describes the implementation of the multiple targets tracking algorithm in FPGA-based vision system. The described algorithm was designed to process such situations as the object trajectories crossing and the temporary object screening by other objects. The source data for this algorithm is a list of the parameters of the previously extracted binary regions from each frame of the sequence. The main idea of this algorithm is to represent the source data as a bipartite graph and split it into insolated elementary graphs corresponding to five situations: object is moving or staying still, a new object detected, object is missed, the pair of the objects is merged into one and the region is divided. These graphs are used to form a new object list. The goal of this work was to implement the described algorithm in small-sized onboard vision system based on the single Xilinx FPGA using MicroBlaze soft processor block. In the proposed implementation of this algorithm recursive procedures were replaced with table-based procedures. The experimental research of the algorithm shows the increasing tracking performance 5 – 9 times on previously described hardware.