CUDA is an advanced massively parallel computing platform that can provide high performance computing power at much more affordable cost. In this paper, we present a parallel graph-based substructure pattern mining algorithm using CUDA Dynamic Parallelism. The key contribution is a parallel solution to traversing the DFS (Depth First Search) code tree. Furthermore, we implement a parallel frequent subgraph mining algorithm based on the subgraph mining techniques used in gSpan and the entire subgraph mining procedure is executed on GPU to ensure high efficiency. This parallel gSpan is functionally identical to the original gSpan and experiment results show that, with the latest CUDA Dynamic Parallelism techniques, significant speedups can be achieved on benchmark datasets, particularly in traversing a DFS code tree.