We construct a previously unknown \(E_2\)-quasi-exactly solvable non-Hermitian model whose eigenfunctions involve weakly orthogonal polynomials obeying three-term recurrence relations that factorize beyond the quantization level. The model becomes Hermitian when one of its two parameters is fixed to a specific value. We analyze the double scaling limit of this model leading to the complex Mathieu equation. The norms, Stieltjes measures and moment functionals are evaluated for some concrete values of one of the two parameters.