Look at the libs under the pkgconfig files, you need to copy everything that is .private to the libs section because FFmpeg will not read those. Also there might be a missing lib or two that you need to search for in the toolchain to link it. Just look for the undefined reference in the lib directory of your toolchain and add the lib to the pkgconfig files at the end.