There are many ways in which you can include external code/ functionality. One of them is to include a (shared) library. Library is a binary file that - given certain knowledge - can be used by other compiled binaries. It is not possible out of the box - in order to access the library, you need a list of all the exported (that is: available to you) methods and classes. This (I'm simplifying a lot here) is done by including the library headers into your code. When those headers are read by the compiler/ linker, they adjust your code so that it can talk with that binary library after compilation. In Qt/qmake, this is done by adding header directories only to INCLUDEPATH variable:
@
INCLUDEPATH += /path/to/library/headers
@

So, you need to compile (or download precompiled) library first, then compile your project while pointing it to those header files. To stress the point: library sources (.cpp) are not present anywhere in this scenario, they are not needed :)

Another way is to use static libraries. This is very similar to above in practice, although it differs in theory. I won't get into that, we don't need to know this at this moment.

Yet another way, possible but seldom used, is to incorporate all the headers and sources into your project (your last attempt does this) using HEADERS and SOURCES variables in qmake. This does not require you to precompile your library, but is usually much harder to do, can cause licensing problems and in general means more work for little gain. Once compiled, the library is not really a library anymore: it is just another part of your application, fully integrated.