You could even connect all 4 wires to your jack. Yellow, black, green and red. There are 4 prongs inside the jack, 2 for one line, and 2 for the other. If you go to Radio Shack, you can purchase a splitter that will actually split line1 and line 2 into two separate jacks. You can plug line 1 phone into line 1, and vice versa.

I just went this morning and picked up a couple of those splitters! I didn't know they existed eventhough I had bought a couple years back a 4wires duplex jack, I didn't even think there would be a 2line splitter. Thanks DarKev for the tip