At very first step
of syndication is to check if both servers are able to communicate each other ,
So I just checked using "ping server1" from server2 and "ping
server2" from server1 and got no pocket loss and thought both are communicating
fine (Infrastructure team overlooked and confirmed that all ports are open , so
didn't validated at port level initially) .

From the above,
what we understood Subscriber is able to communicate with syndicator
(because we got response back) .

Solution in this case

We tried
creating the syndication pair from the server2 (syndicator) to server1
(subscriber) just to make sure the everything fine , then we found that
(server2)Syndicator is not able to communicate with subscriber(server1).
(Server1 to server2 is fine but not reverse).

Gone back
tried to validate the open ports between the servers (not just
"ping") , we tried telnet to that portal default port (10039)
and found that 10039 not open . This is basic thing we should have tried
this to make sure ports are open communicating over those ports are fine.