how to route answers to packets coming in over a particular provider, say Provider 1, back out again over that same provider.

Let us first set some symbolical names. Let $IF1 be the name of the first interface (if1 in the picture above) and $IF2 the name of the second interface. Then let $IP1 be the IP address associated with $IF1 and $IP2 the IP address associated with $IF2. Next, let $P1 be the IP address of the gateway at Provider 1, and $P2 the IP address of the gateway at provider 2. Finally, let $P1_NET be the IP network $P1 is in, and $P2_NET the IP network $P2 is in.

One creates two additional routing tables, say T1 and T2. These are added in /etc/iproute2/rt_tables. Then you set up routing in these tables as follows:

Nothing spectacular, just build a route to the gateway and build a default route via that gateway, as you would do in the case of a single upstream provider, but put the routes in a separate table per provider. Note that the network route suffices, as it tells you how to find any host in that network, which includes the gateway, as specified above.
Next you set up the main routing table. It is a good idea to route things to the direct neighbour through the interface connected to that neighbour. Note the `src' arguments, they make sure the right outgoing IP address is chosen.

Next, you set up the routing rules. These actually choose what routing table to route with. You want to make sure that you route out a given interface if you already have the corresponding source address:
ip rule add from $IP1 table T1
ip rule add from $IP2 table T2

the last hop (router or whatever) that targets you will decide which interface you are contacted on... e.g. if a connection is established from 192.168.1.x, it will not route to the nic connected to 192.168.3.x.

maybe I am missing something here... if this isn't what you want to hear, I think you need to draw your network topology or give us some more information.

Thanks for the reply. I have changed my default route to 192.168.3.X for some specific purpose. That cant be changed. I hope that makes it clear. Also i need to give outside access to the same machine, which doesn't happen even though the forwading is done correctly in the router.
–
rahulNov 27 '12 at 8:12

What is the IP of your router in this example?
–
JustinNov 27 '12 at 11:36

guys, thatnks for the effort. this work. I have put up an answer.
–
rahulNov 29 '12 at 5:59