I have some servers on the public network of my isp, say 192.168.2.0/24. Now my provider gives me additional ip addresses, but unfortunately not in a subsequent range of my first network, say 192.168.4.0/24. I configure the new servers with the new ip range and now they communicate over the default gateway between the subnets, although they are on the same physical network. I add a route to the second network on each server, so they can send packets directly to each other. But if I look in ifconfig, of course I still see only one configured subnet.

Are there any downsides of this configuration? What is the difference to the case where I had the consecutive networks 192.168.2.0/24 and 192.168.3.0/24 and could just configure all interfaces in ifconfig with 192.168.2.0/23 and avoid the extra route?

I could imagine broadcast behavior is maybe different. If i broadcast a network it would only go to half of my servers in one subnet.

Additions:

As I read the first answers, I think my question was maybe no clear enough. The servers are all supposed to be on the public network, I do not want to hide them behind some router. They all also have an internal network connection where most traffic is going over. I was just wondering how you would configure multiple subnets on the same network interface and what the difference is between a setup with consecutive subnets and one with non-consecutive subnets. For me the servers are all in the same public network. It is just that the IP assignment of my provider and the configuration options I see in Linux do not really allow me to configure the servers as such. I have to make the separation between both subnets. I can add additional routes, but will it be the same as if I had one consecutive IP range for all servers?

3 Answers
3

On Public IP, having them in same network or not or consecutive doesn't change much unless you own the whole "sub-part" you can limit with CIDR. If you allow routing of the whole subnet, you'll have external machines routed through your network. Anyways it is never good to route publicly your private data.

To keep your servers connected privately, depending on the number of ports/services you need to access between then, you can:

look if there is no possibility to have a private gateway/router between the servers

build ssh tunnels for each need so you can use to connect for example mysql running on the 2nd server using a local port in the 1st server and the whole connection is encrypted by the tunnel

build a VPN and you servers will be working in a LAN environment within the VPN

There are probably other methods but these are those on top of my mind now and they are secure, simple and easy to apply.

The downside is that all traffic between subnets must pass through a router, rather than being directly switched. In a typical SoHo setup, you have a wire-speed switch (built into your router, connecting its Ethernet ports together), connected to the routing portion that must use its CPU to route packets at process level. A typical SoHo gigabit wireless router can switch 600,000 packets per second but only route 35,000 packets per second.

In addition, that means that all traffic must cross the internal link between the switch portion and the router portion twice, once in each direction. The switching portion is typically fully non-blocking, but you can easily overload the link between the two portions (which is typically an internal Gigabit Ethernet link.

If it has a built-in modem, that's typically connected to the CPU as well. If it has an Internet/WAN Ethernet port, that's sometimes connected to the CPU but more commonly connected also to the non-blocking switch. The internal Gigabit link typically has VLAN support so it can carry both the WAN and LAN traffic.