[OT] Embedded Linux hosts sharing Ethernet broadcast domains with two IP subnets

[Ivan Shmakov]

[Cross-posting to news:comp.protocols.tcp-ip, as the issue at
hand seems more on-topic there. Apologies for some over-quoting
to preserve context.]

I think this is little OT here, because the question is mostly
network oriented. However my nodes will be Linux embedded boxes and
some solutions could be very low-level, so I think many of you could
help me in some way.
I have N Ethernet hosts based on embedded Linux. Each one features
an Ethernet interface dedicated to the final user that could change
the IP address configuration. On that interface a Web server
responds. All the nodes will be configured on the same subnet and
conected to a switch.
Now I need to make an "internal" communication among those hosts.
With the work "internal" I mean the user should ignore the presence
of this communication and relevant details (it should not be
necessarily invisible to a traffic monitoring tools).
For example, when the user changes the IP address of host 1, the new
IP address shouldn't be configured in host 2 too and the internal
communication between host 1 and 2 should continue without
interruption.
I thought about three possible solution, but I'm not a network guru.
The first is to add other logical IP addresses (aliases) used for
internal communication only. Those IP addresses will be hardcoded
and never changed by whoever. The user will be able to configure
only the main IP address for each host. Is this could work with a
simple switch? I think yes, because the switch looks at the MAC
address and doesn't see the IP addresses. Anyway I don't know what
negative effects could be when the same broadcast domain is shared by
two different subnet (internal, with hardcoded and fixed IP
addresses, and external, configurable by the user as he wants).

There're none that I know of. The only trouble to watch out for
is the user (accidentally) configuring the "main" address that
matches the "internal" network.

Another solution is to implement the internal communication staying
at level 2, so avoiding IP addresses, but using only MAC addresses.
I could invent a proprietary layer 2 protocol. This could avoid any
conflict with IP addresses customized by the user, but there are some
disadvantages.

JFYI, there're already MAC-based IPv6 "link-local" addresses --
which, unless you've configured your kernel differently, are
probably already available. Consider, for example:

$ ip addr list
...
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UP group default qlen 1000
link/ether D0:BA:98:76:54:32 brd ff:ff:ff:ff:ff:ff
...
inet6 fe80::D2BA:98ff:fe76:5432/64 scope link
valid_lft forever preferred_lft forever
...
$ ping6 -n -c 3 -- fe80::D2BA:98ff:fe76:5432%eno1 ; ## that is, local host
PING fe80::D2BA:98ff:fe76:5432%eno1(fe80::D2BA:98ff:fe76:5432%eno1) 56 data bytes
64 bytes from fe80::D2BA:98ff:fe76:5432%eno1: icmp_seq=1 ttl=64 time=0.020 ms
...
3 packets transmitted, 3 received, 0% packet loss, time 2042ms
rtt min/avg/max/mdev = 0.020/0.026/0.030/0.006 ms
$

(Note that the exact interface to use -- eno1 in this case --
needs to be specified when referring to these addresses.)

How host 1 could know the MAC address of host 2 for internal
communication? It could be written during delivery of the system,
but replacing one host on the field could be difficult (the MAC
address of replaced host should be changed on the other nodes).
I could invent a proprietary broadcast ARP protocol (what is MAC
address of **host 2**). In order to avoid the issue of knowing the
MAC address of other hosts for internal communication, I could invent
a L2 protocol made by all broadcast frames, considering it will a
low-bandwidth protocol.

There're already a number of "service discovery" protocols
available. You may want to check [1] and follow from there.

[1] http://en.wikipedia.org/wiki/Multicast_DNS

[...]

[h***@gmail.com]

(someone wrote)

I think this is little OT here, because the question is mostly
network oriented. However my nodes will be Linux embedded boxes and
some solutions could be very low-level, so I think many of you could
help me in some way.
I have N Ethernet hosts based on embedded Linux. Each one features
an Ethernet interface dedicated to the final user that could change
the IP address configuration. On that interface a Web server
responds. All the nodes will be configured on the same subnet and
conected to a switch.
Now I need to make an "internal" communication among those hosts.
With the work "internal" I mean the user should ignore the presence
of this communication and relevant details (it should not be
necessarily invisible to a traffic monitoring tools).

The old way would be to use ethernet broadcast, possibly with your
own ethertype. I believe that there is a process for assigning ethertype
values, but usually not for small scale systems like this.

You can't broadcast TCP, but you can UDP. DHCP, for example does
that before it has an assigned IP address.

I think it should be possible with TCP multicast, but I don't
know how you get assigned a multicast address.

For example, when the user changes the IP address of host 1, the new
IP address shouldn't be configured in host 2 too and the internal
communication between host 1 and 2 should continue without
interruption.