Using subnets to extend ESP-Now

ESP-Now is a lightweight networking protocol that uses only two layers (Physical and Data Link) of the OSI model. By doing so it offers high performance and simple programming. In a typical configuration, a single hub device maintains an HTTP connection to the system controller, either as a client or a server, and uses ESP-Now to communicate directly with up to 20 other devices. Commands and queries arrive at the hub on the HTTP interface and are routed to the appropriate device. These messages are sent using a call to a library function that takes 2 arguments: The MAC address of the intended recipient The payload, which can be up to 200 bytes. (A new version increases this limit but I'll ignore that for the purposes of this article.) For a small network, the system offers great performance, impressive reliability and easy programming, but it has two fundamental limitations, being the 20 devices as stated above, and the distance from the hub to the furthest device. As with all things wifi, range is an issue. It can be very hard to find a location for a wifi hub where all of the devices to be controlled can see it reliably. Here's view of a hub with 4 connected devices: The red hatched area is outside the wifi range of the hub, so if new devices are added here they will suffer from poor connections and will consequently be unreliable: One well-known solution to this is to implement a mesh, but this functionality is not offered as part of the ESP-Now spec. In any case, a mesh is often overkill, since what is usually needed is for the hub to drive more devices over a greater range, not for all the devices to talk to each other in arbitrary ways. Fortunately, there's a quite simple solution to this problem, which is to allow any device to handle devices on a subnet of its own. The device count and the range of the network both immediately increase dramatically, albeit with a performance hit, but one that is likely to be insignificant for most small control systems: The way it works is like this. Every device in the system is described by a packet of information, typically a JSON structure, with fields such as the following: "name" - the unique name of the device in the system "mac" - the MAC address of the device "channel" - the wifi channel used "led" - the pin that drives the device LED "led-invert" - a Boolean that governs the ON and OFF polarity "relay" - the pin that drives the device's primary relay "relay-invert" - a Boolean that governs the ON and OFF polarity ... (etc for other hardware and software features of this device) "path" - the path to this device Each device has a JSON file containing its own information, and the central controller holds a file containing all the device structures, indexed by the device names. The last item in the list is the key to the extended networking. For devices close to the hub, nothing is needed here as messages are sent using the MAC address of the device and the payload. But by placing the MAC address of an intermediate device into the field, that device then becomes the parent of the target device (the one the message is intended for). The message is now sent to the parent (the "path" as above), to be forwarded to our required target device. We prefix the payload with the MAC address of the actual target device and structure it so as to make it easy to decode. As an example, let's suppose we want to send a command to turn on the device relay. Let's suppose the MAC address of the device is aa:aa:aa:aa:aa:aa and the payload to turn on the relay is the single word ON. If the network is small and the device is close to the hub we can just send it as is, using a function call such as send(mac, payload) but if the network is full or the device is physically remote we need to relay the message through another device already in the system, one that is closer to our new device. Let's say the MAC address of this parent device is bb:bb:bb:bb:bb:bb. So we send a message to that device, and we adjust the payload to be !aa:aa:aa:aa:aa:aa,ON The function call then becomes (micropython version) send(parent, f'!{mac},{payload}') The exclamation mark is there to signal that the payload is to be routed to the device whose MAC address is given, and the comma is just a separator. Needless to say, this strategy does not allow payloads to begin with an exclamation mark, so you might wish to use a different marker. The code to handle this new message type is remarkably simple. You have to check for the marker, then extract the embedded MAC address and use it to construct a new message to send to the target device. The payload is everything that follows the comma - in this case the original ON - and the reply from the target is simply passed back to your caller. You can see this done in the receive function below. This technique adds no extra devices to the list of "peers" that are held by any given device, and it can be extended almos

Apr 19, 2025 - 17:16

ESP-Now is a lightweight networking protocol that uses only two layers (Physical and Data Link) of the OSI model. By doing so it offers high performance and simple programming.

In a typical configuration, a single hub device maintains an HTTP connection to the system controller, either as a client or a server, and uses ESP-Now to communicate directly with up to 20 other devices. Commands and queries arrive at the hub on the HTTP interface and are routed to the appropriate device. These messages are sent using a call to a library function that takes 2 arguments:

The MAC address of the intended recipient
The payload, which can be up to 200 bytes. (A new version increases this limit but I'll ignore that for the purposes of this article.)

For a small network, the system offers great performance, impressive reliability and easy programming, but it has two fundamental limitations, being the 20 devices as stated above, and the distance from the hub to the furthest device. As with all things wifi, range is an issue. It can be very hard to find a location for a wifi hub where all of the devices to be controlled can see it reliably. Here's view of a hub with 4 connected devices:

The red hatched area is outside the wifi range of the hub, so if new devices are added here they will suffer from poor connections and will consequently be unreliable:

One well-known solution to this is to implement a mesh, but this functionality is not offered as part of the ESP-Now spec. In any case, a mesh is often overkill, since what is usually needed is for the hub to drive more devices over a greater range, not for all the devices to talk to each other in arbitrary ways.

Fortunately, there's a quite simple solution to this problem, which is to allow any device to handle devices on a subnet of its own. The device count and the range of the network both immediately increase dramatically, albeit with a performance hit, but one that is likely to be insignificant for most small control systems:

The way it works is like this. Every device in the system is described by a packet of information, typically a JSON structure, with fields such as the following:

"name" - the unique name of the device in the system
"mac" - the MAC address of the device
"channel" - the wifi channel used
"led" - the pin that drives the device LED
"led-invert" - a Boolean that governs the ON and OFF polarity
"relay" - the pin that drives the device's primary relay
"relay-invert" - a Boolean that governs the ON and OFF polarity
... (etc for other hardware and software features of this device)
"path" - the path to this device

Each device has a JSON file containing its own information, and the central controller holds a file containing all the device structures, indexed by the device names.

The last item in the list is the key to the extended networking. For devices close to the hub, nothing is needed here as messages are sent using the MAC address of the device and the payload. But by placing the MAC address of an intermediate device into the field, that device then becomes the parent of the target device (the one the message is intended for). The message is now sent to the parent (the "path" as above), to be forwarded to our required target device. We prefix the payload with the MAC address of the actual target device and structure it so as to make it easy to decode.

As an example, let's suppose we want to send a command to turn on the device relay. Let's suppose the MAC address of the device is aa:aa:aa:aa:aa:aa and the payload to turn on the relay is the single word ON. If the network is small and the device is close to the hub we can just send it as is, using a function call such as

send(mac, payload)

but if the network is full or the device is physically remote we need to relay the message through another device already in the system, one that is closer to our new device. Let's say the MAC address of this parent device is bb:bb:bb:bb:bb:bb. So we send a message to that device, and we adjust the payload to be

!aa:aa:aa:aa:aa:aa,ON

The function call then becomes (micropython version)

send(parent, f'!{mac},{payload}')

The exclamation mark is there to signal that the payload is to be routed to the device whose MAC address is given, and the comma is just a separator. Needless to say, this strategy does not allow payloads to begin with an exclamation mark, so you might wish to use a different marker.

The code to handle this new message type is remarkably simple. You have to check for the marker, then extract the embedded MAC address and use it to construct a new message to send to the target device. The payload is everything that follows the comma - in this case the original ON - and the reply from the target is simply passed back to your caller. You can see this done in the receive function below.

This technique adds no extra devices to the list of "peers" that are held by any given device, and it can be extended almost indefinitely, by concatenating MAC addresses in the "path" field, such as

bb:bb:bb:bb:bb:bb,cc:cc:cc:cc:cc:cc

which causes the message to be routed through 2 intermediate devices.

I reproduce below the ESP comms class from a Micropython implementation of the above. It relies on a config parameter being supplied during initialization, to provide access to other functions in the system, but mostly it just deals with sending and receiving ESP-Now messages. You will notice that the system holds MAC addresses as human-readable strings, whereas ESP-Now requires formatted binaries, so there's a degree of back-and-forth between the two formats.

import asyncio
from binascii import hexlify,unhexlify
from espnow import ESPNow as E

class ESPComms():

    # Default initializer
    def __init__(self,config):
        self.config=config
        config.setESPComms(self)
        E().active(True)
        self.peers=[]
        print('ESP-Now initialised')

    # Check if a peer is already known
    def checkPeer(self,peer):
        if not peer in self.peers:
            self.peers.append(peer)
            E().add_peer(peer)

    # Send a message to a given MAC address
    async def send(self,mac,espmsg):
        peer=unhexlify(mac.encode())
        self.checkPeer(peer)
        try:
            print(f'Send {espmsg[0:20]}... to {mac}')
            result=E().send(peer,espmsg)
            print(f'Result: {result}')
            if result:
                counter=50
                while counter>0:
                    if E().any():
                        sender,response = E().irecv()
                        if response:
                            print(f'Received response: {response.decode()}')
                            result=response.decode()
                            break
                    await asyncio.sleep(.1)
                    counter-=1
                if counter==0:
                    result='Response timeout'
            else: result='Fail'
        except Exception as e:
            print(e)
            result='Fail'
        return result

    # Handle received messages as they arrive
    # Either deal with them or pass them on
    async def receive(self):
        print('Starting ESPNow receiver on channel',self.config.getChannel())
        while True:
            if E().any():
                peer,msg=E().recv()
                sender=hexlify(peer).decode()
                msg=msg.decode()
                print(f'Message from {sender}: {msg[0:20]}...')
                if msg[0]=='!':
                    comma=msg.index(',')
                    slave=msg[1:comma]
                    msg=msg[comma+1:]
#                    print('Forward to',slave,', msg:',msg)
                    response=await self.send(slave,msg)
                else:
                    response=self.config.getHandler().handleMessage(msg)
                    response=f'{response} {self.getRSS(sender)}'
#                print('Response',response)
                self.checkPeer(peer)
                E().send(peer,response)
            await asyncio.sleep(.1)
            self.config.kickWatchdog()

    # Return the RSS (signal strength) of the received message
    def getRSS(self,mac):
        peer=unhexlify(mac.encode())
        return E().peers_table[peer][0]