How to Deploy Machine Learning Model on Wearable Edge Devices?

I have done mostly machine learning with big data, GPUs on EC2 VMs, K8S clusters etc. But this new assignment is on the other end of the scale. Basically, it is a time series forecasting and regression problem with some body signals. The problem itself is simple enough. I developed some moderate sized transformer/LSTM models using frameworks like tensorflow and darts. But the deployment constraint says the model has to be deployed on a low power proprietary wearable device with

Feb 28, 2025 - 03:42

I have done mostly machine learning with big data, GPUs on EC2 VMs, K8S clusters etc. But this new assignment is on the other end of the scale.

Basically, it is a time series forecasting and regression problem with some body signals. The problem itself is simple enough. I developed some moderate sized transformer/LSTM models using frameworks like tensorflow and darts. But the deployment constraint says

the model has to be deployed on a low power proprietary wearable device with <50 ms latency

So the questions are

What kind of frameworks, or programming languages do I need to support it?
I am not an Android/IOS developer either. Suppose I wrap my model inference (using tensorflow) inside a simple python function that takes some features as arguments and spits out a prediction. So do I assume the device/firmware engineer can invoke the python function to consume my model? Or inside the device I can run a Dockerised service at a port to get the inference (this is what I have done previously in big data context)?
What kind of interface can I use to retrain and update the model regularly? Again, the challenge is in pushing the model to the end device
Or, in some scenarios, may I assume that the device stays internet connected, hence can just make an HTTP request to a server?

I am not sure if I am asking the right questions here, as obviously it is a big context switch from my previous set ups using big cloud services for inference. So any help, resource and standard practices will be greatly appreciated.