With the LLM now up and working on a separate device on my LAN, the next step is to test it remotely to ensure that it works. For this I used the following simple PowerShell on a remote machine

which you will find here:

https://github.com/directorcia/Azure/blob/master/Iot/LLM/echo-ping.ps1

Next, I ran the following PowerShell script:

https://github.com/directorcia/Azure/blob/master/Iot/LLM/echo-test.ps1

which simply runs a standard prompt of;

“Reply with exactly: Hello from Ollama”

and then ensure that I get that reply back from the remote LLM server. This means I have communications to the actual server as well as the LLM.

With all the remote communications confirmed, the next step was to get a device talking to the LLM. For this I had a ESP32-C3-DevKitM-1 hanging around.

The main benefit of this device is that it has inbuilt Wifi. I connected up a LED and resistor to GPIO4 like so:

I then used this code in the device:

https://github.com/directorcia/Azure/blob/master/Iot/LLM/llm-flash.ino

to prompt the LLM for a number of flashes from 1 – 4, which the device would then complete that on the LED. I could also monitor the progress using the terminal, which would look like:

Connecting to Wi-Fi…..
Wi-Fi connected. IP: 192.168.1.42
Requested flash count: 3
Sending request to Ollama (attempt 1)…
HTTP status: 200
Ollama reply: {“flash”:3}

Initially I found that the LLM was returning the same number of flashes, so I needed to adjust the prompting to get some variation. The good news is that I got it all working and the resultant code is above.

So now I have successfully gotten a device talking to a local LLM. I’ll be expanding on this in upcoming articles but very happy that have this basic configuration all working now.