After having AI show my fortune, the next project was having AI navigate my robot car.

To keep things simple I constrained movement to a 5 x 5 grid ( X = 0 – 4 and Y = 0 – 4) and the car could only move N, S, E or W one cell at a time. To get the algorithm right and test this with Ai before actually applying it to the car I simulated the result on the OLED screen I had previously configured.

With that all working I upgraded the code to run with the robot car and you’ll find it here:

https://github.com/directorcia/Azure/blob/master/Iot/LLM/llm-cellmove.ino

The main issue I found was more mechanical with the robot car wheels dragging and not being very precise, so the car woudl easily wander off in different directions. This has more to do with teh quality of the motors and wheels as well as the friction encountered when the wheels start moving. However, aside from that the test worked successfully and the car cycled up the grid and then back down.

Next, I wanted the AI to actually assist with the navigation of the car. I went through plenty of iterations with this. The most important change is that I moved from using a local AI to using Gemini via API calls. The main reason for this was simply speed. As the prompts became larger the local AI model struggled to return the results to the car in enough time to implement effective navigation. I had also wanted to integrate large cloud based LLM so here was the opportunity, so I hooked up Gemini.

The robot car also has an ultrasonic senor connected to a motor at the front so I could sweep it left and right as well for better object detection. However, initially I kept it simple and all on teh car by just using the ultrasonic sensor to detect hazards and try to makeover around them. The code for that is here:

https://github.com/directorcia/Azure/blob/master/Iot/LLM/llm-sweep.ino

I then upgraded the code to integrate the LLM into the navigation process by making decisions on which direction to turn. That code is here:

https://github.com/directorcia/Azure/blob/master/Iot/LLM/llm-sweep.ino

One issues I ran into, that wasted a lot of my time and was totally my own fault, was when I started having issues with the wheel moving the car forward. I blamed the code but in fact, again it was a hardware issue, being the battery charge had become too low to actually drive the wheels acceptably. It is interesting at how quickly that car now actually drains power when fully running.

With the power issues resolved I upgraded the code a number of times to allow the LLM a much higher level of navigation control. You’ll find the final result here:

https://github.com/directorcia/Azure/blob/master/Iot/LLM/llm-assist-nav.ino

The end result of all these experiments is that I have learned that in the full configuration the car now burns a lot of power as it moves, turns the sensor, communicates over wifi and more. all the changes I made to the code would make the car slightly less likely to crash into objects on the floor but a lot more though needs to go into ‘crash free’ navigation. The obvious improvement is to add more sensors to provide the LLM with more information to make better decisions. I also found that the wheel on the car are not precise enough and don’t really provide the best grip. This means they tend to be slow to engage and lag cause the car to veer.

I think all of these can be solved iteratively over time and I am confident that I can get to a situation that allows the robot car to move pretty much crash free around the floor just like a robot vacuum can already. However, the time required is probably not something that I’m willing to invest in just now to get a little incrementally better. I’m happy that my ‘proof of concept’ when it comes to navigation with LLMs works. I think it is time to move onto the next project.