![]()
Visiting guest. Why not sign in?
|
|
A simple behavior arbitration scheme Adjusting the boredom threshold of the Rodney robot | |
A simple behavior arbitration scheme
It's funny how sometimes very simple rules can lead to apparently complex behavior. I've been playing with a little behavior arbitration scheme for my Rodney robot, which allows the machine to decide what "state" it should be in at any point in time and behave accordingly. My aim is eventually to leave the robot switched on all the time, so I wanted to implement a "sleep" mode so that its squeaking servos wouldn't disturb my beauty sleep. I added the facility for the robot to continuously monitor average light levels, such that if the light level drops below a certain level for a certain amount of time the robot becomes dormant. If light levels are sufficient to keep the robot awake then the it looks for interesting moving objects. A level of visual stimulation is calculated from the robots motion detection algorithm, and gradually decays if no moving objects are observed. If visual stimulation drops below a certain threshold then the robot becomes bored and goes into a search mode where its head moves randomly about looking for potential things of interest. If level of visual stimulation exceeds a certain level then the robot goes into a "confused" mode (too many moving targets to decide what to do) and simply remains still until things have quietened down. If visual stimulation is between these two extremes then this means there is a certain manageable amount of visual activity going on, and the robot switches into "curious" mode. In curious mode the robot uses a classifier system in the form of a Kohonen-style SOM to attempt to categorise the observed scene. For fun I could add some other niceties such as speech synthesis ("ooh, I'm bored!", "Good night", etc), maybe using the MS speech SDK. Ciao, |
|
Behaviour-Based AI
God, you've got to stop posting stuff like that! You're making me really want to have my own robot ;) Emotions are a great way to create very interesting emergent behaviours using very simple techniques. In your case, you could model a few emotions and let the bot learn what makes him feel more comfortable... For example, when there's lots going on and he can't understand, he'd feel scared and confused, thereby eventually learning by negative feedback to switch off. The behaviour mechanism you described is very popular among robotics researchers... surprisingly, it's called behaviour based robotics, whereby the robot needs to decide how to behave based on his current state. MIT Media Lab is doing a lot of research into combining all this with emotions (using artificial sheep-dogs for example). Very interesting stuff indeed. |
|
Machine learning
The MIT Kismet robot is probably the best example of vision-based social interaction and modelling of emotional states. This is obviously a lot more sophisticated than Rodney, but the basic principle is the same. To demonstrate a learning algorithm it would be possible to have Rodney's motion tracking ability "emerge" using a reinforcement learning system combined with modelling of emotional states (sometimes refered to as "value systems"). However, for this type of behavior conventional logic suffices quite adequately and the use of RL would probably be overkill and degrade the robots realtime performance. In my opinion its better to reserve these kinds of adaptive system for more difficult tasks such as identifying and classifying *what* the robot is actually looking at. For this task I've used a Kohonen-style SOM. The SOM actually performs very well indeed and if anything learns too rapidly, such that I've had to turn the learning rate and neighbourhood size right down. If the learning rate is too high the SOM is in a continuous state of flux and never really settles into good classifications for any reasonable duration. After some period of time I could turn off the learning and manually assign classifications to areas of the topological map, such that the robot could then make logical inferences about objects being observed and the sequences of observed events. - Bob |
|
Reinforcement Learning and SOFM
I'm actually just looking into RL for my bots. There's quite a bit to be done, especially for speeding up the learning. But as you said, it does have an impact on the performance, and you also notice some fairly erratic behaviour at first! This can be quite dangerous for a physical robot, but taking especial care when modelling the problem seems to do the trick... One thing you might be interested in for learning feature maps, are Neural Gas Networks. They're quite popular at the moment, and rightly so! They can expand to match the dimensionality of your data, and refine and contract as more feedback is obtained from the environment. I'll see if I can find some links for you... |
|
Hyper
For a physical robot its definately useful to have the outputs from whatever adaptive system is being used put through a checking routine to ensure that the desired servo positions won't cause the robot to be damaged. The ability to have the feature space expand or contract depending on the dimensionality of the sensory input received by the robot would certainly be an advantage. One problem with using a SOM type system is that according to the classical algorithm the system gradually moves into an equilibrium state (learning rate and neighbourhood size gradually reducing at some fixed rate) until a solution of some sort is reached. Of course in the real world living systems never reach a state of equilibrium, but are instead in a continuous state of change. You could argue that the ultimate equilibrium state for a living system is death, but even in death biological systems are in a state of being degraded and eventually transformed into other types of organic matter or new forms of life. I recently upgraded my PC from windoze ME to XP, and there does seem to be a dramatic improvement in the performance of the video cameras, to the point where the gains which I was using for motion tracking no longer work. With much faster video processing the robot appears to become hyperactive, with the servos perpetually overshooting and trying to compensate back. It's probably worth making the gains adjustable in the software so that it could potentially be run on different PCs. - Bob |
|
Upgrades
I'm surprised and impressed that the OS upgrade made such a difference! I guess adding parameters to adjust the gain was a good idea anyway, but I'm sure there are plenty other things that you could do with the extra speed... |
|
Utter Boredom
I've refined and improved the Rodney robot's "bored" behavior. Instead on moving the head of the robot either left or right completely at random I've refined this behavior such that although the choice of gaze direction is still a random one the selection is biased towards directions where the robot has previously observed something interesting in the recent past. Biasing the selection of gaze direction in this way allows the robot to more frequently make "lucky guesses", in that the "bored" search routine more often hits upon something of interest. Superficially, it appears as though the robot has "remembered" something that it previously saw and is looking to see if it's still there. - Bob |
|
Robotic Pets
Do you sometimes wonder what the hell it's doing, or is it always obvious what the current 'mode' is? It seems this sort of behaviour is very similar to that of simple mammels, performing on a mostly reactive basis... cats for example aren't too bright, and don't have that much ability for stringing thoughts together. The difference being they have more complex 'evolved' behaviours, which makes them interesting to humans. |
|
The Owl and the Pussycat
I would be more than happy if I could build something with the intelligence level of a cat. Even the most advanced robots built to date probably have substantially less intellect and adaptability than the average insect (certainly far fewer neurons anyway). I don't think that human-level intelligence will be remotely possible for a very long time to come, although people on this discussion group are welcome to prove me wrong :-) - Bob |
|
Artificial Pets
Aiming for something like a cat seems feasible in the medium term... then again (to contradict myself), I'm not sure if copying mammel behaviours is necessarily a 'Good Thing'. Since robots are actually based on completely different 'bodies', I'm sure they will eventually spawn their own kind of complex behaviours - completely different and potentially just as appealing to humans as animals... See the 'Bugs' robots in this previous thread, which is a big step towards this. |
|