![]()
Visiting guest. Why not sign in?
|
|
Audio-visual Finding correspondances between visual and verbal events | |
Audio-visual
Here's some info on the speech recognition that I've been applying to the Rodney robot, together with a few preliminary ideas about how this might be tied in with the Open Mind database. - Bob |
|
Wordnet
It's looking very promising Bob, good work. Quick question, how do you check if he can recognise the objects, and if he associates the object with the right perception? I remember you talking about the visual pattern tree the other week, how does it link together. You may also want to look into Wordnet. I don't know about the other data-bases, but this one seems quite good. |
|
Show and tell
You can check whether he's recognised an object by his verbal questions or pronouncements such as "is that a ball?". I can then give yes or no type replies to confirm or dissociate the identification (a sort of reinforcement learning). The robot will spontaneously make statements if it sees what it thinks is a familiar object. The actual visual/verbal association on Rodney is pretty crude at the moment. I'm using the same short term visual memory as I did previously, which is a system consisting of up to 1000 exemplars. Labels may be assigned to commonly observed exemplars to indicate different object types and so on. This is sufficient for training the robot in the short term (over the course of ten minutes say), but in the longer term the robot will need to analyse the labeled exemplars and try to abstract out the relevant features. For example if the robot picks up ten exemplars labeled with the word "ball" it will then need to compare the images and look for commonalities between them (the "ballness" of the image). The system of verbal interaction is fairly crude at present but looks quite promising in its potential to help the robot build up useful information about its environment. I'm not sure what "wordnet" is, but at the moment I'm planning to use the Open Mind database to help extend the robots logical capabilities. - Bob |
|
Framework
Though you say the visual/verbal association is quite crude, at least you have it setup. It's a good framework to build upon, and when you do get a better algorithm figured out, you can just plug it in. Wordnet is a lexical database in English, which stores all the words you can need, and their association. For example, you can get a list of the meaning of ball, in terms of other words. There are 11 definitions of ball, which quite amazed me. Then you can retrieve synomyms for each definition, and also hypernyms: a ball is a kind of game equipment, instrument, artifact, object... May be useful. I'll look into OpenMind, see what that does. |
|