Heath S., Ball D., Schulz R., and Wiles J., Communication between Lingodroids with different cognitive capabilities, Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, May 6 - 10, 2013
Previous studies have shown how Lingodroids, language learning mobile robots, learn terms for space and time, connecting their personal maps of the world to a publically shared language. One caveat of previous studies was that the robots shared the same cognitive architecture, identical in all respects from sensors to mapping systems. In this paper we investigate the question of how terms for space can be developed between robots that have fundamentally different sensors and spatial representations. In the real world, communication needs to occur between agents that have different embodiment and cognitive capabilities, including different sensors, different representations of the world, and different species (including humans). The novel aspects of these studies is that one robot uses a forward facing camera to estimate appearance and uses a biologically inspired continuous attractor network to generate a topological map; the other robot uses a laser scanner to estimate range and uses a probabilistic filter approach to generate an occupancy grid. The robots hold conversations in different locations to establish a shared language. Despite their different ways of sensing and mapping the world, the robots are able to create coherent lexicons for the space around them.
Ball, D., Heath, S., Wiles, J., Wyeth, G., Corke, P., and Milford, M., (2013). OpenRatSLAM: an open source brain-based SLAM system, Autonomous Robots, 34(3), 149-176
RatSLAM is a navigation system based on the neural processes underlying navigation in the rodent brain, capable of operating with low resolution monocular image data. Seminal experiments using RatSLAM include mapping an entire suburb with a web camera and a long term robot delivery trial. This paper describes OpenRatSLAM, an open-source version of RatSLAM with bindings to the Robot Operating System framework to leverage advantages such as robot and sensor abstraction, networking, data playback, and visualization. OpenRatSLAM comprises connected ROS nodes to represent RatSLAM’s pose cells, experience map, and local view cells, as well as a fourth node that provides visual odometry estimates. The nodes are described with reference to the RatSLAM model and salient details of the ROS implementation such as topics, messages, parameters, class diagrams, sequence diagrams, and parameter tuning strategies. The performance of the system is demonstrated on three publicly available open-source datasets
Heath, S., Schulz, R., Ball, D., and Wiles, J. (2012). Long summer days: Grounded learning of words for the uneven cycles of real world events, IEEE Transactions on Autonomous Mental Development, 4(3), 192-203
Time and space are fundamental to human language and embodied cognition. In our early work we investigated how Lingodroids, robots with the ability to build their own maps, could evolve their own geopersonal spatial language. In subsequent studies we extended the framework developed for learning spatial concepts and words to learning temporal intervals. This paper considers a new aspect of time, the naming of concepts like morning, afternoon, dawn, and dusk, which are events that are part of day-night cycles, but are not defined by specific time points on a clock. Grounding of such terms refers to events and features of the diurnal cycle, such as light levels. We studied event-based time in which robots experienced day-night cycles that varied with the seasons throughout a year. Then we used meet-at tasks to demonstrate that the words learned were grounded, where the times to meet were morning and afternoon, rather than specific clock times. The studies show how words and concepts for a novel aspect of cyclic time can be grounded through experience with events rather than by times as measured by clocks or calendars.
Schulz, R., Wyeth, G., and Wiles, J., (2012). Beyond here-and-now: Extending shared physical experiences to shared conceptual experiences, Adaptive Behavior, 20(5), 360-387
For robots to use language effectively, they need to refer to combinations of existing concepts, as well as concepts that have been directly experienced. In this paper, we introduce the term generative grounding to refer to the establishment of shared meaning for concepts referred to using relational terms. We investigated a spatial domain, which is both experienced and constructed using mobile robots with cognitive maps. The robots, called Lingodroids, established lexicons for locations, distances, and directions through structured conversations called where-are-we, how-far, what-direction, and where-is-there conversations. Distributed concept construction methods were used to create flexible concepts, based on a data structure called a distributed lexicon table. The lexicon was extended from words for locations, termed toponyms, to words for the relational terms of distances and directions. New toponyms were then learned using these relational operators. Effective grounding was tested by using the new toponyms as targets for go-to games, in which the robots independently navigated to named locations. The studies demonstrate how meanings can be extended from grounding in shared physical experiences to grounding in constructed cognitive experiences, giving the robots a language that refers to their direct experiences, and to constructed worlds that are beyond the here-and-now.
Heath, S., Schulz, R., Ball, D., and Wiles, J. (2012) Lingodroids: Learning terms for time, ICRA 2012, The International Conference on Robotics and Automation, Saint Paul, MN, May 2012
For humans and robots to communicate using natural language it is necessary for the robots to develop concepts and associated terms that correspond to the human use of words. Time and space are foundational concepts in human language, and to develop a set of words that correspond to human notions of time and space, it is necessary to take into account the way that they are used in natural human conversations, where terms and phrases such as 'soon', 'in a while', or 'near' are often used. We present language learning robots called Lingodroids that can learn and use simple terms for time and space. In previous work, the Lingodroids were able to learn terms for space. In this work we extend their abilities by adding temporal variables which allow them to learn terms for time. The robots build their own maps of the world and interact socially to form a shared lexicon for location and duration terms. The robots successfully use the shared lexicons to communicate places and times to meet again.
Schulz, R., Whittington, M., & Wiles, J. (2012). Language change in socially structured populations, In: 9th International Conference on the Evolution of Language, Kyoto, Japan, (312-319). World Scientific, 13 - 16 March 2012
Language contact is a significant external social factor that impacts on the change in natural languages over time. In some circumstances this corresponds to language competition, in which individuals in a population choose one language over another based on their social interactions. We investigated the dynamics of language change in two initially separate populations of agents that were then mixed with levels of influence determined by the social classes of the two populations, with 16 different combinations tested. As expected, the study found that how the communities interact with each other impacts on the communal language developed. However, it was also found that the acquisition of new words was substantial even with limited interaction between populations and low levels of influence, and that comprehension could be well established across language groups even when production of words from the other language group was low.
Schulz, R., Wyeth, G., Wiles, J. (2011). Lingodroids: Socially grounding place names in privately grounded cognitive maps, Adaptive Behavior, 19 6: 409-424
For mobile robots to communicate meaningfully about their spatial environment, they require personally constructed cognitive maps and social interactions to form languages with shared meanings. Geographic spatial concepts introduce particular problems for grounding—connecting a word to its referent in the world—because such concepts cannot be directly and solely based on sensory perceptions. In this article we investigate the grounding of geographic spatial concepts using mobile robots with cognitive maps, called Lingodroids. Languages were established through structured interactions between pairs of robots called where-are-we conversations. The robots used a novel method, termed the distributed lexicon table, to create flexible concepts. This method enabled words for locations, termed toponyms, to be grounded through experience. Their understanding of the meaning of words was demonstrated using go-to games in which the robots independently navigated to named locations. Studies in real and virtual reality worlds show that the system is effective at learning spatial language: robots learn words easily—in a single trial as children do—and the words and their meaning are sufficiently robust for use in real world tasks.
Schulz, R., Wyeth, G., Wiles, J. (2011) Are we there yet? Grounding temporal concepts in shared journeys, IEEE Transactions on Autonomous Mental Development, 3(2), 163-175
An understanding of time and temporal concepts is critical for interacting with the world and with other agents in the world. What does a robot need to know to refer to the temporal aspects of events-could a robot gain a grounded understanding of “a long journey,” or “soon?” Cognitive maps constructed by individual agents from their own journey experiences have been used for grounding spatial concepts in robot languages. In this paper, we test whether a similar methodology can be applied to learning temporal concepts and an associated lexicon to answer the question “how long” did it take to complete a journey. Using evolutionary language games for specific and generic journeys, successful communication was established for concepts based on representations of time, distance, and amount of change. The studies demonstrate that a lexicon for journey duration can be grounded using a variety of concepts. Spatial and temporal terms are not identical, but the studies show that both can be learned using similar language evolution methods, and that time, distance, and change can serve as proxies for each other under noisy conditions. Effective concepts and names for duration provide a first step towards a grounded lexicon for temporal interval logic.
Schulz, R., Glover, A., Milford, M., Wyeth, G., and Wiles, J. (2011) Lingodroids: Studies in Spatial Cognition and Language, ICRA 2011, The International Conference on Robotics and Automation, Shanghai, China, May 2011
The Lingodroids are a pair of mobile robots that evolve a language for places and relationships between places (based on distance and direction). Each robot in these studies has its own understanding of the layout of the world, based on its unique experiences and exploration of the environment. Despite having different internal representations of the world, the robots are able to develop a common lexicon for places, and then use simple sentences to explain and understand relationships between places – even places that they could not physically experience, such as areas behind closed doors. By learning the language, the robots are able to develop representations for places that are inaccessible to them, and later, when the doors are opened, use those representations to perform goal-directed behavior.
Schulz, R., Wyeth, G., Milford, and Wiles, J. (2010) Language Change across Generations for Robots using Cognitive Maps, The Twelfth International Conference on the Simulation and Synthesis of Living Systems (ALife XII), Odense, Denmark, Aug 2010
Languages change over time, as new words are invented, old words are lost through disuse, and the meanings of existing words are altered. The processes behind language change include the culture of language acquisition and the mechanisms used for language learning. We examine the effects of language acquisition and learning, in particular the length of the learning period over generations of robots. The robots form spatial concepts related to places in an environment: toponyms (place names) and simple prepositions (distances and directions). The use of spatial concepts allows us to investigate different classes of words within a single domain that provides a clear method for evaluating word use between agents. The individual words used by the agents can change rapidly through the generations depending on the learning period of the language learners. When the learning period is sufficiently long that more words are retained than invented, the lexicon becomes more stable and successful. This research demonstrates that the rate of language change depends on learning periods and concept formation, and that the language transmission bottleneck reduces the retention of words that are part of large lexicons more than words that are part of small lexicons.
Milford, M., Schulz, R., Prasser, D., Wyeth, G., Wiles, J. (2007) Learning spatial concepts from RatSLAM representations, Robotics and Autonomous Systems - From Sensors to Human Spatial Concepts, 55(5), 403-410
RatSLAM is a biologically-inspired visual SLAM and navigation system that has been shown to be effective indoors and outdoors on real robots. The spatial representation at the core of RatSLAM, the experience map, forms in a distributed fashion as the robot learns the environment. The activity in RatSLAM’s experience map possesses some geometric properties, but still does not represent the world in a human readable form. A new system, dubbed RatChat, has been introduced to enable meaningful communication with the robot. The intention is to use the “language games” paradigm to build spatial concepts that can be used as the basis for communication. This paper describes the first step in the language game experiments, showing the potential for meaningful categorization of the spatial representations in RatSLAM.