Concept Mapping Inspired by Physical Navigation

Daniel Angus

Theme 5 draws inspiration from principles of physical navigation. Insights generated by Theme 3 and 4, including how humans use landmarks to assist in executing physical navigation tasks, and the RatSLAM algorithm which is inspired by rodent hippocampal models, make for ideal starting points for investigation into concept navigation.

Theme 5 has been most interested in the development of conceptual mapping techniques which integrate spatial and temporal information. The focus of these techniques has been in trying to understand structure within conversation transcripts, particularly those that relate to speaker specific behaviours and characteristics. An important outcome from this work has been the creation of a conceptual recurrence visualisation system. The conceptual recurrence visualisation system is useful for analysis of transcribed spoken dialogue, and other input texts. The use of this technique in the determination of conversation participant behavioural metrics is the subject of an Australian Provisional Patent, and several papers are currently under review and in preparation.

Global and local map building in a conceptual space

Figure 1. Global and local map building in a conceptual space. This conceptual recurrence plot is built using a transcript of a doctor/patient consultation. Each square on the diagonal refers to a turn in the consultation with the size representing the utterance length. The patient (in red) starts the conversation by outlining various symptoms and this step can be considered as the patient providing the global map. This map is then traversed by the doctor (in blue) and the patient by filling in local details as they go (local refinement). These local refinements recur with the statements made at the beginning of the consultation as well as other statements which refer to the same local detail. At the conclusion of the consultation the doctor offers an overview of the discussion which once again returns discussion to the level of the global map.

Text-based recurrence systems have been used previously to study early-language development by children among other applications, however these systems model language at a simple text (or term-based) level, rather than modelling it at a conceptual level. By working at a conceptual level our system can model conceptual recurrence which captures implied context within a single text. In the example provided we contrast a section of an interview from the SBS Insight program “Emergency” where a mother is discussing her son’s burst appendix. In many of the utterances displayed the term appendix doesn’t appear and as such the host, Jenny Brockie, appears to not recur with the interviewee when asking the question “And the appendix had bust?”. When conceptual information is used, this statement appears to recur strongly as in reality the entire excerpt is related to this boy’s burst appendix.

Term-based Recurrence

a) Term-based Recurrence

Conceptual Recurrence

b) Conceptual Recurrence

Figure 2. Contrasting term-based (term matched) recurrence plotting versus conceptual (context aware) recurrence plotting. In this excerpt from the SBS Insight program “Emergency” a mother (Susan Candlish) is discussing her son’s burst appendix with the program host (Jenny Brockie). The addition of conceptual information allows statements that are conceptually similar to recur. This extra recurrence enables greater accuracy when calculating the degree of engagement between participants, among other behavioural metrics.

The conceptual recurrence visualisation system is currently being used for analysis of a variety of datasets from research and industry areas. Example applications include:

  • Doctor/Patient consultations: Conceptual recurrence has been used as a precursor for the development of several participant behavioural metrics. These metrics correlate with good and poor communication patterns and we have been working in collaboration with colleagues in the School of Psychology to develop protocols for assisting with the assessment of Doctor/Patient communication.
  • Childhood language development: Conceptual recurrence plots are useful in distinguishing language learning difficulties both on a qualitative and quantitative level. This work has been performed in conjunction with colleagues in the Temporal Dynamics of Learning Center in San Diego.
  • Cockpit communication analysis: Analysis of flight-deck transcripts using conceptual recurrence plots show how leadership roles and listening roles are shared by airline staff.
  • Television interviews: Conceptual recurrence plots are able to highlight complex interviewer/interviewee dynamics in a televised interview. Identification of topic engagement and repetition of key concepts can be used to highlight particular interview techniques being employed by an interviewee or interviewer.

Conceptual recurrence plot

Figure 3. Conceptual recurrence plot for a conversation between Andrew Denton and Jeff Kennett on the ABC Enough Rope television program. Kennett is indicated in blue and Denton is indicated in red. Conceptual comparisons between both Kennett and Denton are coloured grey. Dark colour indicates conceptually similar utterances, while white space indicates dissimilar utterances. Using this visualisation we can identify periods of engagement between participants and interact with the system to determine what content they are talking about in these periods. We can also see how particular concepts recur throughout an interview, as in this particular example “depression” appears to be an important concept that is mentioned multiple times.

We have also been collaborating with members of Theme 2 and 3 to study how conceptual mapping techniques can be used to inform about qualities of physical and virtual environments. This work is bridging multiple themes by exploring questions about the role of spatial and episodic memory in physical and conceptual navigation systems. Conceptual mapping algorithms can generate models that can capture qualities about an environment including how landmarks may affect the navigability of the environment.

A virtual environment

Figure 4. A virtual environment that has different landmarks at its four edges is sampled in a 9 x 9 grid pattern (left). The sampling takes panoramic images from each location and these images are placed according to their image similarity using a conceptual mapping algorithm (right). The nodes in the concept map each correspond to a physical location and each node is connected to its nearest neighbour in a Manhattan grid fashion. The shape of the conceptual map changes depending on the arrangement of objects in the environment and their appearance. The topmost map is generated from an environment that does not have any wall, whereas the bottom map is generated from an environment that includes a wall. In the topmost example the conceptual map implies that the corners appear as distinct places within the environment, in the bottommost example the corners have a higher visual similarity due to them being placed in a similar part of the conceptual map.

Ongoing Collaborative Research

Measuring complexity of real and virtual environments using conceptual mapping algorithms. Collaborator: Allen Cheung (Theme 2). This work bridges theme 2 and 5 by using conceptual mapping technologies to provide insight into the complexity of virtual environments that are used in virtual and real-world animal behaviour experimentation.

Publications

  • Angus, D., Smith, A., Wiles, J. (2012). Human Communication as Coupled Time Series: Quantifying Multi-participant Recurrence. IEEE Transactions on Audio, Speech, and Language Processing (in press).
  • Angus, D., Smith, A.E., Wiles, J. (2011) Conceptual Recurrence Plots: Revealing Patterns in Human Discourse, IEEE Transactions on Visualization and Computer Graphics (in press).
  • Angus, D., Woodward, C. (2009) Multiple Objective Ant Colony Optimisation. Swarm Intelligence, 3, 69-85.   (PDF File 426 KB)
  • Angus, D. (2009) Niching for Ant Colony Optimisation. Biologically-Inspired Optimisation Methods: Parallel Algorithms, Systems and Applications. Lewis, A.; Mostaghim, S. & Randall, M. (ed.). Springer.   (PDF File 8000 KB)
  • Angus, D., Deller, A. (2008) Computational Intelligence in Radio Astronomy: Using Computational Intelligence Techniques to Tune Geodesy Models. Simulated Evolution and Learning, 7th International Conference, SEAL08, Springer-Verlag, LNCS 5361, 615-624.   (PDF File 425 KB)
  • Angus, D. (2008) Niching Ant Colony Optimisation. PhD Thesis, Swinburne University of Technology.

Conference Abstracts or Poster

  • Angus, D., Smith, A.E., Wiles, J. (2011) Conceptual Recurrence Plots: Generating Insights into Effective Doctor-Patient Consultations, 4th International Symposium on Recurrence Plots, The Hong Kong Polytechnic University, China.
  • Lai, J., Reilly, J., Wiles, J., Angus, D., Smith, A.E. (2011) Conversational Narratives in School-Age Children With High-Functioning Autism, 2011 American Speech-Language-Hearing Association Convention, San Diego, CA.
  • Watson, B., Angus, D., Farmer, J., Wiles, J., Smith, A.E. (2011) Evaluating Effective Open Disclosure Through Visualisation: What Works and for Whom?, 61st Annual Conference of the International Communication Association, Boston.
  • Angus, D. (2008) Biologically Inspired Concept Navigation, poster presentation at the Brain Plasticity Symposium, QBI, September 2008.

Patents

  • Australian Provisional Patent Application 2010903163 “A communications analysis system and process”, Angus, D., Smith, A. E., Wiles, J. Filing date: 15 July 2010.

Related Activities

  • Invited Speaker at Health Quality and Complaints Commission Network Research Session, Brisbane, Australia. Talk title: Health Data Visualization and Data Mining. Speakers: Wiles, J., Angus, D.
  • Attended the HCSNet Summerfest 2008 at the University of New South Wales. Organised and run by the ARC Human Communication Science Network.

Seminars, Tutorials, Courses Presented

  • Presented a tutorial at the 2009 Animal Navigation Summer School held at the Queensland Brain Institute. Tutorial title: Concept Navigation.
  • Lecturer for ENGG7302: Advanced Computational Techniques in Engineering in 2009 and 2010.

Supervision of Students related to Thinking Systems

  • Andrew Jones, Large Text Corpus Project, 2009.
  • Hazem Alhakami, Investigating the Limitations of Text Similarity Algorithms With Respect to the Amount of Input Text, 2010.

Where to Next?

The Discursis interactive visual text analytic technique we developed in Theme 5 has attracted considerable interest in the social and behavioural science community. In 2012 we will continue to work with our many collaborators within the health communication and journalism communities on the analysis and understanding of human communication. Our work will provide doctors with feedback on what communication strategies are conducive to good patient outcomes in addition to insights about how neural developmental disorders such as autism affect verbal interaction. This work has been made possible due to a Vice-Chancellor's Strategic Initiative in Communication Technologies between the School of Information Technology and Electrical Engineering (ITEE), and the School of Journalism and Communication (SJC) at The University of Queensland (UQ).

 

Navigating Concept Space as a Network: Making Connections between Concepts

Paul Stockwell

The amount of literature available to be read has been increasing dramatically In recent decades. For large or complex domains, automated tools can extract the key concepts from a body of text and represent the
relationships between them in a two dimensional layout. Concept maps are a common method for visualising a domain.

This project uses the physical analogy of navigation in the context of conceptual space. The overall goal of this project is to investigate using pathways to look for probable or causal relationships between concepts or entities that are not directly related. The specific aims are to develop and test a mechanism for decomposing and representing a pathway in a textual domain as a “knowledge pathway”, which can then be used to aid in learning new concepts from a known starting point, or to determine if there is a credible link between two concepts or entities.

The method used in this project starts with extracting the key concepts from a textual corpus using content analysis, and then generating a concept map from it using Leximancer. Network graph techniques such as shortest paths create pathways within a concept map to link concepts that are not directly connected, while minimum spanning trees provide a visual framework that may assist in comprehending complex concept maps.

A series of studies have been completed during the project. The earlier studies looked at algorithmic techniques to provide visual cues and pathways. Next, more evaluative studies were performed, where the “irreducible complexity” of a corpus was investigated where the dimensional reduction of the document no longer accurately captures the underlying “story” of the original text. The current study seeks to compare the algorithmic techniques for generating concept maps against those created by humans. A small pilot study has been performed with the full study currently in final preparation.

This project has produced two publications, a patent, and has been partially commercialised by Leximancer.

Concept Map

Figure 1. Concept Map with minimum spanning tree framework shown in light grey. A “knowledge pathway” is shown in black with arrows from the concept “landmark” to “processing”.

Publications (including refereed conference papers)

Stockwell, P., Colomb, R. M., Smith, A. E., & Wiles, J. (2009) Use of an Automatic Content Analysis Tool: a Technique for seeing both Local and Global Scope. International Journal of Human Computer Studies, 67(5), 424-436.   (PDF File 888 KB)

Conference Abstracts

Stockwell, P., Smith, A. E., Wiles, J. (2009) Displaying a Framework in a Concept Map using Network Graph Techniques. In E. Banissi, et al (Eds.), Proceedings of the 13th International Conference on Information Visualisation (pp. 661-666). Barcelona, Spain: IEEE Computer Society.

Patents

Australian Provisional Patent application 2007906891. “Methods for Determining a Path”, Stockwell, P., Smith, A. E. and Wiles, J.