Innovating Narratives: Teaching machines to adapt using narrative structure
Imagine if you could only communicate by pointing to the words in a dictionary, but then needed to warn about a danger that had not been listed? This analogy indicates why computers struggle to support humans in unexpected situations, such as disaster relief or hazardous workplaces. My research transports information structures from narrative into scientific models, so that artificial intelligence can adapt the knowledge in its machine ‘dictionary’ (ontology) so that new emerging risks can be anticipated and communicated. Narrative has unique qualities that scientific information formats do not, which enable this. I developed new knowledge representation techniques to demonstrate these new operations, first animating them in Keynote and later building a virtual reality modelling platform to demonstrate them. The goal is to model decision-making processes that can adapt in the real world, so that machines to better support us when we need them most, in crisis and unexpected environments.
Queensland leads autonomous systems research, as the home to organisations such as the Australian Research Centre for Aerospace Automation, Reefworks and Boeing’s Loyal Wingman. Drones in Queensland’s reefs and skies would benefit from artificial intelligence that can recognise unexpected entities or collaborate more effectively as a swarm. As a Research Fellow of Trusted Autonomous Systems Defence CRC (TAS), I model knowledge structures from narrative that can enable autonomous systems to perform these adaptations. A goal is to develop the next generation of automated decision-making support tools for Australian industry and defence. A longer goal is to push the frontier of knowledge in science, by including information structures used in the creative arts, and building interdisciplinary teams that can implement them.
At the heart of my research is the representation of contextual change in stories, which I demonstrate by extending the knowledge representation methods on which artificial intelligence models are based. Those structures are presented in animated 2D and 3D formats, including virtual and augmented reality. These formats needed to be developed from scratch. I built a prototype knowledge modelling tool in Unity 3D in collaboration with the Virginia Modelling Analytics and Simulation Centre, and have worked with Disney Research, the University of California Los Angeles and the Eastern Virginia Medical School to apply my approach to multi-system visualisation. For the method itself, I hold a US patent with another pending.
As a TAS Research Fellow in Brisbane, I work with industry partners to address the problem of detecting emergent risk in workplaces. With Downer’s team for risk prevention, I am developing a new approach to automated risk modelling to make it possible to track how changing conditions, unexpected events and even corporate culture can influence accident prevention.
New visualisations of knowledge structures ultimately make it possible to detect new dangers and opportunities. In collaboration with CAE and Griffith University, I have designed a new visual system for CAE’s augmented reality platform, Asgard. This system allows a user to manipulate physical environments as they hover in mid-air, like the holograms in Star Wars. Current platforms like this only represent tangible assets such as equipment. My research introduces ways to “see the invisible”, depicting chains of symbols over a landscape so users can see the cumulative influence of activities and assets over time – the ‘story’.
My unusual fusion of science and creative writing has given me additional techniques to drive innovation. At a State Government level, I advised Queensland’s Acting Director of General Innovation, and the Innovation Advisory Council, on their ‘Strategy for Storytelling about Innovation’ which aims to increase innovation in Queensland’s business and education sectors. I participated in the QLD AI Hub as an invited speaker (the public is invited to these sessions) and facilitated interdisciplinary discussion within the broader TAS community, in a workshop I organized in Brisbane in June 2021. My hope is that my cross-disciplinary expertise can provide more powerful assistance to decision-makers as well as include a broader spectrum of disciplines in STEM research.
I want women and girls to know that the skills that could make them feel alien from traditional STEM domains are probably the insights that are most needed by those fields
My specialised knowledge is anchored in the creative arts. I performed in theatre and won awards for my short stories. When I was in high school, my style of intelligence seemed to be a barrier to participating in STEM fields. However, after working in numerous interdisciplinary projects in the United States and Australia, that background now contributes valuable insights to defence and industry projects. Innovation depends on developing novel ideas, and by definition, the creative arts contain information structures that are misaligned with the sciences. My path enabled me to see that within every perspective that seems incompatible with STEM, there are unheard insights that could address some of its longest-standing problems. It is therefore important for those who might think they would not be welcome in STEM to see role models who demonstrate that they are critically needed.
I thus actively share my odd career trajectory in schools, public talks, conferences and even publications. My goal is to create more scope for creative invention in both scientific and artistic fields, through the example of my work and the interdisciplinary collaboration techniques I develop.
Bringing diverse knowledge into the sciences will become increasingly important to game-changing research as well as community wellbeing. Crises such as covid or extreme climate events impact multiple domains: social, economic, health, environment, supply chain. These disruptions cannot be managed within the lane of a single discipline. A new paradigm is emerging in which researchers must not only master their own field, but be adaptive enough to reimagine the world in collaboration with experts from numerous fields. I represent the frontier of this new paradigm, creating research within a space that did not exist before I arrived.
Due to the pioneering nature of this research, I needed to develop new collaboration techniques to bring teams of other researchers into it. I also needed to develop a paradigm for leadership that did not depend on disciplinary status or peer rank. In both cases, I leverage my understanding of narrative construction, building a ‘shared story’ among collaborators. I have written several chapters for the Australian and US defence communities on these methods and in the process, demonstrate that a leader is the person who can tell the story to the best inclusion of a project’s goals and every team member’s motivations. As a young woman leading communities of established male scientists, I became credible by becoming the storyteller for that tribe: the person who developed the project’s vision and channeled everyone’s interests into it.
A perspective that is not usually part of the traditional research community can become the most compelling and powerful. My rogue approach has enabled me to add a new dimension to STEM, challenging science to broaden its scope to include real-world ambiguities.
STEM did not seem accessible to me when I was in high school. One of the first groups I reached out to was teenage girls in 2016 and 2017, when I was involved in the STEM4Girls workshop in Virginia Beach (organized by the American Association of University Women - AAUW). With this group, I also participated in a career day to represent my cross-over from stories to science. During the STEM4girls workshop, I noticed that sometimes the invited presenters did not make it easy for the teenagers to imagine themselves performing their jobs because they used a lot of jargon and sometimes their exercises were too boring. As part of the committee, I encouraged the presenters to speak about what makes them passionate about their research. Even now, when I teach, I tell the students about the unsolved or challenging aspects of my research, so they can stand on the precipice of new knowledge.
With younger students, I encourage their sense of adventure. In 2019 and 2020 I engaged with two local primary schools in Virginia Beach (United States) as an extension of my participation in a conference in Kanazawa, Japan (the International Society for the Interdisciplinary Study of Symmetry, for which I am on the advisory board). At one of these schools, the children wanted to make their own designs after I’d described the work I’d presented, on the topic of folded and reflected information structures. I included their designs as a slide in my presentation in Japan, then returned to the students to show the video, so they could see that their work had been seen on the other side of the world. I also brought mathematical objects and puzzles from the conference for them to play with. This enabled the young people in that class to see a young woman from their local area connecting their community to academic experts in Japan via geometric symmetry.
Young women who are entering the tech industry stand at the coalface of the economic and social issues that challenge women in STEM. As a consequence, I participated as a speaker at the Young Women in AI conference held at Griffith University in 2019. At that conference, I decided to talk frankly about my own issues with harassment and the challenges of leading a team when you aren’t in a traditional position of power, and gave examples of how I handled these problems. The audience connected emotionally and my 15 minute presentation became 90 minutes long, as the room became an open forum to discuss and debate how to manage these issues day to day, in a career.