Examining Descriptiveness of Narratives Generated using Planning and Large Language Models

Narrative generation is an important field due to the high demand for stories in video game design and also in stories used in learning tools in the classroom. As these stories should contain depth, it is desired for these stories to ideally be more descriptive. There are tools that help with the creation of these stories, such as planning, which requires a domain as input, or GPT-3, which requires an input prompt to generate the stories. However, other aspects to consider are the coherence and variation of stories. To save time and effort and create multiple possible stories, we combined both planning and the Large Language Model (LLM) GPT-3 similar to how they were used in TattleTale to generate such stories while examining whether descriptive input prompts to GPT-3 affect the outputted stories. The stories generated are readable to the general public and overall, the prompts do not consistently affect descriptiveness of outputs across all stories tested. For this work, three stories with three variants each were created and tested for descriptiveness. To do so, adjectives, adverbs, prepositional phrases, and suboordinating conjunctions were counted using Natural Language Processing (NLP) tool spaCy for Part Of Speech (POS) tagging. This work has shown that descriptiveness is highly correlated with the amount of words in the story in general, so running GPT-3 to obtain longer stories is a feasible option to consider in order to obtain more descriptive stories. The limitations of GPT-3 have an impact on the descriptiveness of resulting stories due to GPT-3’s inconsistency and transformer architecture, and other methods of narrative generation such as simple planning could be more useful.