Research confirms it: in image generation, AI creativity is a myth. Although current generative AI models may indeed appear autonomous creative agents, when deconstructing their imaginative process, their lack of true creative abilities becomes apparent. This is the conclusion reached by a new study, published in the scientific journal Advanced Science, led by an international team of researchers from the Brain Cognition and Brain Plasticity group at IDIBELL – UB (Institute of Neurosciences), the Computer Vision Center (CVC)-UAB and the Vienna Cognitive Science Hub. The study, focused on visual creativity and imagination, was born in 2024 during a workshop organized by the Fundació Èpica – La Fura dels Baus, whose work focuses on promoting interdisciplinary collaborations between science, technology, and arts.
As a result of this workshop, the researchers articulated an innovative methodology to study creativity: they prepared a visual-creative imagination task based on abstract stimuli and compared the creative performance of an AI image generation model, with and without human guidance, with that of two groups of people: visual artists and general population. To reproduce a visually comparable drawing style, the generative model was trained using the human participants’ creative productions, and was later tested under two conditions: with and without human guidance, by employing elaborate, concrete prompts in the former case, and more basic ones in the latter. Abstract stimuli and resulting drawings can be seen in Figure 1.
Figure 1. Examples of the Imagination Task’s abstract stimuli and resulting creative images from the four experimental categories (Visual Artists, Non-Artists, Human-Inspired GenAI and Self-Guided GenAI), along with their respective creativity ratings (1-7).
Unanimity among evaluators: human productions are more creative
A group of people and two AIs were in charge of evaluating the level of creativity of the drawings according to 5 different criteria: likeness (to what extent they liked drawing), liveliness, originality, aesthetics and curiosity. In all of them, the results showed a clear gradient: visual artists were rated as most creative, followed by general population and the human-guided AI model, and lastly, in a strong disadvantage, the unguided AI model. “Although the AI model was trained with the creative productions of human participants, it showed poor performance in the production of creative images and, in fact, it did even worse when deprived of human guidance”, explains Dr. Xim Cerdá-Company, researcher at IDIBELL and the CVC/UAB, and co-leader of the study. The scores can be seen in Figure 2.
Figura 2. Examples of drawings from each category, in rows: Visual Artists, Non-Artists, Human-Inspired AI, Self-Guided AI. The values reported correspond to the drawing’s Creativity score by humans, GPT-4o, and Guided-GPT-4o raters, respectively.
Studying creativity as a process, not just for results
For the research team, this study’s contributions to both the fields of AI and cognitive science are manifold. Firstly, it highlights the necessity of employing a diverse range of measures and models when investigating a process as complex and multi-faceted as creativity. “Currently, AI creativity is valued almost exclusively according to verbal creativity tasks, leading to biased results, and even presenting AI models as a creative agents whose performance surpassed that of most humans. With a different approach, directly evaluating the imaginative process from ideation to execution, we have shown that this is not true”, Dr. Cerdá-Company continues. “Creativity must be studied as a process, not just focus on its results”, he adds.
Secondly, by showing that as human guidance decreased, so did the creativity of the models, the study highlighted “that current image-generation models are still far from reproducing independent creative processes”, says Dr. Antoni Rodríguez-Fornells, head of the Cognition and Brain Plasticity group at IDIBELL-UB, ICREA researcher, and co-leader of the study. This highlights the fundamental need for human intervention in multiple stages of the AI creative process, from training to idea generation. “The technical image-generation AI abilities cannot be evaluated solo; the creative process must be explored in its many components. In doing so, it becomes clear that AI depends directly on our intervention,” concludes Dr. Rodríguez-Fornells.
About IDIBELL
The Bellvitge Biomedical Research Institute (IDIBELL) is a research center established in 2004 specialized in cancer, neuroscience, translational medicine, and regenerative medicine. It counts on a team of more than 1.500 professionals who, from 73 research groups, publish more than 1.400 scientific articles per year. IDIBELL is participated by the Bellvitge University Hospital and the Viladecans Hospital of the Catalan Institute of Health, the Catalan Institute of Oncology, the University of Barcelona, and the City Council of L’Hospitalet de Llobregat.
IDIBELL is a member of the Campus of International Excellence of the University of Barcelona HUBc and is part of the CERCA institution of the Generalitat de Catalunya. In 2009 it became one of the first five Spanish research centers accredited as a health research institute by the Carlos III Health Institute. In addition, it is part of the “HR Excellence in Research” program of the European Union and is a member of EATRIS and REGIC. Since 2018, IDIBELL has been an Accredited Center of the AECC Scientific Foundation (FCAECC).