A text-to-image (T2I or TTI) model is a machine learning model which takes an input natural language prompt and produces an image matching that description.
Text-to-image models gradually began to be developed in the mid-2010s during the beginnings of the AI boom, as a result of advances in deep neural networks. In 2022, the output of state-of-the-art text-to-image models—such as OpenAI's DALL-E 2, Google Brain's Imagen, Stability AI's Stable Diffusion, Midjourney, and Runway's Gen-4—began to be considered to approach the quality of real photographs and human-drawn art.
Text-to-image models are generally latent diffusion models, which perform the diffusion process in a compressed latent space rather than directly in pixel space. An autoencoder (often a variational autoencoder) is used to convert between pixel space and this latent representation. These systems typically use a pretrained language or vision–language model to convert the input prompt into a text embedding, and a diffusion-based generative image model that produces images conditioned on that embedding. The most effective models have generally been trained on massive amounts of image and text data scraped from the web. The output of text-to-image models frequently constitutes AI slop, mass-produced content for online engagement at the expense of quality.
In August 2023, a United States district court ruled that AI art is ineligible for copyright due to failure to meet human authorship. In March 2026, it declined to hear a case over whether AI-generated art can be subject to copyright.
History
Before the rise of deep learning in the 2010s, attempts to build text-to-image models were limited to collages by arranging existing component images, such as from a database of clip art.
The inverse task, image captioning, was more tractable, and a number of image captioning deep learning models came prior to the first text-to-image models.
Developments before 2015
Deep learning, characterized by its multi-layer structure that attempts to mimic the human brain, first came about in the 2010s, causing a significant shift in the world of AI art. During the deep learning era, there are mainly these types of designs for generative art: autoregressive models, diffusion models, GANs, normalizing flows.
In 2014, Ian Goodfellow and colleagues at Université de Montréal developed the generative adversarial network (GAN), a type of deep neural network capable of learning to mimic the statistical distribution of input data such as images. The GAN uses a "generator" to create new images and a "discriminator" to decide which created images are considered successful. Unlike previous algorithmic art that followed hand-coded rules, generative adversarial networks could learn a specific aesthetic by analyzing a dataset of example images.
In 2015, a team at Google released DeepDream, a program that uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia. The process creates deliberately over-processed images with a dream-like appearance reminiscent of a psychedelic experience. Later, in 2017, a conditional GAN learned to generate 1000 image classes of ImageNet, a large visual database designed for use in visual object recognition software research. By conditioning the GAN on both random noise and a specific class label, this approach enhanced the quality of image synthesis for class-conditional models.
Autoregressive models were used for image generation, such as PixelRNN (2016), which autoregressively generates one pixel after another with a recurrent neural network. Immediately after the Transformer architecture was proposed in Attention Is All You Need (2018), it was used for autoregressive generation of images, but without text conditioning.
The website Artbreeder, launched in 2018, uses the models StyleGAN and BigGAN to allow users to generate and modify images such as faces, landscapes, and paintings.
2015–2019
The first modern text-to-image model, alignDRAW, was introduced in 2015 by researchers from the University of Toronto. alignDRAW extended the previously introduced DRAW architecture (which used a recurrent variational autoencoder with an attention mechanism) to be conditioned on text sequences. Images generated by alignDRAW were in small resolution (32×32 pixels, attained from resizing) and were considered to be 'low in diversity'. The model was able to generalize to objects not represented in the training data (such as a red school bus) and appropriately handled novel prompts such as "a stop sign is flying in blue skies", exhibiting output that it was not merely "memorizing" data from the training set.
In 2016, Reed, Akata, Yan et al. became the first to use generative adversarial networks for the text-to-image task. With models trained on narrow, domain-specific datasets, they were able to generate "visually plausible" images of birds and flowers from text captions like "an all black bird with a distinct thick, rounded bill". A model trained on the more diverse COCO (Common Objects in Context) dataset produced images which were "from a distance... encouraging", but which lacked coherence in their details. Later systems include VQGAN-CLIP, XMC-GAN, and GauGAN2.
2020s
In the 2020s, text-to-image models, which generate images based on prompts, became widely used, marking yet another shift in the creation of AI-generated artworks.
In 2021, using the influential large language generative pre-trained transformer models that are used in GPT-2 and GPT-3, OpenAI released a series of images created with the text-to-image AI model DALL-E 1. It is an autoregressive generative model with essentially the same architecture as GPT-3. Along with this, later in 2021, EleutherAI released the open source VQGAN-CLIP based on OpenAI's CLIP model. Diffusion models, generative models used to create synthetic data based on existing data, were first proposed in 2015, but they only became better than GANs in early 2021. Latent diffusion model was published in December 2021 and became the basis for the later Stable Diffusion (August 2022), developed through a collaboration between Stability AI, CompVis Group at LMU Munich, and Runway.
In 2022, Midjourney was released, followed by Google Brain's Imagen and Parti, which were announced in May 2022, Microsoft's NUWA-Infinity, and the source-available Stable Diffusion, which was released in August 2022. In August 2022, text-to-image personalization allows to teach the model a new concept using a small set of images of a new object that was not included in the training set of the text-to-image foundation model. This is achieved by textual inversion, namely, finding a new text term that correspond to these images. DALL-E 2, a successor to DALL-E, was beta-tested and released (with the further successor DALL-E 3 being released in 2023). Stability AI has a Stable Diffusion web interface called DreamStudio, plugins for Krita, Photoshop, Blender, and GIMP, and the Automatic1111 web-based open source user interface. Stable Diffusion's main pre-trained model is shared on the Hugging Face Hub.
Ideogram was released in August 2023, this model is known for its ability to generate legible text.
In 2024, Flux was released. This model can generate realistic images and was integrated into Grok, the chatbot used on X (formerly Twitter), and Le Chat, the chatbot of Mistral AI. Flux was developed by Black Forest Labs, founded by the researchers behind Stable Diffusion. Grok later switched to its own text-to-image model Aurora in December of the same year. Several companies, along with their products, have also developed an AI model integrated with an image editing service. Adobe has released and integrated the AI model Firefly into Premiere Pro, Photoshop, and Illustrator. Microsoft has also publicly announced AI image-generator features for Microsoft Paint. Along with this, some examples of text-to-video models of the mid-2020s are Runway's Gen-4, Google's VideoPoet, OpenAI's Sora, which was released in December 2024, and LTX-2 which was released in 2025. Make-A-Video, Imagen Video, Midjourney, and Phenaki can also generate video from text and/or text/image prompts.
In 2025, several models were released. GPT Image 1 from OpenAI, launched in March 2025, introduced new text rendering and multimodal capabilities, enabling image generation from diverse inputs like sketches and text. MidJourney v7 debuted in April 2025, providing improved text prompt processing. In May 2025, Flux.1 Kontext by Black Forest Labs emerged as an efficient model for high-fidelity image generation, while Google's Imagen 4 was released with improved photorealism. Flux.2 debuted in November 2025 with improved image reference, typography, and prompt understanding.
Architecture and training
Text-to-image models have been built using a variety of architectures. The text encoding step may be performed with a recurrent neural network such as a long short-term memory (LSTM) network, though transformer models have since become a more popular option. For the image generation step, conditional generative adversarial networks (GANs) was once widely used in early days. Since 2020, diffusion models have become the popular option. Rather than directly training a model to output a high-resolution image conditioned on a text embedding, a popular technique is to train a model to generate low-resolution images or latent space, and use one or more auxiliary deep learning models to upscale or decode it, filling in finer details.
Text-to-image models are trained on large datasets of (text, image) pairs, often scraped from the web. With their 2022 Imagen model, Google Brain reported positive results from using a large language model trained separately on a text-only corpus (with its weights subsequently frozen), a departure from the theretofore standard approach.
Datasets
Training a text-to-image model requires a dataset of images paired with text captions. One dataset commonly used for this purpose is the COCO dataset. Released by Microsoft in 2014, COCO consists of around 123,000 images depicting a diversity of objects with five captions per image, generated by human annotators. Originally, the main focus of COCO was on the recognition of objects and scenes in images. Oxford-120 Flowers and CUB-200 Birds are smaller datasets of around 10,000 images each, restricted to flowers and birds, respectively. It is considered less difficult to train a high-quality text-to-image model with these datasets because of their narrow range of subject matter.
One of the largest open datasets for training text-to-image models is LAION-5B, containing more than 5 billion image-text pairs. This dataset was created using web scraping and automatic filtering based on similarity to high-quality artwork and professional photographs. Because of this, however, it also contains controversial content, which has led to discussions about the ethics of its use.
Some modern AI platforms not only generate images from text but also create synthetic datasets to improve model training and fine-tuning. These datasets help avoid copyright issues and expand the diversity of training data.
Quality evaluation
Evaluating and comparing the quality of text-to-image models is a problem involving assessing multiple desirable properties. A desideratum specific to text-to-image models is that generated images semantically align with the text captions used to generate them. A number of schemes have been devised for assessing these qualities, some automated and others based on human judgement.
A common algorithmic metric for assessing image quality and diversity is the Inception Score (IS), which is based on the distribution of labels predicted by a pretrained Inceptionv3 image classification model when applied to a sample of images generated by the text-to-image model. The score is increased when the image classification model predicts a single label with high probability, a scheme intended to favour "distinct" generated images. Another popular metric is the related Fréchet inception distance, which compares the distribution of generated images and real training images according to features extracted by one of the final layers of a pretrained image classification model.
Impact and applications
Tools and processes
Approaches
When text-to-image is used, AI generates images based on textual descriptions, using models like diffusion or transformer-based architectures. Users input prompts and the AI produces corresponding visuals. When image-to-image is used, AI transforms an input image into a new style or form based on a prompt or style reference, such as turning a sketch into a photorealistic image or applying an artistic style. When image-to-video is used, AI generates short video clips or animations from a single image or a sequence of images, often adding motion or transitions. This can include animating still portraits or creating dynamic scenes.When text-to-video is used, AI creates videos directly from text prompts, producing animations, realistic scenes, or abstract visuals. This is an extension of text-to-image but focuses on temporal sequences.
Imagery
There are many tools available to people when working with diffusion models. They can define both positive and negative prompts, but they are also afforded a choice in using (or omitting the use of) VAEs, LoRAs, hypernetworks, IP-adapter, and embedding/textual inversions. Artists can tweak settings like guidance scale (which balances creativity and accuracy), seed (to control randomness), and upscalers (to enhance image resolution), among others. Additional influence can be exerted during pre-inference by means of noise manipulation, while traditional post-processing techniques are frequently used post-inference. People can also train their own models.
In addition, procedural "rule-based" image generation techniques have been developed, utilizing mathematical patterns, algorithms that simulate brush strokes and other painterly effects, as well as deep learning models such as generative adversarial networks (GANs) and transformers. Several companies have released applications and websites that allow users to focus exclusively on positive prompts, bypassing the need for manual configuration of other parameters. There are also programs capable of transforming photographs into stylized images that mimic the aesthetics of well-known painting styles.
There are many options, ranging from simple consumer-facing mobile apps to Jupyter notebooks and web UIs that require powerful GPUs to run effectively. Additional functionalities include "textual inversion," which refers to enabling the use of user-provided concepts (like an object or a style) learned from a few images. Novel art can then be generated from the associated word(s) (the text that has been assigned to the learned, often abstract, concept) and model extensions or fine-tuning (such as DreamBooth).
Prompt engineering and sharing
Prompts for some text-to-image models can also include images and keywords and configurable parameters, such as artistic style, which is often used via keyphrases like "in the style of [name of an artist]" in the prompt /or selection of a broad aesthetic/art style. There are platforms for sharing, trading, searching, forking/refining, or collaborating on prompts for generating specific imagery from image generators. Prompts are often shared along with images on image-sharing websites such as Reddit and AI art-dedicated websites. A prompt is not the complete input needed for the generation of an image; additional inputs that determine the generated image include the output resolution, random seed, and random sampling parameters.
Related terminology
Synthetic media, which includes AI images, was described in 2022 as a major technology-driven trend that will affect business in the coming years. Harvard Kennedy School researchers voiced concerns about synthetic media serving as a vector for political misinformation soon after studying the proliferation of AI-generated media on the X platform. Synthography is a proposed term for the practice of generating images that are similar to photographs using AI.
List of notable text-to-image models
Criticism
Despite adoption in certain circles, AI image generation remains artistically and ethically controversial. Notably, many, if not most, images generated by AI are typically considered to be "slop", lacking in any of the genuine cultural value indicative of legitimate works of art. Attempts to use AI in established creative spaces have been met with public and professional backlash in favor of the legitimacy and authenticity offered by human art.
AI visual content also inherits many of the criticisms targeted at wider usage of generative AI, such as reliance on data centers or use for exploitative or non-consensual AI pornography.
Copyright status
In the 2020s, the rapid advancement of deep learning-based generative AI models raised questions about the copyright status of AI-generated works, and about whether copyright infringement occurs when such are trained or used. This includes text-to-image models such as Stable Diffusion and large language models such as ChatGPT. State of the art large language models are trained on all the data which can be scraped from the Internet, often utilizing copyrighted material.As of 2023, there were several pending U.S. lawsuits challenging the use of copyrighted data to train AI models, with defendants arguing that this falls under fair use.
Explanatory notes
See also
Artificial intelligence art
Text-to-video model
AI slop
References
