OpenAI has officially integrated groundbreaking image generation capabilities directly into ChatGPT as of today, a feature aptly named "Images in ChatGPT." Users can now leverage the advanced GPT-4o model to create images seamlessly within the ChatGPT environment. This initial rollout is dedicated solely to image creation and is available across various subscription tiers, including ChatGPT Plus, Pro, Team, and Free versions.
According to spokesperson Taya Christianson, the usage limit for the free tier aligns with that of DALL-E, although she did not disclose specific numbers. This limitation may evolve based on user demand over time. Previously, free users were allowed to generate “three images per day with DALL·E 3.” For enthusiasts of DALL-E, Christianson confirmed that access will continue through a custom GPT, ensuring that loyal users can still enjoy the tool.
Gabriel Goh, the research lead, highlighted that this new model represents a significant advancement over previous iterations. Utilizing the GPT-4o framework, which is "omnimodal," the system can generate various types of data, including text, images, audio, and video. One of the key improvements noted by Goh is the concept of “binding.” This refers to the AI's ability to maintain accurate relationships between different attributes and objects within an image. For instance, while many image models struggle with accuracy when rendering multiple items, this new tool can effectively bind attributes for 15 to 20 objects, showcasing a remarkable enhancement in both accuracy and reliability.
One of the standout features of this new image generation tool is its improved text rendering capabilities. Goh explained that generating coherent text without errors has been a significant challenge in existing tools. Poorly rendered text can render an entire image unusable, making this enhancement crucial. The development team invested many months iterating on this feature. Although not flawless, the text quality has reached a point where it is consistently usable, particularly in larger titles or text elements.
The new system employs an autoregressive approach, generating images sequentially from left to right and top to bottom, mirroring the way text is written. This differs from the diffusion model technique used by most image generators, including DALL-E, which create entire images in one go. Goh speculates that this foundational change contributes to the improved text rendering and binding capabilities of "Images in ChatGPT."
Before the launch, the development team showcased the system's capabilities through various examples, including scientific diagrams like Newton’s prism experiment with accurately labeled components, multi-panel comics featuring consistent characters and text bubbles, and informational posters with precise text. They also highlighted practical uses, such as creating transparent background images for stickers, restaurant menus, and logos.
Jackie Shannon, the multimodal product lead at ChatGPT, articulated how the model incorporates vast world knowledge. This integration means that users can request specific images, like Newton’s prism experiment, without needing to provide extensive explanations. It allows for a more intuitive interaction, enhancing the overall user experience.
While the new image generation process might take longer than previous versions, OpenAI believes this is a worthwhile tradeoff. Shannon noted, “While we certainly have room to improve on latency, the quality of these images, the capability, and the world knowledge truly compensate for the additional seconds users may spend waiting.”
In light of past controversies involving AI-generated content, the OpenAI team has emphasized the robust safeguards integrated into the new system. Shannon assured users that the tool includes measures to prevent watermark removal, block the generation of sexual deepfakes, and refuse requests for child sexual abuse material (CSAM). Although the system does not feature visual watermarks or indicators to denote AI-generated images, all generated images will contain standard C2PA metadata, marking them as creations of OpenAI. The company is also developing internal tools to monitor generated images.
Shannon concluded by stating, “Ultimately, no system is perfect for this type of thing, but we’re continuously improving our safeguards, and we view this as a starting point.” She emphasized that all images generated from ChatGPT are owned by the user, who can use them freely within the bounds of OpenAI's usage policies.
In summary, the launch of "Images in ChatGPT" marks a significant milestone in AI image generation, offering enhanced capabilities and user-friendly features across multiple subscription tiers. The integration of robust safeguards further underscores OpenAI's commitment to responsible AI development.
