CogView4Translation site

What is CogView4

CogView4 is the latest release of the open-source Vincennes graph model from Smart Spectrum AI, marking the first release in theAI image generationAnother major breakthrough in the field. The model not only supports bilingual cue word input, but also has strong complex semantic alignment and command following capabilities, and is able to generate high-quality, high-resolution images and accurately incorporate Chinese characters into the picture, greatly expanding the application scenarios of AI-generated content.CogView4 ranked first in the DPG-Bench benchmark test in terms of overall score, making it the current open source text-generated graphical model's SOTA (state-of-the-art).

CogView4 Main Features

1. Support Chinese and English prompt word input：CogView4的文本编码器升级为GLM-4，全面支持中英文输入，打破了此前open source model仅支持英文的局限，使得中文内容创作者能够更直接、更准确地描述自己的创意需求。
2. Generate high quality images: CogView4 uses advanced diffusion modeling and parametric linear dynamic noise planning, combined with mixed-resolution training techniques, to generate high-quality, high-resolution images that meet users' needs in different scenarios.
3. Arbitrary length cue word supportThe model abandons the traditional fixed-length design and adopts a dynamic text-length scheme, which supports the input of prompt words of any length, enabling users to express their creativity more freely.
4. Powerful text generation: CogView4 is the first open-source wen sheng map model that can generate Chinese characters in the screen, and can accurately incorporate Chinese characters into the generated images, providing more possibilities for advertisements, short videos, creative design and other fields.

CogView4 Technology Principle

1. Text Encoder Upgrade: CogView4 upgrades the text encoder from the English-only T5 encoder to the bilingual-capable GLM-4 encoder, enabling the model to support bilingual input.
2. Mixed Resolution Training: The model adopts a mixed-resolution training technique, combining 2D rotational position coding and interpolated position representation, adapting to different size requirements and supporting the generation of images of arbitrary resolution.
3. Diffusion Generation Modeling: CogView4 is based on a Flow-matching diffusion model and parameterized linear dynamic noise planning forImage Generation, to accommodate the signal-to-noise ratio requirements of different resolution images and further enhance the quality and diversity of the generated images.
4. Multi-stage training strategy: The model is trained using a multi-stage training strategy, including base resolution training, pan-resolution training, high-quality data fine-tuning, and human preference alignment training, to ensure that the generated images are highly aesthetically pleasing and conform to human preferences.

CogView4 Usage Scenarios

1. advertising design: CogView4 is capable of generating high-quality posters, advertising graphics, etc. based on creative descriptions to meet the diverse needs of advertising design.
2. Short video production: For short video creators, CogView4 can generate corresponding screens based on scripts or creative descriptions, improving the efficiency and quality of short video production.
3. art: Artists and designers can utilize CogView4 to generate images with specific styles and moods that inspire creativity and aid in the creation of artwork.
4. Education: Teachers can use CogView4 to generate images related to the teaching content, such as mood maps of ancient poems, scene maps of historical events, etc., to enhance the interest and intuition of teaching.
5. game development: Game developers can utilize CogView4 to generate game graphics and character images according to the game plot and character settings, improving the efficiency and quality of game development.

CogView4 project address

GitHub project address:https://github.com/THUDM/CogView4
HuggingFace Experience Address:https://huggingface.co/spaces/THUDM-HF-SPACE/CogView4