KolorsTranslation site

Background and purpose of the project

Racer's open source Kolors Kolors project aims to advance AI technology in the field of art creation and image generation by providing powerful image generation capabilities. The project is not only a contribution to the technology community, but also a bold push for creative freedom, demonstrating Racer's determination and strength in AI technology.

Project Features and Benefits

1. Bilingual comprehension and generative skills::
   • Kolors Kolors supports bilingual prompt words in English and Chinese, and carries the Generalized Language Model (GLM) as a text encoder, which is capable of understanding and generating both English and Chinese texts, providing creators with a wider creative space.
   • In particular, the processing is optimized for Chinese cultural elements, which makes the generated images closer to Chinese cultural characteristics and meets the localization needs.
2. Long text processing capability::
   • Support for context lengths of up to 256 tokens allows creators to portray what's on their mind, whether it's a complex scene or a rich story, with precision.
3. Massive data training::
   • Trained on billions of text-image pairs, the model has a large knowledge base and is able to generate diverse and accurate images.
4. High quality image generation::
   • Focusing on improving the quality of generation of realistic portraits, artistic styles and complex scenes, the images generated are significantly improved in terms of clarity, detail richness and semantic accuracy.
5. Optimization of Chinese cultural elements::
   • Optimized for Chinese cultural elements in particular, natural landscapes with Chinese characteristics such as the Great Wall and ink landscape paintings, as well as scenes with Chinese cultural symbolism such as ancient streets and the image of the dragon, are accurately reproduced in the images.
6. Chinese Text Generation::
   • Can embed Chinese text in the generated image to add more expression to the image, supports the generation of Chinese fonts and calligraphy.

Technical Architecture and Realization

1. model architecture::
   • Cortu Kolors is based on the SDXL model architecture and incorporates ChatGLM256 technology to enhance bilingual comprehension and text generation.
   • The U-Net structure is used as the backbone model and text encoding is performed through ChatGLM for text-to-image generation.
2. Training Strategies::
   • The training is divided into two phases: a conceptual learning phase and a quality improvement phase.
     • The conceptual learning phase acquires comprehensive knowledge and concepts from large-scale text-image pairs.
     • The quality improvement phase uses millions of pieces of high-quality data selected by machines + humans for training to improve image quality.
   • Introducing a new noise scheduling method to optimize high-resolution image generation.
3. Data sets and assessments::
   • Training was performed using both public datasets (e.g., LAION DataComp, JourneyDB) and proprietary datasets.
   • A category-balanced benchmark dataset, KolorsPrompts, is proposed to guide the training and evaluation of Kolors.

Applications & Experiences

1. AI image creation::
   • Users can generate paintings in a variety of styles and with beautiful quality by entering creative text descriptions.
   • Provide a variety of style templates for users to choose from, to meet different aesthetic needs.
2. AI image customization::
   • Users can upload their own photos and choose different art styles for image customization to generate personalized portraits.
3. Interactive play::
   • In the Racer App, Kolors also supports interactive play such as AI play reviews to increase user engagement and fun.

Open Source Information and Resources

• open source link::
  • Code open source links:https://github.com/Kwai-Kolors/Kolors
  • Model open source links:https://modelscope.cn/models/Kwai-Kolors/Kolors
  • Link to technical report:https://github.com/Kwai-Kolors/Kolors/blob/master/imgs/Kolors_paper.pdf
• Experience the environment::
  • Users can run and experience Kolors models by building ComfyUI environments on platforms such as the Magic Hitch Community.

As the open source image generation model project of Racer, Kolors Kolors excels in bilingual comprehension, long text processing, and high-quality image generation, providing powerful technical support for AI image creation and image customization. Its open source program and rich resources enable more creators and researchers to participate in this field and jointly promote the development and application of AI technology.