Microsoft launches Visual ChatGPT; What is it and how it works, check all details here

Microsoft has released Visual ChatGPT, a system that includes multiple visual foundation models and allows users to communicate with ChatGPT via graphical user interfaces.

The Visual ChatGPT combines various visual foundation models, such as Transformers, ControlNet, and Stable Diffusion, with ChatGPT. With the use of these visual foundation models, ChatGPT can now handle user requests for image generation and editing.

ChatGPT has recently gained popularity due to its ability to respond in a human-like manner. It currently only responds in text format, which means it cannot process, generate, or edit images.


ChatGPT has caught the attention of interdisciplinary researchers because of its extraordinary conversational competence and reasoning abilities in a variety of fields, which makes it an ideal choice for a language interface. However, its linguistic learning restricts it from processing or generating visual images.

Meanwhile, visual foundation models such as Visual Transformers and Stable Diffusion excel in tasks with one-round corrected inputs and outputs, demonstrating exceptional visual comprehension and generating abilities. Combining these two models can result in a new model, such as Visual ChatGPT, that can process and generate visual inputs.

However, the researchers discovered areas of concern in their work, such as inconsistent generation results caused by the failure of visual foundation models (VFMs) and prompt variability.

They came to the conclusion that a self-correcting module is required to ensure that execution outputs match human intentions and to make necessary adjustments. Incorporating such a module, however, may lengthen the model's inference time because of constant course correction. The team intends to look into this further in a future study.

FAQs

Q1. When was ChatGPT launched?
In November 2022

Q2. Who developed ChatGpt?
OpenAI