GPT4Vis
starsforks
How to install GPT4Vis?
whwu95/gpt4vis
$git clone https://github.com/whwu95/GPT4VisWenhao Wu<sup>1,2</sup>, Huanjin Yao<sup>2,3</sup>, Mengxi Zhang<sup>2,4</sup>, [Yuxin Song](https://openreview.net/profile?id=~YuXin_So
README.md
| 1 | <div align="center"> |
| 2 | |
| 3 | |
| 4 | |
| 5 | <h2 align="center"> <a href="https://arxiv.org/abs/2311.15732">GPT4Vis: What Can GPT-4 Do for Zero-shot Visual Recognition?</a></h2> |
| 6 | <h5 align="center"> If you like our project, please give us a star ⭐ on GitHub for latest update. </h2> |
| 7 | |
| 8 | |
| 9 | [](https://arxiv.org/abs/2311.15732) |
| 10 | [](https://zhuanlan.zhihu.com/p/669758735) |
| 11 | |
| 12 | |
| 13 | [Wenhao Wu](https://whwu95.github.io/)<sup>1,2</sup>, [Huanjin Yao](https://openreview.net/profile?id=~Huanjin_Yao1)<sup>2,3</sup>, [Mengxi Zhang](https://scholar.google.com/citations?user=73tAoEAAAAAJ&hl=en)<sup>2,4</sup>, [Yuxin Song](https://openreview.net/profile?id=~YuXin_Song1)<sup>2</sup>, [Wanli Ouyang](https://wlouyang.github.io/)<sup>5</sup>, [Jingdong Wang](https://jingdongwang2017.github.io/)<sup>2</sup> |
| 14 | |
| 15 | |
| 16 | <sup>1</sup>[The University of Sydney](https://www.sydney.edu.au/), <sup>2</sup>[Baidu](https://vis.baidu.com/#/), <sup>3</sup>[Tsinghua University](https://www.tsinghua.edu.cn/en/), <sup>4</sup>[Tianjin University](https://www.tju.edu.cn/english/index.htm/), <sup>5</sup>[The Chinese University of Hong Kong](https://www.cuhk.edu.hk/english/#) |
| 17 | |
| 18 | |
| 19 | </div> |
| 20 | |
| 21 | *** |
| 22 | This work delves into an essential, yet must-know baseline in light of the latest advancements in Generative Artificial Intelligence (GenAI): the utilization of GPT-4 for visual understanding. We center on the evaluation of GPT-4's linguistic and visual capabilities in zero-shot visual recognition tasks. To ensure a comprehensive evaluation, we have conducted experiments across three modalities—images, videos, and point clouds—spanning a total of 16 popular academic benchmark. |
| 23 | |
| 24 | <div align="center"> |
| 25 | <img src="docs/method.png" width="800" /> |
| 26 | </div> |
| 27 | |
| 28 | <details open><summary>📣 I also have other cross-modal projects that may interest you ✨. </summary><p> |
| 29 | |
| 30 | |
| 31 | > [**Revisiting Classifier: Transferring Vision-Language Models for Video Recognition**](https://arxiv.org/abs/2207.01297)<br> |
| 32 | > Wenhao Wu, Zhun Sun, Wanli Ouyang <br> |
| 33 | > [](https://ojs.aaai.org/index.php/AAAI/article/view/25386/25158) [](https://link.springer.com/article/10.1007/s11263-023-01876-w) [](https://github.com/whwu95/Text4Vis) |
| 34 | |
| 35 | |
| 36 | > [**Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models**](https://arxiv.org/abs/2301.00182)<br> |
| 37 | > Wenhao Wu, Xiaohan Wang, Haipeng Luo, Jingdong Wang, Yi Yang, Wanli Ouyang <br> |
| 38 | > [](https://openaccess.thecvf.com/content/CVPR2023/html/Wu_Bidirectional_Cross-Modal_Knowledge_Exploration_for_Video_Recognition_With_Pre-Trained_Vision-Language_CVPR_2023_paper.html) [](https://github.com/whwu95/BIKE) |
| 39 | |
| 40 | |
| 41 | > [**Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?**](https://arxiv.org/abs/2301.00184)<br> |
| 42 | > Wenhao Wu, Haipeng Luo, Bo Fang, Jingdong Wang, Wanli Ouyang <br> |
| 43 | > Accepted by CVPR 2023 as 🌟Highlight🌟 | [](https://openaccess.thecvf.com/content/CVPR2023/html/Wu_Cap4Video_What_Can_Auxiliary_Captions_Do_for_Text-Video_Retrieval_CVPR_2023_paper.html) [](https://github.com/whwu95/Cap4Video)<br> |
| 44 | |
| 45 | |
| 46 | </p></details> |
| 47 | |
| 48 | |
| 49 | |
| 50 | |
| 51 | ## News |
| 52 | <!-- - [x] **[Mar 10, 2024]** We have updated all results in our [report](https://arxiv.org/abs/2311.15732). For accurate predictions, we strongly recommend using single testing with GPT-4V and have accordingly eliminated scripts related to batch testing. --> |
| 53 | - [x] **[Mar 7, 2024]** |
| 54 | Due to the recent removal of RPD (request per day) limits on the GPT-4V API, we'v |
People who liked this also viewed
6 picksType
- model-alignmentModel Alignment is a python library from the PAIR team that enable users to create model prompts through user feedback instead of manual prompt writing and editing. The technique makes use of constitutional principles to align prompts to users' desired values.PromptsOct 2025
- prompt-decoratorsA standardized framework for enhancing how LLMs process and respond to prompts through composable decorators, featuring an official open standard specification and Python reference implementation. Claude Code plugin and MCP server integration.PromptsJul 2026
- tree-promptTree prompting: easy-to-use scikit-learn interface for improved prompting.PromptsOct 2023
- claudifyAI-powered Spotify playlist generator using Claude API. Create personalized playlists from text prompts or transform your liked songs into curated collections. Seamlessly blends AI creativity with music discovery.PromptsJul 2024
- system-prompts-and-models-of-ai-toolsFULL Augment Code, Claude Code, Cluely, CodeBuddy, Comet, Cursor, Devin AI, Junie, Kiro, Leap.new, Lovable, Manus, NotionAI, Orchids.app, Perplexity, Poke, Qoder, Replit, Same.dev, Trae, Traycer AI, VSCode Agent, Warp.dev, Windsurf, Xcode, Z.ai Code, Dia & v0. (And other Open Sourced) System Prompts, Internal Tools & AI ModelsPromptsJul 2026
- CL4R1T4SLEAKED SYSTEM PROMPTS FOR CHATGPT, CLAUDE, GEMINI, GROK, PERPLEXITY, CURSOR, LOVABLE, REPLIT, AND MORE! - AI SYSTEMS TRANSPARENCY FOR ALL! 👐PromptsJul 2026
