Image-to-Speech-GenAI-Tool-Using-LLM

Name: Image-to-Speech-GenAI-Tool-Using-LLM
Author: GURPREETKAURJETHRA

56stars26forks

How to install Image-to-Speech-GenAI-Tool-Using-LLM?

gurpreetkaurjethra/image-to-speech-genai-tool-using-llm

$git clone https://github.com/GURPREETKAURJETHRA/Image-to-Speech-GenAI-Tool-Using-LLM

Installs into the current project.

Summary

Launch App On Streamlit

Files · 1

View on GitHub

README.md

1	# 🖼️Image to Speech GenAI Tool Using LLM 🌟♨️
2	AI tool that generates an Audio short story based on the context of an uploaded image by prompting a GenAI LLM model, Hugging Face AI models together with OpenAI & LangChain. Deployed on Streamlit & Hugging Space Cloud Separately.
3
4	## 📢Run App with Streamlit Cloud
5
6	[Launch App On Streamlit](https://image-to-speech-genai-tool-using-llm.streamlit.app/)
7
8	## 📢Run App with HuggingFace Space Cloud
9
10	[Launch App On HuggingFace Space](https://huggingface.co/spaces/GurpreetKJ/Image-to-SpeechStory_GenAI-Tool)
11
12
13	## 🎯 Demo:
14	![Demo 1: Couple Test Image Output](img-audio/CoupleOutput.jpg)
15
16
17	You can listen respective audio file of this test demo images on respective `img-audio` folder
18
19
20	## 📈System Design
21
22	![system-design](img/system-design.drawio.png)
23
24
25	## 🏆Approach
26	An app that uses Hugging Face AI models to generate text from an image, which then generates audio from the text.
27
28	Execution is divided into 3 parts:
29	- Image to text:
30	an image-to-text transformer model ([Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base)) is used to generate a text scenario based on the on the AI understanding of the image context
31	- Text to story:
32	OpenAI LLM model is prompted to create a short story (50 words: can be adjusted as reqd.) based on the generated scenario. [gpt-3.5-turbo](https://platform.openai.com/docs/models/gpt-3-5)
33	- Story to speech:
34	a text-to-speech transformer model ([espnet/kan-bayashi_ljspeech_vits](https://huggingface.co/espnet/kan-bayashi_ljspeech_vits)) is used to convert the generated short story into a voice-narrated audio file
35	- A user interface is built using streamlit to enable uploading the image and playing the audio file
36
37
38	![Demo 3: Family Test Image Output](img-audio/FamilyOutput.jpg)
39	You can listen respective audio file of this test image on respective `img-audio` folder
40
41
42	## 🌟Requirements
43
44	- os
45	- python-dotenv
46	- transformers
47	- torch
48	- langchain
49	- openai
50	- requests
51	- streamlit
52
53
54	## 🚀Usage
55
56	- Before using the app, the user should have personal tokens for Hugging Face and Open AI
57	- The user should set venv environment and install ipykernel library for running app on local system ide.
58	- The user should save the personal tokens in an ".env" file within the package as string objects under object names: HUGGINGFACE_TOKEN and OPENAI_TOKEN
59	- The user can then run the app using the command: streamlit run app.py
60	- Once the app is running on streamlit, the user can upload the target image
61	- Execution will start automatically and it may take a few minutes to complete
62	- Once completed, the app will display:
63	- The scenario text generated by the image-to-text transformer HuggingFace model
64	- The short story generated by prompting the OpenAI LLM
65	- The audio file narrating the short story generated by the text-to-speech transformer model
66	- Deployed Gen AI App on streamlit cloud and Hugging Space
67
68	![Demo 2: Picnic Vaction Test Image Output](img-audio/PicnicOutput.jpg)
69
70
71	## ▶️Installation
72
73	Clone the repository:
74
75	`git clone https://github.com/GURPREETKAURJETHRA/Image-to-Speech-GenAI-Tool-Using-LLM.git`
76
77	Install the required Python packages:
78
79	`pip install -r requirements.txt`
80
81	Set up your OpenAI API key & Hugging Face Token by creating a .env file in the root directory of the project with the following contents:
82
83	`OPENAI_API_KEY=<your-api-key-here>`
84	`HUGGINGFACE_API_TOKEN=<<your-access-token-here>`
85
86	Run the Streamlit app:
87
88	`streamlit run app.py`
89
90
91	## ©️ License
92
93	Distributed under the MIT License. See `LICENSE` for more information.
94
95	---
96
97	#### If you like this LLM Project do drop ⭐ to this repo and Contributions are welcome! If you have any suggestions for improving this AI Img-Speech Converter, please submit a pull request.💁
98	#### Follow me on [![LinkedIn](https://img.shields.io/badge/linkedin-%230077B5.svg?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/gurpreetkaurjethra/)

People who liked this also viewed

6 picks

Type

outlinesStructured OutputsPromptsJul 2026—15k
GPT-Image2-SkillGPT Image 2 prompt gallery, image prompt library, agentic skill, and CLI for OpenAI image generation/editingPromptsJul 2026—3.9k
PromptEnhancer[CVPR 2026] PromptEnhancer is a prompt-rewriting tool, refining prompts into clearer, structured versions for better image generation.PromptsJun 2026—3.7k
diffusiondbA large-scale text-to-image prompt gallery dataset based on Stable DiffusionPromptsJul 2024—1.4k
stable-diffusion-prompt-readerA simple standalone viewer for reading prompts from Stable Diffusion generated image outside the webui.PromptsMay 2024—1.3k
female-portrait-directorA modular Codex Skill for directing and expanding detailed AI female portrait prompts.PromptsJul 2026—1.1k

Image-to-Speech-GenAI-Tool-Using-LLM

56stars26forks

View on GitHub

How to install Image-to-Speech-GenAI-Tool-Using-LLM?

gurpreetkaurjethra/image-to-speech-genai-tool-using-llm

$git clone https://github.com/GURPREETKAURJETHRA/Image-to-Speech-GenAI-Tool-Using-LLM

Installs into the current project.

Summary

Launch App On Streamlit

Files · 1

View on GitHub

README.md

1	# 🖼️Image to Speech GenAI Tool Using LLM 🌟♨️
2	AI tool that generates an Audio short story based on the context of an uploaded image by prompting a GenAI LLM model, Hugging Face AI models together with OpenAI & LangChain. Deployed on Streamlit & Hugging Space Cloud Separately.
3
4	## 📢Run App with Streamlit Cloud
5
6	[Launch App On Streamlit](https://image-to-speech-genai-tool-using-llm.streamlit.app/)
7
8	## 📢Run App with HuggingFace Space Cloud
9
10	[Launch App On HuggingFace Space](https://huggingface.co/spaces/GurpreetKJ/Image-to-SpeechStory_GenAI-Tool)
11
12
13	## 🎯 Demo:
14	![Demo 1: Couple Test Image Output](img-audio/CoupleOutput.jpg)
15
16
17	You can listen respective audio file of this test demo images on respective `img-audio` folder
18
19
20	## 📈System Design
21
22	![system-design](img/system-design.drawio.png)
23
24
25	## 🏆Approach
26	An app that uses Hugging Face AI models to generate text from an image, which then generates audio from the text.
27
28	Execution is divided into 3 parts:
29	- Image to text:
30	an image-to-text transformer model ([Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base)) is used to generate a text scenario based on the on the AI understanding of the image context
31	- Text to story:
32	OpenAI LLM model is prompted to create a short story (50 words: can be adjusted as reqd.) based on the generated scenario. [gpt-3.5-turbo](https://platform.openai.com/docs/models/gpt-3-5)
33	- Story to speech:
34	a text-to-speech transformer model ([espnet/kan-bayashi_ljspeech_vits](https://huggingface.co/espnet/kan-bayashi_ljspeech_vits)) is used to convert the generated short story into a voice-narrated audio file
35	- A user interface is built using streamlit to enable uploading the image and playing the audio file
36
37
38	![Demo 3: Family Test Image Output](img-audio/FamilyOutput.jpg)
39	You can listen respective audio file of this test image on respective `img-audio` folder
40
41
42	## 🌟Requirements
43
44	- os
45	- python-dotenv
46	- transformers
47	- torch
48	- langchain
49	- openai
50	- requests
51	- streamlit
52
53
54	## 🚀Usage
55
56	- Before using the app, the user should have personal tokens for Hugging Face and Open AI
57	- The user should set venv environment and install ipykernel library for running app on local system ide.
58	- The user should save the personal tokens in an ".env" file within the package as string objects under object names: HUGGINGFACE_TOKEN and OPENAI_TOKEN
59	- The user can then run the app using the command: streamlit run app.py
60	- Once the app is running on streamlit, the user can upload the target image
61	- Execution will start automatically and it may take a few minutes to complete
62	- Once completed, the app will display:
63	- The scenario text generated by the image-to-text transformer HuggingFace model
64	- The short story generated by prompting the OpenAI LLM
65	- The audio file narrating the short story generated by the text-to-speech transformer model
66	- Deployed Gen AI App on streamlit cloud and Hugging Space
67
68	![Demo 2: Picnic Vaction Test Image Output](img-audio/PicnicOutput.jpg)
69
70
71	## ▶️Installation
72
73	Clone the repository:
74
75	`git clone https://github.com/GURPREETKAURJETHRA/Image-to-Speech-GenAI-Tool-Using-LLM.git`
76
77	Install the required Python packages:
78
79	`pip install -r requirements.txt`
80
81	Set up your OpenAI API key & Hugging Face Token by creating a .env file in the root directory of the project with the following contents:
82
83	`OPENAI_API_KEY=<your-api-key-here>`
84	`HUGGINGFACE_API_TOKEN=<<your-access-token-here>`
85
86	Run the Streamlit app:
87
88	`streamlit run app.py`
89
90
91	## ©️ License
92
93	Distributed under the MIT License. See `LICENSE` for more information.
94
95	---
96
97	#### If you like this LLM Project do drop ⭐ to this repo and Contributions are welcome! If you have any suggestions for improving this AI Img-Speech Converter, please submit a pull request.💁
98	#### Follow me on [![LinkedIn](https://img.shields.io/badge/linkedin-%230077B5.svg?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/gurpreetkaurjethra/)

People who liked this also viewed

6 picks

Type