llamaflow
starsforks
How to install llamaflow?
dzhng/llamaflow
$git clone https://github.com/dzhng/llamaflowREADME.md
| 1 | NOTE: this library is deprecated, it is now split into [zod-gpt](https://github.com/dzhng/zod-gpt) and [llm-api](https://github.com/dzhng/llm-api). |
| 2 | |
| 3 | # 🦙 LLamaFlow |
| 4 | |
| 5 | [](https://github.com/dzhng/llamaflow/actions/workflows/test.yml) |
| 6 | |
| 7 | The Typescript-first prompt engineering toolkit for working with chat based large language models (LLMs). |
| 8 | |
| 9 | - [Introduction](#-introduction) |
| 10 | - [Usage](#-usage) |
| 11 | - [Install](#install) |
| 12 | - [Chats](#chats) |
| 13 | - [Prompts](#prompts) |
| 14 | - [Custom Prompts](#custom-prompts) |
| 15 | - [Text Splitter](#-text-splitter) |
| 16 | - [Debugging](#-debugging) |
| 17 | - [Azure](#-azure) |
| 18 | - [API Reference](#-api-reference) |
| 19 | |
| 20 | ## 👋 Introduction |
| 21 | |
| 22 | LLamaFlow is the middleware layer that sits between your software and the AI model, it adds the following capabilities on top of the standard chat completion API: |
| 23 | |
| 24 | - Support for structured outputs from models with complete type safety. All responses are fully validated & typed, works with [zod](https://github.com/colinhacks/zod) as a peer dep. |
| 25 | - Schema definition, serialization / parsing, and **automatically asking the model to correct outputs**. |
| 26 | - Custom content validation hook that allows you to add your own valider for all model outputs, including logic on how to reask the model. |
| 27 | - Nicer API for sending & retriving chat messages from models, no need to keep track of message memory manually. |
| 28 | - Handle rate limit and any other API errors as gracefully as possible (e.g. exponential backoff for rate-limit). |
| 29 | |
| 30 | With LLamaFlow, you can simply query OpenAI's ChatGPT model like so: |
| 31 | |
| 32 | ```typescript |
| 33 | import { OpenAI } from 'llama-flow'; |
| 34 | |
| 35 | const model = new OpenAI({ apiKey: 'YOUR_OPENAI_KEY' }); |
| 36 | |
| 37 | const chat = model.chat({ |
| 38 | systemMessage: |
| 39 | "You are a smart and honest AI assistant. Follow the user's requirements carefully & to the letter, minimize any other prose.", |
| 40 | }); |
| 41 | |
| 42 | const response = await chat.request( |
| 43 | prompt.json({ |
| 44 | message: |
| 45 | 'What are some good names for childrens book about the renaissance? Respond as a JSON array', |
| 46 | schema: z.array(z.string().max(200)), |
| 47 | }), |
| 48 | ); |
| 49 | |
| 50 | console.log(response.content); // content will be typed as string[]; |
| 51 | ``` |
| 52 | |
| 53 | ## 🔨 Usage |
| 54 | |
| 55 | ### Install |
| 56 | |
| 57 | This package is hosted on npm: |
| 58 | |
| 59 | ``` |
| 60 | npm i llama-flow |
| 61 | ``` |
| 62 | |
| 63 | ``` |
| 64 | yarn add llama-flow |
| 65 | ``` |
| 66 | |
| 67 | To setup in your codebase, initialize a new instance with the model you want (only `OpenAI` is suported for now). Note that you can also add default model and chat config (like temperature, timeouts, retries) when initializing. These are just defaults, and can always be overwritten later on a per-chat or per-request basis. |
| 68 | |
| 69 | ```typescript |
| 70 | import { OpenAI } from 'llama-flow'; |
| 71 | |
| 72 | const model = new OpenAI( |
| 73 | { apiKey: 'YOUR_OPENAI_KEY' }, |
| 74 | { model: 'gpt-3.5-turbo' }, |
| 75 | ); |
| 76 | ``` |
| 77 | |
| 78 | ### Chats |
| 79 | |
| 80 | A chat is a conversation between the "user" (your software), and the AI agent. LLamaFlow will take care of managing chat memory, so you can simply continue the conversation by sending another request. Note that different memory management strategies will be added in the future, such as pruning the memory as needed in order to fit the context window. |
| 81 | |
| 82 | ```typescript |
| 83 | const chat = model.chat({ |
| 84 | systemMessage: 'You are an AI writer.', |
| 85 | retainMemory: true, |
| 86 | }); |
| 87 | |
| 88 | // You can ask the AI model with a simple string, or a dedicated `Prompt` object. |
| 89 | const response = await chat.request( |
| 90 | prompt.text( |
| 91 | 'Write a script for a tiktok video that talks about the artistic contribution of the renaissance.', |
| 92 | ), |
| 93 | ); |
| 94 | |
| 95 | // The results, as well as any usage stats, will be returned. |
| 96 | console.log( |
| 97 | `The AI writer's response is: ${response.content}. Token used: ${response.usage.totalTokens}.`, |
| 98 | ); |
| 99 | |
| 100 | // You can follow up on this chat by prompting further, using the `bulletPrompt` object that was created earlier. |
| 101 | const bulletPoints = await chat.request(bulletPrompt); |
| 102 | |
| 103 | // `bulletPoints.content` will be automatically casted in the correct type as defined in the schema field of `bulletPrompt` |
| 104 | console.log( |
People who liked this also viewed
6 picksType
- components-helperBased on the docs to provide code prompt files for vue component libraryPromptsNov 2023
- VibeCode-Protocol-SuiteA comprehensive suite of protocols, meta-prompts, and orchestration tools designed to streamline software development workflows, project management, and team collaboration. Includes the VibeCode Protocol Suite with web interface, user manuals, and reusable templates for efficient development processes.PromptsJul 2026
- Promptable-web-sdkWeb SDK for Promptable Website.PromptsDec 2022
- LLMFlexA python package for developing AI applications with local LLMs.PromptsJan 2025
- cli-repo-to-promptexport any codebase to a single LLM-ready markdown promptPromptsJul 2026
- Files2PromptVS Code extension to copy files for an LLM promptPromptsApr 2025
