cover

Project Introduction

What is SimpleLLMFunc?

SimpleLLMFunc is a lightweight Large Language Model (LLM) application development framework designed to simplify the integration of LLMs into applications. The framework’s design philosophy is “LLM as Function, Prompt as Code,” providing type-safe decorators that allow developers to leverage LLM capabilities in a natural and intuitive way.

Why is SimpleLLMFunc needed?

When developing applications based on large language models, we often face the following challenges:

need to continuously write repetitive API call code
Prompt exists as a string variable in the code, which is not intuitive
Orchestration is constrained by the framework, lacking flexibility
The debugging and monitoring of LLM call processes are difficult

SimpleLLMFunc aims to solve these problems, allowing developers to:

Decorator-driven: Provides decorators such as @llm_function and @llm_chat. All decorators only support asynchronous functions (async def) and are natively adapted for asynchronous calls.
Prompt as logic: Prompt is code, which is the logical implementation of this function.
Type safety: Supports Python type annotations and Pydantic models to ensure correct data structures.
Multimodal Support: Supports mixed input of text, image URLs, and local image paths, innovatively supporting multimodal return of tools.
General model interface: Compatible with any model service that conforms to the OpenAI API format, easy to expand.
API key management: Intelligent load balancing for multiple API keys.
Traffic control: Integrated token bucket algorithm for intelligent traffic smoothing.
Tool system: Supports LLM tool usage, with a simple and easy-to-use tool definition and invocation mechanism, and supports multimodal tool returns.
Complete logs: Support trace_id tracking and searching, facilitating debugging and monitoring.

feature	SimpleLLMFunc	LangChain	Dify
Usability (learning curve)	✅	❌	✅
Intuition	✅	❌	⭕️
flexibility	✅	✅	⭕️
Development Speed	✅	❌	✅
debug	✅	❌	✅
Asynchronous Support	✅	✅	⭕️
Multimodal Support	✅	⭕️	⭕️
Traffic Control	✅	⭕️	⭕️
Type safety	✅	⭕️	❌
Tool Integration	✅🌟	✅	✅
Community and Ecosystem	⭕️	✅	✅

Example Display

Here is a simple example demonstrating the basic usage of SimpleLLMFunc:

⚠️ Decorators such as @llm_function, @llm_chat, and @tool in SimpleLLMFunc can only decorate functions defined with async def. Please call them in an asynchronous context using await.

import asyncio
from typing import List

from pydantic import BaseModel, Field

from SimpleLLMFunc import llm_function, OpenAICompatible

# 定义返回类型
class ProductAnalysis(BaseModel):
    pros: List[str] = Field(..., description="产品优点")
    cons: List[str] = Field(..., description="产品缺点")
    rating: int = Field(..., description="评分（1-5分）")

# 配置 LLM 接口
models = OpenAICompatible.load_from_json_file("provider.json")
llm_interface = models["openai"]["gpt-3.5-turbo"]

# 创建 LLM 函数
@llm_function(llm_interface=llm_interface)
async def analyze_product(product_name: str, review: str) -> ProductAnalysis:
    """
    分析产品评论，提取优缺点并给出评分。
    
    Args:
        product_name: 产品名称
        review: 用户评论
        
    Returns:
        产品分析结果
    """
    pass  # Prompt as Code, Code as Doc

# 使用函数


async def main():
    result = await analyze_product("无线耳机", "音质不错但连接不稳定")
    print(f"优点: {result.pros}")
    print(f"缺点: {result.cons}")
    print(f"评分: {result.rating}/5")


asyncio.run(main())

Asynchronous Support Example

import asyncio

from SimpleLLMFunc import llm_function


@llm_function(llm_interface=llm_interface)
async def async_translate(text: str, target_language: str) -> str:
    """
    将输入文本翻译为目标语言。

    Args:
        text: 要翻译的文本
        target_language: 目标语言

    Returns:
        翻译结果
    """
    pass


async def main():
    result = await async_translate("Hello world", "中文")
    print(result)


asyncio.run(main())

Multimodal Support Example

import asyncio

from SimpleLLMFunc import llm_function
from SimpleLLMFunc.type import Text, ImgPath

@llm_function(llm_interface=llm_interface)
async def analyze_image(description: Text, image: ImgPath) -> str:
    """
    分析图像内容
    
    Args:
        description: 分析要求描述
        image: 本地图片路径
        
    Returns:
        图像分析结果
    """
    pass

# 使用多模态输入
async def run():
    result = await analyze_image(
        description=Text("描述这张图片中的主要内容"),
        image=ImgPath("./photo.jpg")
    )
    print(result)


asyncio.run(run())

Dynamic template parameter example

import asyncio

from SimpleLLMFunc import llm_function

# 万能的代码分析函数
@llm_function(llm_interface=llm_interface)
async def analyze_code(code: str) -> str:
    """以{style}的方式分析{language}代码，重点关注{focus}。"""
    pass


# 万能的文本处理函数
@llm_function(llm_interface=llm_interface)
async def process_text(text: str) -> str:
    """作为{role}，请{action}以下文本，输出风格为{style}。"""
    pass


async def main():
    # 不同的调用方式，适应不同场景
    performance_analysis = await analyze_code(
        python_code,
        _template_params={
            'style': '详细',
            'language': 'Python',
            'focus': '性能优化'
        }
    )

    code_review = await analyze_code(
        js_code,
        _template_params={
            'style': '简洁',
            'language': 'JavaScript',
            'focus': '代码规范'
        }
    )

    # 同一个函数，不同角色
    edited_text = await process_text(
        text,
        _template_params={
            'role': '专业编辑',
            'action': '润色',
            'style': '学术'
        }
    )

    translated_text = await process_text(
        text,
        _template_params={
            'role': '翻译专家',
            'action': '翻译成英文',
            'style': '商务'
        }
    )

    print(performance_analysis, code_review, edited_text, translated_text)


asyncio.run(main())

Core features

Decorator-driven: Use @llm_function and @llm_chat to build LLM-driven features, all of which are natively asynchronous.
DocString as Prompt: Directly define Prompt in function documentation to improve code readability.
Dynamic Template Parameters: Supports dynamically setting DocString template parameters via _template_params during function invocation, allowing a single function to adapt to multiple scenarios.
Type safety: Supports Python type hints and Pydantic models to ensure correct data structures.
Asynchronous Support: @llm_function and @llm_chat natively support asynchronous calls, eliminating the need for additional aliases.
Multimodal Support: Supports multimodal input for text, image URLs, and local image paths, while also supporting tool multimodal returns.
Event Stream and Observability: Obtain the complete ReAct event stream and origin metadata through enable_event=True, facilitating UI routing and performance statistics.
SelfReference + PyRepl Runtime: Provides built-in PyRepl and selfref primitives, supporting persistent memory, fork/spawn/wait and other self-forking capabilities.
Step-by-step decorator pipeline: llm_decorator/steps splits Prompt construction, signature parsing, ReAct and response parsing into composable steps.
Basic Engine Modularization: base/messages, base/tool_call, base/type_resolve evolve independently, with more robust type resolution and multimodal processing.
Generic Model Interface: Compatible with any model service that conforms to the OpenAI API format, easy to extend.
API key management: Intelligent load balancing for multiple API keys.
Traffic Control: Integrate token bucket algorithm for intelligent traffic smoothing.
Tool System: Supports tool invocation, parameter validation, best practice injection, and multimodal return.
Out-of-the-box terminal TUI: Provides the @tui decorator, which can wrap the @llm_chat Agent into a Textual terminal chat interface.
Log Traceability: Supports trace_id association for logs, facilitating debugging and troubleshooting.

Project Architecture

The directory structure of SimpleLLMFunc is as follows:

SimpleLLMFunc/
├── __init__.py                  # 包初始化
├── config.py                    # 全局配置
├── base/                        # 核心执行引擎
│   ├── messages/                # 消息构建与多模态内容生成
│   ├── tool_call/               # 工具调用参数转换、执行与校验
│   ├── type_resolve/            # 类型描述、示例与多模态类型解析
│   ├── post_process.py          # 响应解析与类型转换
│   └── ReAct.py                 # ReAct 协调器
├── builtin/                     # 内置工具
│   ├── pyrepl.py                # Python REPL 工具集
│   └── self_reference.py        # SelfReference 内存实现
├── hooks/                       # 事件流系统
│   ├── events.py                # 事件类型定义
│   ├── stream.py                # 事件/响应流封装
│   ├── event_emitter.py         # 工具自定义事件发射器
│   ├── event_bus.py             # 事件总线
│   └── input_stream.py          # 交互式输入路由
├── interface/                   # LLM 接口层
│   ├── llm_interface.py         # 抽象基类
│   ├── openai_compatible.py     # OpenAI 兼容实现
│   ├── key_pool.py              # API 密钥负载均衡
│   └── token_bucket.py          # 令牌桶流量控制
├── llm_decorator/               # 装饰器与步骤化流水线
│   ├── llm_function_decorator.py
│   ├── llm_chat_decorator.py
│   ├── steps/                   # Prompt/签名/执行/响应拆分
│   │   ├── common/
│   │   ├── function/
│   │   └── chat/
│   └── utils/
│       └── tools.py
├── runtime/                     # 运行时原语与 worker 代理
│   ├── primitives.py
│   ├── builtin_self_reference.py
│   └── worker_proxy.py
├── self_reference.py            # SelfReference 对外接口
├── utils/                       # 通用工具与 TUI 组件
│   └── tui/
├── tool/                        # 工具定义与序列化
│   └── tool.py
├── type/                        # 类型与多模态辅助
│   ├── hooks.py
│   ├── llm.py
│   ├── message.py
│   ├── multimodal.py            # Text / ImgUrl / ImgPath 等
│   └── tool_call.py
├── logger/                      # 日志系统
├── observability/               # Langfuse 等集成
└── py.typed

Module Introduction

LLM interface module

The interface module provides a standard interface for communication with LLM services, supporting any service compatible with the OpenAI API. OpenAICompatible can load multiple model instances through provider.json (provider -> model configuration list); token_bucket.py is responsible for rate limiting, and key_pool.py is responsible for key load balancing.

LLM decorator module

The llm_decorator module is the core of the framework, providing @llm_function and @llm_chat, and splitting Prompt construction, signature parsing, ReAct execution, response parsing and other steps into composable steps in steps/. Multimodal type definitions are located in type/multimodal.py (Text, ImgUrl, ImgPath).

type definition module

The type module exports type definitions related to messages, tool calls, multimodal content, and event streams, making it convenient to directly use types such as Text/ImgUrl/ImgPath in signatures and tools.

Log System

The logger module provides structured logging associated with trace_id, outputs console logs by default and records JSON logs in LOG_DIR/application.log, facilitating troubleshooting and observability.

Tool system

The tool module allows LLMs to invoke external tools and services. Tools are marked with the @tool decorator (only supports async def), and support multimodal returns (text, images, or combinations), while also being able to inject tool best practice prompts.

Event System and Terminal TUI

The hooks module provides a unified event stream (LLM calls, tool calls, custom events) with origin metadata, used for stable routing of main sessions and forked child sessions. utils/tui implements the @tui decorator based on the event stream, which can directly wrap an @llm_chat Agent into an interactive terminal interface; builtin/pyrepl.py provides a default set of available code execution tools.

Target Users

SimpleLLMFunc is especially suitable for the following friends:

LLM Application Development for Beginner Makers: Gentle learning curve, simple content, quick start, intuitive and easy to understand.
Entrepreneurs in rapid prototyping: Need to quickly validate LLM application ideas, shorten development cycles and iteration times.
PMs who know Python: Need to quickly implement LLM application prototypes and validate product ideas.

We also welcome any novices, veterans, or experts who are interested in LLM application development to join our community and explore the infinite possibilities of LLM applications together!