agentsociety.llm.llm

Module Contents

Classes

LLMProviderType

Defines the types of LLM providers.

LLMConfig

LLM configuration class.

LLMActor

Actor class for LLM operations.

LLM

Main class for the Large Language Model (LLM) object used by Agent(Soul).

Data

__all__

MAX_TIMEOUT

API

agentsociety.llm.llm.__all__

[‘LLM’, ‘LLMConfig’, ‘LLMProviderType’]

agentsociety.llm.llm.MAX_TIMEOUT

60

class agentsociety.llm.llm.LLMProviderType

Bases: str, enum.Enum

Defines the types of LLM providers.

  • Description:

    • Enumerates different types of LLM providers.

  • Types:

    • OPENAI: OpenAI and compatible providers (based on base_url).

    • DEEPSEEK: DeepSeek.

    • QWEN: Qwen.

    • ZHIPU: Zhipu.

    • SILICONFLOW: SiliconFlow.

    • VLLM: VLLM.

Initialization

Initialize self. See help(type(self)) for accurate signature.

OpenAI

‘openai’

DeepSeek

‘deepseek’

Qwen

‘qwen’

ZhipuAI

‘zhipuai’

SiliconFlow

‘siliconflow’

VolcEngine

‘volcengine’

VLLM

‘vllm’

class agentsociety.llm.llm.LLMConfig

Bases: pydantic.BaseModel

LLM configuration class.

provider: agentsociety.llm.llm.LLMProviderType

‘Field(…)’

The type of the LLM provider

base_url: Optional[str]

‘Field(…)’

The base URL for the LLM provider

api_key: str

‘Field(…)’

API key for accessing the LLM provider

model: str

‘Field(…)’

The model to use

concurrency: int

‘Field(…)’

Concurrency value for LLM operations to avoid rate limit

timeout: float

‘Field(…)’

Timeout for LLM operations in seconds

serialize_provider(provider: agentsociety.llm.llm.LLMProviderType, info)
validate_configuration()
class agentsociety.llm.llm.LLMActor

Actor class for LLM operations.

Initialization

async call(config: agentsociety.llm.llm.LLMConfig, dialog: list[openai.types.chat.ChatCompletionMessageParam], response_format: Union[openai.types.chat.completion_create_params.ResponseFormat, openai.NotGiven] = NOT_GIVEN, temperature: float = 1, max_tokens: Optional[int] = None, top_p: Optional[float] = None, frequency_penalty: Optional[float] = None, presence_penalty: Optional[float] = None, timeout: int = 300, retries: int = 10, tools: Union[List[openai.types.chat.ChatCompletionToolParam], openai.NotGiven] = NOT_GIVEN, tool_choice: Union[openai.types.chat.ChatCompletionToolChoiceOptionParam, openai.NotGiven] = NOT_GIVEN)

Sends an asynchronous text request to the configured LLM API.

  • Description:

    • Attempts to send a text request up to retries times with exponential backoff on failure.

    • Handles different request types and manages token usage statistics.

  • Args:

    • config: LLM configuration.

    • dialog: Messages to send as part of the chat completion request.

    • response_format: JSON schema for the response. Default is NOT_GIVEN.

    • temperature: Controls randomness in the model’s output. Default is 1.

    • max_tokens: Maximum number of tokens to generate in the response. Default is None.

    • top_p: Limits the next token selection to a subset of tokens with a cumulative probability above this value. Default is None.

    • frequency_penalty: Penalizes new tokens based on their existing frequency in the text so far. Default is None.

    • presence_penalty: Penalizes new tokens based on whether they appear in the text so far. Default is None.

    • timeout: Request timeout in seconds. Default is 300 seconds.

    • retries: Number of retry attempts in case of failure. Default is 10.

    • tools: List of dictionaries describing the tools that can be called by the model. Default is NOT_GIVEN.

    • tool_choice: Dictionary specifying how the model should choose from the provided tools. Default is NOT_GIVEN.

  • Returns:

    • A string containing the message content or a dictionary with tool call arguments if tools are used.

    • Raises exceptions if the request fails after all retry attempts.

class agentsociety.llm.llm.LLM(configs: List[agentsociety.llm.llm.LLMConfig], num_actors: int = min(cpu_count(), 8))

Main class for the Large Language Model (LLM) object used by Agent(Soul).

  • Description:

    • This class manages configurations and interactions with different large language model APIs.

    • It initializes clients based on the specified request type and handles token usage and consumption reporting.

Initialization

Initializes the LLM instance.

  • Parameters:

    • configs: An instance of LLMConfig containing configuration settings for the LLM.

    • num_actors (int): Number of actor instances to create for parallel processing. Defaults to min(cpu_count(), 8).

get_log_list()
clear_log_list()
_get_index()
async atext_request(dialog: list[openai.types.chat.ChatCompletionMessageParam], response_format: Union[openai.types.chat.completion_create_params.ResponseFormat, openai.NotGiven] = NOT_GIVEN, temperature: float = 1, max_tokens: Optional[int] = None, top_p: Optional[float] = None, frequency_penalty: Optional[float] = None, presence_penalty: Optional[float] = None, timeout: int = 300, retries: int = 10, tools: Union[List[openai.types.chat.ChatCompletionToolParam], openai.NotGiven] = NOT_GIVEN, tool_choice: Union[openai.types.chat.ChatCompletionToolChoiceOptionParam, openai.NotGiven] = NOT_GIVEN)

Sends an asynchronous text request to the configured LLM API.

  • Description:

    • Attempts to send a text request up to retries times with exponential backoff on failure.

    • Handles different request types and manages token usage statistics.

  • Args:

    • dialog: Messages to send as part of the chat completion request.

    • response_format: JSON schema for the response. Default is NOT_GIVEN.

    • temperature: Controls randomness in the model’s output. Default is 1.

    • max_tokens: Maximum number of tokens to generate in the response. Default is None.

    • top_p: Limits the next token selection to a subset of tokens with a cumulative probability above this value. Default is None.

    • frequency_penalty: Penalizes new tokens based on their existing frequency in the text so far. Default is None.

    • presence_penalty: Penalizes new tokens based on whether they appear in the text so far. Default is None.

    • timeout: Request timeout in seconds. Default is 300 seconds.

    • retries: Number of retry attempts in case of failure. Default is 10.

    • tools: List of dictionaries describing the tools that can be called by the model. Default is NOT_GIVEN.

    • tool_choice: Dictionary specifying how the model should choose from the provided tools. Default is NOT_GIVEN.

  • Returns:

    • A string containing the message content or a dictionary with tool call arguments if tools are used.

    • Raises exceptions if the request fails after all retry attempts.