Experiment Design¶

This document mainly introduces how to design a social simulation experiment using AgentSociety.

When designing a social experiment in an AgentSociety virtual world, we first need to set the virtual world’s initial state, then influence agent behavior by simulating social operation and applying interventions, and finally obtain agent information through a series of data collection methods to evaluate experimental results.

For example, if we want to study the impact of severe weather (using hurricanes as an example) on citizens, we need to select a coastal city and set citizens’ initial attributes (such as age, occupation, etc.) as the initial state, where citizens should be commuting normally.

After simulating 1 day in the virtual world, when the hurricane strikes, we set the weather variable in the environment to hurricane as an intervention through modifying environment. Citizen agents will process this new information and respond appropriately through the large model.

After the simulation ends, we collect citizen agents’ feelings about the hurricane through questionnaire surveys, and evaluate the hurricane’s impact on citizens based on their responses.

We can also access the database to obtain data such as citizen agents’ locations, moods, cognition, and dialogues during the simulation, and conduct further analysis.

Initial State Setup¶

The initial state setup of the virtual world mainly includes agent profiles and maps.

Agent profiles are descriptions of information for each citizen agent created in the virtual world, including name, gender, age, education level, occupation, marital status, personality role, background story, and other content. This information constitutes the agent’s self-awareness and is the foundation for all agent behaviors and decisions. Therefore, properly setting agent profiles is critical for any social simulation experiment.

Tip

Agent profiles are set through the memory_from_file field in agent configuration, which accepts a JSON file path. See the agent profiles document for file format.

Note

Only citizens type agents accept agent profile input.

Maps will affect the physical space where agents operate, including differences in roads, buildings, and points of interest. They mainly influence agents’ cognitive processes of “where am I” and “what’s around me,” and have important implications for research and experiments on agent spatial behavior.

Tip

Maps are set through the file_path field in map configuration, which accepts a protobuf format map file path. See the custom maps document for new map construction.

Intervention¶

AgentSociety provides multiple ways to intervene in agent behavior, including:

Send message (message): Send a text to a specified agent, which will be processed by the agent’s react_to_intervention(message: str) function to produce an effect. The processing method is implemented by the agent class.
Modify environment (environment): Modify environment variables such as weather, time, temperature, etc. Agents can actively obtain these variables to join the decision-making process, thereby indirectly affecting agent behavior.
Modify agent state (update_state): Directly modify values in the agent’s Key-Value form Status Memory. This method is equivalent to directly “tampering” with the agent’s memory and requires a thorough understanding of the fields contained in the agent’s Status Memory. Generally not recommended.

The intervention process needs to be implemented by adding “intervention” related workflow steps in experiment configuration. See the experiment configuration document for intervention step types and parameters.

Example:

exp:
  name: ...
  workflow:
    - ...
    - type: message
      target_agent: [1, 2, 3, 4, 5]
      message: "马上有恶劣天气，立刻回家！"
    - type: environment
      key: weather
      value: "下雨"
    - type: update_state
      target_agent: [1, 2, 3, 4, 5]
      key: goods_consumption
      value: 1000
    - ...

Data Collection¶

AgentSociety provides multiple ways to collect agent behavior data, including:

Questionnaire survey (survey): Send questionnaires to specified agents. After agents respond, the answers are stored in the database. Recommended for batch collection of agents’ responses to multiple questions.
Interview (interview): Send interviews to specified agents. After agents respond, the answers are stored in the database.
State storage (save_context): Store the agent’s Key-Value form Status Memory into a global Context variable, which is finally stored as a file for subsequent analysis. This method requires a thorough understanding of the fields contained in the agent’s Status Memory. Generally not recommended.

Similar to interventions, the data collection process needs to be implemented by adding “data collection” related workflow steps in experiment configuration. See the experiment configuration document for data collection step types and parameters.

Example:

exp:
  name: ...
  workflow:
    - ...
    - type: interview
      target_agent: [1, 2, 3, 4, 5]
      interview_message: "您好，我是社区工作人员，请问您对社区生活有什么看法？"
    - type: survey
      target_agent:
        agent_class: ["SocietyAgent"]
        filter_str: "${profile.age} >= 18"
      survey:
        id: "550e8400-e29b-41d4-a716-446655440000"
        title: "社区生活满意度调查"
        description: "了解您对社区生活的看法和建议"
        pages:
          - name: "满意度评价"
            elements:
              - name: "overall_satisfaction"
                title: "您对社区生活的整体满意度如何？"
                type: "rating"
                min_rating: 1
                max_rating: 5
              - name: "improvement_areas"
                title: "您认为社区最需要改进的方面有哪些？"
                type: "checkbox"
                choices: ["交通便利性", "环境卫生", "安全状况", "商业配套", "社区活动"]
    - type: save_context
      target_agent: [1, 2, 3, 4, 5]
      key: goods_consumption
      save_as: agent_goods_consumption
    - ...