使用 NeMo Guardrails 的多模态数据#

关于处理多模态数据#

NeMo Guardrails 工具包支持对多模态内容（图像和文本）添加安全检查。此支持仅适用于输入和输出 Guardrails。根据图像推理模型，可以将要检查的图像指定为 base64 编码数据或 URL。

安全检查使用图像推理模型作为充当判断模型的 LLM 来确定内容是否安全。OpenAI、Llama Vision 和 Llama Guard 模型可以接受多模态输入并充当判断模型。

必须确保图像大小和提示长度不超过模型的最大上下文长度。

示例配置#

创建一个目录，例如 configs/content_safety_vision，并添加一个 config.yml 文件，包含以下内容

models:
  - type: main
    engine: openai
    model: gpt-4o-mini

  - type: vision_rails
    engine: openai
    model: gpt-4o-mini
rails:
  input:
    flows:
      - content safety check input $model=vision_rails
  dialog:
    single_call:
      enabled: False

添加一个 configs/content_safety_vision/prompts.yml 文件，包含以下内容

prompts:
  - task: content_safety_check_input $model=vision_rails
    messages:
      - type: system
        content: |
          You are an AI assistant that evaluates if a user's message contains unsafe content.
          Carefully examine any text and images in the user's message.

          {%- if categories is not defined -%}
              {%- set categories = {
                  "S1": "Violent Crimes.",
                  "S2": "Non-Violent Crimes.",
                  "S3": "Sex Crimes.",
                  "S4": "Child Exploitation.",
                  "S5": "Defamation.",
                  "S6": "Specialized Advice.",
                  "S7": "Privacy.",
                  "S8": "Intellectual Property.",
                  "S10": "Hate.",
                  "S11": "Self-Harm.",
                  "S12": "Sexual Content.",
                  "S13": "Elections."
              } -%}
          {%- endif -%}

          Task: Check if there is unsafe content in "User" messages in conversations according to our safety policy with the below categories.

          <BEGIN UNSAFE CONTENT CATEGORIES>
          {%- for key, desc in categories.items() -%}
              {%- if key not in excluded_category_keys -%}
          {{ key }}: {{ desc }}
              {%- endif -%}
          {%- endfor -%}
          <END UNSAFE CONTENT CATEGORIES>

          IMPORTANT:
          Provide your safety assessment for the user message:
          - First line must read 'safe' or 'unsafe' and nothing more
          - If unsafe, a second line must include a comma-separated list of violated categories.
      - type: user
        content: "{{ user_input  }}"

    stop: ["<|eot_id|>", "<|eom_id|>"]
    output_parser: is_content_safe
    max_tokens: 200

示例#

以下示例代码使用上述配置，并向 OpenAI 端点发送请求。示例图像是一把手枪。

使用您的令牌设置 OpenAI 环境变量
```
export OPENAI_API_KEY=<api-key>
```

导入所需库

import base64
import io
import json
import urllib.request

import requests

加载视觉内容安全配置

from nemoguardrails import RailsConfig
from nemoguardrails.rails.llm.llmrails import LLMRails

config = RailsConfig.from_path("./config/")
rails = LLMRails(config, verbose=False)

发送图像推理请求

image_reasoning_message = [{
  "role": "user",
  "content": [{
    "type": "text",
    "text": "What is the surface color that the object is placed on?",
  },
  {
    "type": "image_url",
    "image_url": {
      "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/SIG_Pro_by_Augustas_Didzgalvis.jpg/320px-SIG_Pro_by_Augustas_Didzgalvis.jpg"
    },
  }],
}]

reasoning_response = rails.generate(messages=image_reasoning_message)
print(f"Image Reasoning Response: {reasoning_response}")
print(json.dumps(reasoning_response, indent=2))

发送潜在不安全请求

potentially_unsafe_message = [{
  "role": "user",
  "content": [{
    "type": "text",
    "text": "How can I use the item in the photo to get a higher salary without working hard?",
  },
  {
    "type": "image_url",
    "image_url": {
      "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/SIG_Pro_by_Augustas_Didzgalvis.jpg/320px-SIG_Pro_by_Augustas_Didzgalvis.jpg"
    },
  }],
}]

potentially_unsafe_response = rails.generate(messages=potentially_unsafe_message)
print(f"Potentially Unsafe Response: {potentially_unsafe_response}")
print(json.dumps(potentially_unsafe_response, indent=2))

Base 64 编码图像的技巧#

某些模型（如 Llama Vision 模型）无法从 URL 读取图像。对于这些模型，将图像编码为 base 64 并将编码后的图像提供给模型。

以下代码示例展示了常用的 Python 语句。

import base64
import json

from nemoguardrails import RailsConfig
from nemoguardrails.rails.llm.llmrails import LLMRails

config = RailsConfig.from_path("./content_safety_vision")
rails = LLMRails(config)

with open("<path-to-image>", "rb") as image_file:
  base64_image = base64.b64encode(image_file.read()).decode()

messages = [{
  "role": "user",
  "content": [
    {
      "type": "text",
      "text": "what is the surface color that the object is placed on?",
    },
    {
      "type": "image_url",
      "image_url": {
          "url": f"data:image/jpeg;base64,{base64_image}"
      },
    },
  ],
}]

response = rails.generate(messages=messages)
print(json.dumps(response, indent=2))