在 Hugging Face Spaces 上为 LLM 应用程序追踪设置 Phoenix 可观测性仪表板

作者：Andrew Reed

Phoenix 是 Arize AI 推出的一个开源可观测性库，专为实验、评估和故障排除而设计。它能让 AI 工程师和数据科学家快速可视化数据、评估性能、追踪问题并导出数据以进行改进。

追踪

数据集

实验

在本 notebook 中，我们将了解如何在 Hugging Face Spaces 上部署 Phoenix 可观测性仪表板，并配置它来自动追踪 LLM 调用，从而全面了解您 LLM 应用程序的内部工作情况。

第 1 步：在 Hugging Face Spaces 上部署 Phoenix

虽然 Phoenix 提供了用于本地开发的 notebook 优先选项，但它也可以通过 Docker 部署为一个独立的仪表板。一个长期运行的、托管的仪表板是提供对您的 LLM 应用程序行为的集中视图以及与团队协作的绝佳方式。Hugging Face Spaces 提供了一种简单的方式来托管带可选持久存储的 ML 应用程序，并且它对自定义 Docker 镜像的支持使其成为托管 Phoenix 的绝佳平台 - 让我们看看它是如何工作的！

首先，我们将复制这个演示空间

我们可以将空间配置为私有或公开，并且它可以存在于我们的用户命名空间或组织命名空间中。我们可以使用默认的免费层级 CPU，并且重要的是，指定我们希望为空间附加一个持久性磁盘。

点击“Duplicate Space”后，Docker 镜像将开始构建。这需要几分钟才能完成，然后我们会看到一个空的 Phoenix 仪表板。

第 2 步：配置应用程序追踪

现在我们有了一个运行中的 Phoenix 仪表板，我们可以配置我们的应用程序以使用 OpenTelemetry TracerProvider 自动追踪 LLM 调用。在此示例中，我们将使用 OpenAI 客户端库来检测我们的应用程序，并追踪从 `openai` Python 包到运行在 Hugging Face 无服务器推理 API 上的开放 LLM 的调用。

首先，我们需要安装必要的库

!pip install -q arize-phoenix arize-phoenix-otel openinference-instrumentation-openai openai huggingface-hub

然后，我们将使用 `huggingface_hub` 库登录 Hugging Face。这将使我们能够为我们的 Space 和无服务器推理 API 生成必要的身份验证。请确保用于身份验证的 HF 令牌具有对您 Space 所在组织的正确权限。

from huggingface_hub import interpreter_login

interpreter_login()

现在，我们可以将 Phoenix 客户端配置到我们正在运行的 Phoenix 仪表板上

1. 通过以下方式注册 Phoenix tracer provider
   • 指定我们选择的 `project_name`
   • 将 `endpoint` 值设置为我们 Space 的主机名（可通过仪表板 UI 的“设置”选项卡找到 - 见下文）
   • 将 `headers` 设置为访问 Space 所需的 Hugging Face 标头
2. 检测我们的应用程序代码以使用 OpenAI tracer provider

>>> from phoenix.otel import register
>>> from huggingface_hub.utils import build_hf_headers
>>> from openinference.instrumentation.openai import OpenAIInstrumentor

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="test",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: test
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

第 3 步：进行调用并在 Phoenix 仪表板中查看追踪

现在，我们可以调用一个 LLM 并在 Phoenix 仪表板中查看追踪。我们正在使用 OpenAI 客户端向 Hugging Face 无服务器推理 API 发出调用，该 API 已被检测以与 Phoenix 协同工作。在这种情况下，我们使用的是 `meta-llama/Llama-3.1-8B-Instruct` 模型。

>>> from openai import OpenAI
>>> from huggingface_hub import get_token

>>> client = OpenAI(
...     base_url="https://api-inference.huggingface.co/v1/",
...     api_key=get_token(),
... )

>>> messages = [{"role": "user", "content": "What does a llama do for fun?"}]

>>> response = client.chat.completions.create(
...     model="meta-llama/Llama-3.1-8B-Instruct",
...     messages=messages,
...     max_tokens=500,
... )

>>> print(response.choices[0].message.content)

Llamas are intelligent and social animals, and they do have ways to entertain themselves and have fun. While we can't directly ask a llama about its personal preferences, we can observe their natural behaviors and make some educated guesses. Here are some things that might bring a llama joy and excitement:

1. **Socializing**: Llamas are herd animals and they love to interact with each other. They'll often engage in play-fighting, neck-wrestling, and other activities to establish dominance and strengthen social bonds. When llamas have a strong social network, it can make them feel happy and content.
2. **Exploring new environments**: Llamas are naturally curious creatures, and they love to explore new surroundings. They'll often investigate their surroundings, sniffing and investigating new sights, sounds, and smells.
3. **Playing with toys**: While llamas don't need expensive toys, they do enjoy playing with objects that stimulate their natural behaviors. For example, a ball or a toy that mimics a target can be an entertaining way to engage them.
4. **Climbing and jumping**: Llamas are agile and athletic animals, and they enjoy using their limbs to climb and jump over obstacles. Providing a safe and stable area for them to exercise their physical abilities can be a fun and engaging experience.
5. **Browsing and foraging**: Llamas have a natural instinct to graze and browse, and they enjoy searching for tasty plants and shrubs. Providing a variety of plants to munch on can keep them engaged and entertained.
6. **Mentally stimulating activities**: Llamas are intelligent animals, and they can benefit from mentally stimulating activities like problem-solving puzzles or learning new behaviors (like agility training or obedience training).

Some fun activities you can try with a llama include:

* Setting up an obstacle course or agility challenge
* Creating a "scavenger hunt" with treats and toys
* Introducing new toys or objects to stimulate their curiosity
* Providing a variety of plants and shrubs to browse and graze on
* Engaging in interactive games like "follow the leader" or "find the treat"

Remember to always prioritize the llama's safety and well-being, and to consult with a veterinarian or a trained llama handler before attempting any new activities or introducing new toys.

如果我们导航回 Phoenix 仪表板，我们可以看到我们 LLM 调用的追踪已被捕获并显示！如果您为空间配置了持久性磁盘，那么每次重启空间时，所有的追踪信息都将被保存。

附加内容：使用 CrewAI 追踪多智能体应用程序

可观测性的真正威力来自于能够追踪和检查复杂的 LLM 工作流。在这个例子中，我们将安装并使用 CrewAI 来追踪一个多智能体应用程序。

!pip install -q openinference-instrumentation-crewai crewai crewai-tools

和之前一样，我们将注册 Phoenix tracer provider 并检测应用程序代码，但这次我们还将取消对现有 OpenAI tracer provider 的检测以避免冲突。

>>> from opentelemetry import trace
>>> from openinference.instrumentation.crewai import CrewAIInstrumentor

>>> # 0. Uninstrument existing tracer provider and clear the global tracer provider
>>> OpenAIInstrumentor().uninstrument()
>>> if trace.get_tracer_provider():
...     trace.get_tracer_provider().shutdown()
...     trace._TRACER_PROVIDER = None  # Reset the global tracer provider

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="crewai",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> CrewAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: crewai
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

现在，我们将使用 CrewAI 定义一个多智能体应用程序，以研究并撰写一篇关于 LLM 应用程序中可观测性和追踪重要性的博客文章。

>>> import os
>>> from huggingface_hub import get_token
>>> from crewai_tools import SerperDevTool
>>> from crewai import LLM, Agent, Task, Crew, Process

>>> # Define our LLM using HF's Serverless Inference API
>>> llm = LLM(
...     model="huggingface/meta-llama/Llama-3.1-8B-Instruct",
...     api_key=get_token(),
...     max_tokens=1024,
... )

>>> # Define a tool for searching the web
>>> os.environ["SERPER_API_KEY"] = "YOUR_SERPER_API_KEY"  # must set this value in your environment
>>> search_tool = SerperDevTool()

>>> # Define your agents with roles and goals
>>> researcher = Agent(
...     role="Researcher",
...     goal="Conduct thorough research on up to date trends around a given topic.",
...     backstory="""You work at a leading tech think tank. You have a knack for dissecting complex data and presenting actionable insights.""",
...     verbose=True,
...     allow_delegation=False,
...     tools=[search_tool],
...     llm=llm,
...     max_iter=1,
... )
... writer = Agent(
...     role="Technical Writer",
...     goal="Craft compelling content on a given topic.",
...     backstory="""You are a technical writer with a knack for crafting engaging and informative content.""",
...     llm=llm,
...     verbose=True,
...     allow_delegation=False,
...     max_iter=1,
... )

>>> # Create tasks for your agents
>>> task1 = Task(
...     description="""Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
...   Identify key trends, breakthrough technologies, and potential industry impacts.""",
...     expected_output="Full analysis report in bullet points",
...     agent=researcher,
... )

>>> task2 = Task(
...     description="""Using the insights provided, develop an engaging blog
...   post that highlights the importance of observability and tracing in LLM applications.
...   Your post should be informative yet accessible, catering to a tech-savvy audience.
...   Make it sound cool, avoid complex words so it doesn't sound like AI.""",
...     expected_output="Blog post of at least 3 paragraphs",
...     agent=writer,
... )

>>> # Instantiate your crew with a sequential process
>>> crew = Crew(
...     agents=[researcher, writer],
...     tasks=[task1, task2],
...     verbose=True,
...     process=Process.sequential,
... )

>>> # Get your crew to work!
>>> result = crew.kickoff()

>>> print("------------ FINAL RESULT ------------")
>>> print(result)

# Agent: Researcher
## Task: Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
  Identify key trends, breakthrough technologies, and potential industry impacts.


# Agent: Researcher
## Using tool: Search the internet
## Tool Input: 
"{\"search_query\": \"importance of observability and tracing in LLM applications\"}"
## Tool Output: 

Search results: Title: LLM Observability: The 5 Key Pillars for Monitoring Large Language ...
Link: https://arize.com/blog-course/large-language-model-monitoring-observability/
Snippet: Why leveraging the five pillars of LLM observability is essential for ensuring performance, reliability, and seamless LLM applications.
---
Title: Observability of LLM Applications: Exploration and Practice from the ...
Link: https://www.alibabacloud.com/blog/observability-of-llm-applications-exploration-and-practice-from-the-perspective-of-trace_601604
Snippet: This article clarifies the technical challenges of observability by analyzing LLM application patterns and different concerns.
---
Title: What is LLM Observability? - The Ultimate LLM Monitoring Guide
Link: https://www.confident-ai.com/blog/what-is-llm-observability-the-ultimate-llm-monitoring-guide
Snippet: Observability tools collect and correlate logs, real-time evaluation metrics, and traces to understand the context of unexpected outputs or ...
---
Title: An Introduction to Observability for LLM-based applications using ...
Link: https://opentelemetry.io/blog/2024/llm-observability/
Snippet: Why Observability Matters for LLM Applications · It's vital to keep track of how often LLMs are being used for usage and cost tracking. · Latency ...
---
Title: Understanding LLM Observability - Key Insights, Best Practices ...
Link: https://signoz.io/blog/llm-observability/
Snippet: LLM Observability is essential for maintaining reliable, accurate, and efficient AI applications. Focus on the five pillars: evaluation, tracing ...
---
Title: LLM Observability Tools: 2024 Comparison - lakeFS
Link: https://lakefs.io/blog/llm-observability-tools/
Snippet: LLM observability is the process that enables monitoring by providing full visibility and tracing in an LLM application system, as well as newer ...
---
Title: From Concept to Production with Observability in LLM Applications
Link: https://hadijaveed.me/2024/03/05/tracing-and-observability-in-llm-applications/
Snippet: Traces are essential to understanding the full “path” a request takes in your application, e.g, prompt, query-expansion, RAG retrieved top-k ...
---
Title: The Importance of LLM Observability: A Technical Deep Dive
Link: https://www.linkedin.com/pulse/importance-llm-observability-technical-deep-dive-patrick-carroll-trlqe
Snippet: LLM observability is crucial for any technical team that wants to maintain and improve the reliability, security, and performance of their AI- ...
---
Title: Observability and Monitoring of LLMs - TheBlue.ai
Link: https://theblue.ai/blog/llm-observability-en/
Snippet: LLM-Observability is crucial to maximize the performance and reliability of Large Language Models (LLMs). By systematically capturing and ...
---


导航回 Phoenix 仪表板后，我们可以在新的“crewai”项目中看到我们多智能体应用程序的追踪信息！