在 Hugging Face Spaces 上为 LLM 应用追踪设置 Phoenix 可观测性仪表板

作者：Andrew Reed

Phoenix 是 Arize AI 开发的开源可观测性库，专为实验、评估和故障排除而设计。它允许 AI 工程师和数据科学家快速可视化他们的数据、评估性能、追踪问题并导出数据以进行改进。

追踪

数据集

实验

在本笔记本中，我们将了解如何在 Hugging Face Spaces 上部署 Phoenix 可观测性仪表板，并将其配置为自动追踪 LLM 调用，从而全面了解 LLM 应用程序的内部工作原理。

步骤 1：在 Hugging Face Spaces 上部署 Phoenix

虽然 Phoenix 提供了用于本地开发的 notebook 优先选项，但它也可以通过 Docker 部署为 独立仪表板。长期运行的托管仪表板是提供 LLM 应用程序行为集中视图以及与团队协作的好方法。Hugging Face Spaces 提供了一种使用可选持久存储托管 ML 应用程序的简单方法，并且它对自定义 Docker 镜像的支持使其成为托管 Phoenix 的绝佳平台 - 让我们看看它是如何工作的！

首先，我们将复制演示空间

我们可以将 Space 配置为私有或公共，并且它可以位于我们的用户命名空间或组织命名空间中。我们可以使用默认的免费层级 CPU，并且重要的是，指定我们要将持久磁盘附加到 Space。

点击“Duplicate Space”后，Docker 镜像将开始构建。这将需要几分钟才能完成，然后我们将看到一个空的 Phoenix 仪表板。

步骤 2：配置应用程序追踪

现在我们有了一个正在运行的 Phoenix 仪表板，我们可以配置我们的应用程序以使用 OpenTelemetry TracerProvider 自动追踪 LLM 调用。在本例中，我们将使用 OpenAI 客户端库检测我们的应用程序，并追踪从 openai Python 包到在 Hugging Face 的 Serverless Inference API 上运行的开放 LLM 的 LLM 调用。

首先，我们需要安装必要的库

!pip install -q arize-phoenix arize-phoenix-otel openinference-instrumentation-openai openai huggingface-hub

然后，我们将使用 huggingface_hub 库登录 Hugging Face。这将允许我们为我们的 Space 和 Serverless Inference API 生成必要的身份验证。请确保用于身份验证的 HF 令牌对于您的 Space 所在的组织具有正确的权限。

from huggingface_hub import interpreter_login

interpreter_login()

现在，我们可以将 Phoenix 客户端配置到我们正在运行的 Phoenix 仪表板

1. 通过以下方式注册 Phoenix tracer provider
   • 指定我们选择的 project_name
   • 将 endpoint 值设置为我们的 Space 的主机名（通过仪表板 UI 在“设置”选项卡下找到 - 见下文）
   • 将 headers 设置为访问 Space 所需的 Hugging Face headers
2. 检测我们的应用程序代码以使用 OpenAI tracer provider

>>> from phoenix.otel import register
>>> from huggingface_hub.utils import build_hf_headers
>>> from openinference.instrumentation.openai import OpenAIInstrumentor

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="test",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: test
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

步骤 3：在 Phoenix 仪表板中进行调用并查看追踪

现在，我们可以调用 LLM 并在 Phoenix 仪表板中查看追踪。我们正在使用 OpenAI 客户端调用 Hugging Face Serverless Inference API，该 API 已被检测以与 Phoenix 一起工作。在本例中，我们使用 meta-llama/Llama-3.1-8B-Instruct 模型。

>>> from openai import OpenAI
>>> from huggingface_hub import get_token

>>> client = OpenAI(
...     base_url="https://api-inference.huggingface.co/v1/",
...     api_key=get_token(),
... )

>>> messages = [{"role": "user", "content": "What does a llama do for fun?"}]

>>> response = client.chat.completions.create(
...     model="meta-llama/Llama-3.1-8B-Instruct",
...     messages=messages,
...     max_tokens=500,
... )

>>> print(response.choices[0].message.content)

Llamas are intelligent and social animals, and they do have ways to entertain themselves and have fun. While we can't directly ask a llama about its personal preferences, we can observe their natural behaviors and make some educated guesses. Here are some things that might bring a llama joy and excitement:

1. **Socializing**: Llamas are herd animals and they love to interact with each other. They'll often engage in play-fighting, neck-wrestling, and other activities to establish dominance and strengthen social bonds. When llamas have a strong social network, it can make them feel happy and content.
2. **Exploring new environments**: Llamas are naturally curious creatures, and they love to explore new surroundings. They'll often investigate their surroundings, sniffing and investigating new sights, sounds, and smells.
3. **Playing with toys**: While llamas don't need expensive toys, they do enjoy playing with objects that stimulate their natural behaviors. For example, a ball or a toy that mimics a target can be an entertaining way to engage them.
4. **Climbing and jumping**: Llamas are agile and athletic animals, and they enjoy using their limbs to climb and jump over obstacles. Providing a safe and stable area for them to exercise their physical abilities can be a fun and engaging experience.
5. **Browsing and foraging**: Llamas have a natural instinct to graze and browse, and they enjoy searching for tasty plants and shrubs. Providing a variety of plants to munch on can keep them engaged and entertained.
6. **Mentally stimulating activities**: Llamas are intelligent animals, and they can benefit from mentally stimulating activities like problem-solving puzzles or learning new behaviors (like agility training or obedience training).

Some fun activities you can try with a llama include:

* Setting up an obstacle course or agility challenge
* Creating a "scavenger hunt" with treats and toys
* Introducing new toys or objects to stimulate their curiosity
* Providing a variety of plants and shrubs to browse and graze on
* Engaging in interactive games like "follow the leader" or "find the treat"

Remember to always prioritize the llama's safety and well-being, and to consult with a veterinarian or a trained llama handler before attempting any new activities or introducing new toys.

如果我们导航回 Phoenix 仪表板，我们可以看到来自我们 LLM 调用的追踪已被捕获并显示！如果您使用持久磁盘配置了您的 Space，则每次重启 Space 时，所有追踪信息都将被保存。

奖励：使用 CrewAI 追踪多代理应用程序

可观测性的真正力量来自于能够追踪和检查复杂的 LLM 工作流程。在本例中，我们将安装并使用 CrewAI 来追踪多代理应用程序。

!pip install -q openinference-instrumentation-crewai crewai crewai-tools

像之前一样，我们将注册 Phoenix tracer provider 并检测应用程序代码，但这次我们还将取消检测现有的 OpenAI tracer provider 以避免冲突。

>>> from opentelemetry import trace
>>> from openinference.instrumentation.crewai import CrewAIInstrumentor

>>> # 0. Uninstrument existing tracer provider and clear the global tracer provider
>>> OpenAIInstrumentor().uninstrument()
>>> if trace.get_tracer_provider():
...     trace.get_tracer_provider().shutdown()
...     trace._TRACER_PROVIDER = None  # Reset the global tracer provider

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="crewai",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> CrewAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: crewai
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

现在我们将使用 CrewAI 定义一个多代理应用程序，以研究和撰写一篇关于可观测性和追踪在 LLM 应用程序中的重要性的博客文章。

>>> import os
>>> from huggingface_hub import get_token
>>> from crewai_tools import SerperDevTool
>>> from crewai import LLM, Agent, Task, Crew, Process

>>> # Define our LLM using HF's Serverless Inference API
>>> llm = LLM(
...     model="huggingface/meta-llama/Llama-3.1-8B-Instruct",
...     api_key=get_token(),
...     max_tokens=1024,
... )

>>> # Define a tool for searching the web
>>> os.environ["SERPER_API_KEY"] = "YOUR_SERPER_API_KEY"  # must set this value in your environment
>>> search_tool = SerperDevTool()

>>> # Define your agents with roles and goals
>>> researcher = Agent(
...     role="Researcher",
...     goal="Conduct thorough research on up to date trends around a given topic.",
...     backstory="""You work at a leading tech think tank. You have a knack for dissecting complex data and presenting actionable insights.""",
...     verbose=True,
...     allow_delegation=False,
...     tools=[search_tool],
...     llm=llm,
...     max_iter=1,
... )
... writer = Agent(
...     role="Technical Writer",
...     goal="Craft compelling content on a given topic.",
...     backstory="""You are a technical writer with a knack for crafting engaging and informative content.""",
...     llm=llm,
...     verbose=True,
...     allow_delegation=False,
...     max_iter=1,
... )

>>> # Create tasks for your agents
>>> task1 = Task(
...     description="""Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
...   Identify key trends, breakthrough technologies, and potential industry impacts.""",
...     expected_output="Full analysis report in bullet points",
...     agent=researcher,
... )

>>> task2 = Task(
...     description="""Using the insights provided, develop an engaging blog
...   post that highlights the importance of observability and tracing in LLM applications.
...   Your post should be informative yet accessible, catering to a tech-savvy audience.
...   Make it sound cool, avoid complex words so it doesn't sound like AI.""",
...     expected_output="Blog post of at least 3 paragraphs",
...     agent=writer,
... )

>>> # Instantiate your crew with a sequential process
>>> crew = Crew(
...     agents=[researcher, writer],
...     tasks=[task1, task2],
...     verbose=True,
...     process=Process.sequential,
... )

>>> # Get your crew to work!
>>> result = crew.kickoff()

>>> print("------------ FINAL RESULT ------------")
>>> print(result)

# Agent: Researcher
## Task: Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
  Identify key trends, breakthrough technologies, and potential industry impacts.


# Agent: Researcher
## Using tool: Search the internet
## Tool Input: 
"{\"search_query\": \"importance of observability and tracing in LLM applications\"}"
## Tool Output: 

Search results: Title: LLM Observability: The 5 Key Pillars for Monitoring Large Language ...
Link: https://arize.com/blog-course/large-language-model-monitoring-observability/
Snippet: Why leveraging the five pillars of LLM observability is essential for ensuring performance, reliability, and seamless LLM applications.
---
Title: Observability of LLM Applications: Exploration and Practice from the ...
Link: https://www.alibabacloud.com/blog/observability-of-llm-applications-exploration-and-practice-from-the-perspective-of-trace_601604
Snippet: This article clarifies the technical challenges of observability by analyzing LLM application patterns and different concerns.
---
Title: What is LLM Observability? - The Ultimate LLM Monitoring Guide
Link: https://www.confident-ai.com/blog/what-is-llm-observability-the-ultimate-llm-monitoring-guide
Snippet: Observability tools collect and correlate logs, real-time evaluation metrics, and traces to understand the context of unexpected outputs or ...
---
Title: An Introduction to Observability for LLM-based applications using ...
Link: https://opentelemetry.io/blog/2024/llm-observability/
Snippet: Why Observability Matters for LLM Applications · It's vital to keep track of how often LLMs are being used for usage and cost tracking. · Latency ...
---
Title: Understanding LLM Observability - Key Insights, Best Practices ...
Link: https://signoz.io/blog/llm-observability/
Snippet: LLM Observability is essential for maintaining reliable, accurate, and efficient AI applications. Focus on the five pillars: evaluation, tracing ...
---
Title: LLM Observability Tools: 2024 Comparison - lakeFS
Link: https://lakefs.io/blog/llm-observability-tools/
Snippet: LLM observability is the process that enables monitoring by providing full visibility and tracing in an LLM application system, as well as newer ...
---
Title: From Concept to Production with Observability in LLM Applications
Link: https://hadijaveed.me/2024/03/05/tracing-and-observability-in-llm-applications/
Snippet: Traces are essential to understanding the full “path” a request takes in your application, e.g, prompt, query-expansion, RAG retrieved top-k ...
---
Title: The Importance of LLM Observability: A Technical Deep Dive
Link: https://www.linkedin.com/pulse/importance-llm-observability-technical-deep-dive-patrick-carroll-trlqe
Snippet: LLM observability is crucial for any technical team that wants to maintain and improve the reliability, security, and performance of their AI- ...
---
Title: Observability and Monitoring of LLMs - TheBlue.ai
Link: https://theblue.ai/blog/llm-observability-en/
Snippet: LLM-Observability is crucial to maximize the performance and reliability of Large Language Models (LLMs). By systematically capturing and ...
---


导航回 Phoenix 仪表板后，我们可以在新项目“crewai”中看到来自我们的多代理应用程序的追踪！