OpenBB Copilot

The OpenBB Copilot is an AI-powered companion seamlessly integrated into the OpenBB Workspace, designed to enhance and accelerate your financial analysis workflow.

This sophisticated assistant understands natural language queries, retrieves data from multiple sources, performs complex analysis, and generates actionable insights.

Built for financial professionals, the Copilot transforms your workflow by accelerating analysis and providing context-aware insights based on your specific datasets and workspace configuration.

Basics

Structure

The Copilot interface is designed for intuitive interaction with a clean three-section layout:

• Header: At the top, you'll find controls to manage your conversation. You can see which chat you're in, start a new one, clear the history, expand to full-screen mode, or hide the agent entirely.

• Body: The main chat window where your conversation takes place. It displays the Copilot's answers, its step-by-step reasoning, and any output like charts, tables, or code. The chat automatically scrolls, keeping everything in chronological order. Hover over a message to see its timestamp.

• Footer: Here, you can manage the data and widgets the Copilot uses as context for its answers. You can also add your own custom AI agents or open your prompt library.

Full screen AI-chat

You can resize the AI panel by dragging the border.

In side-panel mode, it provides quick access for simple queries while maintaining your primary workspace view.

When expanded to full-screen mode, you gain maximum real estate for complex conversations, detailed reasoning steps, and large artifacts like comprehensive charts or extensive data tables.

This flexibility allows seamless transitions between quick consultations and deep analytical sessions without losing context or interrupting your workflow.

You can also hide the AI agent entirely if you want to work solely with the dashboard.

Prompt library

The integrated prompt library serves as a productivity multiplier by storing and organizing your most valuable queries. You can save complex prompts and even tag widgets, to ensure the right context is utilized for the right prompt.

Model

The OpenBB Copilot is specifically optimized for financial analysis and data interpretation tasks, leveraging the latest models from OpenAI.

For enterprise deployments, OpenBB provides seamless integration with your organization's Azure OpenAI.

This ensures compliance with internal security and data sovereignty policies.

If you would like full control over your agent capabilities, here's our open source repository with examples of AI custom agents that you can build and integrate into the OpenBB Workspace.

Context

The Copilot employs a sophisticated context understanding system that prioritizes and processes multiple information sources simultaneously.

This hierarchical approach ensures that the most relevant and specific data takes precedence when generating responses, while still maintaining awareness of broader workspace context and conversation history.

Priority

The agent prioritizes context in the following order:

Priority	Context Type	Description
1	Explicit	Widgets added to context specifically.
2	Attached files	Files uploaded directly to the Copilot
3	Dashboard	All widgets currently on your dashboard (in all tabs).
4	Global	All widgets available within the OpenBB Workspace.
5	Conversation	The history of your current conversation.

Explicit context

Explicit context represents the highest priority information source, allowing you to precisely direct the Copilot's attention to specific datasets or widgets. This mechanism is particularly powerful for ensuring accuracy when working with multiple similar datasets or when you need analysis focused on a specific dataset.

When you click on the "Add to context" button on any widget you're creating a direct reference that the Copilot will prioritize above all other available information. After sending the prompt, that widget will remain in context.

When you use the "@" symbol followed by a widget name and the prompt is sent, the widget will disappear from context. This is meant to be a quick tag reference.

Dashboard

The dashboard context provides the Copilot with comprehensive awareness of your currently active dashboard. All widgets on your active dashboard become automatically available as data sources, allowing the Copilot to understand the broader context of your work session. It also has access to the widgets metadata and the current parameters selected.

Attached Files

The attachment system enables the Copilot to process documents and datasets that aren't part of your current workspace widgets. Supported file types include PDF, Excel, CSV, and various other document formats.

The Copilot automatically extracts and indexes content from attached files, making them searchable and referenceable throughout your conversation. This capability is essential for performing ad-hoc analysis or incorporating external datasets and documents into your workflow.

Please note that for PDF files, text must be selectable as Optical Character Recognition (OCR) is not supported. However, you can integrate your own agent that performs OCR.

Global Retrieval

The global retrieval system provides the Copilot with access to the entire OpenBB Workspace widget library, extending far beyond your current dashboard widgets.

When your current dashboard doesn't contain the specific data or analysis tool needed to answer a query, the Copilot can automatically identify and utilize the appropriate widgets from the entire widget library, effectively expanding your analytical capabilities on-demand without manual widget selection.

This is only possible due to the metadata associated with each widget and is why it's important to invest time in specifying the metadata carefully.

Web Search

The web search capability enables the Copilot to access real-time information from across the internet when:

• The available workspace data is insufficient to fully answer your query.
• The user asks to look for the information on the web.

This feature automatically activates as a fallback mechanism, ensuring comprehensive responses even when dealing with breaking news, recent market developments, or information not available in your current workspace widgets.

Conversation

Conversational context enables natural, iterative analysis by maintaining awareness of your entire dialogue history within the current session.

The Copilot tracks your explicit questions and its responses. This memory allows for follow-up queries, refinement of analysis parameters, and building upon previous insights without needing to re-establish context.

The system understands references to "the previous chart", "that analysis", or "the data we discussed" and can seamlessly continue complex analytical workflows across multiple conversation turns.

Data Handling

Structured data

The Copilot excels at processing and analyzing structured financial datasets through multiple specialized capabilities:

• AgGrid (table/chart) widgets: A natural language to SQL translation tool that helps you query tabular data using plain English. Once your data is loaded in a tabular format, text2sql converts your questions into SQL queries to retrieve specific information from these tables. This eliminates the need to write SQL manually while allowing you to explore and analyze your data through simple natural language questions.

• Plotly widgets: Full-featured charting engine that not only generates interactive visualizations but also allows developers to provide the underlying data. The Copilot can extract specific data points, and create derivative analyses from existing visualizations.

Unstructured data

The Copilot's unstructured data processing capabilities enable comprehensive analysis of diverse document types and media:

• Document Processing (MD/PDF): Advanced text extraction and comprehension. The system maintains document structure understanding, preserving context around tables and hierarchical information. This utilizes a sophisticated retrieval system that chunks large documents intelligently, maintains semantic relationships, and provides precise citations. The system can cross-reference information across multiple documents and identify contradictions or supporting evidence.

• Web search:: When the user provides a URL to the AI agent, it converts the web page to markdown for it to be parsed by the model - as done above.

• Image Analysis: Image processing capabilities for charts, screenshots, financial diagrams, and infographics. The Copilot can extract data from visual representations, understand chart types, and incorporate visual information into broader analytical workflows.

Tools

Widgets

The Copilot treats every widget in the OpenBB Workspace as a programmable tool in its analytical toolkit.

The system can chain widget operations, using the output from one widget as input for another, creating sophisticated analytical pipelines automatically.

Widget parameters

Dynamic parameter modification represents one of the OpenBB's Copilot most powerful capabilities.

The system understands the parameter schemas of each widget and can intelligently modify settings like date ranges, asset symbols, calculation periods, and analysis parameters to match your specific requirements.

This eliminates the need for manual widget configuration and enables the Copilot to perform iterative analysis with different parameters automatically, such as comparing the same analysis across different time periods or asset classes within a single conversation.

Web search

The integrated web search capability extends the Copilot's knowledge beyond the OpenBB Workspace ecosystem to access real-time information from across the internet.

This feature automatically activates as a fallback mechanism when information is not available in your workspace, searching financial news sources, regulatory databases, company websites, and other sources.

The system filters and prioritizes results based on relevance and credibility, then integrates external information with your existing workspace data to provide comprehensive answers with proper source attribution.

Step-by-step reasoning

Transparency and auditability are fundamental to the OpenBB's Copilot design, with comprehensive step-by-step reasoning displayed for every analysis.

This detailed process visibility enables users to understand the logical flow, verify data sources, identify potential biases, and ensure compliance with analytical standards.

The reasoning display serves both educational and quality assurance purposes, building user confidence while maintaining the ability to audit and reproduce analytical processes.

Planning

The planning phase demonstrates the Copilot's tactical thinking by decomposing complex analytical requests into logical, sequential sub-tasks. This planning process considers data dependencies, optimal execution order, and resource requirements.

The displayed plan serves as a roadmap that users can review before execution, ensuring alignment with analytical objectives and providing clear expectations for the upcoming analysis workflow.

Querying widgets

Widget query transparency provides complete visibility into data retrieval operations, including the specific widget accessed, all parameters used, and data source information.

This documentation enables users to understand exactly what data was retrieved, verify parameter settings, and reproduce the analysis independently. The system also displays any parameter modifications made automatically, ensuring full awareness of how the Copilot adapted widget configurations to meet query requirements.

Intermediate result artifact

Intermediate artifacts provide crucial visibility into the Copilot's analytical methodology by displaying generated code, SQL queries, calculation formulas, and other technical implementations.

These artifacts serve multiple purposes: enabling technical review and validation, supporting learning and knowledge transfer, facilitating debugging and optimization, and ensuring compliance with analytical standards.

Users can examine and reuse these artifacts, treating them as valuable analytical assets beyond their immediate application.

Artifact generated

Final artifacts represent the culmination of the analytical process, displayed with complete context about their creation methodology. The step-by-step reasoning (status updates) maintain full provenance information, including data sources, transformation steps, and parameter settings used in their generation.

The system preserves the relationship between artifacts and their creation process, enabling users to understand not just what was created, but how and why, supporting both immediate use and future reference or modification.

Output

The Copilot's output system delivers comprehensive, contextual responses that synthesize all available information into actionable insights. Each response is structured to provide immediate answers while supporting deeper investigation, combining direct responses with supporting evidence, data visualizations, and clear pathways for follow-up analysis.

Citations

Comprehensive citation system ensures full traceability and verification of all information sources used in responses. The citation system provides different levels of detail based on source type:

• Widget citations: Direct links to widgets with parameters selected. Smart linking enables "Scroll to widget" functionality for existing dashboard widgets. If the widget doesn't exist in the dashboard or has modified parameters the linking will provide a "Add widget to dashboard" options to facilitate workspace workflow.

• Document citations: Precise page and section references for PDF documents with automatic highlighting of relevant content areas.

• Web citations: Full URL references when web pages are utilized.

Create widgets from AI output

The widget creation system enables seamless integration of Copilot-generated content into your workspace environment.

This capability transforms temporary conversational outputs into permanent analytical assets that can be referenced, shared, and incorporated into ongoing workflows.

The system supports multiple artifact types including formatted text summaries, interactive data tables with sorting and filtering capabilities, and fully functional charts with preserved interactivity.

• Text widget:

• Table widget:

• Chart widget:

User feedback loop

The thumbs up and down buttons in the agent output allow the user to provide feedback in terms of their experience with the copilot. That data can be utilized to create a flywheel that allows developers to understand how the AI agent can be improved.

While the thumbs up does not prompt the user for any additional details.

The thumbs down asks for clarification of what went wrong.