Skip to main content
The web_search tool empowers agents to retrieve, analyze, and synthesize information from the live web. It goes beyond simple keyword matching by employing a multi-stage process: searching, scraping full content, semantically re-ranking results, and finally generating a sourced answer.

​
When to Use This Tool

Use web_search when you need to:
  • Access Real-Time Data: Find information about events, prices, or releases that occurred recently.
  • Verify Facts: Cross-reference internal knowledge with external sources.
  • Broaden Research: Explore topics outside of your internal knowledge base.
  • Find Documentation: Look up the latest API references or technical guides that might not be in your training data.
Reranking & QualityThis tool doesn’t just return a list of links. It fetches the actual content of the top results and uses a specialized “reranker” model to score them based on how well they answer the specific user query. This ensures high-quality, relevant answers even if the initial search keywords were imperfect.

​
Input Parameters

The tool accepts the following parameters:
ParameterTypeRequiredDescription
querystringYesThe search query to send to the web browser. Be specific for best results.
time_rangestringNoFilter results by time. Options: day, month, year, none (default). Only use this if the user explicitly requests a timeframe.
whitelisted_domainsarray of stringsNoA list of domains to strictly filter the search results. If provided, only results from these domains will be returned.

​
Output Structure

The tool returns a JSON object containing the synthesized response and the sources used. 
{
  "query": "latest advancements in solid state batteries 2024",
  "response": "The user is asking about recent advancements in solid-state batteries. I need to find information from 2024 specifically. Looking at the search results, I see several relevant articles...\n\nRecent advancements in solid-state batteries in 2024 have focused on improving energy density and charging speeds. Toyota has announced a breakthrough claiming a range of 745 miles and a 10-minute charge time <web_citation url=\"https://example.com/toyota-news\" title=\"Toyota News\">[1]</web_citation>. Meanwhile, QuantumScape has begun shipping prototypes to automotive partners <web_citation url=\"https://example.com/quantumscape-update\" title=\"QuantumScape Update\">[2]</web_citation>.",
  "sources": [
    {
      "rank": 1,
      "title": "Toyota's Solid State Battery Roadmap",
      "url": "https://example.com/toyota-news",
      "snippet": "Toyota aims to commercialize the technology by 2027...",
      "rerank_score": 0.92,
      "used_in_response": true
    },
    // ... other sources
  ],
  "sources_used": [1, 2],
  "model": "claude-3-7-sonnet-20250219-thinking",
  "reranker": "llm_multimodal_tool_calling_binary_reranker::gpt-4o-mini",
  "total_results": 15,
  "reranked_results": 5,
  "execution_id": "call_zIsjIkJwvPrJTz2GbIRGg8DE"
}

​
Response Fields

FieldDescription
responseThe natural language answer generated by the model, including inline citations and potentially a “Reflection” block where the model plans its answer.
sourcesA list of the web pages that were analyzed. Includes the url, title, snippet, and a rerank_score indicating relevance.
sources_usedAn array of integers corresponding to the rank of the sources that were explicitly cited in the response.
modelThe LLM used to generate the response.
rerankerThe model used to re-order search results by relevance.
execution_idThe unique identifier for this tool execution.

​
Example Usage

Input: 
{
  "query": "current price of bitcoin"
}
Result: The agent will return the current price sourced from a financial news site or exchange data found on the web. Input: 
{
  "query": "earthquake in japan",
  "time_range": "day"
}
Result: The agent restricts results to the past 24 hours, ensuring the information is about recent events rather than historical ones.

​
Domain Whitelisting and Blacklisting

The Web Search tool supports domain whitelisting and blacklisting to give you control over the sources of information. There are two distinct ways this is handled, depending on your setup:

​
1. Strict Whitelisting (Tool Level)

When you configure whitelisted domains directly within the agent’s tool settings, this applies a strict filter.
  • If a whitelisted domain is provided, the tool will only return results from that specific domain.
  • If the search engine does not find any results from the whitelisted domain, no results will be shown to the agent.
  • This is ideal when you want to absolutely guarantee that the agent only uses information from a specific, trusted source (e.g., your own company’s documentation).

​
2. Granular Control & Bonus Scoring (Self-Hosted Secure Browser)

If you want a softer approach—where you still surface general web results but give a “bonus” score to trusted domains, or if you want to completely block certain domains—you can leverage a Self-hosted Secure Browser. By deploying your own browser instance, you can configure WHITELISTED_DOMAINS and BLACKLISTED_DOMAINS at the browser level:
  • Whitelisted Domains (Browser Level): The search engine will actively prioritize and give a bonus to results from these domains, but it will still return other relevant results if the trusted domains don’t have the answer.
  • Blacklisted Domains: The search engine will completely filter out results from these domains (e.g., blocking known spam or competitor sites).
This approach allows for a more flexible and granular control over the search environment without strictly limiting the agent’s ability to find answers.

​
How It Works

  1. Search: The tool queries a search engine to get a broad set of initial results.
  2. Scrape: It visits the top URLs to extract the full text content, not just the search snippet.
  3. Rerank: A specialized “reranker” model compares the user’s query against the full text of each page to determine which ones are truly relevant. 4. Generate: The most relevant chunks of text are passed to an LLM, which synthesizes the final answer and adds citations.

​
Streaming Events

When used in streaming mode, the web_search tool emits real-time events via SSE (Server-Sent Events). This allows you to track the progress of the search pipeline and display the answer as it is generated.

​
Event Types

EventDescription
tool_updateIndicates a progress update (phase change) or provides intermediate data like search results.
tool_partial_updateContains a new text fragment of the generated response (streaming).
errorSignals that a critical error occurred during execution (e.g., search engine timeout).
tool_endSignals the end of the tool execution and provides the full final result.

​
Pipeline Phases (tool_update)

The tool_update event contains a data field with a phase and a status. Here are the possible phases:
  1. WEB_SEARCH
    • status: started / completed
    • data: { "results_count": <int>, "sources": [...] } (on completion)
    • Indicates the initial query to the search engine. The sources array contains the raw search results: 
      // Example 'sources' structure in WEB_SEARCH phase
[
  {
    "rank": 1,
    "title": "Recent Advances in AI",
    "url": "https://example.com/ai-news",
    "snippet": "A summary of the latest breakthroughs...",
    "engine": "google",
    "score": 1.0
  }
]
  2. SCRAPING_URLS
  • status: started / completed
  • data: { "url_count": <int> } (on start), { "scraped_count": <int> } (on completion)
  • Indicates the fetching of full text content from the top URLs.
  • Advanced FeatureScraping full page content is an advanced feature and is disabled by default. When disabled, the tool relies solely on the search engine snippets. If you would like to enable full page scraping for your workspace, please contact the UBIK team at contact@ubik-agent.com.
    1. RERANKING
      • status: started / completed
      • data: { "reranked_count": <int> } (on completion). Note: The refined sources array is emitted in a separate tool_update event immediately after.
      • Indicates that results have been re-ranked by semantic relevance. After this phase completes, additional tool_update events are emitted containing the final reranked_results count and the refined sources array with semantic scores: 
        // Example 'sources' structure after RERANKING phase
[
  {
    "rank": 1,
    "title": "Recent Advances in AI",
    "url": "https://example.com/ai-news",
    "snippet": "A summary of the latest breakthroughs...",
    "rerank_score": 0.85,
    "used_in_response": false
  }
]
    2. GENERATION
  • status: started / completed
  • Indicates that the LLM generation of the final answer is starting.

​
Content Streaming (tool_partial_update)

During the generation phase, tool_partial_update events are emitted for each generated text fragment.
  • content: <string> (The text fragment)
  • output_key: "response"
These fragments must be concatenated to form the complete answer. The text may contain raw citation tags like <source_1> which are replaced with rich <web_citation> tags at the end of the generation.
Handling Large Events (Chunking)If an event payload (such as a large array of search results) exceeds the SSE size limit, it will be split into multiple _delta_sse events. For detailed instructions and code examples on how to buffer and reconstruct these chunked events, please refer to the Streaming Results Guide or the Agent Session Events Guide.

​
Example Event Flow

// Start Web Search
{ "event": "tool_update", "data": { "phase": "WEB_SEARCH", "status": "started", "query": "latest advancements in AI" } }

// Web Search completed
{ "event": "tool_update", "data": { "phase": "WEB_SEARCH", "status": "completed", "data": { "results_count": 10, "sources": [...] }, "output_key": "sources" } }

// Scraping started
{ "event": "tool_update", "data": { "phase": "SCRAPING_URLS", "status": "started", "data": { "url_count": 10 } } }

// Scraping completed
{ "event": "tool_update", "data": { "phase": "SCRAPING_URLS", "status": "completed", "data": { "scraped_count": 8 } } }

// Reranking started
{ "event": "tool_update", "data": { "phase": "RERANKING", "status": "started" } }

// Reranking completed
{ "event": "tool_update", "data": { "phase": "RERANKING", "status": "completed", "data": { "reranked_count": 8 } } }

// Final sources emitted
{ "event": "tool_update", "data": { "reranked_results": 5, "output_key": "reranked_results" } }
{ "event": "tool_update", "data": { "sources": [...], "output_key": "sources" } }

// Generation started
{ "event": "tool_update", "data": { "phase": "GENERATION", "status": "started" } }

// Response streaming (tool_partial_update)
{ "event": "tool_partial_update", "data": { "content": "According", "output_key": "response" } }
{ "event": "tool_partial_update", "data": { "content": " to", "output_key": "response" } }
{ "event": "tool_partial_update", "data": { "content": " the", "output_key": "response" } }
{ "event": "tool_partial_update", "data": { "content": " results <source_1>...", "output_key": "response" } }

// Generation completed
{ "event": "tool_update", "data": { "phase": "GENERATION", "status": "completed" } }

// End
{ 
  "event": "tool_end", 
  "data": { 
    "query": "latest advancements in AI",
    "response": "According to the results <web_citation url=\"...\">[1]</web_citation>...",
    "sources": [
      {
        "rank": 1,
        "title": "Recent Advances in AI",
        "url": "https://example.com/ai-news",
        "snippet": "A summary of the latest breakthroughs...",
        "rerank_score": 0.85,
        "used_in_response": true
      }
    ],
    "sources_used": [1],
    "model": "claude-sonnet-4-20250514",
    "reranker": "bge_text_reranker",
    "total_results": 10,
    "reranked_results": 5
  } 
}
ConfigurationJust like the RAG tool, the models used for generation and reranking in Web Search respect the settings configured in your User Preferences under the RAG section.