Ivy Nodes

A generic placeholder node for callable function references. This node serves as a template or placeholder for custom callable functions that can be passed between nodes. Currently outputs an empty function reference. Use this node when you need to create custom callable functions for use with other nodes like web-scraping (for next_page_rule or content_processor) or prompt-generator (for creator functions).

Converts CSV data from a BytesIO buffer into a human-readable string representation. This node reads CSV data from a buffer (typically from csv-from-data-frame), loads it into a pandas DataFrame, and converts it to a formatted string. Useful for displaying CSV data in logs, reports, or text-based outputs. The output can be used for debugging, logging, or displaying data in text format.

Creates a pandas DataFrame from a list of dictionaries. This is typically the starting point for data processing pipelines. Each dictionary in the list becomes a row in the DataFrame, with dictionary keys becoming column names. Use this node to convert structured data (e.g., from web-scraping or API responses) into a DataFrame for analysis. The output can be used with csv-from-data-frame for export or other data transformation nodes.

Converts a pandas DataFrame into CSV format stored in a BytesIO buffer. Use this node to export data from data processing pipelines. Typically used after data-frame-creator or data transformation nodes. The output buffer can be used with email-builder for attachments, minio-file-upload for storage, or csvto-string-converter for string representation.

Creates a default prompt formatting function that formats context and question in a simple text format. The function takes context and question as parameters and returns a formatted prompt string. The output prompt_creator function is used with prompt-generator node to generate prompts for LLM interactions. This is the simplest prompt format, suitable for most general-purpose AI tasks.

Builds email messages with optional attachments using Python's email library. This node creates MIMEMultipart email objects with sender, recipient, subject, body, and optional file attachments. The output email_message is used with email-provider node to send emails via SMTP. Supports attachments from BytesIO buffers (e.g., from csv-from-data-frame or chart generators). Use this node to prepare emails before sending them.

Sends email messages using an SMTP server. This node requires an email_message object created by the email-builder node. Use this node to deliver emails after building them with email-builder. Supports SMTP authentication with STARTTLS encryption. The node handles connection management and error reporting. Typically used in automated notification or reporting workflows.

Reads files from MinIO object storage and returns them as BytesIO buffers. This node downloads files from a specified bucket and file name, making them available for processing. The output buffer can be used with pdf-processor for PDFs, image-processor for images, or other file processing nodes. Requires minio-provider for the connection. Use this node to retrieve files from object storage for processing pipelines.

Uploads files to MinIO object storage from BytesIO buffers. This node stores file content in a specified bucket with a given file name and content type. The buffer can come from csv-from-data-frame, chart generators, or other nodes that produce file content. Requires minio-provider for the connection. Use this node to persist processed data or generated files to object storage for later retrieval or sharing.

Generates histogram charts from pandas DataFrames using Plotly and exports them as image buffers. This node creates interactive histogram visualizations with customizable titles, axis labels, dimensions, and image formats. The output chart_image buffer can be used with minio-file-upload for storage, email-builder for email attachments, or image-show for display. Requires a DataFrame (typically from data-frame-creator) and column specifications for visualization.

Generates text responses using language models hosted on Hugging Face Hub. Use this node to interact with various LLM models available on Hugging Face. Requires a valid Hugging Face API token. The node supports configurable timeout and token limits. Typically used in AI/ML pipelines for text generation, summarization, or question answering tasks.

Loads and processes images from BytesIO buffers using the Pillow (PIL) library. This node validates image data, loads it into a Pillow Image object, and provides an image processor instance for further operations. Supports common image formats (JPEG, PNG, etc.). The output image_processor is used with image-show for display or can be used for image manipulation tasks. The buffer can come from minio-file-reader or other file sources.

Displays images using the system's default image viewer. This node takes an image_processor instance from image-processor and displays the loaded image. Useful for debugging, visualization, or user interaction workflows. The image is shown in the operating system's default image viewer application. Requires image-processor node output.

Creates a JSON-formatted prompt function that formats context and question as a JSON string. The function returns a JSON object with 'context' and 'question' keys. This format is useful for structured data processing or when the LLM needs to parse the prompt as JSON. The output prompt_creator is used with prompt-generator node.

Ingests text documents into a Milvus collection for retrieval. Embeds each document with a Hugging Face model (CLS pooling + L2 normalize — identical to ivy.node.rag-kb-answer so query and index vectors match), (re)creates the target collection sized to the embedding dimension, and inserts {vector, text} rows. Run this once to populate the knowledge base before querying with rag-kb-answer. Demonstrates the multi-cell program layout (input / implementation / run cells share one kernel session).

Creates and manages a connection to a Milvus vector database. This node establishes a client connection that can be used for vector similarity search operations. Milvus is used for storing and searching high-dimensional vectors (embeddings). The client output is required by milvus-insert and milvus-search nodes. Use this node at the start of vector database workflows.

Inserts text embeddings into a Milvus vector database collection. This node converts text lines into embeddings using a text vectorizer, then stores them in Milvus for similarity search. If the collection already exists, it will be dropped and recreated. Requires milvus-client for database connection, text-vectorizer for embedding generation, and text_lines (typically from pdf-processor). Use this node to build a searchable knowledge base from documents.

Performs vector similarity search on a Milvus collection to find the most similar text embeddings. This node searches for vectors similar to the provided embedding vector and returns the matching text content. Typically used in RAG (Retrieval-Augmented Generation) pipelines where the embedding comes from question node. The search results (context) are used with prompt-generator to create prompts for LLM queries. Requires milvus-client and an embedding vector.

Creates and manages a connection to a MinIO object storage server (S3-compatible). This node establishes a client connection that can be used for file operations. MinIO provides object storage similar to AWS S3. The provider can optionally create a default bucket during initialization. The output minio_provider is required by minio-file-reader and minio-file-upload nodes. Use this node at the start of object storage workflows.

Generates text responses using OpenAI's chat completion API. Supports various models including GPT-4, GPT-4o, and o1 series. This node requires an OpenAI API key and accepts both a prompt object (from prompt-generator) and direct system/user messages. Use this for high-quality text generation, summarization, or conversational AI tasks. Note: This is a high-risk node requiring step-by-step approval due to API costs.

Extracts and splits text content from PDF documents into manageable chunks. This node processes PDF files stored in BytesIO buffers, extracts all text content, and splits it into chunks of 1000 characters with 200 character overlap. The output text_lines array is ideal for vectorization and storage in vector databases. Typically used with text-vectorizer and milvus-insert for building document search systems. The buffer can come from minio-file-reader or other file sources.

Generates formatted prompts using a prompt creator function and validates token limits. This node takes a creator function (from default-prompt-creator, json-prompt-creator, or xml-prompt-creator), context, and question to produce a formatted prompt string. It enforces maximum token limits to prevent exceeding LLM input constraints. The output prompt is typically used with open-ai-client or hugging-face-llm-client nodes for AI interactions.

Converts a textual question into a numerical vector representation using a text vectorizer. This node is essential for RAG (Retrieval-Augmented Generation) pipelines where questions need to be vectorized for similarity search. The output vectorize array is used with milvus-search to find similar content, while the text output preserves the original question. Requires text-vectorizer from text-vectorizer node. Typically used before milvus-search in question-answering workflows.

Single-node retrieval-augmented generation: embeds the question with a Hugging Face model, retrieves the most similar passages from a Milvus collection, assembles a grounded context, and asks an OpenAI chat model for a context-grounded answer. The whole external call is governed (policy risk:high, approval:step) so an agent cannot answer from the knowledge base without approval. All third-party calls run inside the node's Python (compute kernel), matching the existing node library convention.

Splits a long text document into overlapping chunks suitable for embedding and retrieval-augmented generation (RAG). Each chunk is a contiguous slice of the source text bounded by chunk_size with a configurable overlap to preserve context across boundaries. The output chunks list feeds text-vectorizer for embedding before milvus-insert. This node is pure-compute and performs no network or file access.

Creates a text vectorizer instance using a pre-trained tokenizer and model. This node is essential for ML pipelines that require text embeddings. The vectorizer can convert text into numerical vectors for similarity search, classification, or clustering. Requires tokenizer and model from tokenizer-model-loader node. The output vectorizer is used with question node for query vectorization or milvus-insert for storing embeddings.

Loads a pre-trained tokenizer and model from Hugging Face for text embedding generation. This node downloads and initializes both the tokenizer and model required for text vectorization. The outputs are used with text-vectorizer node to create embedding instances. Requires a Hugging Face access token for authentication (especially for private models). This is typically the first step in ML pipelines that require text embeddings.

Scrapes web pages with flexible pagination and content extraction logic. This node uses BeautifulSoup for HTML parsing and supports custom pagination rules and content processors (provided as callable functions). It can scrape single pages or follow pagination automatically. The output scraped_dictionary (list of dictionaries) is typically used with data-frame-creator to convert into a DataFrame for analysis. Requires next_page_rule and content_processor callable functions (can be created with callable node or custom functions).

Creates an XML-formatted prompt function that wraps context and question in XML tags (<context> and <question>). This format is useful for models that understand structured XML markup, providing clear separation between context and question. The output creator function is used with prompt-generator node to generate XML-formatted prompts for LLM interactions.