ai package

The Firebase AI Web SDK.

Functions

Function Description
function(app, ...)
getAI(app, options) Returns the default AI instance that is associated with the provided FirebaseApp. If no instance exists, initializes a new instance with the default settings.
getVertexAI(app, options)
function(ai, ...)
getGenerativeModel(ai, modelParams, requestOptions) Returns a GenerativeModel class with methods for inference and other functionality.
getImagenModel(ai, modelParams, requestOptions) (Public Preview) Returns an ImagenModel class with methods for using Imagen.Only Imagen 3 models (named imagen-3.0-*) are supported.

Classes

Class Description
AIError Error class for the Firebase AI SDK.
AIModel Base class for Firebase AI model APIs.Instances of this class are associated with a specific Firebase AI Backend and provide methods for interacting with the configured generative model.
ArraySchema Schema class for "array" types. The items param should refer to the type of item that can be a member of the array.
Backend Abstract base class representing the configuration for an AI service backend. This class should not be instantiated directly. Use its subclasses; GoogleAIBackend for the Gemini Developer API (via Google AI), and VertexAIBackend for the Vertex AI Gemini API.
BooleanSchema Schema class for "boolean" types.
ChatSession ChatSession class that enables sending chat messages and stores history of sent and received messages so far.
GenerativeModel Class for generative model APIs.
GoogleAIBackend Configuration class for the Gemini Developer API.Use this with AIOptions when initializing the AI service via getAI() to specify the Gemini Developer API as the backend.
ImagenImageFormat (Public Preview) Defines the image format for images generated by Imagen.Use this class to specify the desired format (JPEG or PNG) and compression quality for images generated by Imagen. This is typically included as part of ImagenModelParams.
ImagenModel (Public Preview) Class for Imagen model APIs.This class provides methods for generating images using the Imagen model.
IntegerSchema Schema class for "integer" types.
NumberSchema Schema class for "number" types.
ObjectSchema Schema class for "object" types. The properties param must be a map of Schema objects.
Schema Parent class encompassing all Schema types, with static methods that allow building specific Schema types. This class can be converted with JSON.stringify() into a JSON string accepted by Vertex AI REST endpoints. (This string conversion is automatically done when calling SDK methods.)
StringSchema Schema class for "string" types. Can be used with or without enum values.
VertexAIBackend Configuration class for the Vertex AI Gemini API.Use this with AIOptions when initializing the AI service via getAI() to specify the Vertex AI Gemini API as the backend.

Enumerations

Enumeration Description
AIErrorCode Standardized error codes that AIError can have.
BlockReason Reason that a prompt was blocked.
FinishReason Reason that a candidate finished.
FunctionCallingMode
HarmBlockMethod This property is not supported in the Gemini Developer API (GoogleAIBackend).
HarmBlockThreshold Threshold above which a prompt or candidate will be blocked.
HarmCategory Harm categories that would cause prompts or candidates to be blocked.
HarmProbability Probability that a prompt or candidate matches a harm category.
HarmSeverity Harm severity levels.
ImagenAspectRatio (Public Preview) Aspect ratios for Imagen images.To specify an aspect ratio for generated images, set the aspectRatio property in your ImagenGenerationConfig.See the the documentation for more details and examples of the supported aspect ratios.
ImagenPersonFilterLevel (Public Preview) A filter level controlling whether generation of images containing people or faces is allowed.See the personGeneration documentation for more details.
ImagenSafetyFilterLevel (Public Preview) A filter level controlling how aggressively to filter sensitive content.Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, violence, sexual, derogatory, and toxic). This filter level controls how aggressively to filter out potentially harmful content from responses. See the documentation and the Responsible AI and usage guidelines for more details.
Modality Content part modality.
SchemaType Contains the list of OpenAPI data types as defined by the OpenAPI specification

Interfaces

Interface Description
AI An instance of the Firebase AI SDK.Do not create this instance directly. Instead, use getAI().
AIOptions Options for initializing the AI service using getAI(). This allows specifying which backend to use (Vertex AI Gemini API or Gemini Developer API) and configuring its specific options (like location for Vertex AI).
BaseParams Base parameters for a number of methods.
Citation A single citation.
CitationMetadata Citation metadata that may be found on a GenerateContentCandidate.
Content Content type for both prompts and response candidates.
CountTokensRequest Params for calling GenerativeModel.countTokens()
CountTokensResponse Response from calling GenerativeModel.countTokens().
CustomErrorData Details object that contains data originating from a bad HTTP response.
Date_2 Protobuf google.type.Date
EnhancedGenerateContentResponse Response object wrapped with helper methods.
ErrorDetails Details object that may be included in an error response.
FileData Data pointing to a file uploaded on Google Cloud Storage.
FileDataPart Content part interface if the part represents FileData
FunctionCall A predicted FunctionCall returned from the model that contains a string representing the FunctionDeclaration.name and a structured JSON object containing the parameters and their values.
FunctionCallingConfig
FunctionCallPart Content part interface if the part represents a FunctionCall.
FunctionDeclaration Structured representation of a function declaration as defined by the OpenAPI 3.0 specification. Included in this declaration are the function name and parameters. This FunctionDeclaration is a representation of a block of code that can be used as a Tool by the model and executed by the client.
FunctionDeclarationsTool A FunctionDeclarationsTool is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model.
FunctionResponse The result output from a FunctionCall that contains a string representing the FunctionDeclaration.name and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a FunctionCall made based on model prediction.
FunctionResponsePart Content part interface if the part represents FunctionResponse.
GenerateContentCandidate A candidate returned as part of a GenerateContentResponse.
GenerateContentRequest Request sent through GenerativeModel.generateContent()
GenerateContentResponse Individual response from GenerativeModel.generateContent() and GenerativeModel.generateContentStream(). generateContentStream() will return one in each chunk until the stream is done.
GenerateContentResult Result object returned from GenerativeModel.generateContent() call.
GenerateContentStreamResult Result object returned from GenerativeModel.generateContentStream() call. Iterate over stream to get chunks as they come in and/or use the response promise to get the aggregated response when the stream is done.
GenerationConfig Config options for content-related requests
GenerativeContentBlob Interface for sending an image.
GroundingAttribution
GroundingMetadata Metadata returned to client when grounding is enabled.
ImagenGCSImage An image generated by Imagen, stored in a Cloud Storage for Firebase bucket.This feature is not available yet.
ImagenGenerationConfig (Public Preview) Configuration options for generating images with Imagen.See the documentation for more details.
ImagenGenerationResponse (Public Preview) The response from a request to generate images with Imagen.
ImagenInlineImage (Public Preview) An image generated by Imagen, represented as inline data.
ImagenModelParams (Public Preview) Parameters for configuring an ImagenModel.
ImagenSafetySettings (Public Preview) Settings for controlling the aggressiveness of filtering out sensitive content.See the documentation for more details.
InlineDataPart Content part interface if the part represents an image.
ModalityTokenCount Represents token counting info for a single modality.
ModelParams Params passed to getGenerativeModel().
ObjectSchemaInterface Interface for ObjectSchema class.
PromptFeedback If the prompt was blocked, this will be populated with blockReason and the relevant safetyRatings.
RequestOptions Params passed to getGenerativeModel().
RetrievedContextAttribution
SafetyRating A safety rating associated with a GenerateContentCandidate
SafetySetting Safety setting that can be sent as part of request parameters.
SchemaInterface Interface for Schema class.
SchemaParams Params passed to Schema static methods to create specific Schema classes.
SchemaRequest Final format for Schema params passed to backend requests.
SchemaShared Basic Schema properties shared across several Schema-related types.
Segment
StartChatParams Params for GenerativeModel.startChat().
TextPart Content part interface if the part represents a text string.
ToolConfig Tool config. This config is shared for all tools provided in the request.
UsageMetadata Usage metadata about a GenerateContentResponse.
VertexAIOptions Options when initializing the Firebase AI SDK.
VideoMetadata Describes the input video content.
WebAttribution

Variables

Variable Description
BackendType An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with.These values are assigned to the backendType property within the specific backend configuration objects (GoogleAIBackend or VertexAIBackend) to identify which service to target.
POSSIBLE_ROLES Possible roles.
ResponseModality (Public Preview) Generation modalities to be returned in generation responses.
VertexAIError
VertexAIModel

Type Aliases

Type Alias Description
BackendType Type alias representing valid backend types. It can be either 'VERTEX_AI' or 'GOOGLE_AI'.
Part Content part - includes text, image/video, or function call/response part types.
ResponseModality (Public Preview) Generation modalities to be returned in generation responses.
Role Role is the producer of the content.
Tool Defines a tool that model can call to access external knowledge.
TypedSchema A type that includes all specific Schema types.
VertexAI

function(app, ...)

getAI(app, options)

Returns the default AI instance that is associated with the provided FirebaseApp. If no instance exists, initializes a new instance with the default settings.

Signature:

export declare function getAI(app?: FirebaseApp, options?: AIOptions): AI;

Parameters

Parameter Type Description
app FirebaseApp The FirebaseApp to use.
options AIOptions AIOptions that configure the AI instance.

Returns:

AI

The default AI instance for the given FirebaseApp.

Example 1

const ai = getAI(app);

Example 2

// Get an AI instance configured to use the Gemini Developer API (via Google AI).
const ai = getAI(app, { backend: new GoogleAIBackend() });

Example 3

// Get an AI instance configured to use the Vertex AI Gemini API.
const ai = getAI(app, { backend: new VertexAIBackend() });

getVertexAI(app, options)

Use the new getAI() instead. The Vertex AI in Firebase SDK has been replaced with the Firebase AI SDK to accommodate the evolving set of supported features and services. For migration details, see the migration guide.

Returns a VertexAI instance for the given app, configured to use the Vertex AI Gemini API. This instance will be configured to use the Vertex AI Gemini API.

Signature:

export declare function getVertexAI(app?: FirebaseApp, options?: VertexAIOptions): VertexAI;

Parameters

Parameter Type Description
app FirebaseApp The FirebaseApp to use.
options VertexAIOptions Options to configure the Vertex AI instance, including the location.

Returns:

VertexAI

function(ai, ...)

getGenerativeModel(ai, modelParams, requestOptions)

Returns a GenerativeModel class with methods for inference and other functionality.

Signature:

export declare function getGenerativeModel(ai: AI, modelParams: ModelParams, requestOptions?: RequestOptions): GenerativeModel;

Parameters

Parameter Type Description
ai AI
modelParams ModelParams
requestOptions RequestOptions

Returns:

GenerativeModel

getImagenModel(ai, modelParams, requestOptions)

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

Returns an ImagenModel class with methods for using Imagen.

Only Imagen 3 models (named imagen-3.0-*) are supported.

Signature:

export declare function getImagenModel(ai: AI, modelParams: ImagenModelParams, requestOptions?: RequestOptions): ImagenModel;

Parameters

Parameter Type Description
ai AI An AI instance.
modelParams ImagenModelParams Parameters to use when making Imagen requests.
requestOptions RequestOptions Additional options to use when making requests.

Returns:

ImagenModel

Exceptions

If the apiKey or projectId fields are missing in your Firebase config.

BackendType

An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with.

These values are assigned to the backendType property within the specific backend configuration objects (GoogleAIBackend or VertexAIBackend) to identify which service to target.

Signature:

BackendType: {
    readonly VERTEX_AI: "VERTEX_AI";
    readonly GOOGLE_AI: "GOOGLE_AI";
}

POSSIBLE_ROLES

Possible roles.

Signature:

POSSIBLE_ROLES: readonly ["user", "model", "function", "system"]

ResponseModality

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

Generation modalities to be returned in generation responses.

Signature:

ResponseModality: {
    readonly TEXT: "TEXT";
    readonly IMAGE: "IMAGE";
}

VertexAIError

Use the new AIError instead. The Vertex AI in Firebase SDK has been replaced with the Firebase AI SDK to accommodate the evolving set of supported features and services. For migration details, see the migration guide.

Error class for the Firebase AI SDK.

Signature:

VertexAIError: typeof AIError

VertexAIModel

Use the new AIModel instead. The Vertex AI in Firebase SDK has been replaced with the Firebase AI SDK to accommodate the evolving set of supported features and services. For migration details, see the migration guide.

Base class for Firebase AI model APIs.

Signature:

VertexAIModel: typeof AIModel

BackendType

Type alias representing valid backend types. It can be either 'VERTEX_AI' or 'GOOGLE_AI'.

Signature:

export type BackendType = (typeof BackendType)[keyof typeof BackendType];

Part

Content part - includes text, image/video, or function call/response part types.

Signature:

export type Part = TextPart | InlineDataPart | FunctionCallPart | FunctionResponsePart | FileDataPart;

ResponseModality

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

Generation modalities to be returned in generation responses.

Signature:

export type ResponseModality = (typeof ResponseModality)[keyof typeof ResponseModality];

Role

Role is the producer of the content.

Signature:

export type Role = (typeof POSSIBLE_ROLES)[number];

Tool

Defines a tool that model can call to access external knowledge.

Signature:

export declare type Tool = FunctionDeclarationsTool;

TypedSchema

A type that includes all specific Schema types.

Signature:

export type TypedSchema = IntegerSchema | NumberSchema | StringSchema | BooleanSchema | ObjectSchema | ArraySchema;

VertexAI

Use the new AI instead. The Vertex AI in Firebase SDK has been replaced with the Firebase AI SDK to accommodate the evolving set of supported features and services. For migration details, see the migration guide.

An instance of the Firebase AI SDK.

Signature:

export type VertexAI = AI;

AIErrorCode

Standardized error codes that AIError can have.

Signature:

export declare const enum AIErrorCode

Enumeration Members

Member Value Description
API_NOT_ENABLED "api-not-enabled" An error due to the Firebase API not being enabled in the Console.
ERROR "error" A generic error occurred.
FETCH_ERROR "fetch-error" An error occurred while performing a fetch.
INVALID_CONTENT "invalid-content" An error associated with a Content object.
INVALID_SCHEMA "invalid-schema" An error due to invalid Schema input.
NO_API_KEY "no-api-key" An error occurred due to a missing Firebase API key.
NO_APP_ID "no-app-id" An error occured due to a missing Firebase app ID.
NO_MODEL "no-model" An error occurred due to a model name not being specified during initialization.
NO_PROJECT_ID "no-project-id" An error occurred due to a missing project ID.
PARSE_FAILED "parse-failed" An error occurred while parsing.
REQUEST_ERROR "request-error" An error occurred in a request.
RESPONSE_ERROR "response-error" An error occurred in a response.
UNSUPPORTED "unsupported" An error occured due an attempt to use an unsupported feature.

BlockReason

Reason that a prompt was blocked.

Signature:

export declare enum BlockReason

Enumeration Members

Member Value Description
BLOCKLIST "BLOCKLIST" Content was blocked because it contained terms from the terminology blocklist.
OTHER "OTHER" Content was blocked, but the reason is uncategorized.
PROHIBITED_CONTENT "PROHIBITED_CONTENT" Content was blocked due to prohibited content.
SAFETY "SAFETY" Content was blocked by safety settings.

FinishReason

Reason that a candidate finished.

Signature:

export declare enum FinishReason

Enumeration Members

Member Value Description
BLOCKLIST "BLOCKLIST" The candidate content contained forbidden terms.
MALFORMED_FUNCTION_CALL "MALFORMED_FUNCTION_CALL" The function call generated by the model was invalid.
MAX_TOKENS "MAX_TOKENS" The maximum number of tokens as specified in the request was reached.
OTHER "OTHER" Unknown reason.
PROHIBITED_CONTENT "PROHIBITED_CONTENT" The candidate content potentially contained prohibited content.
RECITATION "RECITATION" The candidate content was flagged for recitation reasons.
SAFETY "SAFETY" The candidate content was flagged for safety reasons.
SPII "SPII" The candidate content potentially contained Sensitive Personally Identifiable Information (SPII).
STOP "STOP" Natural stop point of the model or provided stop sequence.

FunctionCallingMode

Signature:

export declare enum FunctionCallingMode

Enumeration Members

Member Value Description
ANY "ANY" Model is constrained to always predicting a function call only. If allowed_function_names is set, the predicted function call will be limited to any one of allowed_function_names, else the predicted function call will be any one of the provided function_declarations.
AUTO "AUTO" Default model behavior; model decides to predict either a function call or a natural language response.
NONE "NONE" Model will not predict any function call. Model behavior is same as when not passing any function declarations.

HarmBlockMethod

This property is not supported in the Gemini Developer API (GoogleAIBackend).

Signature:

export declare enum HarmBlockMethod

Enumeration Members

Member Value Description
PROBABILITY "PROBABILITY" The harm block method uses the probability score.
SEVERITY "SEVERITY" The harm block method uses both probability and severity scores.

HarmBlockThreshold

Threshold above which a prompt or candidate will be blocked.

Signature:

export declare enum HarmBlockThreshold

Enumeration Members

Member Value Description
BLOCK_LOW_AND_ABOVE "BLOCK_LOW_AND_ABOVE" Content with NEGLIGIBLE will be allowed.
BLOCK_MEDIUM_AND_ABOVE "BLOCK_MEDIUM_AND_ABOVE" Content with NEGLIGIBLE and LOW will be allowed.
BLOCK_NONE "BLOCK_NONE" All content will be allowed.
BLOCK_ONLY_HIGH "BLOCK_ONLY_HIGH" Content with NEGLIGIBLE, LOW, and MEDIUM will be allowed.

HarmCategory

Harm categories that would cause prompts or candidates to be blocked.

Signature:

export declare enum HarmCategory

Enumeration Members

Member Value Description
HARM_CATEGORY_DANGEROUS_CONTENT "HARM_CATEGORY_DANGEROUS_CONTENT"
HARM_CATEGORY_HARASSMENT "HARM_CATEGORY_HARASSMENT"
HARM_CATEGORY_HATE_SPEECH "HARM_CATEGORY_HATE_SPEECH"
HARM_CATEGORY_SEXUALLY_EXPLICIT "HARM_CATEGORY_SEXUALLY_EXPLICIT"

HarmProbability

Probability that a prompt or candidate matches a harm category.

Signature:

export declare enum HarmProbability

Enumeration Members

Member Value Description
HIGH "HIGH" Content has a high chance of being unsafe.
LOW "LOW" Content has a low chance of being unsafe.
MEDIUM "MEDIUM" Content has a medium chance of being unsafe.
NEGLIGIBLE "NEGLIGIBLE" Content has a negligible chance of being unsafe.

HarmSeverity

Harm severity levels.

Signature:

export declare enum HarmSeverity

Enumeration Members

Member Value Description
HARM_SEVERITY_HIGH "HARM_SEVERITY_HIGH" High level of harm severity.
HARM_SEVERITY_LOW "HARM_SEVERITY_LOW" Low level of harm severity.
HARM_SEVERITY_MEDIUM "HARM_SEVERITY_MEDIUM" Medium level of harm severity.
HARM_SEVERITY_NEGLIGIBLE "HARM_SEVERITY_NEGLIGIBLE" Negligible level of harm severity.
HARM_SEVERITY_UNSUPPORTED "HARM_SEVERITY_UNSUPPORTED" Harm severity is not supported.

ImagenAspectRatio

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

Aspect ratios for Imagen images.

To specify an aspect ratio for generated images, set the aspectRatio property in your ImagenGenerationConfig.

See the the documentation for more details and examples of the supported aspect ratios.

Signature:

export declare enum ImagenAspectRatio

Enumeration Members

Member Value Description
LANDSCAPE_16x9 "16:9" (Public Preview) Landscape (16:9) aspect ratio.
LANDSCAPE_3x4 "3:4" (Public Preview) Landscape (3:4) aspect ratio.
PORTRAIT_4x3 "4:3" (Public Preview) Portrait (4:3) aspect ratio.
PORTRAIT_9x16 "9:16" (Public Preview) Portrait (9:16) aspect ratio.
SQUARE "1:1" (Public Preview) Square (1:1) aspect ratio.

ImagenPersonFilterLevel

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

A filter level controlling whether generation of images containing people or faces is allowed.

See the personGeneration documentation for more details.

Signature:

export declare enum ImagenPersonFilterLevel

Enumeration Members

Member Value Description
ALLOW_ADULT "allow_adult" (Public Preview) Allow generation of images containing adults only; images of children are filtered out.Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the Responsible AI and usage guidelines for more details.
ALLOW_ALL "allow_all" (Public Preview) Allow generation of images containing adults only; images of children are filtered out.Generation of images containing people or faces may require your use case to be reviewed and approved by Cloud support; see the Responsible AI and usage guidelines for more details.
BLOCK_ALL "dont_allow" (Public Preview) Disallow generation of images containing people or faces; images of people are filtered out.

ImagenSafetyFilterLevel

This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.

A filter level controlling how aggressively to filter sensitive content.

Text prompts provided as inputs and images (generated or uploaded) through Imagen on Vertex AI are assessed against a list of safety filters, which include 'harmful categories' (for example, violence, sexual, derogatory, and toxic). This filter level controls how aggressively to filter out potentially harmful content from responses. See the documentation and the Responsible AI and usage guidelines for more details.

Signature:

export declare enum ImagenSafetyFilterLevel

Enumeration Members

Member Value Description
BLOCK_LOW_AND_ABOVE "block_low_and_above" (Public Preview) The most aggressive filtering level; most strict blocking.
BLOCK_MEDIUM_AND_ABOVE "block_medium_and_above" (Public Preview) Blocks some sensitive prompts and responses.
BLOCK_NONE "block_none" (Public Preview) The least aggressive filtering level; blocks very few sensitive prompts and responses.Access to this feature is restricted and may require your case to be reviewed and approved by Cloud support.
BLOCK_ONLY_HIGH "block_only_high" (Public Preview) Blocks few sensitive prompts and responses.

Modality

Content part modality.

Signature:

export declare enum Modality

Enumeration Members

Member Value Description
AUDIO "AUDIO" Audio.
DOCUMENT "DOCUMENT" Document (for example, PDF).
IMAGE "IMAGE" Image.
MODALITY_UNSPECIFIED "MODALITY_UNSPECIFIED" Unspecified modality.
TEXT "TEXT" Plain text.
VIDEO "VIDEO" Video.

SchemaType

Contains the list of OpenAPI data types as defined by the OpenAPI specification

Signature:

export declare enum SchemaType

Enumeration Members

Member Value Description
ARRAY "array" Array type.
BOOLEAN "boolean" Boolean type.
INTEGER "integer" Integer type.
NUMBER "number" Number type.
OBJECT "object" Object type.
STRING "string" String type.