使用 Imagen 更換圖片背景


本頁說明如何使用 Firebase AI Logic SDK，透過 Imagen更換圖片背景。

背景更換是以遮罩為基礎的編輯類型 (具體來說是修復)。遮罩是定義要編輯特定區域的數位疊加層。

運作方式：您提供原始圖片和對應的遮罩圖片，定義背景的遮罩 (使用自動背景偵測功能或自行提供背景遮罩)。你也可以提供文字提示詞，描述想要變更的內容。 模型隨即會生成並套用新背景。

舉例來說，你可以變更主體或物體周圍的設定，而不會影響前景 (例如產品圖片)。

跳至自動偵測背景的程式碼 跳至提供背景遮罩的程式碼

事前準備

只有在使用 Agent Platform Gemini API (formerly Vertex AI) 做為 API 供應商時，才能使用這項功能。

如果尚未完成，請參閱入門指南，瞭解如何設定 Firebase 專案、將應用程式連結至 Firebase、新增 SDK、初始化所選 API 供應商的後端服務，以及建立 ImagenModel 執行個體。

支援這項功能的模型

Imagen 可透過 capability 模型編輯圖片：

  • imagen-3.0-capability-001

請注意，Imagen 型號global不支援此位置。

使用自動背景偵測功能更換背景

在試用這個範例之前，請先完成本指南的「事前準備」一節，設定專案和應用程式。

下列範例說明如何使用自動背景偵測功能，更換圖片的背景。你提供原始圖片和文字提示，Imagen 會自動偵測並建立背景遮罩，以修改原始圖片。

Swift

Swift 不支援使用 Imagen 模型編輯圖片。敬請期待今年稍晚推出的功能！

Kotlin

如要使用自動背景偵測功能替換背景，請指定 ImagenBackgroundMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_INSERTION。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // Provide the prompt describing the new background.
    val prompt = "space background"

    // Use the editImage API to replace the background.
    // Pass the original image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        referenceImages = listOf(
            ImagenRawImage(originalImage.toImagenInlineImage()),
            ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and background replacement.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_INSERTION)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如要使用自動背景偵測功能替換背景，請指定 ImagenBackgroundMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_INSERTION。

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your image Bitmap here");

// Provide the prompt describing the new background.
String prompt = "space background";

// Define the list of sources for the editImage call.
// This includes the original image and the auto-generated mask.
ImagenRawImage rawOriginalImage =
    new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage));
// Use ImagenBackgroundMask() to auto-generate the mask.
ImagenBackgroundMask rawMaskedImage = new ImagenBackgroundMask();

ImagenEditingConfig config = new ImagenEditingConfig();

// Use the editImage API to replace the background.
// Pass the original image, the auto-generated masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config),
    new FutureCallback<ImagenGenerationResponse>() {
        @Override
        public void onSuccess(ImagenGenerationResponse result) {
            if (result.getImages().isEmpty()) {
                Log.d("ImageEditor", "No images generated");
            }
            Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap();
            // Process and use the bitmap to display the image in your UI
        }

        @Override
        public void onFailure(Throwable t) {
            // ...
        }
    }, Executors.newSingleThreadExecutor());

Web

網頁應用程式不支援使用 Imagen 模型編輯圖片。敬請期待今年稍晚推出的功能！

Dart

如要使用自動背景偵測功能替換背景，請指定 ImagenBackgroundMask。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.inpaintInsertion。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// Provide the prompt describing the new background.
final prompt = 'space background';

try {
  // Use the editImage API to replace the background.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask.
    ],
    prompt: prompt,
    // Define the editing configuration for inpainting and background replacement.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintInsertion,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Unity 不支援使用 Imagen 模型編輯圖片。敬請期待今年稍晚推出的功能！

使用提供的遮罩更換背景

在試用這個範例之前，請先完成本指南的「事前準備」一節，設定專案和應用程式。

下列範例說明如何使用您提供的圖片中定義的背景遮罩，取代圖片的背景。您提供原始圖片、文字提示詞和遮罩圖片。

Swift

Swift 不支援使用 Imagen 模型編輯圖片。敬請期待今年稍晚推出的功能！

Kotlin

如要使用您提供的遮罩替換背景，請指定 ImagenRawMask 和遮罩圖片。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_INSERTION。

// Using this SDK to access Imagen models is a Preview release and requires opt-in
@OptIn(PublicPreviewAPI::class)
suspend fun customizeImage() {
    // Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
    // Optionally specify the location to access the model (for example, `us-central1`)
    val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1"))

    // Create an `ImagenModel` instance with an Imagen "capability" model
    val model = ai.imagenModel("imagen-3.0-capability-001")

    // This example assumes 'originalImage' is a pre-loaded Bitmap.
    // In a real app, this might come from the user's device or a URL.
    val originalImage: Bitmap = TODO("Load your original image Bitmap here")

    // This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
    // In a real app, this might come from the user's device or a URL.
    val maskImage: Bitmap = TODO("Load your masked image Bitmap here")

    // Provide the prompt describing the new background.
    val prompt = "space background"

    // Use the editImage API to replace the background.
    // Pass the original image, the masked image, the prompt, and an editing configuration.
    val editedImage = model.editImage(
        referenceImages = listOf(
            ImagenRawImage(originalImage.toImagenInlineImage()),
            ImagenRawMask(maskImage.toImagenInlineImage()), // Use ImagenRawMask() to provide your own masked image.
        ),
        prompt = prompt,
        // Define the editing configuration for inpainting and background replacement.
        config = ImagenEditingConfig(ImagenEditMode.INPAINT_INSERTION)
    )

    // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView.
}

Java

如要使用您提供的遮罩替換背景，請指定 ImagenRawMask 和遮罩圖片。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_INSERTION。

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify the location to access the model (for example, `us-central1`)
// Create an `ImagenModel` instance with an Imagen "capability" model
ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1"))
        .imagenModel(
                /* modelName */ "imagen-3.0-capability-001");

ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// This example assumes 'originalImage' is a pre-loaded Bitmap.
// In a real app, this might come from the user's device or a URL.
Bitmap originalImage = null; // TODO("Load your original image Bitmap here");

// This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area.
// In a real app, this might come from the user's device or a URL.
Bitmap maskImage = null; // TODO("Load your masked image Bitmap here");

// Provide the prompt describing the new background.
String prompt = "space background";

// Define the list of source images for the editImage call.
ImagenRawImage rawOriginalImage =
    new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage));
// Use ImagenRawMask() to provide your own masked image.
ImagenBackgroundMask rawMaskedImage =
    new ImagenRawMask(ImagenInlineImageKt.toImagenInlineImage(maskImage));

ImagenEditingConfig config = new ImagenEditingConfig();

// Use the editImage API to replace the background.
// Pass the original image, the masked image, the prompt, and an editing configuration.
Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config),
    new FutureCallback<ImagenGenerationResponse>() {
        @Override
        public void onSuccess(ImagenGenerationResponse result) {
            if (result.getImages().isEmpty()) {
                Log.d("ImageEditor", "No images generated");
            }
            Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap();
            // Process and use the bitmap to display the image in your UI
        }

        @Override
        public void onFailure(Throwable t) {
            // ...
        }
    }, Executors.newSingleThreadExecutor());

Web

網頁應用程式不支援使用 Imagen 模型編輯圖片。敬請期待今年稍晚推出的功能！

Dart

如要使用您提供的遮罩替換背景，請指定 ImagenRawMask 和遮罩圖片。使用 editImage() 並將編輯設定設為使用 ImagenEditMode.INPAINT_INSERTION。

import 'dart:typed_data';
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

// Initialize FirebaseApp
await Firebase.initializeApp(
  options: DefaultFirebaseOptions.currentPlatform,
);

// Initialize the Agent Platform Gemini API backend service (formerly branded Vertex AI).
// Optionally specify a location to access the model (for example, `us-central1`)
final ai = FirebaseAI.vertexAI(location: 'us-central1');

// Create an `ImagenModel` instance with an Imagen "capability" model
final model = ai.imagenModel(model: 'imagen-3.0-capability-001');

// This example assumes 'originalImage' is a pre-loaded Uint8List.
// In a real app, this might come from the user's device or a URL.
final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here.

// This example assumes 'maskImage' is a pre-loaded Uint8List that contains the masked area.
// In a real app, this might come from the user's device or a URL.
final Uint8List maskImage = Uint8List(0); // TODO: Load your masked image data here.

// Provide the prompt describing the new background.
final prompt = 'space background';

try {
  // Use the editImage API to replace the background.
  // Pass the original image, the prompt, and an editing configuration.
  final response = await model.editImage(
    sources: [
      ImagenRawImage(originalImage),
      ImagenRawMask(maskImage), // Use ImagenRawMask() to provide your own masked image.
    ],
    prompt: prompt,
    // Define the editing configuration for inpainting and background replacement.
    config: const ImagenEditingConfig(
      editMode: ImagenEditMode.inpaintInsertion,
    ),
  );

  // Process the result.
  if (response.images.isNotEmpty) {
    final editedImage = response.images.first.bytes;
    // Use the editedImage (a Uint8List) to display the image, save it, etc.
    print('Image successfully generated!');
  } else {
    // Handle the case where no images were generated.
    print('Error: No images were generated.');
  }
} catch (e) {
  // Handle any potential errors during the API call.
  print('An error occurred: $e');
}

Unity

Unity 不支援使用 Imagen 模型編輯圖片。敬請期待今年稍晚推出的功能！

最佳做法和限制

編輯圖片時，建議擴大遮罩範圍。這有助於修飾編輯內容的邊界，讓編輯結果看起來更逼真。一般來說，建議使用 1% 或 2% 的擴張值 (0.01 或 0.02)。


提供有關 Firebase AI Logic 的使用體驗意見回饋