In Firebase Data Connect, bulk data operations are performed using mutations. Even though your Data Connect projects store data in PostgreSQL, you cannot bulk load data using SQL statements or SQL tooling: your Data Connect service and its schemas must stay in sync with your database, and operating directly in PostgreSQL would break this synchronization.
That's why Data Connect provides standard _insertMany
,
_upsertMany
, and _deleteMany
mutations for seeding data and bulk data
management.
When you're prototyping apps and running CI/CD flows, these mutations can be called in a local development environment using the VS Code extension, the Data Connect emulator, and (optionally) a local database instance.
Seed data in local and production instances
In the Get started guide, you set up an app to add a single record to a single table using an ad hoc insert mutation.
To be usable, the movie review app needs data for movies, reviews, and users for prototyping queries and mutations that use joins and other operations on multiple tables with realistic data. You can expand your schema and seed your database.
Your prototyping environment needs code to perform data seeding. This guide provides some samples, illustrating:
- Use of
_insertMany
and_upsertMany
on individual tables - Use of
_insertMany
on related tables
Update the movie review app schema
You can use _insertMany
and _upsertMany
mutations to update individual
database tables one at a time, or update multiple tables related by join
relationships. An expanded movie review app schema that helps illustrate these
use cases and examples is shown below. It expands schema.gql
beyond the
starting Movie
type to include Actor
and MovieActor
types, so we can
prototype more complex queries.
# Actors
# Suppose an actor can participate in multiple movies and movies can have multiple actors
# Movie - Actors (or vice versa) is a many to many relationship
type Actor @table {
id: UUID!
imageUrl: String!
name: String! @col(name: "name", dataType: "varchar(30)")
}
# Join table for many-to-many relationship for movies and actors
# The 'key' param signifies the primary key(s) of this table
# In this case, the keys are [movieId, actorId], the generated fields of the reference types [movie, actor]
type MovieActor @table(key: ["movie", "actor"]) {
# @ref creates a field in the current table (MovieActor) that holds the primary key of the referenced type
# In this case, @ref(fields: "movieId", references: "id") is implied
movie: Movie!
# movieId: UUID! <- this is created by the implied @ref
actor: Actor!
# actorId: UUID! <- this is created by the implied @ref
role: String! # "main" or "supporting"
}
Write mutations to seed zero state data
During prototyping, when your queries and mutations need to be tested against a range of discrete values, you can populate data with multiple records. For example, you might want to add multiple movie records with different types of genres and ratings for testing comparisons and filtering.
Seed data into the Movie
and Actor
tables
Depending on your stage of prototyping, you can use the same technique
introduced in the Get started guide to insert one or two records: that is, you
can use Code Lenses in the VS Code extension to create _insert
mutations,
hard-code data, and Run those mutations in VS Code.
Eventually, it makes more sense to add many records into a table using an
_insertMany
operation. In the movie review app example, this inserts an
initial set of data in Movie
and Actor
.
To execute the following mutations, using the VS Code Firebase extension, in the appropriate file editor view, click the Run (Production) or Run (Local) CodeLens buttons, depending on whether you are prototyping with your production service or a local database.
# insertMany for Movie
# 2 records shown
mutation {
movie_insertMany(data: [
{
id: "550e8400-e29b-41d4-a716-446655440000",
title: "Inception",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/movies%2Finception.jpg?alt=media&token=07b09781-b302-4623-a5c3-1956d0143168",
genre: "sci-fi",
},
{
id: "550e8400-e29b-41d4-a716-446655440001",
title: "The Matrix",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/movies%2Fthe_matrix.jpg?alt=media&token=4975645d-fef8-409e-84a5-bcc1046e2059",
genre: "action",
}
])
}
# insertMany for Actor
# 2 records shown
mutation {
actor_insertMany(data: [
{
id: "123e4567-e89b-12d3-a456-426614174000",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/actors%2Fdicaprio.jpeg?alt=media&token=452e030a-efa5-4ef4-bb81-502b23241316",
name: "Leonardo DiCaprio"
},
{
id: "123e4567-e89b-12d3-a456-426614174001",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/actors%2Fkeanu.jpg?alt=media&token=6056520c-ef3e-4823-aad0-108aab163115",
name: "Keanu Reeves"
}
])
}
Seed data into MovieActor
join table
To test queries and mutations using joins and other complex operations, you can
add multiple records to the MovieActor
table.
Here, when you are updating multiple tables in this kind of relation, you can
add the @transaction
directive to ensure the update completes properly.
mutation @transaction {
movie_insertMany(data: [
{
id: "550e8400-e29b-41d4-a716-446655440000",
title: "Inception",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/movies%2Finception.jpg?alt=media&token=07b09781-b302-4623-a5c3-1956d0143168",
genre: "sci-fi",
},
{
id: "550e8400-e29b-41d4-a716-446655440001",
title: "The Matrix",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/movies%2Fthe_matrix.jpg?alt=media&token=4975645d-fef8-409e-84a5-bcc1046e2059",
genre: "action",
}
])
actor_insertMany(data: [
{
id: "123e4567-e89b-12d3-a456-426614174000",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/actors%2Fdicaprio.jpeg?alt=media&token=452e030a-efa5-4ef4-bb81-502b23241316",
name: "Leonardo DiCaprio"
},
{
id: "123e4567-e89b-12d3-a456-426614174001",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/actors%2Fkeanu.jpg?alt=media&token=6056520c-ef3e-4823-aad0-108aab163115",
name: "Keanu Reeves"
}
])
}
Write a mutation to reset seed data
While prototyping and performing CI/CD, resetting the data to a zero state for executing a new series of tests on a new set of data can be useful.
To do so, if your prototype code does't add records to your tables, use the
_upsertMany
mutation provided by Data Connect.
In the following example, movie_upsertMany
is called with the initial values
to update movie records to their original state.
mutation {
# Execute an upsertMany operation to update the Movie table
movie_upsertMany(data: [
{
id: "550e8400-e29b-41d4-a716-446655440000",
title: "Inception",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/movies%2Finception.jpg?alt=media&token=07b09781-b302-4623-a5c3-1956d0143168",
genre: "sci-fi",
},
{
id: "550e8400-e29b-41d4-a716-446655440001",
title: "The Matrix",
imageUrl: "https://firebasestorage.googleapis.com/v0/b/fdc-quickstart-web.appspot.com/o/movies%2Fthe_matrix.jpg?alt=media&token=4975645d-fef8-409e-84a5-bcc1046e2059",
genre: "action",
}
…
}