1. Overview of Migration Mechanism

1.1 Concept and Role of Migration

Database migration is the process of synchronizing changes in data models to the database structure, which is crucial for data persistence. As the application version iterates, the data model often undergoes changes, such as adding or deleting fields, or modifying indexes. Migration allows developers to manage these changes in a versioned and systematic way, ensuring consistency between the database structure and the data model.

In modern web development, the migration mechanism provides the following benefits:

  1. Version Control: Migration files can track the change history of the database structure, making it convenient to roll back and understand the changes in each version.
  2. Automated Deployment: Through the migration mechanism, database deployment and updates can be automated, reducing the possibility of manual intervention and the risk of errors.
  3. Team Collaboration: Migration files ensure that team members use synchronized database structures in different development environments, facilitating collaborative development.

1.2 Migration Features of the ent Framework

The integration of the ent framework with the migration mechanism offers the following features:

  1. Declarative Programming: Developers only need to focus on the Go representation of the entities, and the ent framework will handle the conversion of entities to database tables.
  2. Automatic Migration: ent can automatically create and update database table structures without the need to manually write DDL statements.
  3. Flexible Control: ent provides various configuration options to support different migration requirements, such as with or without foreign key constraints, and generating globally unique IDs.

2. Introduction to Automatic Migration

2.1 Basic Principles of Automatic Migration

The automatic migration feature of the ent framework is based on the schema definition files (typically found in the ent/schema directory) to generate the database structure. After developers define entities and relationships, ent will inspect the existing structure in the database and generate corresponding operations to create tables, add or modify columns, create indexes, etc.

Furthermore, the automatic migration principle of ent works in an "append mode": it by default only adds new tables, new indexes, or adds columns to tables, and does not delete existing tables or columns. This design is beneficial for preventing accidental data loss and makes it easy to expand the database structure in a forward manner.

2.2 Using Automatic Migration

The basic steps to use ent automatic migration are as follows:

package main

import (
    "context"
    "log"
    "ent"
)

func main() {
    client, err := ent.Open("mysql", "root:pass@tcp(localhost:3306)/test")
    if err != nil {
        log.Fatalf("Failed to connect to MySQL: %v", err)
    }
    defer client.Close()
    ctx := context.Background()

    // Perform automatic migration to create or update the database schema
    if err := client.Schema.Create(ctx); err != nil {
        log.Fatalf("Failed to create database schema: %v", err)
    }
}

In the above code, ent.Open is responsible for establishing a connection with the database and returning a client instance, while client.Schema.Create executes the actual automatic migration operation.

3. Advanced Applications of Automatic Migration

3.1 Delete Columns and Indexes

In some cases, we may need to remove columns or indexes that are no longer needed from the database schema. At this time, we can use the WithDropColumn and WithDropIndex options. For example:

// Run migration with options to drop columns and indexes.
err = client.Schema.Create(
    ctx,
    migrate.WithDropIndex(true),
    migrate.WithDropColumn(true),
)

This code snippet enables the configuration to delete columns and indexes during automatic migration. ent will delete any columns and indexes that do not exist in the schema definition when executing the migration.

3.2 Global Unique ID

By default, the primary keys in SQL databases start from 1 for each table, and different entity types may share the same ID. In some application scenarios, such as when using GraphQL, it may be necessary to provide global uniqueness for the IDs of objects of different entity types. In ent, this can be configured using the WithGlobalUniqueID option:

// Run migration with universal unique IDs for each entity.
if err := client.Schema.Create(ctx, migrate.WithGlobalUniqueID(true)); err != nil {
    log.Fatalf("Failed to create the database schema: %v", err)
}

After enabling the WithGlobalUniqueID option, ent will assign a 2^32 range ID to each entity in a table named ent_types to achieve global uniqueness.

3.3 Offline Mode

Offline mode allows writing schema changes to an io.Writer instead of executing them directly on the database. It is useful for verifying SQL commands before the changes take effect, or for generating an SQL script for manual execution. For example:

// Dump migration changes to a file
f, err := os.Create("migrate.sql")
if err != nil {
    log.Fatalf("Failed to create migration file: %v", err)
}
defer f.Close()
if err := client.Schema.WriteTo(ctx, f); err != nil {
    log.Fatalf("Failed to print database schema changes: %v", err)
}

This code will write the migration changes to a file named migrate.sql. In practice, developers can choose to directly print to standard output or write to a file for review or record-keeping.

4. Foreign Key Support and Custom Hooks

4.1 Enable or Disable Foreign Keys

In Ent, foreign keys are implemented by defining relationships (edges) between entities, and these foreign key relationships are automatically created at the database level to enforce data integrity and consistency. However, in certain situations, such as for performance optimization or when the database does not support foreign keys, you may choose to disable them.

To enable or disable foreign key constraints in migrations, you can control this through the WithForeignKeys configuration option:

// Enable foreign keys
err = client.Schema.Create(
    ctx,
    migrate.WithForeignKeys(true), 
)
if err != nil {
    log.Fatalf("Failed creating schema resources with foreign keys: %v", err)
}

// Disable foreign keys
err = client.Schema.Create(
    ctx,
    migrate.WithForeignKeys(false), 
)
if err != nil {
    log.Fatalf("Failed creating schema resources without foreign keys: %v", err)
}

This configuration option needs to be passed when calling Schema.Create, and it determines whether to include foreign key constraints in the generated DDL based on the specified value.

4.2 Application of Migration Hooks

Migration hooks are custom logic that can be inserted and executed at different stages of migration execution. They are very useful for performing specific logic on the database before/after migration, such as validating migration results and pre-filling data.

Here is an example of how to implement custom migration hooks:

func customHook(next schema.Creator) schema.Creator {
    return schema.CreateFunc(func(ctx context.Context, tables ...*schema.Table) error {
        // Custom code to be executed before the migration
        // For example, logging, checking certain preconditions, etc.
        log.Println("Custom logic before migration")
        
        // Call the next hook or the default migration logic
        err := next.Create(ctx, tables...)
        if err != nil {
            return err
        }
        
        // Custom code to be executed after the migration
        // For example, cleanup, data migration, security checks, etc.
        log.Println("Custom logic after migration")
        return nil
    })
}

// Using custom hooks in migration
err := client.Schema.Create(
    ctx,
    schema.WithHooks(customHook),
)
if err != nil {
    log.Fatalf("Error applying custom migration hooks: %v", err)
}

Hooks are powerful and indispensable tools for complex migrations, giving you the ability to directly control database migration behavior when needed.

5. Versioned Migrations

5.1 Introduction to Versioned Migrations

Versioned migration is a pattern for managing database migration, allowing developers to divide changes to the database structure into multiple versions, each containing a specific set of database modification commands. Compared to Auto Migration, versioned migration provides finer-grained control, ensuring traceability and reversibility of database structure changes.

The main advantage of versioned migration is its support for forward and backward migration (i.e., upgrade or downgrade), allowing developers to apply, rollback, or skip specific changes as needed. When collaborating within a team, versioned migration ensures that each member works on the same database structure, reducing issues caused by inconsistencies.

Auto migration is often irreversible, generating and executing SQL statements to match the latest state of entity models, primarily used in development stages or small projects.

5.2 Using Versioned Migrations

1. Installing the Atlas Tool

Before using versioned migrations, you need to install the Atlas tool on your system. Atlas is a migration tool that supports multiple database systems, providing powerful features for managing database schema changes.

macOS + Linux

curl -sSf https://atlasgo.sh | sh

Homebrew

brew install ariga/tap/atlas

Docker

docker pull arigaio/atlas
docker run --rm arigaio/atlas --help

Windows

https://release.ariga.io/atlas/atlas-windows-amd64-latest.exe

2. Generating Migration Files Based on Current Entity Definitions

atlas migrate diff migration_name \
  --dir "file://ent/migrate/migrations" \
  --to "ent://ent/schema" \
  --dev-url "docker://mysql/8/ent"

3. Application Migration Files

Once the migration files are generated, they can be applied to the development, testing, or production environments. Typically, you would first apply these migration files to a development or testing database to ensure that they execute as expected. Then, the same migration steps would be executed in the production environment.

atlas migrate apply \
  --dir "file://ent/migrate/migrations" \
  --url "mysql://root:pass@localhost:3306/example"

Use the atlas migrate apply command, specifying the migration file directory (--dir) and the URL of the target database (--url) to apply the migration files.