January 2016

Volume 31 Number 1

[Data Points]

EF7 Migrations: Not New but Definitely Improved

By Julie Lerman | January 2016

Julie LermanCode First Migrations has undergone some dramatic changes with Entity Framework 7. While the basic concepts and migration steps remain the same, the workflow has become much cleaner and more straight-forward, and it now works nicely with source control.

EF7 brings two flavors of migrations: the familiar variety you use with NuGet and most .NET projects, and the kind you run as part of the DNX Runtime with ASP.NET 5 apps.

I took a look at these in one of my Pluralsight courses (bit.ly/1MThS4J). Though a few things have changed between the beta 4 version I used in that course and the Release Candidate 1 (RC1) I’m using today, that course remains a great resource for seeing the migrations in action and learning about some of the differences.

But with the release of RC1, which is considered feature complete for the upcoming RTM, it was time to take another close look at EF7 migrations. I’ll begin with the familiar Windows PowerShell commands in the Package Manger Console of Visual Studio. I’ll then discuss these commands as you’d use them with DNX and ASP.NET 5. This article presumes you have some existing knowledge of the database initialization and migrations features in EF6 or earlier versions.

No More Magic Database Initialization—­Use Migrations

Supporting magic is expensive, limiting and creates a lot of baggage. And in the case of DbInitializers, it created a lot of confusion. So EF DbInitializers, along with the default behavior of CreateData­baseIfNotExists, are now gone. Instead, there’s logic that lets you explicitly create and delete data­bases (EnsureDeleted, Ensure­Created), which I’ll be happy to use in integration tests.

EF7 is designed to rely on migra­tions. They should now be your default workflow.

The “enable-migrations” command existed because migrations were tacked onto EF and using them required building out a bit of infrastructure. That command didn’t install any new APIs; it just added a project folder and a new DbMigrationConfiguration class to your project. With EF7, the relevant logic is internal and you don’t have to explicitly tell EF you want to use migrations.

Remember—EF7 is composable. Not everyone will want or need migrations, so if you do, you need to opt in to the migrations commands by installing the package that contains them:

install-package entityframework.commands -pre

The EF7 Migrations Commands

Using the Windows PowerShell Get-Help cmdlet displays the current (as of RC1) migration commands, as shown in Figure 1. This list should be the same for the RTM.

The Migrations Commands as Listed Via the NuGet Package Manager Get-Command
Figure 1 The Migrations Commands as Listed Via the NuGet Package Manager Get-Command

Note also that Update-Database no longer has the “-script” parameter. It will do only what its name suggests: update the database. As always, using this command on production databases isn’t recommended. Figure 2 displays details about the Update-Database command.

Update-Database Syntax and Parameters
Figure 2 Update-Database Syntax and Parameters

To create scripts, you can use the new Script-Migration command. And there’s no longer a secret recipe for creating idempotent scripts (introduced in EF6). Idempotent is a parameter, as you can see in Figure 3.

Script-Migration Parameters
Figure 3 Script-Migration Parameters

Big Changes for Migration History and Model Snapshots 

Automatic migrations has been removed from EF7. Thanks to that, managing migrations is simpler and source control is now supported.

In order for migrations to deter­mine what needs to happen, a history of previous migrations has to exist. Each time you run the add-migration command, the API will look at that historical information. Previously, migrations stored an encoded version of a snapshot of the model for each migration in the database, in a table (most recently) named _MigrationHistory. This is what EF has used to figure out what a migration needed to accomplish. Each time add-migration was run, it meant a trip to the database to read this history, and that was the source of a lot of problems.

With EF7, when a migration is created (via add-migration), in addition to the familiar migration file inside the Migrations folder, EF7 also creates a file to store the current model snapshot. The model is described using the Fluent API syntax used for configurations. If you make a change to your model and then add a migration, the snapshot gets revised with the current description of the full model. Figure 4 shows a migration file that introduces a new string called “MyProperty.”

Figure 4 The Complete newStringInSamurai Migration Class

public partial class newStringInSamurai : Migration
  {
    protected override void Up(MigrationBuilder migrationBuilder)
    {
      migrationBuilder.AddColumn<string>(
        name: "MyProperty",
        table: "Samurai",
        nullable: true);
    }
    protected override void Down(MigrationBuilder migrationBuilder)
    {
      migrationBuilder.DropColumn(name: "MyProperty", table: "Samurai");
    }
  }

And here’s the portion of the updated ModelSnapshot class, where you can see that “MyProperty” is now part of the description of the Samurai class:

modelBuilder.Entity("EF7Samurai.Domain.Samurai", b =>
  {
    b.Property<int>("Id")
      .ValueGeneratedOnAdd();
    b.Property<string>("MyProperty");
    b.Property<string>("Name");
    b.HasKey("Id");
  });

So, individual migrations describe what has changed, as they always have. But now, in the project, you always have a description of the full model. This means that when you add a new migration, EF doesn’t have to go to the database to find out what the previous model looked like. What it needs is right at hand. And even more important, this works much better with source control because it’s right there in your code and can be diffed and merged by source control like any other file. It’s not tucked away in a database that may not even be accessible to your teammates. Before EF7, there was no good way to manage migrations in source control. You’ll find guidance about migrations for versions ear­lier that EF7 in “Code First Migrations in Team Environments” (bit.ly/1OVO3qr). This article begins with the advice: “Grab a coffee, you need to read this whole article.”

With EF7, the migration history table in the database (Figure 5) now stores only the migration’s id and the version of EF that created it. Again, this is a big change from earlier versions, which stored the snapshot itself into this table. Update-Database will check this table to see which migrations have been applied and applies any that are missing, even those that aren’t consecutive. As long as this operation needs to touch the database, I see no problem with migrations doing a quick pre-check against the database in this scenario.

The dbo_EFMigrationsHistory Displaying Migrations Added Table
Figure 5 The dbo_EFMigrationsHistory Displaying Migrations Added Table

You no longer have to deal with that nonsense about not being able to add a migration if the previous one hasn’t been applied to the database yet. That was a side effect of automatic migrations and caused many of us to stumble into a tangled web of confusion. Now you can build up migrations in your solution as needed and then update the database when you’re ready.

Because of the model snapshot that EF7 stores in the project, you shouldn’t just delete a migration file from your project because the snapshot would then be incorrect. So, along with the change to the migrations workflow comes a new command: remove-migration. Remove-migration is designed to undo a migration that you most likely just added and then changed your mind about before it was applied to the database. The command does two things in your project: It removes the latest migration file and it then updates the snapshot file in the migrations folder. Interestingly, this doesn’t rebuild the snapshot based on the model. Instead, it modifies the snapshot based on the model snapshot tucked away in the Designer.cs file that’s created along with each snapshot. I tested this out and saw that if I didn’t change the model (that is, I didn’t remove the unwanted property from the Samurai class), the snapshot was still fixed up according to the migrations. But don’t forget to fix your model, too. Otherwise the model and the snapshot will be out of sync and, most likely, your database schema will also be out of sync with the model.

As a side note, Brice Lambson, the EF team member responsible for migra­tions, shared with me that if you do delete a migration file manually, the very next call to remove-­migration will see that and fix up the snapshot without deleting another migration file.

Like add-migration, remove-­migration doesn’t affect the database. However, it does check in the database’s migration history file to see if the migration has already been applied. If it has, then you can’t use remove-migration, not yet at least.

Sounds confusing, I know. But I still like remove-migration because the alternative is to just keep moving forward and adding a migration whenever you change your mind. That’s potentially confusing to other team members or to the future you. Plus, it’s a small price to pay for the benefit of allowing migrations to participate in source control.

To help clarify, Figure 6 has guidance for unwinding an unwanted migration. The chart presumes that “Migration1” has been applied to the database and “Migration2” was created, but never applied to the database.

Figure 6 Dealing with an Unwanted Migration

State of Database Commands Tasks Performed by Command
Update-Database not run after Migration2 Remove-Migration

1. Verifies Migration2 is not in history table

2. Deletes Migration2.cs

3. Fixes Snapshot.cs to reflect previous migration

Update-Database was run after Migration2

Update-Database Migration1

Remove-Migration

1. Runs SQL for “Drop” method of Migration2

2. Removes Migration2 row from history table

1. Deletes Migration2.cs

2. Fixes Snapshot.cs to reflect previous migration

If you need to undo multiple migrations, start by calling update-database followed by the name of the last good migration. Then you could call remove-migration to roll back migrations and the snapshot, one at a time.

With all of this focus on the migrations, don’t forget that those migrations are a reflection of changes to the model. So here is one last reminder that you must manually undo the changes to the model and then remove any migrations that reflect those changes.

Reverse Engineer with Scaffold-DbContext

Previous versions of EF provided the ability to reverse engineer a database into Code First models. Microsoft made this capability available through the Entity Framework Power Tools and, starting with EF6, through the EF Designer. There are also third-party tools such as the popular (free) extension, EntityFramework Reverse POCO Generator (reversepoco.com). Because EF7 doesn’t have a designer, the scaffold command was created to do this job.

Here are the parameters for Scaffold-DbContext:

Scaffold-DbContext [-Connection] <String> [-Provider] <String>
                   [-OutputDirectory <String>] [-ContextClassName <String>]
                   [-Tables <String>] [-DataAnnotations] [-Project <String>]
                   [<CommonParameters>]

I’m a big fan of fluent API configurations, so I’m happy that these are once again the default and you need to opt in to using data annotation instead with the DataAnnotations flag. The following is a command to reverse engineer from my existing database using that parameter (I could also select which tables to scaffold, but I won’t bother with that here):

Scaffold-DbContext
   -Connection "Server = (localdb)\mssqllocaldb; Database=EF7Samurai;"
   -Provider entityframework.sqlserver
   -OutputDirectory "testRE"

As Figure 7 shows, this resulted in a new folder and several files to be added to my project. I have a lot more examining of those classes to do because this is an important feature, but I’ll leave that for the future.

Result of Scaffold-DbContext
Figure 7 Result of Scaffold-DbContext

The DNX Versions of Migrations Commands

The commands you run in the Package Manager Console are Windows PowerShell commands, and Windows PowerShell is, of course, part of Windows. One of the most important features of ASP.NET 5—and EF7—is they are no longer dependent on Windows. DNX is a lightweight .NET execution environment that can be run on OS X and Linux, as well as Windows. You can read more about it at bit.ly/1Gu34wX. The entityframework.commands package contains migrations commands for the DNX environment, as well. Here’s a quick look at those commands.

You can execute the commands directly in the Package Manager Console if you happen to be using Visual Studio 2015, which is what I’ll show here. Otherwise, you execute them at the command line of your relevant OS. Check out Nate McMaster’s Channel 9 video where he codes up EF7 and migrations on a Mac. (bit.ly/1OSUCdo)

First, you need to be sure you’re in the correct folder—that’s the folder for the project where your model resides. The Default project dropdown isn’t related to your file path. Use the dir command to see where you are and then just cd to the correct folder. Tab completion is your friend here. In Figure 8 you can see that although the default project is src\EF7WebAPI, in order to execute dnx commands in that directory, I have to explicitly change the directory to the path for this project that contains my EF7 model.

Getting to the Correct Directory Before Running DNX Commands
Figure 8 Getting to the Correct Directory Before Running DNX Commands

Once there, I can execute commands. Every command, by the way, needs to be prefaced with dnx, and you can use dnx ef to list all of the available commands. Note that ef is a shortcut I’ve set up in project.json. Check out my Pluralsight video, “Looking Ahead to Entity Framework 7” (bit.ly/PS_EF7Alpha), to see more about the setup.

The core commands are database, dbcontext and migrations; migrations has the following sub-commands:

add     Add a new migration
list    List the migrations
remove  Remove the last migration
script  Generate a SQL script from migrations

Each command has parameters you can query by adding --help, like this:

dnx ef migrations add --help

For example, if the migration name is initial, then the command is:

dnx migrations add initial

Remember that these are the same commands you saw earlier in this article, so you’ll find parameters that allow you to specify the project, the DbContext and other details. The script command has the idempotent parameter as described earlier. But one thing that’s different about the dnx script command is that, by default, the script is simply listed in the command window. You need to explicitly specify an output <file> parameter to push the script to a file.

The following command will apply the migrations to the database: 

dnx ef database update

The dbcontext command has two subcommands: dbcontext list will list all of the DbContexts that can be discovered in the project. This is here because you can’t get tab-expansion to provide the options easily like you can when using the commands in Windows PowerShell; scaffold is the dnx version of the DbContext-Scaffold command you learned about earlier.

Overall a Much Better Migrations Experience with EF7

With the removal of automatic migrations, migrations have been simplified in EF7. Although the basic concepts are the same, the workflow is a lot cleaner. Removing the barriers to using migrations in a team environment was critical. I like the way this was fixed by bringing the model snapshot into the source code. And allowing reverse engineering with the scaffold command is a great way to provide this important capability without relying on a designer.

The July 23 Design Meeting Notes from the EF team (bit.ly/1S813ro) include a nice table with all of the Windows PowerShell and dnx commands after the naming was tightened up. I expect there will be an even cleaner version in the documentation eventually, but I still find this one useful.

I wrote this article using the RC1 version of EF7, but this version is considered to be feature complete, so it’s likely this information will align with the release of EF7 that’s coming along with ASP.NET 5 in early 2016.


Julie Lerman is a Microsoft MVP, .NET mentor and consultant who lives in the hills of Vermont. You can find her presenting on data access and other .NET topics at user groups and conferences around the world. She blogs at thedatafarm.com/blog and is the author of “Programming Entity Framework” (2009), as well as a Code First and a DbContext edition, all from O’Reilly Media. Follow her on Twitter: @julielerman and see her Pluralsight courses at juliel.me/PS-Videos.

Thanks to the following Microsoft technical expert for reviewing this article: Brice Lambson