• Article

February 2012

Volume 27 Number 02

NuGet - Creating a NuGet Gallery

By Clark Sell | February 2012

This is the third and last article in our series on NuGet, a new package management ecosystem for developers. NuGet, a project of the Outercurve Foundation, gives .NET developers a way to consume, author and publish packages. So far, the series showed you how to manage project libraries with NuGet (msdn.microsoft.com/magazine/hh547106) and how to become a NuGet author (msdn.microsoft.com/magazine/hh708753). In this article, we’ll take a look at hosting your own NuGet gallery and creating a build process to help manage your packages.

NuGet.org already exists as a public repository for NuGet packages, so you might question the point of hosting a gallery. That’s understandable, but what about leveraging that ecosystem infrastructure within the walls of your own dev environment? Why not set up your own private gallery not only to facilitate your product’s development ecosystem but, better yet, to tie it directly to your build and deployment processes? Moreover, anything you see at NuGet.org is available to you free.

In this article, we’ll explore the steps necessary to create and consume your own NuGet gallery. As we’ve said before, incorporating NuGet into your development lifecycle isn’t complicated and will be well worth the effort. Once you’ve done it, your package consumption, creation and distribution problems will become no more than a distant memory.

Before diving in, let’s explore a use case that most developers have encountered: versioning. Your product, depending on its overall size and complexity, is very likely the result of many teams, differing release schedules and a variety of assemblies. Of course the released product is nicely bundled for your customers, but its journey to completion included hundreds if not thousands of builds, churning out many different versions of product assets.

Over the past decade, development practices—including automated build and deployment systems—helped reduce the pain, but they always fell short in terms of distribution and choice. Package management systems provide the solution for this.

Think of your own development ecosystem, in which different teams work independently to publish the versions they choose for consumption. Each team, with its own build and release schedule, wants control and an easy way to publish its products. With a package management system, each team gets to decide what the product is and how it should get installed, and the customers get to decide when they should consume it.

To some degree, this is very similar to how open source communities operate, using proven practices that easily scale. Although the packages you find on NuGet.org are fairly “large grained,” you can follow those same practices for your building your own applications. A good, concrete example of the type of software this would be useful for might be a helper library your company builds and maintains for its public products.

Maybe your company restricts only the libraries and versions its developers may use. This is another great reason for you to host your own proprietary gallery with the approved packages and package versions.

Fast and Simple

The easiest way to host your own NuGet gallery is to expose a file share. This might sound a bit rudimentary, but if you need something fast and simple, this method works extremely well. The share can be structured as you see fit, placing your packages—that is, your *.nupkg files—anywhere on the share. You can refer to the share as a package source.

With the package source created, let’s add it to the development environment. You can have as many package sources as desired and easily move between each. To edit your package sources in Visual Studio, navigate to Tools | Options | Package Manager | Package Sources (see Figure 1). Note that as of this writing, version 1.5 was current. You should expect slight differences across NuGet versions.

Adding a Package Source in Visual Studio
Figure 1 Adding a Package Source in Visual Studio

Here you can add, remove and change the default package source. If you prefer, you can do the same in WebMatrix, a free development environment. Just click on the NuGet Gallery icon on the main menu and select Add Source, as shown in Figure 2.

Adding a Package Source in WebMatrix
Figure 2 Adding a Package Source in WebMatrix

As you can see, both figures list two package sources. You can choose to use just one or all at any given time.

While the file system package source is very easy to set up and start using, it breaks down pretty quickly when you start scaling out. Luckily, NuGet.orgwas built with exactly this in mind, to provide an open platform for people to take, extend and build on for their own needs. You can find the NuGet Gallery project at github.com/NuGet/NuGetGallery. The NuGet Gallery is an ASP.NET MVC 3 project built on top of Razor, OData, SQL Server and Azure.

The best way to understand the features and functionality the NuGet Gallery has to offer your development ecosystem is to simply explore NuGet.org.

Before you can create and publish a package, you need to be a registered user. Once you’ve registered, you have full rights to upload and manage your packages and edit your profile. You can also generate the unique API key that lets you automate publishing packages to the Gallery (see Figure 3). For more information about publishing a package, please see the previous article in this series, “Becoming a NuGet Author.”

A Registered User Account in NuGet
Figure 3 A Registered User Account in NuGet

When you upload a package to the NuGet Gallery, it automatically receives its own place in the Gallery, as shown in Figure 4.

A Package Uploaded to the NuGet Gallery
Figure 4 A Package Uploaded to the NuGet Gallery

This is where you can view all of the important information about a package. For each package, the NuGet Gallery tracks the version downloads, total downloads, dates, licenses and so on. Of course, a package on the NuGet.orgsite will most likely yield considerably larger download numbers given its ecosystem, but these statistics are vital to any package author, regardless of project size.

The package feed is clearly the most important feature in the overall stack. It’s an OData feed you’ll find at https://YourGallery/api/v2. This is the same URL you’ll use as your package source. Without this feed, all of the power of NuGet is lost.

Getting Started

Before getting started, verify you have the following installed:

At the time of this writing, there are no installers for the Gallery and the project is under very active development. That means it’s always best to go to the project’s homepage for the latest information. You’ll need to download the source and build it, which is simple.

To get the source, you can either just download a zipped copy of the latest master branch or clone the source base with git, like so:

$ git clone https://github.com/NuGet/NuGetGallery.git

When you’ve done this, you’ll have the entire source base locally. At its root you’ll find a Windows PowerShell script called Build-Solution.ps1. Run this script to set up your machine and build the source.

If you run the source, you’ll notice it looks and feels exactly like NuGet.org. You might be thinking of tweaking the UI, perhaps modifying the homepage. That’s understandable, but be cautious, especially if your intentions are to stay in sync with the master source base.

The next logical step is integrating your NuGet gallery with your existing build processes. If you don’t have a build process and you’re looking for a little “getting started” guidance with Team Foundation Server (TFS) build processes, check out the Microsoft Visual Studio ALM Rangers Build Customization Guide at rabcg.codeplex.com.

All right: You have a library you want to make available on a gallery, which could be the official NuGet Gallery, a proprietary gallery or even a local file share. No matter which, you’ll have to perform certain tasks to create the NuGet package, and they’re generally the same tasks no matter where the gallery resides.

Through the course of developing your product, you’ll repetitively go through the process of coding, compiling, packaging, pushing and publishing. You’ll also, we hope, include a couple of healthy doses of testing; results from that testing might make you deviate from publishing to the gallery if you find any issues (see Figure 5).

The NuGet Development Process
Figure 5 The NuGet Development Process

Keeping consumers happy with working software is a good thing, and you’ll have an easier time doing so using a managed process. Remember, though, the managed process can’t be entirely automated; there’s no replacement for human verification. Don’t assume you can publish just because your software passes all the automated tests. You could easily publish a package that isn’t ready and make your consumers angry. Moreover, they won’t be happy if you publish constantly and they can’t keep up with the ever-changing versions. Make publishing a conscious decision.

Use the build process to build, package and push your NuGet library to a gallery where you perform final tests and do that human verification. That way, you can choose continuous integration or a scheduled build to automatically push the package and make it ready for you to test. How might you do that? Here’s an example using TFS and its workflow-based automated build capability.

NuGet.exe is a stand-alone, parameter-driven console application, but it can easily be called by another process, such as TFS Team Build. Team Build executes a workflow; in TFS terms, this is a set of custom activities controlled through a build template and initiated by a build definition. One packaged solution for automating the NuGet development process is the NuGetter project at nugetter.codeplex.com. This project provides an automated shell around the NuGet.exe application. We’ll use this package to show what can be done and things to think about as you work toward automating your own NuGet packages.

As Figure 6 shows, the build definition in TFS is where you provide all the necessary information to do the packaging, push and publish. Building a flexible and repeatable build process for NuGet requires several pieces of data and the switches that turn portions of the process on or off. The data is organized into categories that are titled to give you a sense of the order in which the steps are executed.

Nugetter Build Template
Figure 6 Nugetter Build Template

The PrePackaging section is used for complicated or multiframework NuGet packaging that requires some initial preparation. NuGetter can call a Windows PowerShell script (if you desire) to organize the library files to make packaging easier. This step isn’t required, so you can use the flag parameter “Invoke PowerShell Script” to tell the process whether to execute the script.

You can use a .nuspec file directly to define how your package will be created or you can do so indirectly through a .csproj file. It’s up to you; use the approach that better suits your needs.

As you can see, all of the parameters required for packaging are included. Version, base path (source folder for packaging), API Key and gallery location (called Source) are all provided as part of the build definition. However, the security of this information might be of great importance to you. For example, the API Key is what allows you and only you to push NuGet packages to the gallery. If that key becomes public, anyone could push bogus packages in your name. Because of this, NuGetter allows you to provide either the actual API Key or a path to a file that contains the key. If you use a file path, the build process reads the file, extracts the key and uses it without listing the key in the log, and security is maintained. Obviously, the build service ID must have read access to the file for this to work.

Another potential issue for you is managing the version of your package. In Figure 6, the version number is provided in the build definition. But, do you really want to keep changing that number in the build definition? What happens when you have several builds (continuous integration [CI], daily, other scheduled and various manual builds)? NuGetter gives you the choice to enter the version directly or, as with the API Key, you can provide a file path that multiple build definitions can use. They will all use the same version number and you can manage it in just one place.

The Push Destination in the example is the NuGet Gallery, but this is where you provide the build destination even if you’re pushing your package to an internal gallery or to a local file store. This is another case where the build process might need to intervene. NuGet.exe expects a URL as the destination and if you’re pushing to a local store—which won’t be a URL—the process needs to interpret the destination format. In this case, instead of using NuGet.exe to do the pushing, you have the build process do it.

For reasons of completeness and feature parity with NuGet.exe, NuGetter does provide for automated publishing. Just keep in mind our earlier caveats before deciding to use this capability.

By the way, you don’t have to worry that your consumers might miss any package updates. The community has this covered with NuGetFeed (github.com/NuGetFeed/NuGetFeed). NuGetFeed lets you build a custom RSS feed based on the packages you select. Your consumers can add a custom feed to their favorite RSS reader and be informed of any updates.

Another option for creating a NuGet feed is MyGet at MyGet.org. MyGet might be perfect for those of you who want a private NuGet feed for your list of approved packages but don’t want to spend the time, money, and effort creating and maintaining your own gallery infrastructure. MyGet is a hosted solution that allows you to create galleries very quickly that are specific to your needs. Refer to the MyGet site for more detailed information.

Finally, as we mentioned earlier, NuGet and NuGet Gallery are open source projects. This means you’re empowered to contribute anything from documentation to features, or smash a few bugs along the way. On the github homepage, the project team has detailed the exact steps you can take to contribute.

Wrapping Up

NuGet has no doubt changed the way we think about our .NET package management ecosystem, both public and private. Both NuGet Gallery and NuGetFeed are essential components to completing that ecosystem. Bringing these tools into your daily routine opens new possibilities. As noted, NuGet Gallery is in active development, so make sure to visit github.com/NuGet/NuGetGallery for the most up-to-date information.

You can find all of the links used in this article and more at on.csell.net/HostingNuGet. 

Clark Sell is as a senior Web evangelist for Microsoft outside of Chicago. He podcasts at DeveloperSmackdown.com, blogs at csell.net and can be found on Twitter at twitter.com/csell5.

Mark Nichols is as a senior software consultant for Microsoft outside of Chicago. He podcasts at DeveloperSmackdown.com, and can be found on Twitter at twitter.com/mark_nic.

Thanks to the following technical experts for reviewing this article: David Ebbo, Phil Haack and Brandon Satrom