How to: Load and Unload Assemblies (C# Programming Guide)
The assemblies referenced by your program will automatically be loaded at build time, but it is also possible to load specific assemblies into the current application domain at runtime. For more information, see Application Domains.
There is no way to unload an individual assembly without unloading all of the application domains that contain it. Even if the assembly goes out of scope, the actual assembly file will remain loaded until all application domains that contain it are unloaded.
If you want to unload some assemblies but not others, consider creating a new application domain, executing the code inside that domain, and then unload that application domain. For more information, see How to: Execute Code in Another Application Domain.