Setting a button as a CancelButton in the Visual Studio designer also has the side effect of setting the DialogResult of the button to Cancel. For a modal dialog, this will automatically cause the form to close when the button is clicked. To prevent the form from closing either clear the DialogResult property or use this code in your click handler:
this.DialogResult = DialogResult.None;