I do not know of a built-in way to do what you want... but you can easily
define a class that inherits from System.Windows.Forms.TextBox, and adds
that functionality.  Simply define all the textboxes on your form to derive
from your class instead of directly from System.Windows.Forms.TextBox

That's not so hard.

Keep the value of the field in a class member or in a variable.  Then in
the validating event, if the value in the textbox is not valid restore the
textbox from the variable.  If the value is ok, then in the validated
event, set the variable with the value in the text box.

Something like this (pseudocode):

Private txtBoxValue As String

Private Sub TextBox1_Validating(...)
  If Not TextBoxContentsValid Then
     TextBox1.Text = txtBoxValue    'Restore previous value
     e.Cancel = True
  End If
End Sub

Private Sub TextBox1_Validated(...)
  txtBoxValue = TextBox1.Text          'Save the good value
End Sub

Never use interface elements (textboxes, etc.) to be the storage for your
data.  Always use a class or variable and only use interface elements to
reflect what is contained in the variable.  

As far as not being able to close the app if the field is not valid,  I
tend to use a close button that has it's CausesValidation property set to
False so that validation does not occur when I click that button.

HTH