RenderTransformOrigin has a somewhat nonstandard use of the Point structure value, in that the Point does not represent an absolute location in a coordinate system. Instead, values between 0 and 1 are interpreted as a factor for the range of the current element in each x,y axis. For example, (0.5,0.5) will cause the render transform to be centered on the element, or (1,1) would place the render transform at the bottom right corner of the element. NaN is not an accepted value.
Values beyond 0 and 1 are also accepted, and will result in more unconventional transform effects. For instance, if you set RenderTransformOrigin to be (5,5), and then apply a RotateTransform, the rotation point will be well outside the bounds of the element itself. The transform will spin your element around in a big circle that originates beyond bottom right. The origin might be somewhere inside its parent element and could possibly be possibly out of frame or view. Negative point values are similar, these will go beyond the top left bounds.
Render transforms do not affect layout, and are typically used to animate or apply a temporary effect to an element.