When you license the Filetransfer Appliance, it will be licensed to that particular hostname. You can continue to use the VMware installation or move to a physical server, as long as the hostname doesn't change.
Please try the VMware image first to determine that the Filetransfer Appliance suit your needs. If you buy a license and realize afterwards that it doesn't work for you, for whatever reason, please open a Support Ticket within 60 days and the money will be refunded instantly.