You can buy a horse in town. Also it seems they can disappear if you close down your server. If you ride fast you should not jump down from it because it would probably run off. Be careful to not press and hold shift (sprint button) nearby guards because the horse might do trample damage to them and they would kill your horse then. Don’t forget to put some grass into your horse if you go out with it and don’t want it to starve to death. To access the inventory of your pet or any container while you are mounted press R.
You can kill the horses in the stable to make them respawn, they will give you hide and meat.