No. Going onto/into private property would be trespassing, with or without horse. You can't take a horse on footpaths, it has to be a bridleway, at least. So most 'open' land would be out of bounds, as well.
You can ride on any road, except of course, a motorway. You should not ride on pavements although sometimes you may need to get off the road because of road works or some other hazard, but if you do you must give way to pedestrians at all times. As Catso says, you cannot ride on footpaths, only bridleways. You can go into town centres as long as there are no signs forbidding riding, but not many riders would want to.