Next door neighbours porch

[Suefoulkes]

My next door neighbours have recently started building a solid brick porch which is adjacent to my front window. They come into my garden whenever they wish to do work on the structure without asking my permission and have damaged plants in my garden by walking all over them. Are they allowed to just enter my garden when they wish or do I have the right to deny access?