If mortal death does not mean the end of your existence, then you should find yourself in a spiritual realm, for want of a better term, and so expansion in a physical universe isn't likely to change owt where you find yourself, should it ?
How can there be a greater distance to travel when you are considering distances increasing here, and yet after death you are no longer here any more ? How do you know the illusion of time still exists wherever you find yourself after death anyway ?