It's a form of aphantasia, which I share with you.
I can only remember that my best friend is heavily overweight through recalling talking about it. (I can't picture him in my mind).
I can't tell you anything about the looks of people I've known in the past (including my parents), or I know now, unless, at some time (for example), I've actually thought to myself "Oh, they're tall" or "Isn't his hair receding these days". (I have no pictures of them in my mind).
If I recall incidents where a friend was with with me (even just a minute or so after they've left) my 'visualisation' of the scene in my mind only includes a vague impression of them being present (or I might hear their words). I don't see them in my mind.
Aphantasia is common among people on the autism spectrum (as I am).
You can take an aphantasia test here:
http://www.bbc.co.uk/news/health-34039054