Probably not the worst I ever read, but whenever a question such as this comes up my mind immediately goes to one of the Tarot books by Piers Anthony. I don’t remember which one, it was just in a pile of books people left in a dorm one time and I had nothing to read. I finished it, but I can’t tell you anything about it other than the vague recollection that I hated it.
My girlfriend in high-school had an imaginary friend when she was a child. By the time we had met and got together he was in jail for murder I think. That was the end of the friendship.