What reasons does Holden give for being "sort of sorry" for visiting Mr. Spencer?

Holden has received a letter from Mr. Spencer asking him to come and visit. Mr. Spencer, a history teacher, knows that Holden has been asked to leave Pencey. However, almost immediately upon entering Mr. Spencer's room—Mr. Spencer is sick with the "grippe"—Holden realizes that he is "sort of" sorry he came. He sees 70-year-old Mr. Spencer in his robe, which reveals his "bumpy chest" and old person's legs. He also breathes in the medicinal scent of Vicks and overall feels disgust at the scene. But beneath his repulsion at confronting age and sickness Holden experiences compassion, for Holden is nothing if not a caring person.

By the time Mr. Spencer asks Holden to hand him his failed history exam paper, Holden, who likes to nuance his feelings, moves to "you can't imagine how sorry I was getting" for coming to visit. He is mortified that he had written a note on the unfinished exam saying it was OK if Mr. Spencer flunked him, which he did because he didn't want his teacher to feel bad about it. Yet, though he very much wishes Mrs. Spencer wouldn't bring up the exam at all, he continues to want to protect Mr. Spencer's feelings:

Well, you could see he really felt pretty lousy about flunking me. So I shot the bull for a while. I told him I was a real moron ...

Holden feels he has to get away from Mr. Spencer, in part because the older man sees the world from such a different perspective, but Holden also understands that his teacher genuinely is worried for him and cares about what happens to him:

"I'd like to put some sense in that head of yours, boy. I'm trying to help you. I'm trying to help you, if I can." He really was, too. You could see that. But it was just that we were too much on opposite sides of the pole, that's all.

Holden reassures Mr. Spencer twice not to worry about him and that he will be OK, even going so far as, though it must repulse him, to "sort of put my hand on his shoulder."

Holden is "sort of" sorry he goes to see Mr. Spencer not so much because his teacher is old and sick, though he does feel sorry about that, but because he recognizes that Mr. Spencer cares about him, and yet they view the world too differently for Holden to be able to genuinely communicate with him.