My brother is 3 years older than me and has Aspergers. Obviously, he struggles greatly with social issues to the point where his only contact outside of work is me.
The problem is, I am so busy. I work full time during the week and then work 6 evenings a week, plus during the day most Saturdays and Sundays. I find myself getting asked 'can you come round' SO much - I don't live far to be fair but it's just such a drag, he has no sense of humour and always wants to talk about how depressed he is, how awful everything is. Even if I a, low myself and he tries to empathise it just ends up being all about him so a 'bit under the weather, I've got a cold,' turns into him saying 'oh yeah I've got flu; it's awful, I was in agony, couldn't possibly go to work ...' 
I feel so torn as I love him but find him hard to like. I just feel guilty when I have to say no to him.
Can anyone empathise? I just need some understanding I think.