I love being with you is a polite way to tell someone you like them but are not in love with them
It is an accepted social response to being told "I love you". There is a whole episode of Friends about this when Ross is dating Mona.
Ross: Hey! You were a closed book! Okay? I’m not a mind reader! Besides, I hate those conversations. I’m horrible at them. Really! Maybe-maybe I need kind of a gesture. Y’know, something that says we’re moving forward without having to talk about it.
Monica: Like asking her to move in with you?
Ross: Smaller than that.
Monica: Making her a mixed tape?
Ross: Uh, bigger than that.
Phoebe: Give her a key to your apartment.
Ross: Whoa-hello! We were closer with the mixed tape.
Monica: All right. Have you said, "I love you?" You could say, I love you.
Ross: Yeah I-I don’t-I don’t think I’m quite there yet, but I could say I looove spending time with you.
Phoebe: No, we hate that.
Monica: That is a slap in the face.