How do I deal with this guy I'm really into? Im so scared I will get hurt!?

There's this guy from my work that I like. We met months ago on a work get together but work in diff. states. We talk now and again by phone and have even got together once, kissed, he wanted more but I said 'not now' (as I need time but didnt explain this to him & he cancelled an invite he had given me the next day..maybe he felt rejected? But now he has called me and wants to come see me on his next vacn. I asked him what he was up to for the weekend and he told me that a girl will call by to work on a project with him. I asked him if this was a girlfriend and he said no it is a friend. His voice didn't hint of any feelings for her. I mean, why would he call me & say he's coming to visit if he's into another girl? He does like travelling though and would love to see my city. He will also call to another guy friend on his travels. Am I worrying unecessarily? I know he has loads of female friends and one I know is def only platonic as she is also a friend of mine. Is he interested?