Congrats on building a better relationship thus far. You can ask if he'd like to go casually along with you somewhere or do something together. You can say something like, "If you have some board games, when you are together you can ask if he wants to play." Or, "I want to ride my bike but I'd feel better with a man along. Do you want to come?" If you know he likes a certain sport or team, try remembering some stats and bring up the topic. The key is to get him to enjoy time with you that is not just physical. At the very least you will build a better bond where he will feel at ease with you.