Alright, so it might be that there is a negative atmosphere tied to the house, regardless of whether or not you consciously think about it. That in itself is a very tragic thing and it may be tied to your lose. Have you felt that you have healed from that experience?
Now that could mean that you need to shift that atmosphere to a positive one, remembering the good time you shared with your brother or the happy times that had in general. I know that someone passing away so tragically can be hard to move away from, and you do need to let yourself heal no matter how long that will take. You also need to understand that you are still alive and your brother would want you to live the life you wanted and if that means move on, then you should move on.

Now if you truly feel that you want to travel, not to escape from you past or the town or anything, your past may be keeping you there because you do not have closure, or you could have just lost confidence to travel alone because it is a scary thing to go places alone not knowing anyone. With that, you need to heal and be patient with yourself. You may need to wait until you build back yourself or you may just need to take that risky jump and see what happens. Like I said, you need to determine your own priorities and work toward everyday, listen to what you think is the best choice to do that.