Don't Confederate flags meen your racist...?

So when I was back in school there was this guy who I used to see every once in a while. He had like a Confederate backpack and Confederate bumper stickers all over his truck. He used to stare at me all the time but I thought it was because I'm black/mixed and I thought he hated black people. But it turned "out" that he likes me, saying how beautiful I am, calling me, wanting to hang out. But I'm kinda weary of this guy because I thought that Confederate flags and whatnot ment that your racist. I'm not trying to label people but that's just what I was told living in a mostly white community in the south. I think he's really cute but I'm not to sure about this. Help? Sorry for spelling I'm on my phone. Thanks(: