A question about STD's?

My boyfriend and I became sexually active recently. So far, it has only been oral. My boyfriend is not a virgin but I am. Whenever we had oral, we never used any protection because I trusted him and there were no visible signs of STD's (no bumps, rashes, etc.) Before we became active, he told me that whenever he had intercourse, he used condoms but for oral, he didn't use condoms because he didn't think it was necessary. The girls that he had oral with were virgins, so... I didn't make a big fuss. However, I heard a rumor at school that one of the girls he had oral with did stuff with other guys and she wasn't honest with my boyfriend about it. I am a bit paranoid now because I hear that some STD's have no symptoms whatsoever. I don't have any symptoms... my boyfriend doesn't have any. Should I be concerned? I am kind of panicking because I heard that chlamydia has no symptoms but if it is left untreated, you could become infertile? What do I do?