According to the American Latex Allergy Association, there are around 1% in the population who are suffering from latex allergy. These people will experience rashes, itching, and even burning pain after contacting with the protein in latex (rubber in general) products.
So if you don't feel well after using latex condom, you may try the non-latex condoms. They include polyurethane, polyisoprene, or even lamb skin! After all, they are considered a better and newer material than latex. So why don't you have a trial?