What's wrong with my ear?

About a week and a half ago I got back from a Florida vacation in which on several occasions, I got water stuck in my ear, and each time I did a crappy job of getting the water out. After a day or two when we got back my ear started hurting and I didn't really pay it any attention until it became quite bothersome whenever I would scratch my ear and a shooting pain would run through it. I thought it was swimmer's ear so I took some swimmer's ear drops and even though I let them spill out after about 5 minutes of having them in (due to the burning pain they were causing in my ear) the next day the pain was pretty much gone. Another day went by with no pain. And then I think we're to yesterday, when I was cleaning my ears out with a q-tip after the shower. I was going pretty deep in my ear like I always do, thinking nothing of it, but later on in the day and today as well, my ear has started hurting again, this time even more so than before. So I know it's either swimmer's ear, an infection, or I harmed my ear while cleaning it. Which is more likely?