What's the best way to deal with beggars?

OK, I work in a major city centre, and there seems to be loads of beggars. I don't know if I look vulnerable or not but they always seem to target me. I don't give them anything, as I believe they would only spend it on drugs/alcohol, and it is better to give it directly to a charity, but I always seem to get abuse off them. I've tried just ignoring them, and I get the abuse. I've tried saying, no sorry, and I get the abuse. It is really starting to put me off walking through the city because it is every day now that someone approaches me for money. How do you deal with beggars and what do you find works. Some of them can be quite intimidating as they have nothing to lose with being homeless, so they could easily rob you if you don't give them anything.