City specialization

I've read from many different sources that I should be specializing my cities, however I'm unsure as to how exactly I should be doing this and my current "method" of specializing my cities doesn't seem to be working that well - I always find large generic mega-cities to be vastly more effective than any of my specialized cities.

For example, I tend to find that cities with low population don't produce much of anything unless I focus efforts on food (so that I have the population to work the surrounding tiles) and production (so that I can produce the buildings needed to specialize my city).

Conversely, although I know that I should be trying to produce buildings that provide a percentage bonus (e.g. to production or research) in those cities that have high numbers in the first place, whenever I look at the economy screen I tend to find that cities with higher population produce more of everything, and so in my larger cities I end up building everything (as they have the excess production required to do so).

The end result is that all of my cities look pretty much the same apart from size. How can I effectively specialize my cities?