The problem here is that everyone has a different definition of “middle class”.

I think you’re using the definition of “middle management”, and Carlin was using the definition of “working class”. The lower class would be the poor who have less than full time employment (inconsistent or on welfare, etc), and “the rich” would be people who don’t actually have to work for a living.

The terms are inconsistent.