Well, that's not entirely what I meant either. If you have a header that does what you want it to do, the bigger the better for the rest of the system, yes. If you don't have a header, or if the header is a generic one, very big pipes may or may not produce the desired results.

I'm really just trying to answer the original question, and the answer is that a free-flowing muffler cannot have adverse effects on performance at any rpm.