Well, on a very basic level and just for starters, polyglycol brake fluid absorbs moisture. Up to 2% of its volume per year if you're talking about DOT 3 spec. Even though the braking system should indeed be a sealed system, the air that brake fluid can be exposed to is a form of a contaminant as it carries moisture for the fluid to absorb. If you have any leaks in the system this potential is magnified. This is why it is recommended to always use brake fluid out of a sealed container.

Over time, accumulated moisture in the brake fluid has the potential to lower the boiling point and raise the freezing point of your brake fluid, which generally makes the brakes less reliable. Especially true if the brakes are subjected to severe use. Water in the brake fluid also has the potential to corrode the metal parts of the braking system, which can also affect the reliability of the system and lead to accelerated wearing and replacement of parts.

Plus the occasional flush removes any dirt which may have collected in the system.

Not saying that you ever have, or ever will, experience any of the above problems, but FWIW that's the theory behind the recommendation to periodically flush and replace fluid.