The flange is bright, the pipe is bright, and every bolt head holding them together is brown. The rust may be a surface film or the heads may be visibly eaten away. It is the same complaint either way, and it points at the fastener rather than at the system behind it.
The distinction matters because the repair is different. A pipe that is failing has to be replaced. A fastener that is failing needs a different fastener, and the flange underneath it is usually in perfectly good condition.
When two different metals are joined and wetted, the less noble one corrodes. A bolt is small; the flange and the pipe it clamps are large. Even a modest difference in alloy puts the bolt on the losing side of that ratio, and a small area giving up electrons to a large one corrodes quickly.
Hardware is often not the same grade as the part it holds. Plain steel, coated steel and lower-alloy stainless all turn up on exhaust assemblies, and any of them will fail while the stainless either side of them is still untouched.
A bolt is a set of crevices. The thread is a spiral gap, the underside of the head traps water against the flange, and a washer makes it worse by adding a second gap beside the first. Crevices hold liquid and exclude oxygen, and stainless in a low-oxygen chloride crevice is not protected the way it is on an open face.
Add the mechanical side. A fastener is under load, and the head is the part that meets the spanner. Coating damaged during fitting exposes bare metal at exactly the point that also stays wet, which is why the head goes first and the shank follows.
Not all brown on a bolt started on the bolt. Rust running down from a mild steel bracket above, or from a shield fixing, leaves a stain trail that begins higher up. Check the top of the head against the bottom: clean on top and rusty underneath means the source is above it.
Embedded free iron from a wire brush or a grinding wheel is the other impostor. It looks like rust on the fastener, but it wipes off and leaves sound metal, and it appears on the parts that were worked on rather than on the ones that were not.
Touch a magnet to the head and then to the flange. A strong pull on the bolt and nothing on the pipe tells you the two are different materials. Check whether the rust is confined to the heads and the exposed threads while the shanks are clean, which is the crevice pattern rather than a material fault.
Wipe a head with a cloth. Surface rust on a coated fastener comes off as a brown smear over grey metal that is still sound. A pitted head has lost material, its hex edges are rounded, and that one will not come undone when you need it to.
The simplest way to avoid the pattern is to specify fasteners with the part, in a grade that matches what they clamp, rather than buying them at the end of a build from whatever is in the drawer. A stainless bolt in the wrong grade, or a coated one used where it will sit in water, fails while the exhaust is still sound.
It is worth looking at how an assembly is supplied for that reason. Exhaust hardware and fittings chosen alongside the pipe keep the joint on one side of the galvanic scale, and a stainless steel exhaust muffler supplied with matching fixings will not show this stain at all.