New to carbide….what am I messing up here?

I recently picked up an entry level carbide face mill with TPU style inserts. I am not looking for a mirror finish with this tool, but noticed the surface finish changed for the worse mid-cut on one of my first cuts with it. I had used it a few times prior for some moderate cuts in mild steel and had decent results.

Tool is a 2.5″ dia 4 insert (TPU) face mill turning at 325rpm with a speed of ~212fpm (a bit slow, yes, but next step up is double that). Chips had a honey brown color and were well formed.

Mill head trammed just before starting and described a ~9 inch circle with less than 1 thou variation.

Started squaring up this mild steel block and ended up with the following surface finish. At the top of the block, in this picture, I was feeding from right to left with a 20 thou depth of cut at ~8ipm feed rate. First cut of the job. Halfway through the surface finish got worse and stayed that way for the rest of the job. Took a few more cuts and got the same rougher surface.

I’m new to carbide…am I making any obvious mistakes? Fixture (vise, on parallels, seated with hammer after clamping) and tool was tight.