I would not use abrassives on mill for two reasons: wear and it actually does not increase accuracy.
Surface grinders are next logical step from the mill, but to produce more flat, they are different machine, more accurate way geometry and more accurate spindle, but less load carrying capacity on other directions. Therefore surface grinder attachemets on the mill are usually used when material it too hard to mill, not much extra accuracy is gained.
Your observation on fly cutter vs. end mill is correct. Tramming affects less on cutters of smaller diameter.
You need to define how flat you need that piece first. Here you might resort to scraping, least you need to check it with blue.
Abrassives will then even out surface quality, but will not improve much flatness, easily decrease it actually, specially on the edges.
Lapping on cast iron plate is probably easiest and cheapest - specially if you do have the cast iron plate.