Adjusting cutters on the Record 44 plane

Great little plane is the Record 44 for grooves and I tend to use it for small boxes a bit. The cutters start at 1/8 inch then jump up to 3/16 etc. I often find that my panels are around the 1/8 inch maybe a bit over. If I use the 1/8 inch cutter the grooves are too tight. My question is if you have a cutter that is a little too tight, do you just make another pass with a slight adjustment of the fence (I've found this to be easier said than done once the initial groove is made)? I have the other bigger cutters which I never use, maybe they can be ground down to fit? I would like just a little slop in the groove so that the thin panels can float slightly.