The .quad file contains the actual instructions the print driver uses that defines the amount of ink to lay down at each of the tonal steps along the grayscale. That .quad file is created with behind the scenes scrips based on the input values in the .qdif file (quad ink descriptor file). If those input values are incorrectly set then you could get the problems you are having—either the ink limits are set too high or the overlaps are not accurate. You can't actaul edit the .quad file (at least not easily) so you need to edit the .qdif file (or .txt file on a mac).
To better help understand what you are attempting to so and to better help you out here: Are you creating a profile from scratch or trying to linearize one of the raw profiles included with QTR? and are you using a Mac or PC? Did you print, measure and define the ink limits using the ink separate file printed with the QTR calibration mode? What printer and paper are you using (and what QTR profile are you using to print the grayscale stepwedge?)
I know this can seem overwhelming at first, especially if it isn't clear what is happening behind the scenes.
All the best,
Richard Boutwell