When you say you calibrated QTR with a gamma 1.8 do you mean that the step wedge you used had an embedded gray gamma 1.8? or that some other setting of 1.8 in a photoshop levels adjustment (Mike Ware approach), or in the print dialog box? Or, was this in the Gray_Gamma input in the QTR profile creation tools (or text file).
The gamma from your digital camera is based on the color settings in the camera, or in the editing/raw conversion process (assuming you coming from raw capture). You can choose to use ProPhoto RGB as an export color space, which has a gamma of 1.8, which would translate directly to your current QTR calibration. I am working more with capture 1 and different process recipes for different output types depending on where things are ending up. The same can be done in Adobe Camera Raw.
If your digital camera is just spitting out jpegs then they are probably either Adobe RGB 1998 or sRGB, both of which have a gamma of 2.2. If your gray working space in photoshop is 1.8, then if you just convert the color files to grayscale it should convert it directly to the working space and it should look correct (although depending on how they are inverted prior to sending them to QTR it might end up wonky)
The nice thing about having a linear space in QTR, especially when working with digital negatives, is that the gray curve correction and linearization step for each process and setup will make the the whole range print straight so there is no worry about the shadows at gg 1.8 or 2.2 being flipped to the highlights and causing that to go weird in the print. Of course all that is dependent on everything being the same throughout the process... which seems to be what is monkeying things up here.
I just posted about a spreadsheet tool for curve creation for inkjet prints, but I did create one for working with a densitometer and converts density to LAB so that it can work easily with QTR and digital negatives.
You can find both here: http://www.bwmastery.com/qtr-correction-curve-tool
Richard Boutwell