White and Mid Grey Match
 

In this idea - I made a WORK COPY - then OPEN my ORIGINAL in my HUE/SATURATION TOOL.
First off - I REMOVED my COLOR (SATURATION) by making my VALUE BOX NUMBER to "0".
Then - I REMOVED my BRIGHTNESS by making my VALUE BOX NUMBER to "0".
Then - I SLOWLY begin to REPLACE BRIGHTNESS - by holding the WHITE of my 10 POINT GRAY SCALE (under even houselights) AGAINST the WHITE in the GRAY SCALE in the IMAGE to MATCH the SAME BRIGHTNESS in the WHITES.
Then - I SLOWLY REPLACE my COLOR - until my COLOR until I REACH the SAME BRIGHTNESS as in point directly above. Then I may add a touch of HUE as well or I may not depending if it is needed. I can also use the GREY CARD and do the same with that as a double check.
Then - when finished that - I may add a touch of CONTRAST, SHARPNESS or ..... only AS of IF NEEDED.
Then - I CROP and RESIZE and SAVE the FINAL RESULT.
One thing I do like - this cropped image is much closer to what my naked eyes saw than the original and my whites are not wiped - or close to being wiped out - with the texture still in the clothes.

I don't really see how what you are doing with this can work as the white on your grey scale card does not seem to be pure white (seems a fair way off to my eye). The white of the subjects top is much closer to pure white and still seems to have retained detail so is not blown. When comparing the white on the card in the image to the white on the card held to your monitor there are many potential issues. The brightness setting of the monitor will greatly influence the final output as will any colour shift due to an uncalibrated monitor. To my eye the brightness, whites and colour tones in the original are much better than in the final processed image.

I also find the way that the background has been removed to be very distracting as it has created a very harsh, unnatural outline to your subject. I would prefer to get a distraction free background in camera rather than relying on removing it in processing. If you do need to deal with a BG a touch of selective blurring would give a more natural look than replacing it with a block colour. When taking a portrait it is often worth spending as much (often more) time looking at the background than the subjects. Shooting to get a shallow depth of field will help a lot as will avoiding having bold coloured objects in the background as these will distract the eye when viewing the photo. If the background has distracting objects it can often be solved by shifting your angle, if not then move your subject!

My thanks Prem for the kind words. They are much appreciated.