Nikon 1 V1.
 

I'm getting decent results with this camera but feel I could do more with something a bit better. I like the system compacts because they are a nice size and not heavy, I don't mind going a bit bigger but not to a full size DSLR, any suggestions.

I doubt that you will see a huge leap if you change to another CSC, but I do far prefer the Panasonic's over the other options. I recently went from a J1 (which should be the same for IQ as your V1) to a Panasonic GF5 and am glad that I did. The slightly smaller sensor in the Nikon was always an issue for me as although it handled noise well the images had a very digital feel. For me they had that look that you get with compacts rather than the cleaner look from a DSLR. I have not tried any of the newer Nikon 1 series cameras so don't know if this has been improved. What lenses do you use with you V1? It might be that a new lens makes more of a change than a new camera/system.