Quick XDMCP Question (Headless System)

Google might have the answer for this somewhere, but I'm not finding the appropriate query that susses it out...

Basically, what I want is to run my system at a level akin to run-level 3 (where the console device(s) are purely character-based), but still have the ability to `ssh -X` into my host and do an `Xnest :1 -query localhost` and get served a full graphical desktop login. Overall, I'm looking to avoid having the full desktop handler stack running except for "on demand" usage cases.

Bonus: if I can achieve that, how much of the X11 environment normally loaded via the "basic-desktop" yum group can I delete and still get the normal desktop experience granted through XDMCP?