Mostly I try to install the absolute minimum graphics software on a server. Most installs should be possible without graphics. For instance installing an Oracle database you can use silent mode and a response file.
Now and then you can't get around this. If you have the luxury of just remotely displaying a xterm back to your Unix desktop (or Windows with X-Window software like Xming etc) then that should be enough. If you have a slow link over a WAN then displaying X remotely becomes quite impossible. This is because of the way the X-Window system updates pixels, keyboard and mouse clicks. In this case you can try VNC.
Note since you have installed minimal graphics software on the server you should not expect a nice GNOME type desktop awaiting you upon a successful VNC connection.
This is an example of using VNC to connect to a Solaris 11 LDOM.
Install VNC server:
# pkg search vncserver INDEX ACTION VALUE PACKAGE basename file usr/bin/vncserver pkg:/firstname.lastname@example.org root@host11:~# pkg install pkg:/email@example.com
Run VNC Server as the user you need to use:
dev1_a@host11:~$ vncserver You will require a password to access your desktops. Password: Verify: New 'host11:2 (dev1)' desktop is host11:2 Creating default startup script /export/home/dev1/.vnc/xstartup Starting applications specified in /export/home/dev1/.vnc/xstartup Log file is /export/home/dev1/.vnc/host11:2.log
Now connect with your VNC client to host11:2.