I have struggled with this before and now I will make a note of it to save myself time when I encounter this again.
I have a setup on Solaris 11 where I am running a custom filter to accept mail on port 25 and then pass it on to sendmail for processing. My filter need the sendmail daemon and I am running the sendmail daemon on port 10026. In a normal sendmail setup you are not stealing port 25 so I doubt you will have local mail submission problems. And you might not need local mail submission with mail or mailx anyhow. If you do need local mail submission and defaults are not working as was in my case below is my fix.
Make sure you are running the sendmail client.
# svcs -a | grep sendmail online Jun_30 svc:/network/smtp:sendmail online 6:43:35 svc:/network/sendmail-client:default
Generate /etc/mail/submit.cf. In my case I copied and edited custom_submit.mc. Note I had to use "MSA".
# pwd /etc/mail/cf/cf # diff submit.mc custom_submit.mc [...] < FEATURE(`msp', `[127.0.0.1]')dnl --- > FEATURE(`msp', `[127.0.0.1]',`MSA')dnl # /usr/ccs/bin/m4 ../m4/cf.m4 custom_submit.mc > /etc/mail/submit.cf
# svcadm disable svc:/network/sendmail-client:default # svcadm enable svc:/network/sendmail-client:default