Too bad most SSH clients don't seem to support SRV records, they would've been perfect for this:

  ;; Domain:     mydomain.com.
  ;; SSH running on port 2999 at host 1.2.3.4

  ;; A Record
  vm1928.mydomain.com. 1 IN A 1.2.3.4

  ;; SRV Record
  _ssh._tcp.vm1928.mydomain.com. 1 IN SRV 0 0 2999 vm1928.mydomain.com.

If supported it would result in just being able to do "ssh vm1928.mydomain.com" without having to add "-p 1928"