ttymsg: Overhaul

* Instead of an error string, return the usual 0 or -1 and let the caller
  figure out what, if anything, to tell the user.

* Avoid string manipulations by opening /dev first and using openat()
  with O_RESOLVE_BENEATH.

* Add a boolean argument which, if false, causes ttymsg() to return
  without sending the message if the tty's group-writable bit is not
  set.  This saves programs that respect this setting (like syslogd(8))
  from having to check before calling ttymsg().

* Update all callers.

The observable effect of this change is minimal except for slightly
different error messages when ttymsg() fails.  However, syslogd(8) will
no longer print spurious error messages on the console after trying and
failing to write a log message to an X11 session.

PR:		295171
MFC after:	1 week
Reviewed by:	jfree, markj
Differential Revision:	https://reviews.freebsd.org/D57018

libexec/talkd/announce.c

@@ -40,6 +40,7 @@
 
 #include <errno.h>
 #include <paths.h>
+#include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
@@ -152,7 +153,7 @@ print_mesg(const char *tty, CTL_MSG *request,
 	 * stack up processes trying to write messages to a tty
 	 * that is permanently blocked.
 	 */
-	if (ttymsg(&iovec, 1, tty, RING_WAIT - 5) != NULL)
+	if (ttymsg(&iovec, 1, tty, RING_WAIT - 5, true) != 0)
 		return (FAILED);
 
 	return (SUCCESS);