Prevent crash in isatty()

The existing code allocated 2 bytes (char*), but gtty() needs 6 bytes
(struct sgttyb), so isatty() smashed the stack and corrupted its
return address, probably causing SIGBUS or SIGSEGV.

Fix by switching to TIOCGETD, which needs 2 bytes.  TIOCGETD isn't in
the manual for tty(4), but does appear in
https://minnie.tuhs.org//cgi-bin/utree.pl?file=V7/usr/sys/dev/tty.c

This fixes hilo_c.pdpv7 and hilo_mod.pdpv7 in simh-pdp11.
This commit is contained in:
George Koehler 2018-06-15 00:48:29 -04:00
parent 5de5611c24
commit 5db312a1c0

View file

@ -1,8 +1,8 @@
int isatty(int fd)
{
char* p;
unsigned u;
if (gtty(fd, &p) < 0)
if (ioctl(fd, /*TIOCGETD*/(('t'<<8)|0), &u) < 0)
return 0;
return 1;
}