*Important:* Do `make clean` to work around a problem and prevent
infinite rebuilds, https://github.com/davidgiven/ack/issues/68
I edit tokens.g in util/LLgen/src, so I regenerate tokens.c. The
regeneration script bootstrap.sh can't find LLgen, but I can run the
same command by typing the path to llgen.
Silence warning from clang at `if (ch2 = ...)`
Delete `|| rm %{outs}` in build.lua, because it hid the exit status of
tabgen, so if tabgen failed, the build continued and failed later.
Edit build.lua for programs losing their private assert.h, so they
depend on a list of .h files excluding assert.h.
Remove modules/src/assert; it would be a dependency of cpp.ansi but we
didn't build it, so cpp.ansi uses the libc assert.
I hope that libc <assert.h> can better report failed assertions. Some
old "assert.h" files didn't report the expression. Some reported a
literal "x", because traditional C expanded the macro parameter x in
"x", but ANSI C89 doesn't expand macro parameters in string literals.
Because of the accidental deletion, mcgg on my machine followed a
garbage pointer, and never wrote calls to emit_fragment.
A wrong call to `data->emit_reg(0, 0)` instead of the correct
`data->emit_fragment(0)` caused PowerPC mcg to emit an empty string
instead of `8(fp)`, causing a syntax error in PowerPC as.
The wrong `data->emit_reg(0, 0)` called the function emit_reg() in
mach/proto/mcg/pass_instructionselection.c, but that function
unfortunately has `if (vreg) { ... }`. The call had vreg == NULL
because the fragment wasn't a vreg, but emit_reg() ignored the problem
and emit nothing.
@dram reported a build failure in FreeBSD at
https://github.com/davidgiven/ack/issues/1#issuecomment-273668299
Linux manual for getopt(3) says:
> If the first character of optstring is '-', then each nonoption
> argv-element is handled as if it were the argument of an option with
> character code 1....
>
> The use of '+' and '-' in optstring is a GNU extension.
GNU/Linux and OpenBSD handle '-' in this special way, but FreeBSD
seems not to. If '-' is not special, then em_ego can't find its input
file, so the build must fail. This commit stops using '-' in both
em_b and em_ego, but doesn't change mcg.
Also fix em_ego -O3 to not act like -O4.
This is more useful when looking for patterns; lino - 1 is probably
the line number in the patterns file. DIAGOPT is off by default but
one can edit optim.h to enable it.
The other changes just clean up whitespace.
In util/ncgg, add two more errors for tables using reglap:
- "Two sizes of reg_float can't be same size"
- "Missing reg_float of size %d to contain %s"
In mach/proto/ncg, rename macro isregvar_size() to PICK_REGVAR(), so
the macro doesn't look like a function. This macro sometimes doesn't
evaluate its second argument.
In mach/powerpc/ncg/mach.c, change type of lfs_set to uint32_t, and
change the left shifts from 1U<<regno to (uint32_t)1<<regno, because
1U would be too small for machines with 16-bit int.
This relocation is specific to PowerPC. @davidgiven suggested the
name RELOPPC_LIS in
https://github.com/davidgiven/ack/pull/52#issuecomment-279856501
Reindent the list in h/out.h and util/led/ack.out.5 because
RELOLIS_PPC is a long name. I use spaces and no tabs because the tabs
looked bad in the manual page.
If the ncg table uses reglap, then regvar($1, reg_float) would have
two sizes of registers. An error from ncgg would happen if regvar()
was in a token that allows only one size. Now one can pick a size
with regvar_w() for word size or regvar_d() for double-word size.
Add regvar_d and regvar_w as keywords in ncgg. Modify EX_REGVAR to
include the register size. In ncg, add some checks for the register
size. In tables without reglap, regvar() works as before, and ncg
ignores the register size in EX_REGVAR.
This merges several fixes and improvements from upstream. This
includes commit 5f6a773 to turn off qemuppc. I see several failing
tests from qemuppc; this merge will hide the test failures.
The new feature "reglap" allows two sizes of floating-point register
variables (reg_float), if each register overlaps a single register of
the other size. PowerPC ncg uses reglap to define 4-byte instances
of f14 to f31 that overlap the 8-byte instances.
When ncgg sees the definition of fs14("f14")=f14, it removes the
8-byte f14 from its rvnumbers array, and adds the 4-byte fs14 in its
place. Later, when ncg puts a variable in fs14, if it is an 8-byte
variable, then ncg switches to the 8-byte f14. The code has
/* reglap */ comments in util/ncgg or #ifdef REGLAP in mach/proto/ncg
reglap became necessary because my commit a20b87c caused PowerPC ego
to allocate reg_float in both 4-byte and 8-byte sizes.
same base name and generate multiple files based on it, we can't really use
mkstemp() for every temporary file. Instead, use mkstemp() once on a
placeholder, then generate temporary names based on this. (And delete the
placeholder once we've finished.)
The size of a reg_float isn't in the descr file, so ego doesn't know.
PowerPC and SPARC are the only arches with floating-point registers in
their descr files. PowerPC and SPARC registers can hold both 4-byte
and 8-byte floats, so I want ego to do both sizes.
This might break our SPARC code expander because ego doesn't know that
8-byte values take 2 registers in SPARC. (So ego might allocate too
many registers and deallocate too much stack space.) We don't build
the SPARC code expander, and its descr file is already wrong: its list
of register save costs is too short, so ego will read past the end of
the array.
This commit doesn't fix the problem with ego and PowerPC ncg. Right
now, ncg refuses to put 4-byte floats in registers, but ego expects
them to get registers and deallocates their stack space. So ncg emits
programs that use the deallocated space, and the values of 4-byte
floats become corrupt.
Use f14 to f31 as register variables for 8-byte double-precison.
There are no regvars for 4-byte double precision, because all
regvar(reg_float) must have the same size. I expect more programs to
prefer 8-byte double precision.
Teach mach/powerpc/ncg/mach.c to emit stfd and lfd instructions to
save and restore 8-byte regvars. Delay emitting the function prolog
until f_regsave(), so we can use one addi to make stack space for both
local vars and saved registers. Be more careful with types in mach.c;
don't assume that int and long and full are the same.
In ncg table, add f14 to f31 as register variables, and some rules to
use them. Add rules to put the result of fadd, fsub, fmul, fdiv, fneg
in a regvar. Without such rules, the result would go in a scratch
FREG, and we would need fmr to move it to the regvar. Also add a rule
for pat sdl inreg($1)==reg_float with STACK, so we can unstack the
value directly into the regvar, again without a scratch FREG and fmr.
Edit util/ego/descr/powerpc.descr to tell ego about the new float
regvars. This might not be working right; ego usually decides against
using any float regvars, so ack -O1 (not running ego) uses the
regvars, but ack -O4 (running ego) doesn't use the regvars.
Beware that ack -mosxppc runs ego using powerpc.descr but -mlinuxppc
and -mqemuppc run ego without a config file (since 8ef7c31). I am
testing powerpc.descr with a local edit to plat/linuxppc/descr to run
ego with powerpc.descr there, but I did not commit my local edit.
I hastily chose the name RELOLIS for this relocation type. If we want
to rename it, we only need to edit these files:
- h/out.h
- mach/powerpc/as/mach5.c
- util/amisc/ashow.c
- util/led/ack.out.5
- util/led/relocate.c
The new relocation type RELOLIS handles these instructions:
lis RT, ha16[expr] == addis RT, r0, ha16[expr]
lis RT, hi16[expr] == addis RT, r0, hi16[expr]
RELOLIS stores a 32-bit value in the program text. In this value, the
high bit is a ha16 flag, the next 5 bits are the target register RT,
and the low bits are a signed 26-bit offset. The linker replaces this
value with the lis instruction.
The old RELOPPC relocated a ha16/lo16 or hi16/lo16 pair. The new
RELOLIS relocates only a ha16 or hi16, so it is no longer necessary to
have a matching lo16 in the next instruction. The disadvantage is
that RELOLIS has only a signed 26-bit offset, not a 32-bit offset.
Switch the assembler to use RELOLIS for ha16 or hi16 and RELO2 for
lo16. The li32 instruction still uses the old RELOPPC relocation.
This is not the same as my RELOPPC change from my recent mail to
tack-devel (https://sourceforge.net/p/tack/mailman/message/35651528/).
This commit is on a different branch. Here I am throwing away my
RELOPPC change and instead trying RELOLIS.
This commit slightly improves the formatting of the manuals. My
OpenBSD machine uses mandoc(1) to format manuals. I check the manuals
with `mandoc -T lint` and fix most of the warnings. I also make
other changes where mandoc didn't warn me.
roff(7) says, "Each sentence should terminate at the end of an input
line," but we often forgot this rule. I insert some newlines after
sentences that had ended mid-line.
roff(7) also says that blank lines "are only permitted within literal
contexts." I delete blank lines. This removes some extra blank lines
from mandoc's output. If I do want a blank line in the output, I call
".sp 1" to make it in man(7). If I want a blank line in the source,
but not the output, I put a plain dot "." so roff ignores it.
Hyphens used for command-line options, like \-a, should be escaped by
a backslash. I insert a few missing backslashes.
mandoc warns if the date in .TH doesn't look like a date. Our manuals
had a missing date or the RCS keyword "$Revision$". Git doesn't
expand RCS keywords. I put in today's date, 2017-01-18.
Some manuals used tab characters in filled mode. That doesn't work.
I use .nf to turn off filled mode, or I use .IP in man(7) to make the
indentation without a tab character.
ack(1) defined a macro .SB but never used it, so I delete the
definition. I also remove a call to the missing macro .RF.
mandoc warns about empty paragraphs. I deleted them. mandoc also
warned about these macro pairs in anm(1):
.SM
.B text
The .SM did nothing because the .B text is on a different line. I
changed each pair to .SB for small bold text.
I make a few other small changes.
---snip---
The ELF spec at http://www.sco.com/developers/gabi/ says, "In each
symbol table, all symbols with STB_LOCAL binding precede the weak and
global symbols," and that sh_info is the index of the first non-local
symbol.
I was mixing local and global symbols and setting sh_info to zero. I
also forgot to set the type of the .shstrtab section.
---snip---
assembler directives, ha16() and has16(), for the upper half; has16() applies
the sign adjustment. .powerpcfixup is now gone, as we generate the relocation
in ha*() instead. Add special logic to the linker for undoing and redoing the
sign adjustment when reading/writing fixups. Tests still pass.
Convert each ack.out symbol to ELF, preserving its name and value, and
some but not all other symbol info. The ELF symbol table comes with
ELF section headers. If the input file has no symbols (ack -Rled-s),
then the output file has no ELF section headers.
Append the symbol table and section headers to the ELF file. Linux
ignores this appended info when it execs the file. The info might
pollute the last page of the ELF segment, but Linux clears this
pollution. Tools like nm and gdb can read the ELF symbol table.
I include code to translate debugger symbols to an ELF .stab section.
I did not test this code, because I did not find a platform that can
put S_STB symbols in the ack.out file.
Put end of sentence at end of line. This is roff(7) syntax so the
formatter can make spacing between sentences.
Use the macro .PP to break paragraphs. Use bold for the command name
in SYNOPSIS, to match other ack manuals.
No change to linuxppc and qemuppc. They continue to run ego without
any descr file.
I copied m68020.descr to powerpc.descr and changed some numbers. My
numbers are guesses; I know little about PowerPC cycle counts, and
almost nothing about ego. This powerpc.descr causes most of the
example programs to shrink in size (without descr -> with descr):
65429 -> 57237 hilo_b.osxppc -8192
36516 -> 32420 hilo_c.osxppc -4096
55782 -> 51686 hilo_mod.osxppc -4096
20096 -> 20096 hilo_p.osxppc 0
8813 -> 8813 mandelbrot_c.osxppc 0
93355 -> 89259 paranoia_c.osxppc -4096
92751 -> 84559 startrek_c.osxppc -8192
(Each file has 2 Mach segments, then a symbol table. Each segment
takes a multiple of 4096 bytes. When the code shrinks, we lose a
multiple of 4096 bytes.)
I used "ack -mosxppc -O6 -c.so" to examine the assembly code for
hilo.mod and mandelbrot.c, both without and with descr. This reveals
optimizations made only with descr, from 2 ego phases: SP (stack
pollution) and RA (register allocation). In hilo.mod, SP deletes some
instructions that remove items from the stack. These items get
removed when the function returns. In both hilo.mod and mandelbrot.c,
RA moves some values into local variables, so ncg can make them into
register variables. This shrinks code size, probably because register
variables get preserved across function calls. More values stay in
registers, and ncg emits shorter code.
I believe that the ego descr file uses (time,space) tuples but the ncg
table uses (space,time) tuples. This is confusing. Perhaps I am
wrong, and some or all tuples are backwards. My time values are the
cycle counts in latency from the MPC7450 Reference Manual (but not
including complications like "store serialization").
In powerpc.descr, I give the cost for saving and restoring registers
as if I was using chains of stw and lwz instructions. Actually ncg
uses single stmw and lmw instructions with at least 2 instructions.
The (time,space) for stmw and lmw would be much less than the
(time,space) for chains of stw and lwz. But this ignores the pipeline
of the MPC7450. The chains of stw and lwz may run faster than stmw
and lmw in the pipeline, because the throughput may be better than the
latency. By using the wrong values for (time,space), I'm trying to
tell ego that stmw and lmw are not better than chains of stw and lwz.
This brings in David Given's PowerPC changes, including the addition
of the modern code generator (mcg) for PowerPC.
Resolve minor conflicts in top build.lua and util/led/main.c
This uses uint32_t for the base, file offset, and alignment of each
section, to be consistent with the usage of uint32_t in h/out.h
Also declare setbit() as static.
corresponding invocation in the ncg table so the same helpers can be used for
both mcg and ncg. Add a new IR opcode, FARJUMP, which jumps to a helper
function but saves volatile registers.
unlink("core") doesn't work with OpenBSD, where core dumps have names
like "ncg.core". Users who don't want core dumps can turn them off
with "ulimit -c 0" in sh(1). Then the system doesn't write a core
dump. That's better than writing core then unlinking it.
This commit changes how ack(1) parses backslashes in its descr files.
Before this commit, ack set the high bit of each character escaped by
a backslash, and later cleared all high bits in command arguments, but
this lost the high bits in non-ASCII filenames. After this commit,
ack keeps backslashes in strings while processing them. Functions
scanvars(), scanexpr(), doassign(), unravel(), addargs() now
understand backslashes. I remove from ack_basename() the warning
about non-ASCII characters.
This commit makes some incompatible changes for backslashes in descr
files. None of our descr files uses backslashes, except for those
backslashes that continue lines, and there are no changes for those
backslashes. The problem with non-ASCII filenames had its cause in a
feature that we weren't using.
With this commit, ack now understands backslashes after the = sign in
both "var NAME=value" and "mapflag -flag NAME=value". Before, ack
never scanned backslashes in "var" lines, so "var A=\{B}" failed to
prevent expansion of B. Now it does. Before, ack did scan for
backslashes in the "-flag NAME=" part of "mapflag" lines. Now it
doesn't, so it is no longer possible to map a flag that contains a
literal space, tab, or star "*".
I removed the expansion of "{{" to "{". One can use "\{" for a
literal "{", and "\{" now works in "var" lines. Before and now, ack
never expanded "{" in flags for "mapflag", so the correct way to map a
literal flag "-{" remains "mapflag -{ ...", not "mapflag -{{ ...".
(The other way "mapflag -\{ ..." stops working with this commit.)
Backslashes in strange places, like "{NA\ME}", probably have different
behavior now.
Backslashes in "program" lines now work. Before, ack scanned for
backslashes there but forgot to clear the high bits later.
Escaping < or > as \< or \> now works, and prevents substitution of
the input or output file paths. Before, ack only expanded the first <
or > in each argument. Now, it expands every unescaped < or > in an
argument, but this is an accident of how I rewrote the code. I don't
suggest to put more than one each of < or > in a command. The code no
longer optimizes away its recursive calls when the argument is "<".
The code continues to set or clear the high bit NO_SCAN on the first
characters of flags. This doesn't seem to be a problem, because flags
usually begin with an ASCII hyphen '-'.
If fork() fails, then report a fatal error. Don't spin the cpu
retrying fork() until it succeeds. It can fail when we reach a limit
on the number of processes. Spinning on the cpu would slow down other
processes when we want them to exit. This would get bad if we had a
parallel build with multiple ack processes spinning.
new= newvar(name) takes ownership of the string and might free its
memory. Don't print name. Do print new->v_name.
Also #include <string.h> for strcmp().
Declare most functions before using them. I declare some functions in
ack.h and some in trans.h (because trans.h declares type trf). I
leave declarations of scanb() and scanvars() in .c files because they
need type growstring. (I can't #include "grows.h" in another header
file as long as grows.h doesn't guard against multiple inclusion.)
Functions used within one file become static. I remove a few tiny
functions. I move a few functions or declarations to more convenient
places. Some functions now return void instead of a garbage int.
I feel that keyword "register" is obsolete, so I removed it from where
I was editing. This commit doesn't touch mktables.c
register 'type'; now use int/float/long/double throughout to identify
registers. Lots of register allocator tweaks and table bugfixes --- we now get
through the dreading Mathlib.mod!
and epilogues. mcgg now exports some useful data as headers. Start factoring
out some of the architecture-specific bits into an architecture-specific file.
attributes when allocating. Unfortunately, backward edges don't work (because
the limited def-use chain stuff doesn't work across basic blocks). Needs more
thought.
turned into generic ones (as they'll be useful everywhere). Node arguments for
predicates require the '%' prefix for consistency. Hex numbers are permitted.
This feature has never been used since its introduction, more than 3
years ago, in David Given's commit c93cb69 of May 8, 2013. The commit
was for "PowerPC and M68K work". I am not undoing the entire commit.
I am only removing the stackadjust and stackoffset() feature.
This commit removes the feature from my branch kernigh-linuxppc. This
removal includes the mach/proto/ncg parts. The default branch already
removed most of the feature, but kept the mach/proto/ncg parts. That
removal happened in commit 81778b6 of May 13, 2013 (which was a merge;
git diff af0dede81778b6). The branch dtrg-experimental-powerpc
merged the default branch but without the removal. That merge was
commit 4703db0f of Sep 15, 2016 (git diff 8c94b134703db0). My branch
kernigh-linuxppc is off branch dtrg-experimental-powerpc, so I can no
longer get the removal by merging default.
David Given described the stackadjust feature in
https://sourceforge.net/p/tack/mailman/message/30814691/
The instruction stackadjust would add a value to the offset, and the
function stackoffset() would return this offset. One would use this
to track sp - fp, then omit the frame pointer by not keeping fp in a
register.
mcg can track individual hop inputs and outputs (needed for live range
analysis!); the register allocator now puts the basic blocks into the right
order in preparation for live range analysis.
to make special nodes like NOP work properly). Realise that the way I'm dealing
with the instruction selector is all wrong; I need to physically copy chunks of
tree to give to burg (so I can terminate them correctly).
inasmuch as it looks better before register allocation. Basic blocks now know
their own successors and predecessors (after a certain point in the IR
processing).
functions. Not convinced that semantic types are actually working --- there are
still problems with earlier statements leaving things in the wrong registers.
instructions can be turned on and off based on their parameters. New lexer
using a lexer. Now quite a lot of the way towards being a real instruction
selector.
Upon enabling the check, mach/powerpc/ncg/table fails to build as ncgg
gives many errors of "Previous rule impossible on empty stack". David
Given reported this problem in 2013:
https://sourceforge.net/p/tack/mailman/message/30814694/
Commit c93cb69 commented out the error in util/ncgg/cgg.y to disable
the Hall check. This commit enables it again. In ncgg, the Hall
check is checking that a rule is possible with an empty fake stack.
It would be possible if ncg can coerce the values from the real stack
to the fake stack. The powerpc table defined coercions from STACK to
{FS, %a} and {FD, %a}, but the Hall check didn't understand the
coercions and rejected each rule "with FS" or "with FD".
This commit removes the FS and FD tokens and adds a new group of FSREG
registers for single-precision floats, while keeping FREG registers
for double precision. The registers overlap, with each FSREG
containing one FREG, because it is the same register in PowerPC
hardware. FS tokens become FSREG registers and FD tokens become FREG
registers. The Hall check understands the coercions from STACK to
FSREG and FREG. The idea to define separate but overlapping registers
comes from the PDP-11 table (mach/pdp/ncg/table).
This commit also removes F0 from the FREG group. This is my attempt
to keep F0 off the fake stack, because one of the stacking rules uses
F0 as a scratch register (FSCRATCH).
calculated incorrectly because of overflow errors.
Replace it with an extended RELOPPC relocation which understands addis/ori
pairs; add an la pseudoop to the assembler which generates these and the
appropriate relocation. Make good.
--HG--
branch : dtrg-experimental-powerpc-branch
cf/cf_loop.c and share/put.c tried to read the next pointer in an
element of a linked list after freeing the element. ud/ud_copy.c
tried to read beyond the end of the _defs_ array: it only has
_nrexpldefs_ elements, not _nrdefs_ elements.
These bugs caused core dumps on OpenBSD. Its malloc() put _defs_ near
the end of a page, so reading beyond the end crossed into an unmapped
page. Its free() wrote junk bytes and changed the next pointer to
0xdfdfdfdfdfdfdfdf.
and generate invalid calls to the optimisers.
Previously ego would generate a temporary file template that looked like
/tmp/ego.A.BB.XXXXXX, call mktemp() on it to randomise the XXXXXX, and then
replace A and BB with data.
However, it used strrchr to find the A and B. Which would fine, except when
mktemp produced an A or a B in the randomised part...
This code was written on 4 March 1991. I was 16.
This needed lots of refactoring to ego --- not all platforms have ego descr
files, and ego will just crash if you invoke it without one. I think originally
it was never intended that these platforms would be used at -O2 or above.
Plats now only specify the ego descr file if they have one.
the number of types of relocation possible in the object file. (Now,
hopefully, working.)
Also change the object serialiser/deserialiser to never try to read or
write raw structures; it's way safer this way and we don't need the
performance boost any more.
--HG--
branch : default-branch
