----------------------- DIARY FOR PARROT Z-MACHINE DEVELOPMENT -----------------
----------------------------- September 15, 2003 -------------------------------
I realized I can for now steal txd's output, cuz that has all (?) the
information I need from the Z file, including the opcode & its arguments.
Use txd -n because that gives addresses instead of naming the routines. (I
could use names too, but eventually I want to go back to true addresses when
I'm running actual bytecode, so why go away from that?).
Use -w 0 so we don't wrap lines and can use while(<>).
Use -a so we get ^ instead of actual newline. This might change, though.
Is it illegal to jump into the middle of a subroutine? Wouldn't the ret break
the stack? Also, you couldn't jump to the top of a sub cuz the arguments would
confuse the which was expecting commands. This means I should be able to
implement the routines as routines with actual calls, as opposed to the subs
which'll really be subs.
But are we sure that everything txd thinks is a sub is really a sub?
It's legal (but "bad practice" and confuses txd) to jump into a subroutine. I
think I'll make the command decision that the Perl version of topaz won't deal
with things like that. A straight Parrot version probably wouldn't care. IMCC?
Well, I should probably code it not to use subs. (Or maybe it doesn't care
either.) Jumping into a sub doesn't change the call stack. Maybe IMCC/PASM
would be fine with that.
Managed to get a whole bunch of ops working. Simplified version of test.inf
works!
----------------------------- September 16, 2003 -------------------------------
Spec 2.2: Numbers are sometimes regarded as signed, in the range $-32768$ to
$32767$. In effect $-n$ is stored as $65536-n$ and so the top bit is the sign
bit.
The operations of numerical comparison, multiplication, addition, subtraction,
division, remainder-after-division and printing of numbers are signed; bitwise
operations are unsigned. (In particular, since comparison is signed, it is
unsafe to compare two addresses using simply jl and jg.)
t2.inf is working!
I spent a ridiculous amount of time before I found out what arith & log shift
meant. Now I know, and finally it passes t3.inf.
Added inc, dec, inc_chk, dec_chk, store, art_shift, log_shift
----------------------------- September 17, 2003 -------------------------------
I realized that I can use the same skeleton to translate to PASM.
E.g. Topaz::PASM and Topaz::Perl.
Each has a translate_ops, translate_header, etc. sub that outputs the right
info (to TOUT file handle)
Moved much more stuff into subroutines in order to effect the above. It
probably makes things cleaner anyway. I believe the main program is now
output-language agnostic.
Created Topaz::Language::Generic along with TL::Perl and TL::IMCC
(and a factory to make them).
Added opcode 'test'.
Started working on IMCC translate_command!
----------------------------- September 18, 2003 -------------------------------
Added ret, new_line, push, pop, pull, store (low-handing fruit). Should test!
----------------------------- September 19, 2003 -------------------------------
Added load, print_char, ret_popped
Passes t4.inf and in-progress t5.inf.
Working on test program.
----------------------------- September 22, 2003 -------------------------------
I basically need to store static AND dynamic memory in my output file. The
problem is that the top of static memory is stored NOWHERE in the file. And
routines from high memory can be interleaved with data (yuck!). So really, we
should just bite the bullet and read 64K into the file. It doesn't hurt,
except by adding 200K (2 bytes plus a space per byte, plus line number) and
4000 lines to every single output file. Bleah.
Truth is, the program probably would've ended up being that long anyway. I
don't really know the format of everything in the static memory, so how would
I store it in variables anyway?
Add hooks for simplify().
Add hexified first 64K of story file to the output file.
Still passes current t5.inf.
----------------------------- September 23, 2003 -------------------------------
It would be nice to store stuff from dynamic memory in pretty arrays & stuff,
but note that we can't entirely do that because we need to rewrite a Z-code
dynamic memory for "save". Maybe when I write the dynamic memory parser, which
infodump is currently doing, then I'll write the outputter too. For now,
though, I can basically copy e.g., Rezrov's ZObject.pm.
It looks like I don't need to know much about static memory. Obviously, the
code needs to be able to access it, but I won't need to parse it much, because
most of the parsing is done by the Z-code. The Z-spec memory map has in static
memory: grammar table, actions table, pre-action table, adjectives table,
dictionary. Of those, I think dictionary is the only one that I need to access
in ways other that a simple loadb.
Would it be faster to take $dynamic = substr($static, $dynamic_size) and
translate loadb as substr($addr > $dynamic_size ? $static : $dynamic, $addr, 1)?
I don't actually know that vec/substr access is much faster on a smaller
string, and 64K won't really impress Perl anyway in terms of slowing down arg
passing or whatever.
LOTS of work on t5.inf. Broke it into subs that test different pieces
(arithmetic, print_* etc.). Added a bunch of tests.
Added loadb/w, storeb/w. Started on t6.inf.
----------------------------- September 24, 2003 -------------------------------
Changed Topaz to Plotz: Pol(l)y-Lingual Opcode Translator for the Z-machine.
Made PlotzInclude.pm.
Implemented random.
----------------------------- September 25, 2003 -------------------------------
t6.inf: count tests & prettify output.
Copied rezrov's decode_text. Trying to get print_addr to work.
MAJOR PROBLEM with using txd!!! When it gets a constant in, e.g., a je
command, it finds the string that starts there (sometimes?!). But I want to
compare a value to a value, not to the string that's there! I'm pretty sure
this means I need to decode at least those hexcodes pretty early on to be able
to run Zork or Advent.
----------------------------- September 29, 2003 -------------------------------
Move a bunch of stuff to PlotzParse.pm, which will take care of
(language-independent, of course) parsing of the command.
It will place stuff in a big hash that'll get passed to simplify and
then to translate_command.
Implement simplify() for ret_popped and print_ret. Ugly but workable
format_commands().
FYI, there's 18 opcodes that're only version 6.
There's 15 or so object/property opcodes
I implement 50-something.
----------------------------- September 30, 2003 -------------------------------
Realized that txd puts quotes in in lots of circumstances. So I need
to read the hex codes after all.
Work on PlotzParse::parse_command
------------------------------ October 01, 2003 --------------------------------
More work on parse_command. And on getting PlotzTranslator to understand
the resulting hash.
One problem I'm having is that I can't apply something to all args, like
make_var. And I can't join all existing args to e.g. call a sub. FOr now, I'm
just doing "random" => "PlotzRuntime::z_random(Z_RANGE)" but that could
get annoying with more complicated subs. Oh well.
It's working! Got t7.inf to work.
Now I should be set to create Header code, and then do objects!
------------------------------ October 02, 2003 --------------------------------
Turned out my fancy get_word stuff didn't work: vec($a, $index, 16) MUST have
$index even! Went back to Edmonson's vec<<8 + vec.
Almost got get_parent working!
------------------------------ October 03, 2003 --------------------------------
get_parent, get_child, get_sibling work. test_attr and prop stuff SHOULD be
pretty simple, now that I got the general object stuff out of the way
set_attr, clear_attr, test_attr, print_paddr
------------------------------ October 05, 2003 --------------------------------
print_obj
Other object stuff isn't quite as simple as I thought.
------------------------------ October 07, 2003 --------------------------------
Thought a lot about save/restore, looked at Quetzal stuff.
Thought about call stack, which I'll need to implement.
------------------------------ October 08, 2003 --------------------------------
jin, get_prop_addr, get_prop_len, get_next_prop
------------------------------ October 09, 2003 --------------------------------
get_prop, put_prop, remove_obj, insert_obj
------------------------------ October 10, 2003 --------------------------------
Changed parse & runtime to use @Memory instead of $Low_Mem. It means MANY
fewer calls to vec (especially when I correct global vars to always write to
memory, which they don't now), and why not store it as numbers?
verify, piracy (ha!)
Change t7.inf to t8.inf cuz we finished objects.
Call it v0.4
Added version-specific syntax to parse_command. Syntax does change between
versions for the same command sometimes, but I believe in all but one case
(erase_line) you just get extra arguments, which won't hurt anything.
Btw, seems like the parsing is running slower today. Could it be that @Memory
access is significantly *slower* than vec? Really shouldn't be for just an 8K
array. Or maybe it slowed down a while ago and I didn't notice? In any case,
it doesn't matter if parsing is slow. If runtime ends up very slow, I could
try & change back to vec in PRP & see if it speeds up. Bleah.
Added abbreviation support.
Changed print & print_ret to just store address of the string and use
print_addr.
------------------------------ October 13, 2003 --------------------------------
Read hexcodes from memory instead of using txd.
I'm also testing at every command and routine header whether my PC matches
that of txd. The only place I don't check is the very first routine.
Seems to work for t8, minizork, and advent! (It was a bit ugly cuz of orphan
code framgents. Note that if I only read addresses of routine headers, I won't
have the orphan problem & can revert to the simpler code in plotz.pl's main
loop.
Handle pre-V5 local variable initialization.
"call sp" was being translated to &rtn$stack[$sp--] which breaks Perl.
Changed it to &{'rtn' . ($stack[$sp--])} which works as long as you turn off
strict refs. (Not worth changing those few sub calls to create a block turning
off strict refs cuz (1) it's not trivial - you might be using the sub return
value - and (2) eventually these'll just be computed gotos, which won't
break Perl (perhaps sadly).)
Zork now breaks because of an indirect variable, while Advent gets a bit
further (if you define e.g. sub unimplemented_set_text_style {}) and even
prints some stuff out but then gives infinite uninitialized value messages.
If I "fix" the indirect variable in Zork (and set @Global_Var from @Memory at
the top of the program) then we print some good and some gibberish welcome
text, and then die due to trying to set $stack[$sp] when $sp is -2! That
sucks! OTOH, if I'm really lucky, that problem will go away when I fix other
things.
Another problem I noticed. There's a variable that starts out as -1 and gets
incremented. Perl is incrementing it to 65536, 65537, etc. I guess I *do* need
to set to unsigned_word() when doing load/store/etc.? Because the system tries
to access an array [global_var() + local11*2]. If local11 is 65K, that breaks.
Then again, I get the same error if I do local11%=65K in there.
------------------------------ October 15, 2003 --------------------------------
Allow je to take multiple args.
Fixing global variables should be easy: just call global_var(num) or
global_var(num, value). However, it gets hard because of the interplay of
several factors:
- I was using ++ in several places, and Z_foo=Z_bar in others, which would
break global_var(), unless we made it an lvalue subroutine, which isn't
entirely standard Perl.
- exactly when is it an lval?
- Bug in existing code: inc sp should be equivalent to $stack[1]++,
but it was popping the stack, then incrementing the thing it popped off (and
throwing it away).
- Bug: store was being translated to "Z_VARIABLE = Z_VALUE", which means
store sp would change to $stack[--$sp] = $value when it should be
$stack[$sp++]!!
Ended up fixing (?) all of these by changing make_var to get_var, and then
calling a var_to_lval routine which changes rvals to lvals!
Right now, I'm printing out "Useless use of..." every time I do a -> sp.
I could fix it by using $stack[] throughout instead of push/pop, but what a
pain!
The bad news is, this may totally break for indirect variables...
------------------------------ October 17, 2003 --------------------------------
Fix "Useless use" problem by using "$stack[@stack] =" instead of push @stack.
Start changing t8.inf to work with v3 or v4, using
"#Iftrue #version_number >= 5; version5 stuff...; #Endif;"
Problem is that call_vn and call_vn2 didn't exist until v5. I had to
change my strategy and use Inform calls more so they translate to
call/call_vn depending on the version.
Other problem is that v3 has max 3 arguments, which breaks assert2 and other
subs.
With a bit more hacking, I managed to get v3/v4 to compile that way - and
plotz works on the v4 and v3 files too!
I set all set_vars to set the value to ($rval)%0x10000. I'm not sure whether
this is right or not.
Found some bugs. I'm getting a little further in translating zork. To get it
to start up, I need to do "no strict 'refs'" for a sub call, and replace two
instances of [sp]. Problem is that now I'm getting a bug where it says it
tries to call 0, and I'm not sure whether it's supposed to get that or not. If
I say "return 0" instead of calling zero, then it doesn't print "There is a
small mailbox here". OTOH, if I don't do %0x10000 it gets farther, but only
cuz it's sending a 65536 in as an argument - and in that case it prints a
bunch of gibberish, followed by "a small mailbox". Argh! And I don't really
know how to debug this...
------------------------------ October 20, 2003 --------------------------------
Fixed computed calls ("call sp"), which were breaking due to strict refs. (Put
them in a "do {no strict refs; &{...}}" block.)
I THINK I got indirect vars working! I still need to write a bunch more tests
for it, but it passes a number of tests (which frotz, dzip, and rezrov fail!)
Minizork now runs directly from compilation. It prints the header and the
first message, then dies at L27721 because it's trying to goto routine0, using
the result of a get_prop. (If I return 0 at that point, it proceeds, but
doesn't print the "There is a small mailbox here.") If I comment out the
routine call AND the "return 1" line after it, then it continues, without
printing that there's mailbox, until it prints the ">", then complains about
unimplemented_read. I.e., there's still at least one bug, but we're making
progress! I think I'm going to need to actually trace the entire program
running (with the "-d" I'm going to add to the runtime) to figure out what's
going on.
------------------------------ October 21, 2003 --------------------------------
I tracked down the mini-Zork problem! It turns out it was calling 0. I thought
forever that "call 0" means "return 0", but really it means "0", i.e. DO
NOTHING but give a false value if it was a call_*s routine. Once I fixed that,
I can run minizork, and it correctly prints out everything and dies trying to
do "read" after printing '>'!
Advent also works (modulo several more unimplemented (io) opcodes), although I
had to put in a Zstrict test.
------------------------------ October 22, 2003 --------------------------------
Moved stuff from PlotzParse header reader to PlotzIOPerl::update_header, since
it's all IO related. Hacked a few things because we combined files into one
file, but otherwise was able to get it to run. Didn't test it at all yet.
Need to move output_stream and a bunch of other subs into there.
E.g., Stream class that does stream stuff.
Actually, can I do any language-independent stream stuff? E.g., put rules
about streams in PlotzIO? Printing to memory? Might require too many
dependent/independent sub calls.
------------------------------ October 23, 2003 --------------------------------
Staretd working on "read"!
------------------------------ October 24, 2003 --------------------------------
More work on read.
------------------------------ October 27, 2003 --------------------------------
Work on encode_text
------------------------------ October 28, 2003 --------------------------------
Figured out some stuff about indirect variables.
Much more on read/dictionary/encode_text subs.
After much debugging, we're able to match words in the dictionary.
read seems to work!
Zork is getting there! Direction commands seem to work. This includes "north"
and "go north". However, "open mailbox" or any two-word command seems to break
with "you used 'mailbox' in a way I don't understand". It does find the second
word, though. Still, I may be writing it to the parse buffer incorrectly.
I think I may need to trace it.
------------------------------ October 29, 2003 --------------------------------
I may have finally really fixed indirect variables. Yuck!
t8.inf now tests all indirect opcodes pretty carefully.
------------------------------ October 30, 2003 --------------------------------
Aha! Finally found a good acronym for t8: CZECH!
Current best candidates for what the acronym stands for:
- Check Z-machine Emulator Compliance Handily (Hastily?)
- Comprehensive/Complete Z-machine Emulator CHecker
- Comprehensive Z-machine Emulator Compliance H
In Zork1: 19778 is "w", 16362 (3fea) is "e"
Found the bug! I was missing a () in "test" opcode. Look at all the other
opcodes to make sure precedence OK!
------------------------------ October 31, 2003 --------------------------------
Cleaned up and tested czech 0.8. It's ready for release.
Zippped up Plotz 0.5 while I'm at it.
Next big move is I/O. output_stream, buffer, (newline ASCII 13 stuff?),
then input_stream. Then test read extensively. Clean up PerlIO and release
a version (0.6? 0.7?)
Then all I have left is save/restore (big) for v0.8,
Win32/Curses for v0.9 (if necessary)
Cleanup Module structure for v1.0!!!
----------------------------- November 03, 2003 --------------------------------
Baby step towards directory cleanup: create PlotzPerl/ and move Runtime and IO
into it (with minor module renaming).
----------------------------- November 04, 2003 --------------------------------
Copied a huge chunk of code from Games::Rezrov into PlotzPerl::IO.
Tried to get it to work.
----------------------------- November 05, 2003 --------------------------------
Found some bugs. We're now able to (I think) correctly do t9.z5.
----------------------------- November 06, 2003 --------------------------------
Broke czech.inf into a bunch of include files.
- test.h has assert, p/f in it.
- io.h has I/O tests (right now, just test_print)
- non_io.h has all non-I/O tests in it.
- dynamic.h is a stub for a file that'll eventually test dynamic memory
- test_all.inf test I/O and non-I/O
- test_io.inf test I/O only
- test_non_io.inf test non-I/O only
Copied test_*.inf to parrotZ/ta.inf et al. I can now compile
them with "inf +czech ta".
Worked a while on buffering and paging. They seem to work now.
Having copied huge amounts of code from Games::Rezrov::StoryFile,
I'm actually now able to put the following opcodes into PlotzTranslator:
output_stream
input_stream
split_window
set_window
erase_window
get_cursor
set_cursor
set_text_style
!!!
OTOH, I haven't really tested them. Now I need to write a bunch of tests in
io.h to see if these things work.
Btw, since I set save_undo to return -1 (and do nothing), I'm now able to
compile and run Advent.z5 without any changes to the source code. Again, no
guarantees it works.
----------------------------- November 07, 2003 --------------------------------
Added -r, -c options to Plotz output program. Promptly ignored them and set
default to 24x80 on dumb terminal.
Working on io.h stream tests.
frotz seems to be asking for a filename each time I open stream 2.
Isn't that against the spec?
----------------------------- November 10, 2003 --------------------------------
Stream 2 is at least partly working.
----------------------------- November 11, 2003 --------------------------------
I've decided to make the czech test separation "interactive" vs.
"non-interactive". So for non-int I'm going to use input_stream 1
and test read. I may use output_stream 2. Can definitely use output_stream 3.
Start working on 'read' test. I'm able to read a line & compare it to
an array of bytes.
----------------------------- November 12, 2003 --------------------------------
Put read test into a subroutine do_ni_read. Then I can test multiple strings.
Created ti.in to input the strings with.
----------------------------- November 13, 2003 --------------------------------
Infocom-ized the read tests. Btw, I'm running into a problem because Inform
encodes a quotation mark in the dictionary as a ZSCII char instead of an A2
char, so my encode_text (and Winfrotz'!) aren't finding it.
Able to run tn.pl with Win32!
Also, when I run my program in a 1000-line window, it prints the buffer on
line zero, which is invisible way up at the top of the scrollbar! Shouldn't
the system be using Window() instead of Size() and then writing things to
(Size)[1]-(Window)[1]?
Alternatively, you could set Size equal to Window, which will get rid
of the scrollbar. Still, it seems convenient to have the scrolling available.
I did the latter for now, cuz it only requires changing one subroutine
(tho it would be polite to restore the size (and color) in DESTROY!)
Able to run minizork.z3, too!
Advent's status line isn't working correctly, though. Is it getting
buffered?
----------------------------- November 17, 2003 --------------------------------
Found a STUPID bug in verify which happened to work always until now because
dynamic memory's last byte was always a zero.
rezrov doesn't buffer output stream 2. This is a good excuse to refactor the
whole IO module.
Put plans in IO section in NOTES.txt for now.
Release v0.6. It's not really fully tested but I should freeze things before
breaking everything. Tests seem to work right now.
----------------------------- November 19, 2003 --------------------------------
Separated PlotzPerl::Input from the rest of PlotzPerl::IO. In theory, I can
now separate out PlotzPerl::Output, tho that's harder. (Almost everything now
in PPI will go in there!)
----------------------------- November 20, 2003 --------------------------------
Looks like stuff in the upper window doesn't get written to stream 2, but DOES
get written to stream 3.
----------------------------- November 25, 2003 --------------------------------
Thinking about streaming
----------------------------- November 26, 2003 --------------------------------
Wrote lots of notes on streaming plans. I think I have a workable system.
Text is turned into PP::Text objects which are blessed scalarrefs; the
subclasses determine whether the text was input or output, etc.
PP::Output tries to write to every stream (but returns immediately if it
wrote to stream 3). Each OutputStream tests itself to see whether it's
selected, and also to see whether there are other reasons not to accept
the stream. If the stream does accept, then the text is streamed to it.
If buffering is on for that stream/window/buffer_mode, then it gets
buffered, otherwise simply written. Only stream 1 ever knows about
the Screen object (although stream 2 needs to know which window we're
writing too since some windows don't get written to the transcript).
----------------------------- December 01, 2003 --------------------------------
Moved a bunch of stuff to PlotzPerl::Output.
I'm able to run czech on it with the new class in place.
I'm running into some ugliness: should windows, cursor be in Output class?
If not, then we're doing lots of cross-class talking. If yes, then Output
class is really just the IO class (cuz Windows aren't only Output). In THAT
case, we should get rid of output class and just give IO class an array of
OutputStreams, an array of Windows, InputStreams, etc.
----------------------------- December 04, 2003 --------------------------------
Start creating OutputStream class & subclasses.
----------------------------- December - February ------------------------------
[RL]
----------------------------- February 24, 2004 --------------------------------
Added rudimentary XML output capability. Depends on XML::Simple
Should make XML::Simple use'd only if I input .xml as the target, tho.
------------------------------ March 04, 2004 ----------------------------------
Export needed subs from PP::Runtime, to make output program a bit more
readable. I should do the same for PPI, once it settles down.
------------------------------ March 08, 2004 ----------------------------------
Working on adding restart opcode.
Wrote simple Perl script to read quetzal files.
It looks like Frotz2002 may have a bug for extremely simple programs - the
local vars in the save file don't seem to be what they should be.
Rezrov always makes 15 local variables in each frame of the save file.
Dzip seems to get things right, or at least almost right.
------------------------------ March 09, 2004 ----------------------------------
More work on OutputStream stuff. Get it to write transcript bits if we change
them using storew instead of output_stream.
Put Plotz into cvs at home!
Fixed a package bug that was causing register_newline not to be used at all,
such that [MORE] wasn't happening. Then fixed a couple package bugs that made
register_newline not work. We seem to be back to passing tests again.
------------------------------ March 10, 2004 ----------------------------------
Put main() into a while loop and an eval block.
Which allows for restart/restore.
Basically, restart/restore do a die() with a special string that means "repeat
the while loop, which will rerun the Z-machine from scratch, possibly with
some extra stuff happening."
@quit re-implemented as a die() with a string that means "This die is OK."
(Using die for restart/restore also means if I call restart I'm not pushing
the whole new game onto the (Perl) callstack that I was in when I called
restart. die pulls me all the way out of the call stack before restart puts me
back in.)
The eval/while conveniently allows me to call cleanup routine on the ZIO, with
the knowledge that it'll run whether I die or exit gracefully. So, e.g.,
Win32 port now returns the window to black & white & puts the scroll bar back
after running.
restart opcode appears to work! At least, it saves fixed pitch bit and does
appear to start from the beginning again.
Because restart would otherwise rerun the entire program, I'm creating a
restart.inf separate from the regular test programs.
I need to add to the test program, tho.
------------------------------ March 11, 2004 ----------------------------------
Changed PlotzParse to read "txd|" all in one gulp and store its data.
Then our main plotz program just calls that at the beginning. Afterwards,
in our subroutine loop, the system just uses each value from the stored
txd data.
Fixed restart bug. It now actually resets dynamic memory. (Tested in
stationfall, zork.)
Moved a bunch of code around IO.pm. New OutputStream code
now compiles, although it won't run at all yet. Most of the necessary
skeleton is there, but the key is to *gradually* replace
existing stuff with the new stuff. E.g., do select/deselect stuff first, then
deal with actual printing to the streams. I'll need to be hybrid for a while,
with e.g., references to $PlotzPerl::Output::zios etc.
------------------------------ March 12, 2004 ----------------------------------
Slight improvement to main while loop.
Add stuff to PPI::Cleanup. Namely, it calls PP::Input & PP::Output::cleanup.
PPO::Cleanup in turn calls cleanup on each outputstream.
Which will flush buffers and/or close filehandles.
More work on select/deselect/is_selected.
Cleanup seems to work.
Call Output::new (which calls new() on all the streams). Starting the
migration to the new objects!
------------------------------ March 13, 2004 ----------------------------------
Move @output_stream functionality to select/deselect methods for each stream
------------------------------ March 15, 2004 ----------------------------------
Fix a bug in @output_stream. Seems to basically work!
Start moving write_chunk to stream_line subs.
------------------------------ March 16, 2004 ----------------------------------
Moved stream 4 to OutputStream. Seems to work.
Also got rid of the hack where we explicitly echoed certain streams after
reading them in. Instead, we now just set the text to be the right kind of
PP::Text::Input object, and call Output::output in read_command to echo it to
the right place.
We do have the problem now that lines from the file aren't being echoed to the
screen, because PP::OS::Screen::stream is an empty sub. Might as well pull
buffer_zchunk (and all stuff dependent on $buffer) out of ZIO - there's no
reason for ZIO to know that stuff, and it can all be handled by OS::Buffered.
(Prompt_buffer may be a special case, but I've already moved that to PP::IO.)
It's especially worth doing this because some of the ZIO code was calling PP
code already, and this way we'll have less overlap.
Then I can fix OS::Screen::stream.
Then I THINK I can get rid of write_text and most of write_zchar!
Also $zios, $selected_streams.
And move flush/register_newline to PP::OS, which makes PP::Output very small.
Then test and release!
Wrote some tests into restart.inf
------------------------------ March 17, 2004 ----------------------------------
Got rid of Output::write_text and mostly got rid of write_zchar! Everything is
now going through PP::OutputStreams.
Moved buffer_zchunk to OutputStream::Screen.
I should move flush/register_newline to PP::OS.
Then consolidate the various write'ing routines. It seems like there may be
more newline()s than we really need.
We should meld stream, buffer_zchunk, etc.
Fixed recursion bug in transcript's is_selected.
Moved register_newline to OS::Screen. Making some progress.
Even once we move flush, there's still a mess.
Someday in the future, we should try and reduce all the calls
to PlotzPerl::IO::foo, PlotzPerl::Output::foo scattered around the code.
I think it's a question of correctly isolating what happens in the ZIO and
what happens in the game.
We've got a bug where certain newline's aren't being registered, I think,
cuz the test program waits too long to print a [MORE] at one point.
------------------------------ March 20, 2004 ----------------------------------
Trying to consolidate OS::Screen's buffer_zchunk with the stream/stream_line
scheme I had tried to build in OS::Buffered. Realized that the buffer_zchunk
method is smarter:
if buffering {
add_to_buffer the whole string
} else {
just write the string
}
Inside add_to_buffer, I add to the buffer. Then I word wrap the new larger
buffer. IF there are any Z_NEWLINEs in the buffer (either because the game
explicitly wrote them to the buffer or because word_wrap added them) then I
need to flush.
Note that changing screen size, font changes the number of chars needed to
finish the line so I need to call maybe_flush (which flushes only if nec.)
from add_to_buffer as well as any time I change one of the above.
I need to call flush for sure any time I switch windows OR from, e.g.,
filename_prompt or read_command, where I want to make sure the user sees
something so that they can input. (Just call it from get_input instead?)
------------------------------ March 21, 2004 ----------------------------------
More work to consolidate buffer_zchunk & other write()s.
------------------------------ March 22, 2004 ----------------------------------
More work to consolidate buffer_zchunk & other write()s.
Added word wrapping to transcript, but in a hackish way (using prompt_buffer).
Transcript needs to store its own "what I've written since the last newline".
Wait: that's just length(buffer())! But the problem is, if we call flush when
there's no \n, then stuff we print next needs to be in later columns. I think
I may need to use ZIO-like get_position() for any buffered output. Move ZIO's
position stuff to there? Cuz it's not only dependent on the GUI! OTOH, might
that break things? But otherwise we can't really wrap transcript correctly.
Screen is currently wrapping even when it's not supposed to. (See very first
couple prints in Czech.)
Output now looks worse, although guts are slightly better.
------------------------------ March 23, 2004 ----------------------------------
Fix stupid bug in Screen word wrapping.
Hack around problem where dying in cleanup makes other error message disapper.
Concatenate errors. (But maybe I should fix what's making it die in the first
place!)
Cleanup cleanup() and deselect a bit.
Flush buffers before deselecting a stream or quitting. Need to test this!
------------------------------ March 24, 2004 ----------------------------------
Zork basically seems to be working. Problem with 'script' though
(Which I think is also happening with restart.inf).
Note how 'Here' gets printed BEFORE the filename prompt.
This is probably a casualty of the fact that Zork is editing the
transcript bit explicitly, so we don't actually open the file
until we're told to print something! filename_prompt is calling
IO::flush. First of all, flush currently has $transcript->flush
commented out. But even if you uncommented it, you'd have this worse problem
that you're trying to print to an unopened file!
Possible hack to fix it: tie $PlotzMemory[FLAGS2] to a variable
that calls select/deselect when it's edited. This is pretty sexy, actually.
Less sexy hack: Transcript::is_selected will be called from output
the first time someone tries to print anything (unless stream 3 is
selected), because Output::output calls $stream->output foreach $stream (1..4)
That means we have at most one string that's not getting printed. And even
better, we know it's going to happen DURING the print statement, not
sometime afterward. And even better better, $text is passed to output().
SO we can override Transcript::output to pass $text to is_selected,
which is already overridden, and add to is_selected that it should
call $transcript->add_to_buffer that string.
This hack WON'T fix the problem where we're printing the string 'Here'
to the screen before filename_prompt, though. That's caused by
the flush() in filename_prompt.
Hack 3: Instead of checking the transcript bit in is_selected, we could check
is any time that we do output() (no matter which stream's selected).
If we call check_transcript_bit in Output::output BEFORE we loop through
the streams, and call filename prompt from there if we see the bit
was changed, then we're guaranteed that the very first time we try to output()
ANYwhere after modifying the transcript bit, we'll open the file. That is,
system won't even add anything to any stream's buffer, let alone print it out,
before we've done filename_prompt. So there's zero danger that anything will
be missed in the transcript.
This is probably the easiest solution, tho I should keep hack 1 in the
comments and/or in my notes, cuz it's sexy.
(This does make @print a wee bit slower, which hack1 wouldn't. So?)
See update, 3/28 to hack 3.
------------------------------ March 25, 2004 ----------------------------------
Added Title setting. Works for at least win32.
Work on que.pl: now able to write a file exactly the same
as dzip's save file (for a simple file)!
Still not handling stack.
Also not xor'ing with orig memory on read OR write.
Next step: read Z-file and xor, make sure save files still the same.
Then push stack in program, make sure save files still the same.
Then change que.pl subs to PlotzRuntime::Save subs!
------------------------------ March 26, 2004 ----------------------------------
Adding eval stacks to b.inf works fine in que.pl.
Compressing/uncompressing memory seems to work, too.
FC shows no differences between sav files for b.z5 AND story\minizork.z3
when I load the file into que.pl and then write it out again. In THEORY, this
means that my reading and writing are correct; of course, it's also possible
that somewhere in the middle I'm making a symmetrical mistake.
Note that que.pl's middle is essentially the @Save_Stack
and @PlotzMemory::Memory that'll be used in Plotz.
------------------------------ March 28, 2004 ----------------------------------
Re the transcript problem from before, I'm pretty sure I have a hack 4 that's
SUPER simple and will work, even though it's less sexy. All I need to do is
try to write to the transcript BEFORE writing to the other streams! That way,
I check the transcript's is_selected and possibly call (de)select on the
transcript BEFORE I print anything the screen.
Note that filename_prompt will call IO::flush which will call
Transcript->flush (should it?) which will try to flush the transcript before
we've picked a filename for it. I guess if filename_prompt will only be
written to the screen, it SHOULDN'T flush the transcript. Alternatively,
flush() could do nothing if it's called on a non-selected stream. After all,
deselect will flush the buffer and after that you can't write anything to the
buffer until you select it.
AHA! SYSTEM doesn't call newline() when I do <>!! E.g.,
[Filename: c.cmd]:
User hits a newline but the game never register_newline's it!
------------------------------ March 29, 2004 ----------------------------------
No. Looks like above is NOT the problem. register_newline seems to get called
every time I do a newline.
WAIT! It's NOT a bug! Games::Rezrov set things up so that when you do an
@read, the number of lines written is reset to zero. Obviously, the user got
to enter stuff there, so it's OK. Of course, IF we are using MORE prompts for
input_stream(1), then we do NOT want to reset the number there. But that's
part of fixing the larger input_stream(1) bug.
------------------------------ March 30, 2004 ----------------------------------
Finally bit the bullet and searched for matching parens for signed word
conversion.
------------------------------ April 01, 2004 ----------------------------------
Move czech to a separate CVS repository.
Export PlotzPerl::IO functions to avoid output program ugliness.
Also fix bug in signed_word()
------------------------------ April 05, 2004 ----------------------------------
Wrote CallStack.pm
Implemented parts of save/restore!
Copied que.pl and stuff from notes into Callstack.pm.
Modified them as seemed necessary, without trying to run them.
Removed "if $Restoring" stuff from main loop, cuz it seemed simpler, and
shouldn't hurt stuff.
Wrote Callstack::z_call, which implements calling and adds in stack calls.
z_call is working correctly - tn.pl still works.
(This allowed me to simplify some logic in PlotzTranslator, for call*.)
PlotzParse creates a next_pc key for save and call, cuz we need them.
Starting to work on @save.
Still need to call update_stack.
Also need to implement resetting dynamic memory. Right now, we just restore
original memory. Really, we need to restore orig memory for restart, modified
memory for restore. I could call reset_dynamic_memory from CallStack, but make
sure it doesn't get re-overwritten! (And make sure we save the flag2 bits.)
------------------------------ April 16, 2004 ----------------------------------
More work on @save and @restore.
Tried to fix dynamic memory issues.
Moved most of @checksum to PlotzMemory.
Minor fiddling with main loop, moved more stuff to PlotzMemory.
More work on routine_start.
I THINK I've written all the needed code, and just need to bugfix now.
------------------------------ April 17, 2004 ----------------------------------
Fix bugs. We now get identical saves for dzip & plotz with an extremely simple
z5 file.
------------------------------ April 18, 2004 ----------------------------------
Actually got save to output same exact file as Dzip, for a simple output file.
Zork save was restore-able by dzip, although during writing there
were two errors where i had a non-byte passed into "pack 'C'". I think this is
due to variables or stack things that are -1.
------------------------------ April 19, 2004 ----------------------------------
Got rid of silly hex/unhex in the quetzal stuff.
The -1 bug is bigger. Internal variables -- i.e., local variables and the
stack -- are really allowed to be negative or stored in Swahili or whatever I
want, as long as we translate them correctly whenever they get used.
That is, whenever we use them in arithmetic OR store them, we need to
correctly sign OR unsign them.
When I put this into b.inf:
l = 0; k = l-2; @storeb 500 0 k; !-2;
I get the "pack 'C'" error mentioned above. @storeb stores -2 in the byte.
It then tries to pack 'C', -2, which is an error cuz -2 has too many digits.
zork1, start game, open mailbox, save. It writes two -2's with storew.
SO:
@storeb, @storew should be doing &0xff.
global_var shold be doing it (because it writes to memory), and IS doing it.
dec, inc should NOT be doing &0xffff, which they ARE currently doing in
var_to_lval.
Do I need to explicitly significate bitwise things? If I add two numbers to
get a result > 32K and then bitwise or with something, what should I get? No.
The variables will naturally be unsigned as Perl stores them.
What if I have -1 in a variable and do a >>? art_shift or
log_shift may actually change their behavior here if I haven't put the right
things into the variables. Ah. @dec a variable to -1 (current code changes it
to 65535) and then shift it and see that the sign bit is incorrect.
Do we also get errors because the number of bits is wrong? I think so!
If we log_shift -1 3 places, we should get (-1&65535)>>3 = 65535 >> 3 = 8191,
but our current code probably yields -1>>3 which in Perl is 2^32-1 >> 3 which
is 2^29-1 which will be -1 if we print it out.
Call log_shift tests with variable = -9 as well as 65525 or whatever.
Also call inc on -1 and 65535.
The following are safe cuz they only go variable to variable (or stack):
store, ret, ret_popped, pull, push
loadb, loadw are ok cuz they put <65K values INTO variables
The reason I added %64K to var_to_lval is that I had a variable that was
started at 65535, and it was being incremented. This broke global_var,
which tried to get $PlotzMemory::Memory[GLOBAL + 2*$var] which is too
big when $var is 64K. But I can do $var &= 0xffff in there. Just need to
see whether any other problems arise.
Make test cases for these weird negative situations we got.
Note that the n1 constant from test.h (used e.g., in no_inter.h) is
replaced in the code with #ffff. Which means we're testing what the code
does with the constant #ffff very nicely, but we're not testing a
variable like "dec 0 -> nv;" I should really be doing both in jump AND
other kinds of tests, to make sure we handle negative variables
correctly. Also inc -1 and use that in an index for a store or load.
------------------------------ April 20, 2004 ----------------------------------
Apparently forgot to copy all my changes to Work. Had to re-find the problem
of get_orig_dynamic_memory.
My discard bit was upside down, too.
Also, comparing my save file to dzip's, I get differences in the header.
These differences aren't surprising, but I wrote above that I got "same exact
output".
Turns out this was because fc on winMe decides binary files may be the
same even if they have differences, unless you do fc /b!
Hm. The differences now are: flags1 capabilities, interpreter #,
screen size, default colors. I think all those are OK.
------------------------------- May 11, 2004 -----------------------------------
Finished a draft of a test program that uses czech/test.h to test
restore.
Problem with Zork save file. Turns out my save/restore plan doesn't work if
you do @call sp (because stack gets re-popped during restore), or if sp is one
of the args to the subroutine.
------------------------------- May 12, 2004 -----------------------------------
Made a big mess trying to get save/restore working.
climb_stack is now (correctly?) figuring out which sub to call, and
calling z_call from there. (In z_call, we don't push frame stacks if we're
being called from climb_stack.)
------------------------------- May 13, 2004 -----------------------------------
Possibly cleaning up the mess, or making it worse.
Add a main() that generates the dummy stack frame.
save generates a save file (in c.z5) that dzip212 reads and passes c.z5 tests
with. Restore is definitely still screwed.
------------------------------- May 14, 2004 -----------------------------------
Rearranging which subs call which, and what needs to happen in
routine_start.
Got rid of the main() I had added. I was just confusing matters, because in
real life, the first sub you call is main. Make the dummy frame by hand again.
Restore passes test!
------------------------------- May 17, 2004 -----------------------------------
More rearranging of save/restore.
------------------------------- May 18, 2004 -----------------------------------
dxt.pl - read z files without txd! (So far does almost nothing.)
------------------------------- May 19, 2004 -----------------------------------
dxt.pl does a lot more! It actually finds all called subs in c.z5!
Note that it doesn't find subs that never get called. How could it?
Well, in the future if we have gaps in the subs, we'll try to fill them in.
Still, there's nothing really wrong with it - well, unless they get called by
"@call sp" or something, which is of course possible.
But it only finds 66 of the 341 routines in minizork, and 100 or so of
300 for Advent :(
------------------------------- May 20, 2004 -----------------------------------
We can make the algorithm better! See notes.
------------------------------- May 21, 2004 -----------------------------------
Pretty up the output of dxt, compare it with txd. Now I'm able to see
explicitly that we're missing 280 routines from minizork. However, I could
catch a lot of them if I went out on a limb and parsed gaps in my code.
(I still won't find all the routines that are after the last routine I found
unless I parse all the strings and parse the gap from the last sub to the
first string. I STILL won't find the one sub that's before... no wait, I will,
cuz it's called from another sub. But I wouldn't be guaranteed to find it if
it was a computed call.
------------------------------- May 24, 2004 -----------------------------------
Made the algorithm better. Also realized I wasn't finding as many
subs as I could. I now automatically try the sub after finding a 0 byte.
Now we find 133. Still not so many subs, but double what we had!
From Spec 1 Remarks:
---
Inform never compiles any overlap between static and high memory (it places
all data tables in dynamic memory). However, many Infocom games group tables
of static data just above the high memory mark, before routines begin; some,
such as 'Nord 'n' Bert...', interleave static data between routines, so that
static memory actually overlaps code; and a few, such as 'Seastalker' release
15, even contain routines placed below the high memory mark.
---
This means my '0' finding is suspect. However, I think other parts of the
algorithm are still OK. I THINK that after the high memory mark, it would be
USELESS to have any data other than code & strings, so the '0' finding should
still work after static data is over. OTOH, I don't know how to find the top
of static data...
We die on Advent because we run into string territory.
Find the minimum start of strings. This tells me when to stop reading subs.
Doesn't quite work cuz we don't see the min string address until after reading
the last sub.
Changed the die to a warn (will need to reread last sub, eventually) for now.
Anyway, we now find 220 subs in Advent!
And still no subs found incorrectly!
------------------------------- May 25, 2004 -----------------------------------
We find 64/64 routines in tn.z5! And we get their locations right, too!
217/341 in minizork.
I moved code around a bit (breaking things).
Now, when we end a sub with a ret/rfalse, etc. we assume the next byte
starts a sub.
Good news: we now find ALL 341 subroutines in minizork! And every
sub has the correct begin and end address!
Good news 2: I find ALL 387 subs correctly in Advent!
(Last sub is off by two but that's because of padding zeroes and the way I
calculate txd's end.)
Bad news: ALL print_paddr opcodes in minizork print to variables, so I have
NO way of knowing where the strings start! So I find one extra junk sub. Hm.
Same with zork1 (440), with the added problem that I don't find the
very first sub.
------------------------------- May 28, 2004 -----------------------------------
Got rid of main() in dxt.pl by moving stuff in packages into subs (so it
doesn't matter that they're not parsed at compile time in a 'use').
Code still works.
PlotzUtil is now included in dxt.pl, and is only used to hold @Memory and
%Constants. I can put them in main:: (or PlotzParse?) eventually.
-------------------------------- June 18, 2004 --------------------------------
I THINK I've correctly implemented a change to the way restore climbs the call
stack. Instead of calling each Z-code sub, and having that sub call
climb_stack, and going back and forth into & out of Z-code,
I essentially have z_call do this:
if (restoring) {
$stackref = next frame's eval stack (taken from save file's call stack)
@args = next frame's local vars (taken from save file's call stack)
$result = z_call(sub from next frame);
$first_arg = [$result, $next_PC, $stackref]
} else {
create and push a new stack frame
# $first_arg = [], @args = args passed in to z_call
}
next_sub($first_arg, @args)
So z_call recursively calls itself to climb up to the top of the call stack
without ever calling Z-code subs. When it gets to the top, it calls the sub
that called @save, and jumps to the @save, which sets restoring(0). As it
finishes each sub, it pops a layer off (the stack and) the recursive z_call
calls. (As it happens it'll never undo all the recursion, because main()
does z_call(first sub), and that sub will never return; it'll just quit.)
The best part is, I even described this in the POD for z_call.
I think things are at least a bit more simple than they used to be.
restore functionality is really isolated to CallStack.pm, except for the
necessary goto in each Z-code sub.
Note that the old version sort of forced the Z-machine back into the Perl
call stack early in the process. Now, we've got a hybrid stack (part of it
will be implemented with recursion, the top part is with regular z_calls) for
the whole time the game is played (from a restore). Who cares?
-------------------------------- June 21, 2004 --------------------------------
Don't bother recursing to the top frame. Just work our way from the top frame
down the stack. Each time, call the sub in that frame and jump to the right
point in it. (Note that during that sub, we may call other subs, so call stack
needs to be in correct status at the time.) When we get to the end of the sub,
call into the next lower frame's sub, at the point where that sub returned.
This is even more like Z-machine and a bit less like the real Perl stack -
just jump into each sub at a certain place, finish that sub, jump into the sub
one frame lower in the stack etc - but I think it's worth it for the
simplicity.
while (top frame .. bottom frame) {
my $result; # for very top frame, it's undef
get saved frame
args = locvar in frame
stack = stack in frame
$next_PC = &_get_next_PC();
$sub_to_call = get_sub_to_call($next_PC or $restore_PC);
$first_arg = [$next_PC, $result, @$stackref];
@args = @$locref;
$result = &{"main::rtn$sub_address"}($first_arg, @args)
pop saved frame
}
-------------------------------- June 22, 2004 --------------------------------
I finally found an expression that significates a number without using
unpack/pack or using the variable twice. It's "($x+0x8000) % 0x10000 - 0x8000".
It's one or two characters *more* than unpack/pack, though, and doesn't look
much simpler. (It is probably faster, but I have a feeling that's not the
bottleneck in running code - and I have no idea how fast converted programs
run right now anyway.) It works for -64K .. +64K so whether or not the
original number we're starting with is signed or unsigned or even out of
range, we end up with a signed word.
Moved the main code I was putting in the Perl program into a "Z_machine"
sub in Runtime, cuz why not?
Created start_machine, which calls z_call for first sub OR does the
whole restore thing (looping over restored call stack, jumping into the middle
of subs).
-------------------------------- June 23, 2004 --------------------------------
Setting the store_var in resume_execution (the new sub that used to be in
start_machine) means I don't need "R123" labels anymore, except for @save.
-------------------------------- June 24, 2004 --------------------------------
Took this from zork1 translation:
L26092: $stack[@stack] = z_call(26306, \@locv, \@stack, 26098, 0, pop(@stack));
It's the translation of:
65ec: e0 2f 33 61 00 00 call 66c2 sp -> sp
The crazy thing is, I think it's actually doing the right thing, including
passing the correct (i.e., popped) stackref to z_call!
Fixed log_shift, storew, storeb translations to correctly & 0xff(ff).
Still need to add test for it to czech. (Do I need to fix dec? Does it matter
that it does &0xffff?)
-------------------------------- June 25, 2004 --------------------------------
Read that if caller() is called within the DB package, then it sets @DB::args
equal to the arguments given in that frame. That information is enough that we
don't need to build our own call stack. If only I had worked this out two
months ago! Anyway, I was able to write a build_call_stack routine that
identically reproduces @Save_Stack (as far as I can tell).
Need to:
- change resume_execution to use only @Restore_Stack. I believe I can
then get rid of the global @Save_Stack, and create a local @Save_Stack
variable. (@Restore_Stack still needs to stay global so it works
after die()). Note that build_call_stack needs to go on top of any
@Restore_Stack (or push onto @Restore_Stack w/in the sub, whatever).
If I want to get rid of z_call entirely:
- move DEBUG statement to first line of each routine
- create a rtn0 that returns 0
- change regexp in build_call_stack
- call rtns with first arg containing stackref, locref, nextPC, store_var.
But note that the called sub should NOT use those to initialize @locv, @stack
(unlike calls from resume_execution).
- change call_* translation
Might not be worth it: requires complications in Z-subs like
calling 0, no strict refs, what to do with extra args.
Even if I don't get rid of it, I've simplified the system by not storing
the Save_Stack.
-------------------------------- June 27, 2004 --------------------------------
Created directory for Language::Zcode using:
`h2xs -AX --compat_version=5.6.1 -n Language::Zcode`;
Problem: Zcode.pm was created in '.' which means that "use Language::Zcode"
won't work in that dir. I can put it in a dir called 'Language' and say "use
lib .." in plotz.pl. Or put 'Language/Zcode.pm' into the Makefile.pl and
not need any use lib at all!
-------------------------------- June 29, 2004 --------------------------------
Changing directory hierarchy.
Only part of the way through, but I've put all the .pm's in the correct
locations and I'm able to translate a z5 and run the resulting .pl file.
I still need to go through all the pm's and replace all PlotzPerl, PlotzParse,
etc. with the correct stuff. Busy work.
Then I can just port all the pm's to the Language/Zcode directory that
h2xs created.
-------------------------------- June 30, 2004 --------------------------------
Removed leftover cruft from explicit building of the call stack. I think things
are cleaner now. (I'm still using z_call, tho it's much smaller now.)
-------------------------------- July 07, 2004 --------------------------------
Divided Translator.pm into a bunch of files in Translator/.
Things still seem to work.
-------------------------------- July 13, 2004 --------------------------------
Splitting Parser modules into Language::Zcode. I'm pulling them out of
dxt.pl (now renamed tuxedo.pl) because I had updated that code a bit.
tuxedo.pl: supposed to be a more formal version of txd. In the end, since I'm
lazy, it'll probably just be a black-and-white version of txd's full-color
functionality. In the end, the only real benefit of tuxedo is that it's Open
Source. (I was told on the Z-machine mailing list that, while txd's source is
distributed free, in order to copy any of the code, I would need to get
permission from everyone who ever wrote any of the code. All of which happened
prior to 1992.)
-------------------------------- July 14, 2004 --------------------------------
Finished "porting" tuxedo.pl. I'm now able to reproduce the old results, i.e.,
I get Advent.z5 perfect and break on minizork because I never recognize the
strings.
-------------------------------- July 15, 2004 --------------------------------
Created LZ::Parser::Routine, which stores and parses one Z-code routine.
Changed newplotz.pl to use it.
Changed tuxedo.pl to use it (partly), and it works!
-------------------------------- July 16, 2004 --------------------------------
tuxedo.pl is even better. I now look for the address of the last command
in the sub, so txd and Perl are matching perfectly. The last command
address is needed anyway for parsing a routine. (We can't parse up to the
'end' address, because there might be padding zeroes.)
newplotz.pl mostly works.
For totally inexplicable reasons, we're getting a "Substitution loop"
in PlotzTranslator.pm when running newplotz, even though it seems to be doing
exactly what plotz.pl was doing at that point!
-------------------------------- July 19, 2004 --------------------------------
Got rid of the weird Substitution loop bug by just using _SW#...#
instead of _SW(...), which required fancy balanced paren matching.
Did simple profiling with:
perl -d:DProf newplotz.pl
dprofpp (-I sorts by cumulative secs)
Found out that moving all the init stuff from parse_command out of the
sub made things run twice as fast!
Also realized that there's only 7000 commands in Advent, which is 90K.
So there can't be more than 40000 commands even in Curses. Maybe I ought
to save the results of parsing after all. Well, won't do it for now,
but I could. At current speeds, it would shave maybe 20-30% off the total
running speed.
-------------------------------- July 20, 2004 --------------------------------
Useless mini-tweak to z_call.
I can identify orphan fragments! If command we just read was a jump/ret
(sub-ending opcode) and no earlier command jumped past it AND we're not
starting a new sub, then we must be in an orphan fragment!
And don't return a sub at all when I get an error reading that sub. Now
we even get the correct subs for minizork!
-------------------------------- July 21, 2004 --------------------------------
I figured out that it's very easy to mix Perl & Inform in one file.
The Inform preprocessor completely ignores the Perl, and as a bonus,
we can even get all of the Inform code into the Perl!
This will be very convenient for writing L::Z tests.
I can write Inform code to test, then compile the file with inform
(possibly in different Z versions) and read the .z? file with L::Z::Parser!
#Ifdef PERL;
my $Z_version = 5; # could use 3 or 4 or whatever
(my $Inform_file = $0) =~ s/\.t/.z$Z_version/;
my @Inform = grep {/./} <DATA>;
...
#endif;
__DATA__
[ Main i;
i = 1;
@print_num i;
quit;
];
More cleanup work on tuxedo.pl & LZ::Parser::Perl. I'm getting happier
with it.
The bad news: zork1's first sub is never called explicitly. So I'm
going to have to do the "read everything after the dictionary" trick.
Other bad news: zork1 has a 0xb0 in sub 90d4 after a quit, instead
of padding with a friendly 0x0. How can I tell that that b0 is padding
rather than a command?! Actually, I argue that txd is wrong, and it
IS an orphan (rtrue) command!
The last sub is wrong too - it thinks that the start of strings is
an orphan command. It then tries to start a sub after that, but soon breaks.
So the only extra stuff is that one orphan command which, of course,
won't affect anything.
-------------------------------- July 22, 2004 --------------------------------
VERY complicated thinking today. Eventually, I believe I figured out the
Right Way to do the z_call stuff. I put the recursion back in, but now
it's in combination with the @DB::args stuff.
Also, I haven't exactly tested it, or even run it at all. But I'm pretty sure
I've got the algorithm right, at least.
-------------------------------- July 26, 2004 --------------------------------
Clean up and rename Language::Zcode::Runtime::MachineState. It seems to
work!
Split out Language::Zcode::Runtime::Quetzal. Why not?
-------------------------------- July 27, 2004 --------------------------------
Figured out that for read_char on win32, we can open a new
Win32::Console(STD_INPUT_HANDLE) without creating the whole Win32 terminal.
Then we can change the mode so it'll read just a char. See console.pl
Add end_of_dictionary sub to Parser::Perl. With it, I can add to @todo
the first address after the dictionary is over. (I should probably add
it with score MAYBE or something, cuz there's no guarantee it starts
a sub. OTOH, we're positive it's not in the middle of a sub. Still, we
wouldn't want to *think* we're finding real code that ends up overlapping
where a real sub starts. Main point is just to find the few if any
subs before main(), which is not even a concern for Inform.)
Seems to work for zork1, Advent, and minizork, which is all I have right now.
-------------------------------- July 29, 2004 --------------------------------
Continue end_of_dictionary stuff.
We can now plotz Advent, minizork (random data after dictionary), AND zork1!
Fixed bug in ZIO::Win32 that would give warning whenever switching
window focus in/out of the ZIO.
-------------------------------- July 30, 2004 --------------------------------
show_status: handle case where location global var isn't set (e.g., test
program)
Fixed Advent bug: it was a mistranslation of set_cursor.
------------------------------- August 02, 2004 -------------------------------
Counting opcodes using Games::Rezrov.
Played a few turns of Advent.
1 compare_jz 22152
2 compare_je 21823
3 increment 12095
4 jump 11552
5 compare_jg 11106
6 set_variable 10498
7 compare_jl 8642
8 get_property_addr 8199
9 call_func 6359
10 get_word_index 6159
11 loadb 5085
12 rtrue 3968
13 call_proc 3836
14 decrement 3438
15 ret 2731
16 write_zchar 2472
17 store_byte 2317
18 get_child 1984
19 test_attr 1926
20 divide 1873
Total 147711. First ten: 105437
minizork:
1 loadb 5967
2 store_byte 5261
3 compare_jz 4985
4 dec_jl 3640
5 add 3246
6 compare_je 2871
7 get_word_index 2270
8 set_variable 2234
9 store_word 1932
10 multiply 1861
11 call_func 1717
12 inc_jg 1542
13 test_attr 1512
14 jump 1258
15 get_sibling 604
16 get_child 577
17 get_property_addr 568
18 compare_jg 486
19 test_flags 357
20 subtract 337
zork1:
1 compare_jz 3590
2 compare_je 2675
3 set_variable 1712
4 call_func 1450
5 test_attr 1356
6 jump 1179
7 get_word_index 1148
8 add 619
9 store_word 600
10 inc_jg 558
11 loadb 496
12 get_sibling 429
13 get_child 394
14 get_property_addr 370
15 compare_jg 332
16 test_flags 273
17 get_parent 266
18 get_property 257
19 compare_jl 240
20 routine_push 239
Total 19389. First ten: 13963
ballerina.z8:
1 jump 48624
2 compare_jg 46201
3 compare_je 42785
4 compare_jz 41683
5 increment 40224
6 set_variable 38142
7 compare_jl 25840
8 jin 20642
9 get_word_index 18918
10 get_property_addr 18087
11 call_func 14929
12 loadb 8574
13 store_word 7941
14 subtract 7737
15 routine_push 7408
16 add 7388
17 rtrue 5147
18 divide 5005
19 ret 4621
20 store_byte 4340
21 bitwise_and 4277
Total 451702. First ten: 341146
In all four cases, nearly all the first 10 opcodes -- which make up 70-75% of
the opcodes -- are translated into pure Perl, hence should be pretty fast.
(And many of the next ten, too, depending on the game.) Sub calls will have
some overhead due to z_call, although not TOO much since I'm now using Perl's
(presumably optimized) call stack instead of making my own. rezrov also lots
tons of time (tho was able to deal with changing Z-files!) by re-parsing every
opcode every one of those thousands of times.
The upshot: I really don't think optimization is going to be necessary!
20K lines of Perl including a bunch of subs can run pretty darn fast.
(And jumps are probably pretty fast even if they're deprecated.)
And here's a few turns of a translated Advent.pl:
%Time ExclSec CumulS #Calls sec/call Csec/c Name
28.9 1.263 1.209 15562 0.0001 0.0001 Language::Zcode::Runtime::Opcodes:
:thing_location
16.0 0.702 0.574 44205 0.0000 0.0000 Language::Zcode::Runtime::Opcodes:
:global_var
8.27 0.361 36.721 9543 0.0000 0.0038 Language::Zcode::Runtime::MachineS
tate::z_call
5.75 0.251 0.251 1 0.2510 0.2510 PlotzMemory::read_memory
5.73 0.250 0.220 8641 0.0000 0.0000 Language::Zcode::Runtime::Opcodes:
:get_prop_info
5.73 0.250 1.013 7293 0.0000 0.0001 Language::Zcode::Runtime::Opcodes:
:get_prop_addr
4.13 0.180 0.141 11034 0.0000 0.0000 PlotzPerl::Output::get_stream
4.10 0.179 0.760 2621 0.0001 0.0003 PlotzPerl::Output::output
3.92 0.171 1.518 1304 0.0001 0.0012 main::rtn30520
So if we do need to optimize, it's not for the regular opcodes. Of course I'd
want to run for more turns to make sure, but this looks pretty good.
OTOH, ballerina.pl wouldn't even run. Is it cuz it's five meg, or some other
problem? (Well, it runs but immediately dies. It won't run under Dprof.)
------------------------------- August 03, 2004 -------------------------------
Consolidated home & work versions of IO.pm. We're now almost totally consistent!
Split out Text.pm and put in POD. (Eventually could go from LZ::Runtime::Text
to just LZ::Text. Put decode_text in there, to be called by Parser AND
Runtime.)
Tracked down 2 sneaky bugs.
First, z_call will break during restores, because it calls subs with extra
args. (It changes @args to set @locv.) This will break check_arg_count (which
just tests @_). I need to move @locv back into $frame_info and set @locv more
carefully.
Second, I need to add "unsigned_word" (aka & 0xffff) to a bunch of locations.
The one that broke ballerina was loadb, which was trying to load a number
> 32K, so it tried loading a negative offset and broke.
But there's a whole bunch of places that could theoretically be negative (like
the memory locations you call read, get_cursor, and print_*addr with).
It might be that I should fix this by putting a + or a - next to the
args in Parser::Opcode, and add the SW/UW there instead of cluttering
all the translations in Translator. (Also, that way we'll have the info for
all the other languages.)
------------------------------- August 04, 2004 -------------------------------
Fixed the save bug, and created a test for it.
------------------------------- August 10, 2004 -------------------------------
Change cvs directories. Moved LZ to a lib directory, moved scripts to
scripts/.
Changed MachineState.pm to State.pm
Language::Zcode -> Games::Zmachine, based on discussions on
module-authors@perl.org
------------------------------- August 11, 2004 -------------------------------
Got PIR translation working! Admittedly, only for a few ops, and we're
totally ignoring globals, signed word conversion, and a zillion other things,
but it DOES create a .imc file that compiles and runs!
------------------------------- August 12, 2004 -------------------------------
Now able to:
- create Inform file from Perl ("perl t\20_parser.t Inform")
- compile it ("inf +code_path=t t\p20_arser.t")
- run test ("perl t\p20_arser.t"
or "perl -MTest::Harness -e "runtests('t\parser.t')")
or "make test"!
parser.t tests a couple constants and that a couple commands get the right
values into the right slots.
------------------------------- August 13, 2004 -------------------------------
Start on trans_perl.t, which tests that command hashes like those coming out
of a Parser get translated into the right Perl.
Work on perl_test.h, which changes asserts etc. to output "(not )?ok \d+"
that Test::Harness will like!
------------------------------- August 14, 2004 -------------------------------
Add Makefile.PL, MANIFEST to CVS. We're now MakeMaker-compatible!
------------------------------- August 17, 2004 -------------------------------
Got Test::Harness to play nice with a plotz'd file!
Issues:
- need to set rows/columns in IO.pm explicitly. (It's possible to run
tests with parameters passed, but complicated. Need to set
Test::Harness:switches. And of course I only want it to happen for this
one test. How hard is it to change what'll be in the Makefile?)
- If rows/columns is 24 by default, we'll get lots of [MORE]. Even if
we pipe a 'perl -e "print qq{\n}"' to the call to runtests, the test
during which it gets to the [MORE] fails. Can I track this down? And/or,
can I set rows to 255? Which - if I add the code to IO.pm - will never
[MORE]ify. (Note that in that case I also need to say don't bother doing
the [MORE] printing test, since it'll fail.)
There's still some potential issues with rows/columns and [MORE], but
we're able to run 193 successful tests in a .inf file that -> .z -> .pl!
Added sub, mul, print_char, new_line to PIR.
------------------------------- August 23, 2004 -------------------------------
Add parameter passing to PIR. call* at least sort of work.
Change local vars to "local1" etc., just like Inform names them.
Add signed word support.
Change Games::Zmachine BACK to Language::Zcode, based on yet more discussions
on module-authors@perl.org. I was right all along!
Did a diff with all files from work (from 8/12), so we're pretty much in synch
now.
make test works!
------------------------------- August 24, 2004 -------------------------------
-t option so we can choose "dumb" or "win32".
-r 255 doesn't print any [MORE] prompts
Make _ZMemory a global variable. (Still reading it from file, tho.)
Added store, load: they're easy if you ignore indirect variables!
Added storeb, loadb, storew, loadw.
All of this needs further testing, of course.
------------------------------- August 25, 2004 -------------------------------
Write new 40_trans_perl.t which basically does plotz. Didn't write its tests,
though.
80_all.t runs the translated file with the correct inputs (-r 255 -t dumb).
Works!
80_all.t could read output from 40_trans_perl.t.
Mostly untested push, pop, pull, and print.
(Print cheats right now & prints the decoded string)
------------------------------- August 27, 2004 -------------------------------
_read_memory now correctly (?) reads in memory from the generated PIR file,
using _add_mem in main.pir.
- I rolled my own 'hex' function
- I call it with a string of 16 space-separated numbers and parse those
16 nums out, setting an array value for each one, then re-set
global "_Z_Memory" = mem.
I'm not sure that's the best way to do it. Ask on the list.
------------------------------ September 01, 2004 -----------------------------
I think I came up with a significantly better way of handling substition of
global vars / sp into the commands as I translate them to PIR.
Seems to work for a bunch of cases, but I need to test a bunch
more, like call foo(g01, g01, g01, sp, sp, sp)
------------------------------ September 02, 2004 ------------------------------
Terribly ugly hack to allow running on non-Win32 systems. eval the entire
GR::ZIO_win32 module. The eval will die for non-win32 systems, which don't
find "use Win32", but that's fine, unless you try to do -t win32, which SHOULD
break. I really need to fix this more nicely someday (AFTER 0.8).
------------------------------ September 03, 2004 ------------------------------
20_parser.t:
loadb/w, storeb/w, inc(_chk), dec(_chk)
ret, ret_popped, push, print_char, piracy
Add objects from test.h so we can test object opcodes. Add V5PLUS
definition.
pull, verify, test, check_arg_count, buffer_mode, input_stream, output_stream
------------------------------ September 04, 2004 ------------------------------
print, print_ret, print_addr, print_paddr
(Added nop to Translator::Perl!)
------------------------------ September 07, 2004 ------------------------------
Got 40/80 tests working. 40 translates, 80 runs it.
Added object, property, attr opcodes to 20_parser
------------------------------ September 08, 2004 ------------------------------
Language::Zcode can't read a Windows command input file (input stream
1) when it's running on Unix. It gets a \r\n instead of a \n. (Is the
answer as simple as s/\r\n/\n/ before chomping in z_read?) Note that
reading a Unix input file on Windows SEEMS to have worked. Make sure
read_char works too!
Kludged around the bugs. I can now run tests successfully on Solaris.
Finish (first pass) all (supported) opcodes in 20_parser.t!
That means we're done with simple tests!
From Spec1.1: Section 7.1.2 is incorrect - output streams 3 and 4 are
present only in Version 5 and later.
That means I'm now fully implementing v3!!! (Except maybe a few ZSCII
characters, and sound, of course.)
------------------------------ September 09, 2004 ------------------------------
Adding POD et al.
hello.inf (Hello, World!) for an example file.
------------------------------ September 10, 2004 ------------------------------
Created languages/Perl|XML|PIR with some sample code & PIR/XML library files.
------------------------------ September 12, 2004 ------------------------------
Working on Infidel, which appears NOT to have random elements.
------------------------------ September 13, 2004 ------------------------------
Able to do perl -Ilib story\INFIDEL.pl -r 255 < story\infidel.sol and
win the game!
Bugfix in checksum because Infocom games have non-zero padding after
the official end of the file.
Major updates to zcode.xsl (and a couple supporting updates to
Translator::XML). We now have a bunch of opcodes for which we get
output that looks sort of like Inform!
vim: tw=78