SCF/CCF flags - new discovery
I can't believe that no one is interested in discussing the additional, previously undocumented internal state of the Z80 CPU I have discovered recently. I have posted the details in this thread so far - perhaps its title just made it to slip your radar...
Patrik
Patrik
Post edited by Patrik Rak on
Thanked by 1ZjoyKiLer
Comments
Bytes:Chuntey - Spectrum tech blog.
The tests made so far make me believe that the Z80 CPU has an internal register, which I called Q for now, where it assembles the new content of the F register, before moving it back to F. When the instruction doesn't modify the flags, this register is reset instead (or in case of NEC CPU's set to some partially undefined values, as Woody experienced before).
Normally, you never see the content of this register. But it leaks the info through in case of the the SCF/CCF instructions, which seem to set the content of the 5+3 flags of F to ((Q^F)|A).
In other words, the content of A is copied to flags 5+3 after SCF/CCF if the previous operation did set the flags, whereas it is ORed in there if it didn't set the flags. That explains why the previous studies regarding this never got it right, because they assumed it depends only on the state of the known flags and registers.
Of course, it raises the question if this hypothetical register affects only those two instructions, or if it might actually affect some other which we took for granted for years...
Patrik
Have you checked these relevant sources?
The Undocumented Z80 Documented (by Sean Young)
MEMPTR, esoteric register of the ZiLOG Z80 CPU
Programming the Z80 (by Rodnay Zaks)
Also there is this page, which shows 2 copies of WZ (along with BC/DE/HL, so swapped with EXX instruction).
Z80 CPU architecture
So this WZ might exist as a double register pair, or just a single one that's shared between whatever BC/DE/HL set you have selected currently. :-? I have no idea either way...
For emulation purposes, there's this "instruction exerciser" program called ZEXALL (along with ZEXDOC), which is pretty much king to decide how accurate Z80 emulation is. If a Z80 emulator passes this 100%, AND you can still find behaviour different from a real Z80, you'll be very, very hard pressed to find any software that would stumble over that. Even in the ZX Spectrum's huge library. :)
So go ahead, but improving emulation accuracy beyond ZEXALL (if that is possible at all !) would be hardly useful for emulation purposes. And might even show differences between real Z80 chips. Perhaps that's why interest in this subject seems so limited... :wink: (and mostly a waste of time IMHO - I tend to regard information like the above mostly useful as a tool to improve ZEXALL test results in case <100%)
Thanks for the info, but we are well beyond MEMPTR now, which is documented in detail here. The z80tests from the other thread surpass ZEXALL in this regard and test MEMPTR thoroughly.
What I write about here is new info (although Woody was quite close, too), about as subtle as MEMPTR was. But many emulators have implemented MEMPTR now nevertheless...
Of course, I agree that this is well beyond being practically useful, like MEMPTR emulation is. This was more like a call to other emulator authors...
Patrik
I'm following this thread, among other similar threads discussing the very internal details of the Z80, with great attention because this information helps understanding how the Z80 is behaving internally. This is very usefull to write accurate hardware descriptions of the chip (the T80 core is accurate, but not enough, as the I/O cycle signals doesn't behave the same as in a real Z80).
Things like the behaviour of MEMPTR have allowed me to see that the [+/-1] combinational module present in the middle of the 16-bit path, inside the Z80, is uded by MEMPTR (e.g. the WZ register) too. This new discover about SCF/CCF (must read carefully) may allow me better understand how flags updates are transferred to the F register.
Knowing all the details of the datapath of the Z80 is usefull to write a hardware description that does behave like the chip itself: a microprogrammed machine, instead of the usual description method, considering each instruction on its own. A microprogrammed-based description is surely much more compact and can be used on smaller (=cheaper) configurable devices.
So, thanks for the info :)
Patrik
Correcting myself earlier: dug up this old thread, which suggests there's only one WZ, not a pair of 'm (exchanged with either EX AF,AF' or EXX instructions). Which makes sense: the last link in my previous post mentions the Zaks book as reference, but I couldn't find that exact image in said book. :oops:
Also Intel 8080A datasheet has a block diagram which has similarities to those in Zaks book. Maybe not useful for understanding what exactly the Z80 does, but perhaps helpful in figuring out how things might work. And write Z80 tests / emulator code to compare emulator with behavior of real Z80 chips (like that "z80tests.tap" posted in the other thread).
Agree, a hardware description represents internal construction, and is usually the most compact way to fully describe observed behavior. Emulator writers may feel uncomfortable using that as a starting point, but it should be straightforward to write an emulator using that as reference. The other way around (emulator code -> a hardware model) doesn't work well, if at all.
So indeed if all this helps to understand Z80 internals, that's useful even if no application software depends on the exact details. Personally I'm not so sure there's another internal register (or just a flip-flop?) here. It might just be an internal state that's left somewhere as result of executing a previous instruction (like in the case of instruction or IX/IY prefixes). Or an incomplete understanding of how this WZ (MEMPTR) register is used. :???:
I have a Mostek Z80 (I thought I had two, but either I lost the other one or I was imagining it) which seems quite a rare manufacturer for the chip. The Mostek Z80 shows some different behaviour too, for example programs written with the SP1 library have a tendency to crash; I suspect a difference in how the undocumented instructions behave.
It'll be interesting to see if anything changed with the CMOS versions of the chip. The undocumented instructions all work (I suspect by the time CMOS variants came out, so many people were using the undocumented instructions that they couldn't really change it).
The only way of getting truly to the bottom of this is a chip de-encapsulation exercise on various makes of Z80. I heard the Visual 6502 team had de-encapsulated a Z80, I wonder if they have done so for just the Zilog model or others too...
Well, if we are lucky, it will behave in such a way that single bit is enough to represent this state. OTOH, when you consider how F register works, it would make sense if there was some place to assemble new F, and then just store it to F. The way how F is paired with A for PUSH/POP AF and EX AF,AF' furthermore makes F seem like an ordinary register, so having another place for flag assembly makes sense.
Anyway, I went on and updated the ZEXALL code with few more test vectors, to test SCF and CCF alone as well as SCF+CCF and CCF+SCF. I have also taken the liberty of updating the CRCs to really match the real Spectrum values, something J.G.Harston never got too. I also moved the most interesting tests to the beginning, and initialized I to 255, just to be sure. Otherwise I didn't change anything, even if it calls for cleanup and optimization. You can find this ZEXALL2 here. The first 10 or so tests passed on the real 48k Spectrum I have available so far, and I am sure the rest will too once it finishes, as it is the old stuff. Of course, two of the first four fail in every emulator I tried, which was expected.
Now I plan to create two other versions, one which executes CCF before each tested instruction and the other one which executes CCF after. That should really put it to test. It's already getting late, though, so it will have to wait for tomorrow.
Patrik
Doesn't the ALU on the Z80 have a TMP register, could this be used to assemble the new F?
If we add up the bytes for the all the known registers doesn't it come to an odd number like 29 or something, which I would guess would leave three bytes (32) that we don't know about?
As for this 'Q' register, if it exists at all, chances are it's simply an existing part of timing / control logic / a temporary buffer inside the Z80 that's always been there. Just not programmer-visible, or "named" if you will.
My impression is that the predominant CPU people mention finding in Speccies are NEC ones?
Meanwhile, I got frustrated by how inefficiently is the original written, and decided to come up with my own implementation. Stay tuned.
Patrik
Somewhere I read not long ago, that while the CPU is performing internal operations (i.e. it's not using the external buses), the data bus outputs whatever it's hapenning in the internal 8-bit Z80 bus
Gah. Now I'm going to have to get out the high speed pod for my logic analyzer again to test this...
The conclusion is that the CCF being executed before other instruction doesn't affect any other instrcution except CCF and SCF. So that's good.
Meantime, I have finished writing my own ZEXALL replacement, now I just need to fill in all the test vectors. I was hoping that I could at least reuse those from the old one, but as I already found several errors in them, I will be rather creating my own. BTW, I have also noticed that the original doesn't compute a proper CRC either - the CRC table is stored in big endian order but used as if it was little endian, so the result is merely a hash, rather than proper polynomial residue.
Patrik
Damn those blasted big-endian machines!
The test vectors took quite some time to create, but now it's finally finished. The test now features about 150 test vectors and thoroughly tests almost all Z80 instructions The good news is that it runs in a fraction of time compared to ZEXALL, while using even more specific tests at the same time. I'll be releasing it in few days, after I add bit of documentation and wrap the whole package.
I have also prepared a variant which ran CCF before each tested instruction and another variant which ran CCF after each tested instruction. Both were run on real Spectrum as well as various emulators.
As I have reported before, the pre-CCF variant indicates that no other instruction except CCF and SCF is affected by the Q register, which is good. It means that single bit is enough to keep track of its state, which simplifies things.
The post-CCF variant proved what I have stated before - if an instruction modifies the flags, the immediately following SCF/CCF does move of bits 5+3 from A to F, whereas if an instruction doesn't modify the flags, the SCF/CCF does OR of bits 5+3 from A to F.
Finally, I also explicitly tested how an interrupt might affect the Q register. And it turns out that an interrupt clears the Q register as well, so regardless of the instruction interrupted, the SCF/CCF executed as first instruction of the interrupt handler always does OR, rather than move. (At least for IM 2 mode. I believe that IM 0 does the same thing, as does standard IM 1 case, but I can only speculate what happens if weird instruction is on the bus in case of IM 1.)
In either case, I think this is enough information for anyone willing to update their Z80 emulation core to match the real CPU behavior.
Patrik
Patrik
Just for the record, the release was just announced here.
Patrik
BTW, to make it explicitly clear - by modifying flags I mean updating the individual flag bits as part of the instruction operation. The EX AF,AF' and POP AF instructions, which technically change the value of F as well, are not considered modifying the flags in this regard, so the following SCF/CCF does OR.
Of course, the Z80 CCF test I posted would detect incorrect implementation, but I thought I'll mention it explicitly so the available info is as clear as possible.
Patrik
BTW, for anyone interested, Jon Needle has included this in his SZX specs back in 2012, even though he has not put the new version online, as he has more things to incorporate yet. But you can take this flag for granted.
Patrik
Patrik
I've done my best to add the new information. I've kept the part about NEC clones behaving differently as we still don't have enough information to go on, there. I've amended the mention of Ian Greenway's investigation to place bounds on the certainty of his findings, although I think it's good to leave it in there as it seems a nice to document the history of this investigation (even if I am somewhat biased in that opinion).
I've also added a link to ZEXALL2.
The (Q ^ F) | A logic seems odd to me but I guess there'll be an explanation for this!
I'm looking forward to the day we get a 'Visual Z80', and our understanding will finally be complete. Even if there are aspects of some sources of Z80 that are temperature dependent, at least we can then agree not to worry too much about the specifics of those Z80s!
Since this has reminded me, I've also updated the Wiki page with the little known fact that OUT (C),0 actually behaves as OUT (C),255 when using a CMOS Z80. It's probably worth updating assemblers and disassemblers to warn about the instruction. I imagine the last thing anyone demo coder would want would be to see this manifest itself as an undesired white background when presenting at a compo!
http://fuse-emulator.sourceforge.net/
Thanks, zub. The Q register was just a broader concept along the way of analyzing this. While it conceptually exists (it is the place where the new flags are assembled before they are moved to the F register), we have no evidence how it works at the gate level. And it is not necessary for explaining the end effect, so I have taken the opportunity and simplified the explanation to the bare minimum.
The ZEXALL2 was actually superceded by my own rewrite. However, not being an editor, I could not add new links. Would you mind adding the following text after the ZEXALL2 sentence?
Thanks.
Cool, I didn't know that. You learn something new every day! :)
Patrik
I did realise this, but you're right, it's simpler the way you've phrased it. I doubt the XOR operation would have any basis in the Z80's actual logic in this regard, so the Q register seems very unlikely to exist in hardware, even if it is a helpful concept.
For Fuse, if/when we change the flags implementation, we may well model this as a 'Q' register as it will be cheaper to perform an XOR with 'Q' than to add an extra branch to each instruction that updates the flags register, but that's an implementation detail of the emulator and I realise this has nothing to do with how the hardware works internally. In actual fact, we'll probably model our Q register in such a way that Q takes the value 0 if flags change, but takes the value F & ((1 << 3) | (1 << 5)) when flags are left unchanged.
There's the open question about the behaviour following a NONI instruction as described in (http://www.z80.info/decoding.htmDecoding Z80 Opcodes (an invalid instruction with an ED-prefix). I'd also like to be sure that this affects all instructions that don't modify flags, not just documented instructions, and that a valid NOP is also included in this, along with NOP with redundant ED prefixes, the IM instruction. Behaviour of POP AF also seems of interest, as this might be argued as being an instruction that doesn't so much modify flags as just restore them from the stack.
Okay, I have done, although I've removed mention of the old program entirely as I think it just confuses matters.
Glad to have helped. Imagine this will be appearing in a whole load of emulators any time soon! :-) I will at least be adding it to add it to Fuse at some point.
http://fuse-emulator.sourceforge.net/
The test vectors in my tester cover all these. Except all those ED-NOPs, I just couldn't be bothered to cherry pick them all. :)
And POP AF and EX AF,AF' are additionally mentioned few posts above.
Thanks. BTW, in the referral URL, you may perhaps want to point to this post of this thread rather than the Winston's reply.
Well, I will stick with the OUT 0 behavior, but it's good to know.
Patrik
Apologies, I missed that. TBH, I found it a little hard to follow with the information being split up into separate postings, although I guess that's the point of updating the Wiki! :-)
I assume then that the ED-NOP(s) that you did test, IM instructions, etc. add did not count as flag updating instructions. In that case, this seems simple enough to implement. I guess at the start of each M1 cycle, I will save the previous 'flags updated' state and then reset it for the new instruction... then just search for assignments to F and set 'flags updated' whereever they occur, and adjust CCF/SCF accordingly.
MEMPTR is another thing I'll need to implement, but hopefully this shouldn't be much of a pain.
I guess most Spectrums have the NEC Z80s, where the SCF/CCF behaviour isn't so clear, and MEMPTR behaves differently, but it seems worthwhile to emulate a model of Z80 accurately, even if it's a less common one, as this will provide the best possible compatibility with other emulators for RZX recordings.
Thanks, have done!
It's certainly something to be careful of. BTW, I appreciate your reordering of the sections ? it hadn't quite occured to me that the behaviour of OUT 0 was a major variation, whereas the behaviour of flag bits that nobody understood until recently and therefore nobody has relied upon is less of a direct concern!
I can see that to a lot of users, a CMOS Z80 option would be yet another confusing option with no clear purpose. Still, Fuse aims for completeness, at least in relation to the original Sinclair machines, so I will make an effort to the option added.
I guess the main thing is to enourage authors of assemblers and debuggers to include warnings about this instruction. I certainly plan to modify the Z80 patches for GDB and ask in fuse-devel about this too, but it's not clear to me how best to display the instruction in disassemblies. Any thoughts?
http://fuse-emulator.sourceforge.net/
I agree that both of these things are a bit esoteric, however, there are situations where out (c),0 is an absolutely priceless instruction and emulation of its correct behaviour is a must, I believe. And given that the described bug is the only way to reliably detect CMOS Z80 on a 48K spectrum, I would definitely LOVE to see added support for both of these quirks in good emulators.