ZX-Assembler file handling concept

edited September 2014 in Emulators
clausjahn wrote: »
Dear readers,

At this time, it's a good time to ask you, the future users, how to implement the file usage. Let's suggest some cases; none of the following suggestions is implemented now:

Case 1: the user opens and edits an ASM file:
=============================================
Doing this, ZX-Assembler can be used as a pure editor. Compiling the code will check the syntax, determine the compiled code (bytes), but nothing special will happen.
If some output commands are present, they would produce output files, e.g. .DAT or .BIN files. Also, TAP or TZX is imaginable here.
On the other hand, if no output file has been defined, ZX-Assembler could produce a so-called 'list of datablocks', as you know from my other programs, e.g. ZX-Blockeditor. Of course with cut-and-paste abilities.
This would be a good decision if you want to run the code with another program (e.g. ZX-Blockeditor).

Saving an ASM file would put the data from my cell editor back into a formatted file, with labels just at the beginning, followed by the important column data, comments and (not yet) implemented repeaters.

Case 2: the user opens and edits a SNA file:
============================================
Doing this, ZX-Assembler would open any byte-based datablock, including SCREEN$ blocks, and all that. As the SNA reader now recognizes the BASIC program and variables area, as well as RAMTOP address, the memory data can be split-down to several sub-blocks, as in the following example (taken from ZX-Blockeditor):

zxassembler21.jpg

When opening such a datablock, ZX-Assembler will import the code, produces auto-labels, with names like sub_... for called labels or Lbl_... for normal labels, and - which cannot be prevented - maybe lot of senseless code (occurs on data sections or strings), like that:

zxassembler22.jpg

Editing such files produces a common problem: When the user edits the file (which is no problem), e.g. he adds new labels, changes sections or inserts macro and all that meta-code-stuff, where should I store the changed source code?
The edited SNA could take the binary compiled code of course, but the changed source file is lost.
On the other hand, in such cases of changed SNA file, I could generate an ASM file containing the complete changed source code.
Opening the SNA again then should check whether a matching ASM file is present in the file folder.
That's a real problem. It would be better solution, not being able to saving back a SNA file, but rather saving the file to a new file format, e.g. .ASMPRJ, containing all that file data (here: the modified SNA data as well as the ASM source code).

Case 3: the user opens and edits a tape or disk file:
=====================================================
ZX-Assembler is now already able to open any multi-block file, e.g TAP, TZX, DSK and so on.
If you read one datablock, it is red into ZX-Assembler's editor (pretty much as seen in case 2).
Editing a datablock could be theoretically be done without problems, as a shorter compiled result could replace the datablock value easily.
As in case 2, I have the same problem: where to place the modified ASM source code?
For this file type the user could have changed many of the datablocks, resulting in probably numerous changed ASM source codes.

The modified ASM source files could be stored as well inside the previous mentioned .ASMPRJ project files that holds the TAP (or whatever) file together with those ASM files.

I hope you as the future users could help me to find the best way to solve the case 2 and 3 problems.

Has someone ever thought about my suggestions for ZX-Assembler's file handling?
I must generate a good concept now, there are some ideas in my brain but really nothing that can be called 'ready to go'. I'm now short before a 0.5 beta release but would be nice for the user to have much more than a theoretic output.
Post edited by clausjahn on

There are no problems, only solutions (K. Flynn)

Visit my ZX-Modules homepage with lot of free programs!
Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
«1

Comments

  • edited September 2014
    ****! I wrote a detailed answer that WoS lost. I'm not writing all that again. :(

    Anyone subscribing to this thread should have received a copy of my post by email. Could someone please copy/paste it here?
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • edited September 2014
    I never noticed that posts are disappearing earlier in the year, until aowen didnt mention it.
    In this case, this becomes a serious problem.
  • edited September 2014
    ****! I wrote a detailed answer that WoS lost. I'm not writing all that again. :(

    Anyone subscribing to this thread should have received a copy of my post by email. Could someone please copy/paste it here?

    Aaargh! If I ever had copied your reply this morning before I went to work...I wanted to reply to it this afternoon, but - as you said - it vanished. :-(
    I've red it quickly this morning but unfortunately I did not receive an email notification, so I cannot restore it.

    From my brain (about 80% accurate) and very short:

    You told me that you would not use ZX-Assembler for program development, nor for game analysis. You told me the need of SkoolKit support to make the work easier for you.

    I wanted to reply to you that I cannot use SkoolKit on my Windows machine, because I cannot run *.py files.
    Even when I yesterday took a closer look to SkoolKit 4.1 and to its example rresults, I saw that this tool does a wonderful job. I assume that the comments of the sectioned program and data sections do not come from SkoolKit itself (I took a look at the SkoolDaze example), but even so, it seems to be really powerful. I must agree - I see no need for you to use ZX-Assembler.

    Even though the time may be too early to decide whether ZX-Assembler is useful for some purpose or not, there is no need to somebody to use my programs at all. I never intended to generate this 'one-for-all dinosaur tool'.
    I really know that there may be better and more powerful tools around, so I recommend always: Use the tools you want and need - as we all do.

    It'n not only because I definitely cannot realize everything.

    If I could (can I with Windows?) run SkoolKit I would be very happy to implement some of its capabilities, but if not, why not continuing using SkoolKit?

    I would say that ZX-Assembler will be good as a code editor for ZX-Blockeditor and ZX-Paintbrush (for editing the code blocks rather than byte-by byte editing in the block editor), in the way you already can edit BASIC programs and screens. And it will be good for learning Z80, as well as for developing small or bigger projects. Finally, users of SjASM or SjASMPlus could find it useful.

    Or not...

    But my question here was not the question how useful it will be, but to find a good and usable way of file storage.

    I think I made a good progress yesterday: I added a file dropbox toolwindow with which you can import files from disk and back to disk. E.g. the INCBIN command looks into that dropbox for a given file name, loads it and uses the bytes and includes them into the code position.
    OUTPUT command will store its output file(s) to it. Looks like a good solution.

    I thought about a new file extension for edited files:
    ASM --> ASM.ZXPRJ or
    TAP --> TAP.ZXPRJ

    and so on could be a way to save all edited information that comes across when editing a file and generating the above mentioned meta data to prevent of a loss of it.

    There are no problems, only solutions (K. Flynn)

    Visit my ZX-Modules homepage with lot of free programs!
    Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
  • edited September 2014
    clausjahn wrote: »
    I wanted to reply to you that I cannot use SkoolKit on my Windows machine, because I cannot run *.py files.

    Even when I yesterday took a closer look to SkoolKit 4.1 and to its example rresults, I saw that this tool does a wonderful job. I assume that the comments of the sectioned program and data sections do not come from SkoolKit itself (I took a look at the SkoolDaze example), but even so, it seems to be really powerful. I must agree - I see no need for you to use ZX-Assembler.
    To use SkoolKit on any OS, you need Python. You can get it for Windows or Mac OS X here. I recommend version 2.7.x, but 3.4.x should work too. (I've just added a 'Requirements' section to the README on GitHub to make this clear.)

    If I remember correctly, Einar said it would be good if ZX-Assembler could be used as an IDE for SkoolKit, and I agree - though, of course, I may be biased. :) SkoolKit can produce disassemblies, but it doesn't provide any sort of environment to assist with the reverse engineering process (other than its own ASM/HTML output). You are left to your own devices (and your own favourite text editor) for that. There might eventually be some overlap between ZX-Assembler's and SkoolKit's features, but in any case, being able to import a SkoolKit source file (and export to one) would be a handy feature.

    Anyway, this doesn't really have anything to do with your question about file handling, so let me derail this topic no further.
    SkoolKit - disassemble a game today
    Pyskool - a remake of Skool Daze and Back to Skool
  • edited September 2014
    clausjahn wrote: »
    Aaargh! If I ever had copied your reply this morning before I went to work...I wanted to reply to it this afternoon, but - as you said - it vanished. :-(
    I've red it quickly this morning but unfortunately I did not receive an email notification, so I cannot restore it.

    Thanks for trying anyway :)
    clausjahn wrote: »
    You told me that you would not use ZX-Assembler for program development, nor for game analysis. You told me the need of SkoolKit support to make the work easier for you.

    Just to make it clear, my point wasn't really about my personal needs.

    I was arguing that ZX-Assembler can be a good tool to develop new code or modify an existing snapshot, but those are tasks that existing tools already perform just fine. Yet another tool for this purpose is welcome, but not really necessary. I may even use it myself to do it, but it won't make much difference.

    On the other hand, if ZX-Assembler could work as a user-friendly IDE for SkoolKit, that's something we really need, and would help us a lot. The idea is loading a snapshot into ZX-Assembler, then optionally loading code execution maps from emulators (so it will know how to interpret each block area), then manually refining block definitions, renaming labels, adding comments, etc. ZX-Assembler could directly use SkoolKit file format internally, or provide an option to import/export from/to SkoolKit.

    I'm not suggesting SkoolKit support to be provided instead of the ability to assemble files and modify snapshots, but in addition to it...
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • edited September 2014
    SkoolKid wrote: »
    If I remember correctly, Einar said it would be good if ZX-Assembler could be used as an IDE for SkoolKit, and I agree - though, of course, I may be biased. :) SkoolKit can produce disassemblies, but it doesn't provide any sort of environment to assist with the reverse engineering process (other than its own ASM/HTML output). You are left to your own devices (and your own favourite text editor) for that. There might eventually be some overlap between ZX-Assembler's and SkoolKit's features, but in any case, being able to import a SkoolKit source file (and export to one) would be a handy feature.

    Exactly.
    SkoolKid wrote: »
    Anyway, this doesn't really have anything to do with your question about file handling, so let me derail this topic no further.

    No, I think these issues are directly related to the original question.

    In cases 2 and 3, if ZX-Assembler simply produces a snapshot or tape file as output, it will be wasting all the extra "semantic" information provided by the user, just like any other tool.

    Therefore I'm arguing that ZX-Assembler should have an internal format where all this extra information is preserved.
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • edited September 2014
    If I understand correctly you want to do something that nobody has ever done before - work directly on opened .sna, .tap or tzx.

    I suppose that would involve a lot of work and I wonder if it is worth it.

    If somebody develops a game, he will work on .asm files and each time during compilation create a new .sna. I can't see any need to disassembly an existing .sna at any moment here.

    As for editing tape images - well, it could be useful for some hack, creating a version with infinite lives etc. but you can do such small jobs in hex editor as well. And I can't think of any bigger job of working on tape images. Again, when you create your own program you make .tap at the last moment when everything is ready.

    Still people have their habits of working which emerged because existing tools could do X and couldn't do Y. Maybe such a tool could help to make some things easier and faster, just we don't know about it yet :)
  • edited September 2014
    Ralf wrote: »
    If I understand correctly you want to do something that nobody has ever done before - work directly on opened .sna, .tap or tzx.
    Emuzwin have a built-in assembler with possibility to store code, etc., which assemble direct to memory.

    I do not understand why to invent same things once again...

    But I agree, having Sjasm and Pasmo and all other cross-assemblers, I see no point in any new assembler... As actually all things, which assembler should do - is to produce assembled routines, which could be written to tape or to disc or whatever you want. This could handle any assembler nowadays, actually, using other tools like BIN2TAP etc.
  • edited September 2014
    Emuzwin have a built-in assembler with possibility to store code, etc., which assemble direct to memory.
    As does Spin.
    I wanna tell you a story 'bout a woman I know...
  • edited September 2014
    SkoolKid wrote: »
    To use SkoolKit on any OS, you need Python. You can get it for Windows or Mac OS X here. I recommend version 2.7.x, but 3.4.x should work too. (I've just added a 'Requirements' section to the README on GitHub to make this clear.) If I remember correctly, Einar said it would be good if ZX-Assembler could be used as an IDE for SkoolKit, and I agree - though, of course, I may be biased. :) SkoolKit can produce disassemblies, but it doesn't provide any sort of environment to assist with the reverse engineering process (other than its own ASM/HTML output). You are left to your own devices (and your own favourite text editor) for that. There might eventually be some overlap between ZX-Assembler's and SkoolKit's features, but in any case, being able to import a SkoolKit source file (and export to one) would be a handy feature.

    Anyway, this doesn't really have anything to do with your question about file handling, so let me derail this topic no further.

    You did not derail this topic - quite the reverse! As Einar concluded below, it's really on-topic. I agree to him. :-) Also thanks for the detailed descriptions!
    I will download Python and try to bring SkoolKit to life on my machine. I will tell you my progresses in this topic here.
    Thanks for trying anyway :) Just to make it clear, my point wasn't really about my personal needs.

    I was arguing that ZX-Assembler can be a good tool to develop new code or modify an existing snapshot, but those are tasks that existing
    tools already perform just fine. Yet another tool for this purpose is welcome, but not really necessary. I may even use it myself to do it,
    but it won't make much difference.

    So far agreed.
    On the other hand, if ZX-Assembler could work as a user-friendly IDE for SkoolKit, that's something we really need, and would help us a lot. The idea is loading a snapshot into ZX-Assembler, then optionally loading code execution maps from emulators (so it will know how to interpret each block area), then manually refining block definitions, renaming labels, adding comments, etc. ZX-Assembler could directly use SkoolKit file format internally, or provide an option to import/export from/to SkoolKit.

    I'm not suggesting SkoolKit support to be provided instead of the ability to assemble files and modify snapshots, but in addition to it...

    Of course in addition. Well, let's see what I can find portable from SkoolKit and what next to do (useful) for the editing and file storage of files...
    The problem of data sections displayed as Z80 mnemonics is another thing that has to be solved (somehow), I assume, that SkoolKit has a good way
    storing such memory areas. So, probably I would not need to invent the wheel twice. :-)

    In cases 2 and 3, if ZX-Assembler simply produces a snapshot or tape file as output, it will be wasting all the extra "semantic" information provided by the user, just like any other tool.Therefore I'm arguing that ZX-Assembler should have an internal format where all this extra information is preserved.

    Yes, that's what I meant. Opening a snapshot or tape/disk file and make changes could either produce a new version of the file when saving - while losing the
    extra information as label names - or to a so-called-project (or meta data) file that contains all around that edited file. Or perhaps into a SkoolKit format,
    if this makes more sense (I cannot state to this now, as I need the detail analysis of it first).

    Ralf wrote: »
    If I understand correctly you want to do something that nobody has ever done before - work directly on opened .sna, .tap or tzx.

    I suppose that would involve a lot of work and I wonder if it is worth it.

    Well, it's often used on the other ZX-Modules. With ZX-Editor, you can edit basic programs of tapes or disks. It's (nearly) the same fact.
    Implementation of editing tapes and disks is already working in ZX-Assembler (as it all has been re-used from ZX-Editor/Blockeditor/Paintbrush etc.).
    Creating snapshots will be necessary not even for the INCSNA command, but also for a 'run program' feature. Running machine code for testing is done
    at its best with snapshots, that are stated with your desired emulator (my opinion). So this feature will be implemented anyway.
    Ralf wrote: »
    If somebody develops a game, he will work on .asm files and each time during compilation create a new .sna. I can't see any need to disassembly
    an existing .sna at any moment here.


    Okay, this may probably of less use, I agree with you. Even for studying Z80 algorithms it could be useful. More important is indeed the way you describe:
    Writing ASM -> compiling it -> INCSNA -> Having an SNA output (export) but still the ASM source.
    Ralf wrote: »
    As for editing tape images - well, it could be useful for some hack, creating a version with infinite lives etc. but you can do such small
    jobs in hex editor as well. And I can't think of any bigger job of working on tape images. Again, when you create your own program you make
    .tap at the last moment when everything is ready.

    Yes, may be true...
    Ralf wrote: »
    Still people have their habits of working which emerged because existing tools could do X and couldn't do Y. Maybe such a tool could help
    to make some things easier and faster, just we don't know about it yet :)

    Exactly. Let's wait another while until these things are more clear.
    Emuzwin have a built-in assembler with possibility to store code, etc., which assemble direct to memory.
    I do not understand why to invent same things once again...

    Do you personally have the opinion that using the built-in assembler, that WinZ80, EmuZWin and Spin, etc. contain (I know all of them and like them very much) is enough?
    Don't we need a program like ZX-Assembler that will have a IDE, lists of named labels, auto-recognization of calls and jumps, including assembly of most
    of SjASM commands, starting and exploring own asm files, run them into your fav. emulator with one mouse-click? Don't we really need this? (this is not meant ironic, it's a real question to you all users of assemblers)
    But I agree, having Sjasm and Pasmo and all other cross-assemblers, I see no point in any new assembler...
    As actually all things, which assembler should do - is to produce assembled routines, which could be written to tape or to disc or
    whatever you want. This could handle any assembler nowadays, actually, using other tools like BIN2TAP etc.

    If that goes for everybody here, it seems to be useless to keep on the (hard) work on writing ZX-Assembler...
    Even if I would not really stop my work until a fully working beta release, I definitely will not produce another 'dead' project like ZX-Central, ZX-Favourites and ZX-Editor S.E.

    There are no problems, only solutions (K. Flynn)

    Visit my ZX-Modules homepage with lot of free programs!
    Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
  • edited September 2014
    I think some of the comments on here are very negative.

    Claus is building a complete set of tools that help the spectrum community and it's called 'zx modules' and I use some of them myself but to say don't bother is shocking....

    so lets tell all the spectrum emulator builders on here not to bother because we can all use Spectaculator (why re-invent the wheel)

    ignore them Claus (some of use like it :) )
  • edited September 2014
    ASH-II wrote: »
    so lets tell all the spectrum emulator builders on here not to bother because we can all use Spectaculator (why re-invent the wheel)
    Absolutely not, Spectaculator can't produce the bugs Spud can...
    I wanna tell you a story 'bout a woman I know...
  • edited September 2014
    ASH-II wrote: »
    I think some of the comments on here are very negative.

    Claus is building a complete set of tools that help the spectrum community and it's called 'zx modules' and I use some of them myself but to say don't bother is shocking....

    so lets tell all the spectrum emulator builders on here not to bother because we can all use Spectaculator (why re-invent the wheel)

    ignore them Claus (some of use like it :) )

    Thank you for your nice words, ASH-II, but I don't feel bothered. I do really need negative comments. I do want a reflection of potential users, and non-users. Don't worry, I do not intend to stop my work only because of one negative comment. Your comment about Spectaculator is true: We do need many tools, not only one big tool. :-)

    BTW: Making ZX-Assembler still makes lot of fun! :-)

    There are no problems, only solutions (K. Flynn)

    Visit my ZX-Modules homepage with lot of free programs!
    Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
  • edited September 2014
    clausjahn wrote: »
    BTW: Making ZX-Assembler still makes lot of fun! :-)
    Which is exactly why there are quite a few of us who have written emulators!!
    I wanna tell you a story 'bout a woman I know...
  • edited September 2014
    clausjahn wrote: »
    Do you personally have the opinion that using the built-in assembler, that WinZ80, EmuZWin and Spin, etc. contain (I know all of them and like them very much) is enough?
    Claus, EmuzWin can do that all with built-in assembler. It have sort of IDE, calls, etc.
    Don't we need a program like ZX-Assembler that will have a IDE, lists of named labels, auto-recognization of calls and jumps, including assembly of most
    of SjASM commands, starting and exploring own asm files, run them into your fav. emulator with one mouse-click? Don't we really need this? (this is not meant ironic, it's a real question to you all users of assemblers)
    Almost every text editor could do this. Sublime Text 2 (for example you can check this integration http://zx-pk.ru/showthread.php?t=21731 from a feature list I see it could do all you mentioned), Notepad++, MultiEdit, UltraEdit, etc... You can attach Sjasm even to Visual Studio, Netbeans and Eclipse, getting benefits of all their features!

    Why waste time on making same thing once again? Better to work on integration with existing IDEs, by me.
    If that goes for everybody here, it seems to be useless to keep on the (hard) work on writing ZX-Assembler...
    Even if I would not really stop my work until a fully working beta release, I definitely will not produce another 'dead' project like ZX-Central, ZX-Favourites and ZX-Editor S.E.
    If we check the list of projects, which are done here, most probably 95% of them written in C (for engines like Churrera), Basic and AGD. So remaining 5% would use their preferring tool anyway (as I know, there is some % of ppl who would not use for work anything except EmuzWin). I do not think I personally would use any IDE by now, as I am quite happy how I do this already.

    I just trying to be realistic.
  • edited September 2014
    Agree with Ash-II, speaking as someone who's very new to z80 programming (and a big fan of ZX-Modules already) this looks like exactly the sort of tool I would use. And as someone who likes hacking about in games I like the idea of being able to edit .sna/.tzx files directly.

    It's your project though, entirely up to you what you do with it!
    The comp.sys.sinclair crap games competition 2015
    "Let's not be childish. Let's play Spectrum games."
  • edited September 2014
    clausjahn wrote: »
    Do you personally have the opinion that using the built-in assembler, that WinZ80, EmuZWin and Spin, etc. contain (I know all of them and like them very much) is enough?
    Don't we need a program like ZX-Assembler that will have a IDE, lists of named labels, auto-recognization of calls and jumps, including assembly of most
    of SjASM commands, starting and exploring own asm files, run them into your fav. emulator with one mouse-click? Don't we really need this? (this is not meant ironic, it's a real question to you all users of assemblers)

    I can see zx-assembler perhaps having some light utility in the way you described -- quick editing of snas, a disassembly view of code blocks and a means to edit it. If you incorporated Skoolkit type functionality it could be used to disassemble projects.

    If you're talking about writing new sw, perhaps another avenue is having zx-asm act as a front-end for other assemblers. You could provide your own directives and automatically translate to other assembler formats and then those assemblers would be used to do the assembling. Pasmo and sjasm have some nice features that are not available in z80asm and asz80, for example, but on the other hand neither pasmo nor sjasm are able to support large projects with shared code so it's not possible to use those assemblers in those kinds of projects.

    The large projects (z88dk, spectranet, etc) have thousands of assembly subroutines in them. The assemblers they are using have sections and linkers to manage code libraries. An IDE that documented and made those functions searchable might be interesting. I think a means to use shared code in an IDE environment is missing.

    I'm just thinking out loud here, not sure if these are good ideas or not.
  • edited September 2014
    on the other hand neither pasmo nor sjasm are able to support large projects with shared code so it's not possible to use those assemblers in those kinds of projects.
    What do you mean here? There are very big projects, written in sjasm, with more than 100 files - not enough big for you? :)
  • edited September 2014
    What do you mean here? There is very big projects, written in sjasm, with more than 100 files - not enough big for you? :)

    sjasm does not have a linker or sections support.

    The linker allows you to pull subroutines from libraries individually into the binary and it will make sure there is only one copy of the function in the final binary.

    I'll just give you a brief example:
    PUBLIC __strtoul__
    
    EXTERN l_valid_base, l_eat_ws, l_eat_sign, l_eat_digits
    EXTERN l_neg_dehl, l_char2num, l_mulu_40_32x8, l_eat_base_prefix
    
    INCLUDE "clib_cfg.asm"
    
    __strtoul__:
    
       ; strtol, strtoul helper
    

    This function exports one global symbol "__strtoul__" and calls other externally defined functions "l_valid_base, l_eat_ws, ...". It also has many local labels not seen outside this file. The complete source is here.

    Another function:
    PUBLIC asm_atol
    
    EXTERN l_eat_ws, l_eat_sign, l_neg_dehl, l_atoul
    
    asm_atol:
    

    One global symbol is exported "asm_atol" and several external functions are called "l_eat_ws, etc". It also has many local labels not seen outside this file. The complete source is here.

    If I write the non-sensical program:
    org 32768
    
    EXTERN asm_atol, __strtoul__
    
    start:
    
    call asm_atol
    call __strtoul__
    ret
    

    Inside that code fragment, only "asm_atol" and "__strotoul__" become part of the local namespace. In addition to the code from "asm_atol" and "__strtoul__", the final binary will contain the following functions needed by those calls:

    l_valid_base, l_eat_ws, l_eat_sign, l_eat_digits
    l_neg_dehl, l_char2num, l_mulu_40_32x8, l_eat_base_prefix
    l_atoul

    The linker will also look at each of those functions and see if anything else is required. For example, it will see that "l_eat_ws" needs to call "asm_isspace" so "asm_isspace" will also be added to the binary. (Source of l_eat_ws). The linker will then go into "asm_isspace" and see if anything is needed for that function, and so on.

    Additionally, the linker will only pull in one copy of each function needed. So although both "asm_atoul" and "__strtoul__" need "l_eat_ws", only one copy of "l_eat_ws" will be attached to the binary.

    We have thousands of functions like this. If we had to put them all into a binary, the binary would be about 150k on the spectrum. You can see it's not possible for large projects to be used with simpler assemblers -- linkers are required :)


    Sections allow code to be assigned to different blocks of memory. I know sjasm has something like that. In the context I have in mind, I want to be able to assign code to ROM, initialized data to a DATA section and uninitialized data to a BSS section. Then I can map those sections to relevant areas in memory. You can go further to fit code and data to multiple banks in bankswitched systems, for example.
  • edited September 2014
    In Sjasmplus there is a MODULE definition, which actually does sort of that - all labels became local labels. You can read docs for that, seems this is what you want. For shifting data there is PHASE/UNPHASE functions.

    And actually when I start a new project, I make a sort of kernel with all needed subroutines (lets be honest - there is no universal routine in Z80 programming - almost every routine had to be modified in one or another way in every project). So things like you proposed - needed only in C. Java or similar. In assembler you do all these things related to your real tasks, modify behaviour and stuff - this why there is no need of such mechanics...

    That said, I spoke with lot ASM programmers and all were quite happy without things you told. Custom macroses cover 99% of things needed.
  • edited September 2014
    That said, I spoke with lot ASM programmers and all were quite happy without things you told. Custom macroses cover 99% of things needed.

    Somehow I remembered this anecdote:

    Two salesmen from shoe companies came to Africa to check if they could sell shoes there.

    The first one wrote: it's great market, nobody has shoes here yet.
    The second one wrote: it's no use, people don't wear shoes here.

    Which one was right? ;)

    The same is here. People lived without some feature in their tool forever. Will they find new feature usuful? Only experience may tell us.
  • edited September 2014
    It is not same. Sjasmplus have already some tools, which does similar things, just in different way.

    Anyway, in this case would be better to use Sjasmplus repository and add needed things directly there (I assume AA could code C? correct?), rather than making completely new assembler with built-in frontend part (which could take ages to code, actually).
  • edited September 2014
    In Sjasmplus there is a MODULE definition, which actually does sort of that - all labels became local labels. You can read docs for that, seems this is what you want. For shifting data there is PHASE/UNPHASE functions.

    Yes MODULE can hide locals and PHASE can create a new binary block but they are only partial substitutions.

    The key feature of linkers is that they can extract from libraries only those functions you are using and they will not duplicate any of the code. This is essential for code reuse, whether you program in a high level language or assembler.

    It's how assemblers have worked since the 1960s. The home computers of the 1980s were working under constrained resources and writing linking assemblers was not practical so we got simpler assemblers without linking. That's what many people who grew up on the Spectrum are used to and they may not even be aware that there are such things as linkers. Even if they program professionally, in the last 10-20 years how programs are linked together has been made hidden behind IDEs so many professional programmers have no idea how assemblers, linkers, compilers and the operating system interact. But underneath it all, static linking as I described above is how it's done for statically linked programs.

    I'll give you one example specific to the z80 and then one specific to the Spectrum.

    Some z80 applications need a floating point library for their programs. For this reason, z88dk supplies one. It consists of the following floating point functions:

    acos, addhalf, amax, amin, asin, atan, atan2, atof, ceil, compare, cos, cosh.c, dadd, dcompar, ddiv, deq, dge, dgt, div1, div14, dleq, dlt, dmul, dne,, dsub, dswap, evenpol, exp, f_no, f_yes, fabs, fadd, fdiv, float, floor, fmod, fmull, fradd, fsub, ftoa.c, ftoe.c, hladd, hlsub, ifix, incr, int2, ldbcfa, ldbchl, ldfabc, log, log10, minusbc, minusfa, norm, norm4, norma, odd, pack, pack2, poly, pow, pushfa, rand, rshift, seed, sgn, sin, sinh, sqrt, tan, tanh, ufloat, unpack, init_floatpack2, pi

    All these functions put together might occupy 3-4k. I could bundle it all into a module and anyone who wants to use float math has to swallow the full 3-4k to use it. Even if the program only needs to multiply and add floats, it's stuck with every function in the float package. Worse, those float functions also contain common functions like "multiply these two ints", "convert this ascii char to a digit" and will contain its own implementation. The main program may also use those functions, as might other libraries being used in the program. Each will get a copy of the same function so that the same code is duplicated several times in the binary.

    An alternative is the programmer goes in there and copies needed code into his own project by hand. He will have to start with functions he calls directly, then functions called by those functions, functions called by functions called by those functions, etc. He'll have to modify the code to hide local names so that they don't interfere with names used in the rest of the program. This is time consuming, error prone, and ridiculous if you have to sort through thousands of functions. Code duplication can still easily occur.

    With a linker, if your program only multiplies and adds floats, only the part of the floating point library that does that will be added to the program. There will be no duplicated code as any program or other libraries that use, eg, the "multiply two ints" subroutine will use the same subroutine.


    Specific to the spectrum now. z88dk contains a sprite engine that has been used in many games. It supplies many functions, listed in this header file. If every function of the library were used in a game, it might occupy 4-5k (pure guesswork). However, games do not need to use every function and they are unlikely to need to draw every kind of sprite: masked, OR, XOR, LOAD, ATTR. If this were bundled as a MODULE a program would have no choice but to add all the code, even stuff it does not use, to the final binary. With a linker, only the parts of the library that are used get added to the binary. That's pretty important on a memory constrained system like the Spectrum and doubly important because the design of this library requires a lot of RAM for workspace.


    I don't think PHASE and UNPHASE is sufficiently general to substitute for sections. With section support, we need to be able to define any number of independent memory regions and we need to be able to place library code, program code, and data in specific user-defined sections. This is because there are many different memory maps on z80 machines. I know sjasm is specifically targetted at the spectrum so maybe it doesn't have to worry about that but z88dk in particular is targetted at any z80 machine. But just one "textarea", for example, would not be enough for every z80 target.
    And actually when I start a new project, I make a sort of kernel with all needed subroutines (lets be honest - there is no universal routine in Z80 programming - almost every routine had to be modified in one or another way in every project). So things like you proposed - needed only in C. Java or similar. In assembler you do all these things related to your real tasks, modify behaviour and stuff - this why there is no need of such mechanics...

    I disagree :) Most code gets reused unmodified. It's only performance related code that might get revised significantly. Stuff like sprite engines when the particular game is cpu constrained. So you might write from scratch a small portion of the total code you are using in any given project, unless you do not have a large set of subroutines to select from already.
    That said, I spoke with lot ASM programmers and all were quite happy without things you told. Custom macroses cover 99% of things needed.

    I think it's because there is very little code reuse going on in the Spectrum community. The people who are producing have their own stuff and reuse their own stuff. They don't share much or use other's work much.

    But not everyone is working that way and, indeed, most zx sw that is produced these days is making use of code libraries written by others.
  • edited September 2014
    Claus, EmuzWin can do that all with built-in assembler. It have sort of IDE, calls, etc.

    The built-in assembler of EmuzWin ist okay so far, even if EmuZWin makes some trouble on my machine. I did not yet explore all of it's abilities, and I agree: it's very powerful.
    Almost every text editor could do this. Sublime Text 2 (for example you can check this integration http://zx-pk.ru/showthread.php?t=21731 from a feature list I see it could do all you mentioned), Notepad++, MultiEdit, UltraEdit, etc... You can attach Sjasm even to Visual Studio, Netbeans and Eclipse, getting benefits of all their features!

    The sublime text add-on looks indeed _very_ nice. The black background and nice colours and syntax highlighting this way is just perfect. In opposite to you I don't know about most of the tools you mentioned, and probably that's why I started an own project. It probably would never reach the flexibility you have with all those tools, that's clear and I agree.
    Why waste time on making same thing once again? Better to work on integration with existing IDEs, by me.

    I cannot disagree. :-o
    If we check the list of projects, which are done here, most probably 95% of them written in C (for engines like Churrera), Basic and AGD. So remaining 5% would use their preferring tool anyway (as I know, there is some % of ppl who would not use for work anything except EmuzWin). I do not think I personally would use any IDE by now, as I am quite happy how I do this already.

    I just trying to be realistic.

    No problem. I'm interested in all point of views here. Don't think your comments will be completely ignored, even if I may continue my project (probably by wasting time). Your probably realistic comments gave me an important view about the other side of the medal...

    There are no problems, only solutions (K. Flynn)

    Visit my ZX-Modules homepage with lot of free programs!
    Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
  • edited September 2014
    I can see zx-assembler perhaps having some light utility in the way you described -- quick editing of snas, a disassembly view of code blocks and a means to edit it. If you incorporated Skoolkit type functionality it could be used to disassemble projects.

    Agreed. I did not intend to produce another cross-platform-system, but rather a ZX-Spectrum and ZX-Modules depended tool. So, I guess, the powerful behaviors for users with big projects of thousands of linked ASM files, and the need of a linker, is probably not what I can make. Perhaps in a later release but of course not in an early release.
    If you're talking about writing new sw, perhaps another avenue is having zx-asm act as a front-end for other assemblers. You could provide your own directives and automatically translate to other assembler formats and then those assemblers would be used to do the assembling. Pasmo and sjasm have some nice features that are not available in z80asm and asz80, for example, but on the other hand neither pasmo nor sjasm are able to support large projects with shared code so it's not possible to use those assemblers in those kinds of projects.

    As there are many different assemblers, emulators with built-in-assemblers around, there are of course lot of different syntaxes out now, different commands, etc. Usually I try to support as much commands and syntaxes as possible rather than using my tool as a front-end to other assemblers. (which indeed would be a nice additional feature, anyway).
    The large projects (z88dk, spectranet, etc) have thousands of assembly subroutines in them. The assemblers they are using have sections and linkers to manage code libraries. An IDE that documented and made those functions searchable might be interesting. I think a means to use shared code in an IDE environment is missing.

    I'm just thinking out loud here, not sure if these are good ideas or not.

    Also - no problem. We all are right now just thinking and suggesting. :-)

    About linking: So far, no linker is needed for ZX-Assembler, because it just now produces ready-to-go outputs. external and include clauses will be necessary anyway, so ZX-Assembler does the linking on itself.
    Having already modules, conditional compiling, defines and macros as well as binary and ASM includes (refer to the detailled topic "ZX-Modules app report") most of the things we need will be available already.
    I also see good potential in supporting these control files (CTL) of SkoolKit, which I started to study yesterday.

    There are no problems, only solutions (K. Flynn)

    Visit my ZX-Modules homepage with lot of free programs!
    Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
  • edited September 2014
    leespoons wrote: »
    Agree with Ash-II, speaking as someone who's very new to z80 programming (and a big fan of ZX-Modules already) this looks like exactly the sort of tool I would use. And as someone who likes hacking about in games I like the idea of being able to edit .sna/.tzx files directly.

    It's your project though, entirely up to you what you do with it!

    That's the group of people I want to write ZX-Assembler for. At this state I really have to take care about many different purposes (e.g. SkoolKit support), but I don't want, nor intend to reach the high potential, that some of the already mentioned tools offer.
    At it first stages, ZX-Assembler will be a very rudimentary assembler in comparision to other tools. As well as the optic may be not as cute as in the examples, Shadow Maker showed us (a good idea to re-design the the input editor of ZX-Assembler before the first release).

    There are no problems, only solutions (K. Flynn)

    Visit my ZX-Modules homepage with lot of free programs!
    Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
  • edited September 2014
    The key feature of linkers is that they can extract from libraries only those functions you are using and they will not duplicate any of the code. This is essential for code reuse, whether you program in a high level language or assembler.

    It's how assemblers have worked since the 1960s. The home computers of the 1980s were working under constrained resources and writing linking assemblers was not practical so we got simpler assemblers without linking. That's what many people who grew up on the Spectrum are used to and they may not even be aware that there are such things as linkers. Even if they program professionally, in the last 10-20 years how programs are linked together has been made hidden behind IDEs so many professional programmers have no idea how assemblers, linkers, compilers and the operating system interact. But underneath it all, static linking as I described above is how it's done for statically linked programs.

    I'll give you one example specific to the z80 and then one specific to the Spectrum.

    Some z80 applications need a floating point library for their programs. For this reason, z88dk supplies one. It consists of the following floating point functions:

    acos, addhalf, amax, amin, asin, atan, atan2, atof, ceil, compare, cos, cosh.c, dadd, dcompar, ddiv, deq, dge, dgt, div1, div14, dleq, dlt, dmul, dne,, dsub, dswap, evenpol, exp, f_no, f_yes, fabs, fadd, fdiv, float, floor, fmod, fmull, fradd, fsub, ftoa.c, ftoe.c, hladd, hlsub, ifix, incr, int2, ldbcfa, ldbchl, ldfabc, log, log10, minusbc, minusfa, norm, norm4, norma, odd, pack, pack2, poly, pow, pushfa, rand, rshift, seed, sgn, sin, sinh, sqrt, tan, tanh, ufloat, unpack, init_floatpack2, pi

    All these functions put together might occupy 3-4k. I could bundle it all into a module and anyone who wants to use float math has to swallow the full 3-4k to use it. Even if the program only needs to multiply and add floats, it's stuck with every function in the float package. Worse, those float functions also contain common functions like "multiply these two ints", "convert this ascii char to a digit" and will contain its own implementation. The main program may also use those functions, as might other libraries being used in the program. Each will get a copy of the same function so that the same code is duplicated several times in the binary.

    An alternative is the programmer goes in there and copies needed code into his own project by hand. He will have to start with functions he calls directly, then functions called by those functions, functions called by functions called by those functions, etc. He'll have to modify the code to hide local names so that they don't interfere with names used in the rest of the program. This is time consuming, error prone, and ridiculous if you have to sort through thousands of functions. Code duplication can still easily occur.

    With a linker, if your program only multiplies and adds floats, only the part of the floating point library that does that will be added to the program. There will be no duplicated code as any program or other libraries that use, eg, the "multiply two ints" subroutine will use the same subroutine.


    Specific to the spectrum now. z88dk contains a sprite engine that has been used in many games. It supplies many functions, listed in this header file. If every function of the library were used in a game, it might occupy 4-5k (pure guesswork). However, games do not need to use every function and they are unlikely to need to draw every kind of sprite: masked, OR, XOR, LOAD, ATTR. If this were bundled as a MODULE a program would have no choice but to add all the code, even stuff it does not use, to the final binary. With a linker, only the parts of the library that are used get added to the binary. That's pretty important on a memory constrained system like the Spectrum and doubly important because the design of this library requires a lot of RAM for workspace.

    Interesting, very interesting example. I agree with your example that states for a need of a linker - vice versa to what I said above. But I would say, that what you described will be really necessary for big projects with many reuses of kernel routines. But I really understand what you described (I know the linking procedure from my Borland Delphi anyway).
    Probably in the early stages not important for the issues ZX-Assembler will be made for. But for big projects, you are really right here....
    I don't think PHASE and UNPHASE is sufficiently general to substitute for sections. With section support, we need to be able to define any number of independent memory regions and we need to be able to place library code, program code, and data in specific user-defined sections. This is because there are many different memory maps on z80 machines. I know sjasm is specifically targetted at the spectrum so maybe it doesn't have to worry about that but z88dk in particular is targetted at any z80 machine. But just one "textarea", for example, would not be enough for every z80 target.

    ....

    I think it's because there is very little code reuse going on in the Spectrum community. The people who are producing have their own stuff and reuse their own stuff. They don't share much or use other's work much.

    But not everyone is working that way and, indeed, most zx sw that is produced these days is making use of code libraries written by others.

    Yes, for the Spectrum community, the usage of a linker seems to be overdressed, but I will not forget this feature and these needs. So let me say thanks for that interesting example!

    There are no problems, only solutions (K. Flynn)

    Visit my ZX-Modules homepage with lot of free programs!
    Or visit my music-related website if you're interested in synthesizer music or computer animations and movies I've created
  • edited September 2014
    Yes MODULE can hide locals and PHASE can create a new binary block but they are only partial substitutions.

    The key feature of linkers is that they can extract from libraries only those functions you are using and they will not duplicate any of the code. This is essential for code reuse, whether you program in a high level language or assembler.


    Snip

    I think it's because there is very little code reuse going on in the Spectrum community. The people who are producing have their own stuff and reuse their own stuff. They don't share much or use other's work much.

    But not everyone is working that way and, indeed, most zx sw that is produced these days is making use of code libraries written by others.

    GREAT post AA!

    I was a Z80 hobby coder on the Sam Coupe and am now working in a high level language. I used to hand code and optimise everything in Z80, but completely missed the intermediate steps between assembly and high level stuff.

    The discussions here and on the z88dk forum have been very enlightening.

    Thanks!
  • edited September 2014
    AA, actually there is tons of way to execute routine in Z80. And I personally use all sorts - popping from stack, adding a number and pushing back and doing ret, using shift tables, using direct memory modifications... So this "linker" would certainly fail to find all "unused" code and it would be more a problem, rather than a benefit. Also duplicated code - well, 90% of screen output HAVE duplicated code for speeding output.

    Maybe what you tell is useful for some novice programmer or someone, who came from Java, but in reality such automatic tool would not work normally - and could be more an issue for programmer, rather than a help. Indeed, there would be no warnings in assembler, but the program would just fail to work :) I will curse creators of such madness a lot!

    So for me such linker would bring more problem for real asm programmer, if he do not use only "CALL" and "JP" all the time, and indeed, if you check code for modern games, written directly in assembler, I am sure you will find, that 90% of them use non-direct execution of calls (actually most of games from the past use that too).

    You just thinking like a z88dk user, that's the problem :) C is not assembler, and obv yes, in C your asm routine had to be kept mostly same. I guess you never coded big game project in assembler?

    Moreover, if you really want this for some reason, for such thing it is possible to write a pre-processor, which would just analyze text code and comment out (or remove) unused data in your text files, possible to write it even for unexperienced person, even in Python or PHP or whatever fits for good text parsing. No real need to integrate this into assembler, much better to do as separate tool (as for example I would never use this :)). And you actually could write this yourself, without any problems I believe :)
  • edited September 2014
    I think it's because there is very little code reuse going on in the Spectrum community. The people who are producing have their own stuff and reuse their own stuff. They don't share much or use other's work much.

    But not everyone is working that way and, indeed, most zx sw that is produced these days is making use of code libraries written by others.
    You can give me example of library, written by others in assembler, used by others in assembler? As I know, most ppl, who do not want to write something, just search for a code snippet to output a sprite, then choose which one is better for them (someone need fast, others need less memory, or need 50/50 - there is different situations). If problem is supposed to be fast, for example 50hz, it become a problem, and a struggle between code size/execution time begins.

    This why I can say, that only routine I reuse mostly unmodified, is a conversion from numeric to text. All other, including text print and sprite output - is modified, depending on what I want to get.

    Of course, if you always make arcade-like game, which look like usual 3-day coded Churrera game, then obv 90% of code would not change, but what coder really interested in such work? And this would not need any linker too.

    So much better would be not creating libraries, but make a site with tons of snippets, to be included in the code, by me...
Sign In or Register to comment.