Compressors are available for Speccys ?

edited September 2013 in Development
Hi folks,

although I was an wasn't strictly a speccy coder in the glory days (amstrad & amiga for the most part), I have secides to arse about with tommygun & ZX spin (it's cheaper than the pub)

Anyway I was wondering what packers are available for the speccy (I wrote an rle compressor many years ago for z80 machines).

I currently have
pletter (which seems to corrupt sectins or ram)
megalz (which I will be playing with later today)
bitbuster12 (don't have a c compiler atm so can't compile the packer)

Any other suggestions ?

Also I don't suppose anybody has any tape loaders (fancy ones) kicking around that I could play with (I do have the Jon North one)

Regards,

Rob...
Post edited by Kilby on
«1

Comments

  • edited July 2007
    I'm assuming that I can use the standard bitbuser compressor, and then use the extreme version of the decompression routine.

    I may be being thick, but I don't ghave the binarys for the butbuster extreme compression util, and no C compiler to compile them with.

    Rob...
  • edited July 2007
    Personally I use PuCrunch, it's fast, and has a very good compression ratio.
  • edited July 2007
    Take a look in the Utils section on my website. If there's no link to the files you want, let me know and I'll get something sorted out. It's still a work in progress at the moment.
    Oh bugger!<br>
  • edited July 2007
    There's also a decompressor available for aPPack, the aPLib library example.

    You can find the library here:
    http://www.ibsensoftware.com/download.html

    and here's a compiled compressor together with a z80 decompressor:
    http://www.smspower.org/maxim/smssoftware/aplib.html

    On the Spectrum, it has been used for Cannon Bubble.
  • edited July 2007
    Thanks folks so far i have been got megaLz, bitbuster and pletter operational

    Though I'm having a bit of fun getting used to tommygun, plasmo and sjasm (too used with tasm, maxam and devpac over the years)

    I will try out the others over then next few hours, as I'd rather have a decompressor a few bytes longer, as long as it is well behaved.

    Currently I like megaLz as it's 100% relocatable so I can throw it into screen memory before executing it (old habits die hard, especially when microdrives steal some vital RAM ;) )
  • edited July 2007
    File Compressor
    (for advanced ZX Spectrum users)

    http://user.sezampro.yu/~mstancevic/File_Compressor/File_Compressor.htm
  • edited July 2007
    Kilby wrote: »
    I'm assuming that I can use the standard bitbuser compressor, and then use the extreme version of the decompression routine.
    Not quite... the standard Bitbuster compressor dumps out the uncompressed length as the first four (or is it two?) bytes of its output, which isn't necessary at all. The Bitbuster Extreme compressor doesn't do that - but that's the one and only difference as far as the compressor goes. I readily admit that 'Extreme' is a completely over-the-top description for such a trivial change :-)

    I've really wanted to give MegaLZ a go, as it's meant to give much better compression ratios than BB Extreme for a not-much-bigger decompressor, but my most recent projects have all required in-place compression (i.e. compressed data at 25000, unpacked to 25000) which MegaLZ doesn't do. To be fair, I don't think BB Extreme does either... instead, I've used Hrust (in the form of my command-line hack chrust - not sure if anyone's successfully compiled it for Windows incidentally), which gives excellent compression at the cost of a whopping 256-byte decompressor routine.
  • edited July 2007
    Thanks gentlemen (I assume)

    I finally managed to get aplib-z80 assembling with both pasmo and sjasmplus (the author made way too much use of wla features).

    So far aplib is the winner as regards inital compression ratio, but the unpack routine is huge @ 496 bytes but hacking out the sega vram decompress reduces to 241 bytes (still huge)

    Anyway heres some figures using the game code from Volcanic Planet which starts off @ 8300 bytes (small isn't it)

    Packer Compressed file Packer size Total
    APlib 6361 214 (or 496) 6575 (or 6857)
    MegaLz 6438 116 6554
    Pletter 6516 113 6629
    bitbuster 6596 112 6596

    I also feel that the aplib decompression is a lot slower than the others. I'm sure it can be size optimised but I need to get some practice coding before I consider doing anything about it.

    So far MegaLz would be my preferred option as I can ldir it to whatever position I want and then call it and it's reasonably small

    I still have to try the RNC (Rob Northern Computing for all you Amiga and ST folks)

    bitbuster extreme (heh), PUcrunch, hrust and chrust, will have to wait till I get a hold of some time and some compiled .exe files

    Now has anybody gor any turbo loaders I could abuse ?

    Rob...

    PS incase you are wondering, I loved Volcanic Planet way back in 83, plus it's so small (it's a 16L game) I'm considering using hacking it to bits as I have 32K free for my own code.

    A very colourful decompressor is the first on the list, just for a quick bit of hack & slash coding
  • edited July 2007
    Kilby wrote: »
    bitbuster extreme (heh), PUcrunch, hrust and chrust, will have to wait till I get a hold of some time and some compiled .exe files

    If it's any use to you I've uploaded BBE compiled for Win32 console here (although as gasman says, if you use it you'll only save 4 bytes over the regular BB).

    As for turbo loaders, this site has loading and saving routines for some commercial loaders.
  • edited July 2007
    StuBruise wrote: »
    If it's any use to you I've uploaded BBE compiled for Win32 console here (although as gasman says, if you use it you'll only save 4 bytes over the regular BB).

    As for turbo loaders, this site has loading and saving routines for some commercial loaders.

    Many thanks

    Hey I a bit of a completist at times, plus it's easier to frig about with code that somebody contactable has worked on ;)

    I spent more than a little time on that site already ;)
    I did write a turboloader on the Amstrad in 86 but I certainly couldn't be arsed to do it again.

    I was hoping to crib a few ideas regarding how the data for the screen was held in the Alcatraz loaders (where the positions held in tables), or where the address and screen data interleaved when loaded

    Rob...
  • edited July 2007
    I would also recommend the File Compressor... it is VERY good - especially when it comes to WHAT you can compress and where you put it. You don't need to take care of any relocation or similar. And the compression level is decent too ....
    http://user.sezampro.yu/~mstancevic/File_Compressor/File_Compressor.htm
  • edited July 2007
    Kilby wrote: »
    Hi folks,

    although I was an wasn't strictly a speccy coder in the glory days (amstrad & amiga for the most part), I have secides to arse about with tommygun & ZX spin (it's cheaper than the pub)

    Anyway I was wondering what packers are available for the speccy (I wrote an rle compressor many years ago for z80 machines).
    Any other suggestions ?

    I made some simple LZW . You may see that almost all titles here are compressed:

    http://www.ppest.org/qaop/qa.php

    Depacker is in REM line usually - short, about 45 bytes, and fast. I post you packer with sources if want...
  • edited July 2007
    piters wrote: »
    I made some simple LZW . You may see that almost all titles here are compressed:

    http://www.ppest.org/qaop/qa.php

    Depacker is in REM line usually - short, about 45 bytes, and fast. I post you packer with sources if want...

    you could put depacker source here. 45B is pretty short.

    btw. pmc is rather advanced LZ+Huffman compressor. the most of mentioned packer (except Pucrunch) are LZSS variations. i did not see a LZW implementation on speccy yet.
  • edited July 2007
    Here is depacker:

    DEKOMPRESS

    ;ENDPOINTER IS ZERO

    LD HL,50000 ;SOURCE (COMPRESSED DATA)
    LD DE,#4000 ;DESTINATION

    MLOOP LD A,(HL)
    INC HL
    LD C,A
    OR A
    JP M,DISPL
    RET Z
    LD B,0
    LDIR
    JR MLOOP
    DISPL RLCA
    RLCA
    RLCA
    AND 3
    LD B,A
    LD A,C
    LD C,(HL)
    INC HL
    PUSH HL
    LD H,D
    LD L,E
    AND #1F
    SBC HL,BC
    ADD A,3
    LD C,A
    LD B,0
    LDIR
    POP HL
    JR MLOOP

    Actually, I'm not sure that it is right LZW, or just something similar :-) I made it alone, without literature, in 1991.

    Complete source is here: http://www.ppest.org/zx/COMP_DEC.ZIP
  • edited July 2007
    piters wrote: »

    Actually, I'm not sure that it is right LZW, or just something similar :-) I made it alone, without literature, in 1991.

    Complete source is here: http://www.ppest.org/zx/COMP_DEC.ZIP
    no, it is not LZW ;)
  • edited July 2007
    Tom-Cat wrote: »
    I would also recommend the File Compressor... it is VERY good - especially when it comes to WHAT you can compress and where you put it. You don't need to take care of any relocation or similar. And the compression level is decent too ....
    http://user.sezampro.yu/~mstancevic/File_Compressor/File_Compressor.htm


    fc_35_td.tap link is broken. Anyone has this file?
    bitbuster extreme (heh), PUcrunch, hrust and chrust, will have to wait till I get a hold of some time and some compiled .exe files

    PuChrunch win32 console version: http://www.cs.tut.fi/~albert/Dev/pucrunch/pucrunch_x86.zip (use this with -c0 -d parameters)

    you can test those on volcanic planet so we can see which one is best.

    I tested it on the "Spellbound Dizzy" loading screen, PuChrunch gives 4937 bytes, while BBE gives 5058 bytes.
  • edited July 2007
    Arda wrote: »
    fc_35_td.tap link is broken. Anyone has this file?



    PuChrunch win32 console version: http://www.cs.tut.fi/~albert/Dev/pucrunch/pucrunch_x86.zip (use this with -c0 -d parameters)

    you can test those on volcanic planet so we can see which one is best.

    I tested it on the "Spellbound Dizzy" loading screen, PuChrunch gives 4937 bytes, while BBE gives 5058 bytes.

    Many thanks, I'm on holiday ATM so the speccy stuff is on the back burner, yeah I know I'm the opposite of everybody else).

    also having to deal with a slightly flooded house :(

    Anyway I will knock together a few more compressed vesions of the test file, as I have noticed some of the decompressors have huge differences in decompress speeds.
  • edited October 2007
    Kilby wrote: »
    I also feel that the aplib decompression is a lot slower than the others.

    I've optimized the getbit routine and it now takes 373 less states for each control byte. It will probably still be the slowest, but not for so much difference now.
    ; aPPack decompressor 
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ldi
    		xor	a
    		ld 	(lwm),a
    		ld	a,128
    		ld	(ap_byte),a
    
    aploop:		call 	ap_getbit
    		jp 	nc,apbranch1
    		call 	ap_getbit
    		jr 	nc,apbranch2
    		call 	ap_getbit
    		jr 	nc,apbranch3
    					
    		xor 	a		;LWM = 0
    		ld 	(lwm),a
    		
    		ld 	bc,0		;get an offset
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		ld 	a,b
    		or 	c
    		jr 	nz,apbranch4
    		xor 	a  		;write a 0
    		ld 	(de),a
    		inc 	de
    		jr 	aploop
    apbranch4:	ex 	de,hl 		;write a previous bit (1-15 away from dest)
    		push 	hl
    		sbc 	hl,bc
    		ld 	a,(hl)
    		pop 	hl
    		ld 	(hl),a
    		inc 	hl
    		ex 	de,hl
    		jr 	aploop
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl		
    		rr 	c
    		ret 	z		;if a zero is encountered here, it's EOF
    		ld 	b,2
    		jr 	nc,ap_dont_inc_b
    		inc 	b
    ap_dont_inc_b:	ld 	a,1		;LWM = 1
    		ld 	(lwm),a
    		push 	hl
    		ld 	a,b
    		ld 	b,0
    		ld 	(r0),bc		;R0 = c
    		ld 	h,d
    		ld 	l,e
    		or 	a
    		sbc 	hl,bc
    		ld 	c,a
    		ldir
    		pop 	hl
    		jr 	aploop
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec 	bc
    		dec 	bc
    		ld	a,(lwm)
    		or	a
    		jr	nz,ap_not_lwm
    		ld 	a,b			;bc = 2?
    		or 	c
    		jr 	nz,ap_not_zero_gamma	;if gamma code is 2, use old r0 offset,
    		call 	ap_getgamma		;and a new gamma code for length
    		push 	hl
    		ld 	h,d
    		ld 	l,e
    		push 	bc
    		ld 	bc,(r0)
    		sbc 	hl,bc
    		pop 	bc
    		ldir
    		pop 	hl
    		jr 	ap_finishup
    ap_not_zero_gamma:
    		dec bc
    ap_not_lwm:
    		;do I even need this code?
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,c
    		ld 	c,(hl)
    		inc 	hl
    		ld 	(r0),bc
    		push 	bc
    		call 	ap_getgamma
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    		
    		ld 	hl,1279
    		or 	a
    		sbc 	hl,de
    		jr 	nc,skip2
    		inc 	bc
    skip2:		ld 	hl,127
    		or 	a
    		sbc 	hl,de
    		jr 	c,skip3
    		inc 	bc
    		inc 	bc
    skip3:		pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    		sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    		ldir
    		pop 	hl
    ap_finishup:	ld 	a,1
    		ld 	(lwm),a
    		jp 	aploop
    
    apbranch1:	ldi
    		xor 	a
    		ld 	(lwm),a
    		jp 	aploop
    
    ap_getbit:	ld	a,(ap_byte)
    		and	a
    		rl	a
    		ld	(ap_byte),a
    		ret	nz
    		ld	a,(hl)
    		inc	hl
    		rla
    ap_endit:	ld	(ap_byte),a
    		ret
    
    ap_getbitbc: 	sla 	c		;doubles BC and adds the read bit
    		rl 	b
    		call 	ap_getbit
    		ret 	nc
    		inc 	bc
    		ret
    
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbitbc
    		call 	ap_getbit
    		jr 	c,ap_getgammaloop
    		ret
    		
    ; required data
    ap_byte: 	defb 	0
    lwm:		defb 	0
    r0:		defw 	0
    
  • edited October 2007
    A few more optimizations:
    ; aPPack decompressor
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ld	a,128
    		ld	(ap_byte),a
    apbranch1:	ldi
    		xor	a
    		ld 	(lwm),a
    aploop:		call 	ap_getbit
    		jp 	nc,apbranch1
    		call 	ap_getbit
    		jr 	nc,apbranch2
    		call 	ap_getbit
    		jr 	nc,apbranch3
    					
    		xor 	a		;LWM = 0
    		ld 	(lwm),a
    		ld 	b,a		;get an offset
    		ld	c,a
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		ld 	a,b
    		or 	c
    		jr 	nz,apbranch4
    		ld 	(de),a		;write a 0
    		inc 	de
    		jr 	aploop
    apbranch4:	ex 	de,hl 		;write a previous bit (1-15 away from dest)
    		sbc 	hl,bc
    		ld 	a,(hl)
    		add	hl,bc
    		ld 	(hl),a
    		inc 	hl
    		ex 	de,hl
    		jr 	aploop
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl		
    		rr 	c
    		ret 	z		;if a zero is encountered here, it's EOF
    		ld 	b,2
    		jr 	nc,ap_dont_inc_b
    		inc 	b
    		or 	a
    ap_dont_inc_b:	ld 	a,1		;LWM = 1
    		ld 	(lwm),a
    		push 	hl
    		ld 	a,b
    		ld 	b,0
    		ld 	(r0),bc		;R0 = c
    		ld 	h,d
    		ld 	l,e
    		sbc 	hl,bc
    		ld 	c,a
    		ldir
    		pop 	hl
    		jr 	aploop
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec 	bc
    		dec 	bc
    		ld	a,(lwm)
    		or	a
    		jr	nz,ap_not_lwm
    		ld 	a,b			;bc = 2?
    		or 	c
    		jr 	nz,ap_not_zero_gamma	;if gamma code is 2, use old r0 offset,
    		call 	ap_getgamma		;and a new gamma code for length
    		push 	hl
    		ld 	h,d
    		ld 	l,e
    		push 	bc
    		ld 	bc,(r0)
    		sbc 	hl,bc
    		pop 	bc
    		ldir
    		pop 	hl
    		jr 	ap_finishup
    ap_not_zero_gamma:
    		dec bc
    ap_not_lwm:
    		;do I even need this code?
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,c
    		ld 	c,(hl)
    		inc 	hl
    		ld 	(r0),bc
    		push 	bc
    		call 	ap_getgamma
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    		ld 	hl,1279
    		sbc 	hl,de
    		jr 	nc,skip2
    		inc 	bc
    		or 	a
    skip2:		ld 	hl,127
    		sbc 	hl,de
    		jr 	c,skip3
    		inc 	bc
    		inc 	bc
    skip3:		pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    		sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    		ldir
    		pop 	hl
    ap_finishup:	ld 	a,1
    		ld 	(lwm),a
    		jp 	aploop
    
    ap_getbit:	ld	a,(ap_byte)
    		and	a
    		rl	a
    		ld	(ap_byte),a
    		ret	nz
    		ld	a,(hl)
    		inc	hl
    		rla
    ap_endit:	ld	(ap_byte),a
    		ret
    
    ap_getbitbc: 	sla 	c		;doubles BC and adds the read bit
    		rl 	b
    		call 	ap_getbit
    		ret 	nc
    		inc 	c
    		ret
    
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbitbc
    		call 	ap_getbit
    		jr 	c,ap_getgammaloop
    		ret
    		
    ; required data
    ap_byte: 	defb 	0
    lwm:		defb 	0
    r0:		defw 	0
    

    and using the index registers instead of variables:
    ; aPPack decompressor 
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ld	ixl,128
    apbranch1:	ldi
    		ld	ixh,0
    aploop:		call 	ap_getbit
    		jp 	nc,apbranch1
    		call 	ap_getbit
    		jp 	nc,apbranch2
    		call 	ap_getbit
    		jp 	nc,apbranch3
    
    		ld 	bc,0		;get an offset
    		ld	ixh,c		;LWM = 0
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		call 	ap_getbitbc
    		ld 	a,b
    		or 	c
    		jr 	nz,apbranch4
    		ld 	(de),a		;write a 0
    		inc 	de
    		jr	aploop
    apbranch4:	ex 	de,hl 		;write a previous bit (1-15 away from dest)
    		sbc 	hl,bc
    		ld 	a,(hl)
    		add	hl,bc
    		ld 	(hl),a
    		inc 	hl
    		ex 	de,hl
    		jr	aploop
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl		
    		rr 	c
    		ret 	z		;if a zero is encountered here, it's EOF
    		ld 	b,2
    		jr 	nc,ap_dont_inc_b
    		inc 	b
    		or	a
    ap_dont_inc_b:	ld	ixh,1
    		push 	hl
    		ld 	a,b
    		ld 	b,0
    		ld	iyh,b
    		ld	iyl,c
    		ld 	h,d
    		ld 	l,e
    		sbc 	hl,bc
    		ld 	c,a
    		ldir
    		pop 	hl
    		jr	aploop
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec 	bc
    		dec 	bc
    		ld	a,ixh
    		or	a
    		jr	nz,ap_not_lwm
    		ld 	a,b			;bc = 2?
    		or 	c
    		jr 	nz,ap_not_zero_gamma	;if gamma code is 2, use old r0 offset,
    		call 	ap_getgamma		;and a new gamma code for length
    		push 	hl
    		ld 	h,d
    		ld 	l,e
    		push 	bc
    		ld	b,iyh
    		ld	c,iyl
    		sbc 	hl,bc
    		pop 	bc
    		ldir
    		pop 	hl
    		jr 	ap_finishup
    ap_not_zero_gamma:
    		dec bc
    ap_not_lwm:
    		;do I even need this code?
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,c
    		ld 	c,(hl)
    		inc 	hl
    		ld	iyh,b
    		ld	iyl,c
    		push 	bc
    		call 	ap_getgamma
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    		ld 	hl,1279
    		sbc 	hl,de
    		jr 	nc,skip2
    		inc 	bc
    		or 	a
    skip2:		ld 	hl,127
    		sbc 	hl,de
    		jr 	c,skip3
    		inc 	bc
    		inc 	bc
    skip3:		pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    		sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    		ldir
    		pop 	hl
    ap_finishup:	ld	ixh,1
    		jp 	aploop
    
    ap_getbit:	ld	a,ixl
    		and	a
    		rl	a
    		ld	ixl,a
    		ret	nz
    		ld	a,(hl)
    		inc	hl
    		rla
    ap_endit:	ld	ixl,a
    		ret
    
    ap_getbitbc: 	sla 	c		;doubles BC and adds the read bit
    		rl 	b
    		call 	ap_getbit
    		ret 	nc
    		inc	c
    		ret
    
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbitbc
    		call 	ap_getbit
    		jr 	c,ap_getgammaloop
    		ret
    
  • edited October 2007
    Very nice and useful.
    I find apLib to give the best compression usable on Spectrum, and I did test a lot of packers. Sometimes PUCrunch gives a slightly better compression ration, but the Z80 unpacker I found is bigger (around 300 bytes) and it's not worth it.
    I did some modifications of Maxim's version of the depacker (http://www.smspower.org/maxim/smssoftware/aplib.html) aiming to decompress SCREEN$s directly into video memory, data being ordered by columns. I used 2 approaches: 1) for each depacked byte, calculate the SCREEN$ line/column by dividing the offset by 192, and it was slow; 2) find references of already written data using an incremental method of calculating the SCREEN$ address ordered by columns, and it was even slower, because the offset can be anything from 1 to $ffff. So the fastest approach seems to be using an extra temporary buffer (that may overlap the compressed data) to decompress the SCREEN$ and after depacking, reorder data back from column based to Spectrum video memory.
    I plan to write a cross-compressor of SCREEN$ and data, to minimize the storage requirements and increase loading speed of Spectrum programs loading from tape and disk. So this is very interesting for me.
    Cheers!
  • edited November 2007
    Even more optimized
    ; aPPack decompressor 
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ld	ixl,128
    apbranch1:	ldi
    		ld	ixh,0
    aploop:		call 	ap_getbit
    		jr 	nc,apbranch1
    		call 	ap_getbit
    		jr 	nc,apbranch2
    		call 	ap_getbit
    		jr 	nc,apbranch3
    
    		ld	bc,16		;get an offset
    		ld	ixh,b		;LWM = 0
    get4bits:	call 	ap_getbit
    		rl 	c
    		jr	nc,get4bits
    		jr 	nz,apbranch4
    		ld 	a,b
    		ld 	(de),a		;write a 0
    		inc 	de
    		jp	aploop
    apbranch4:	and	a
    		ex 	de,hl 		;write a previous byte (1-15 away from dest)
    		sbc 	hl,bc
    		ld 	a,(hl)
    		add	hl,bc
    		ld 	(hl),a
    		inc 	hl
    		ex 	de,hl
    		jp	aploop
    
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl		
    		rr 	c
    		ret 	z		;if a zero is encountered here, it's EOF
    		ld 	b,2
    		jr 	nc,ap_dont_inc_b
    		inc 	b
    		or	a
    ap_dont_inc_b:	ld	ixh,1
    		push 	hl
    		ld 	a,b
    		ld 	b,0
    		ld	iyh,b
    		ld	iyl,c
    		ld 	h,d
    		ld 	l,e
    		sbc 	hl,bc
    		ld 	c,a
    		ldir
    		pop 	hl
    		jp	aploop
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec 	c
    		ld	a,ixh
    		or	a
    		jr	nz,ap_not_zero_gamma
    		dec	c
    		jr 	nz,ap_not_zero_gamma	;if gamma code is 2, use old r0 offset,
    		call 	ap_getgamma		;and a new gamma code for length
    		push 	hl
    		ld 	h,d
    		ld 	l,e
    		push 	bc
    		ld	b,iyh
    		ld	c,iyl
    		sbc 	hl,bc
    		pop 	bc
    		jp 	ap_finishup
    
    ap_not_zero_gamma:
    		dec 	c
    		;do I even need this code?
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,c
    		ld 	c,(hl)
    		inc 	hl
    		ld	iyh,b
    		ld	iyl,c
    		push 	bc
    		call 	ap_getgamma
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    		ld	a,4
    		cp	d
    		jr 	nc,skip2
    		inc 	bc
    		or	a
    skip2:		ld 	hl,127
    		sbc 	hl,de
    		jr 	c,skip3
    		inc 	bc
    		inc 	bc
    skip3:		pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    		sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    ap_finishup:	ldir
    		pop 	hl
    		ld	ixh,1
    		jp 	aploop
    
    ap_getbit:	ld	a,ixl
    		add	a,a
    		ld	ixl,a
    		ret	nz
    		ld	a,(hl)
    		inc	hl
    		rla
    		ld	ixl,a
    		ret
    
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbit
    		rl 	c
    		rl 	b
    		call 	ap_getbit
    		jr 	c,ap_getgammaloop
    		ret
    
  • edited November 2007
    Arda wrote: »
    fc_35_td.tap link is broken. Anyone has this file?

    All versions of File Compressor, 1 tap file:

    http://user.sezampro.yu/~mstancevic/File_Compressor/FC.tap.zip

    File Compressor v3.50 is first on the tap file

    LOAD "" CODE 16384 : RANDOMIZE USR 18432
  • edited December 2007
    My last versions:

    1 - Speed optimized, 186 bytes:
    ; aPPack decompressor 
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ld	ixl,128
    apbranch1:	ldi
    aploop2:	ld	ixh,1
    aploop:		call 	ap_getbit
    		jr 	nc,apbranch1
    		call 	ap_getbit
    		jr 	nc,apbranch2
    		call 	ap_getbit
    		jr 	nc,apbranch3
    
    		ld	bc,16		;get an offset
    apget4bits:	call 	ap_getbit
    		rl 	c
    		jr	nc,apget4bits
    		jr 	nz,apbranch4
    		ld 	a,b
    		ld 	(de),a		;write a 0
    		inc 	de
    		jp	aploop2
    apbranch4:	and	a
    		ex 	de,hl 		;write a previous byte (1-15 away from dest)
    		sbc 	hl,bc
    		ld 	a,(hl)
    		add	hl,bc
    		ex 	de,hl
    		ld 	(de),a
    		inc 	de
    		jp	aploop2
    
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl
    		rr 	c
    		ret 	z		;if a zero is encountered here, it's EOF
    		ld	a,2
    		ld 	b,0
    		adc	a,b
    		ld	ixh,0
    		push 	hl
    		ld	iyh,b
    		ld	iyl,c
    		ld 	h,d
    		ld 	l,e
    		sbc 	hl,bc
    		ld 	c,a
    		ldir
    		pop 	hl
    		jp	aploop
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec	c
    		ld	a,c
    		sub	ixh
    		jr	z,ap_r0_gamma
    		dec	a
    
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,a
    		ld 	c,(hl)
    		inc 	hl
    		ld	iyh,b
    		ld	iyl,c
    
    		push 	bc
    		
    		call 	ap_getgamma
    
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    
    		ld	a,4
    		cp	d
    		jr 	nc,apskip2
    		inc 	bc
    		or	a
    apskip2:	ld 	hl,127
    		sbc 	hl,de
    		jr 	c,apskip3
    		inc 	bc
    		inc 	bc
    apskip3:	pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    		sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    		ldir
    		pop 	hl
    		ld	ixh,b
    		jp 	aploop
    
    ap_r0_gamma:	call 	ap_getgamma		;and a new gamma code for length
    		push 	hl
    		push 	de
    		ex	de,hl
    
    		ld	d,iyh
    		ld	e,iyl
    		sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    		ldir
    		pop 	hl
    		ld	ixh,b
    		jp 	aploop
    
    ap_getbit:	ld	a,ixl
    		add	a,a
    		ld	ixl,a
    		ret	nz
    		ld	a,(hl)
    		inc	hl
    		rla
    		ld	ixl,a
    		ret
    
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbit
    		rl 	c
    		rl 	b
    		call 	ap_getbit
    		jr 	c,ap_getgammaloop
    		ret
    

    2 - Size optimized, 163 bytes
    ; aPPack decompressor
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ld	ixl,128
    apbranch1:	ldi
    aploop0:	ld	ixh,1		;LWM = 0
    aploop:		call 	ap_getbit
    		jr 	nc,apbranch1
    		call 	ap_getbit
    		jr 	nc,apbranch2
    		ld 	b,0
    		call 	ap_getbit
    		jr 	nc,apbranch3
    		ld	c,16		;get an offset
    apget4bits:	call 	ap_getbit
    		rl 	c
    		jr	nc,apget4bits
    		jr 	nz,apbranch4
    		ld 	a,b
    apwritebyte:	ld 	(de),a		;write a 0
    		inc 	de
    		jr	aploop0
    apbranch4:	and	a
    		ex 	de,hl 		;write a previous byte (1-15 away from dest)
    		sbc 	hl,bc
    		ld 	a,(hl)
    		add	hl,bc
    		ex 	de,hl
    		jr	apwritebyte
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl
    		rr 	c
    		ret 	z		;if a zero is encountered here, it's EOF
    		ld	a,2
    		adc	a,b
    		push 	hl
    		ld	iyh,b
    		ld	iyl,c
    		ld 	h,d
    		ld 	l,e
    		sbc 	hl,bc
    		ld 	c,a
    		jr	ap_finishup2
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec 	c
    		ld	a,c
    		sub	ixh
    		jr 	z,ap_r0_gamma		;if gamma code is 2, use old r0 offset,
    		dec 	a
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,a
    		ld 	c,(hl)
    		inc 	hl
    		ld	iyh,b
    		ld	iyl,c
    
    		push 	bc
    		
    		call 	ap_getgamma
    
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    
    		ld	a,4
    		cp	d
    		jr 	nc,apskip2
    		inc 	bc
    		or	a
    apskip2:	ld 	hl,127
    		sbc 	hl,de
    		jr 	c,apskip3
    		inc 	bc
    		inc 	bc
    apskip3:	pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    ap_finishup:	sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    ap_finishup2:	ldir
    		pop 	hl
    		ld	ixh,b
    		jr 	aploop
    
    ap_r0_gamma:	call 	ap_getgamma		;and a new gamma code for length
    		push 	hl
    		push 	de
    		ex	de,hl
    
    		ld	d,iyh
    		ld	e,iyl
    		jr 	ap_finishup
    
    
    ap_getbit:	ld	a,ixl
    		add	a,a
    		ld	ixl,a
    		ret	nz
    		ld	a,(hl)
    		inc	hl
    		rla
    		ld	ixl,a
    		ret
    
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbit
    		rl 	c
    		rl 	b
    		call 	ap_getbit
    		jr 	c,ap_getgammaloop
    		ret
    
  • edited November 2012
    Even faster & smaller:

    1 - Speed optimized, 185 bytes:
    ; aPPack decompressor 
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ld	a,128
    apbranch1:	ldi
    aploop2:	ld	ixh,1
    aploop:		call 	ap_getbit
    		jr 	nc,apbranch1
    		call 	ap_getbit
    		jr 	nc,apbranch2
    		call 	ap_getbit
    		jr 	nc,apbranch3
    
    		ld	bc,16		;get an offset
    apget4bits:	call 	ap_getbit
    		rl 	c
    		jr	nc,apget4bits
    		jr 	nz,apbranch4
    		ex	de,hl
    		ld 	(hl),b		;write a 0
    		ex	de,hl
    		inc 	de
    		jp	aploop2
    apbranch4:	ex	af,af'
    		ex 	de,hl 		;write a previous byte (1-15 away from dest)
    		sbc 	hl,bc
    		ld 	a,(hl)
    		add	hl,bc
    		ld 	(hl),a
    		ex	af,af'
    		ex 	de,hl
    		inc 	de
    		jp	aploop2
    
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl
    		ex	af,af'
    		rr 	c
    		ret 	z		;if a zero is found here, it's EOF
    		ld	a,2
    		ld 	b,0
    		adc	a,b
    		push 	hl
    		ld	iyh,b
    		ld	iyl,c
    		ld 	h,d
    		ld 	l,e
    		sbc 	hl,bc
    		ld 	c,a
    		ex	af,af'
    		ldir
    		pop 	hl
    		ld	ixh,b
    		jp	aploop
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec	c
    		ex	af,af'
    		ld	a,c
    		sub	ixh
    		jr	z,ap_r0_gamma
    		dec	a
    
    		;do I even need this code?
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,a
    		ld 	c,(hl)
    		inc 	hl
    		ld	iyh,b
    		ld	iyl,c
    
    		push 	bc
    
    		call 	ap_getgamma2
    
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    
    		ex	af,af'
    		ld	a,4
    		cp	d
    		jr 	nc,apskip2
    		inc 	bc
    		or	a
    apskip2:	ld 	hl,127
    		sbc 	hl,de
    		jr 	c,apskip3
    		inc 	bc
    		inc 	bc
    apskip3:	pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    		sbc 	hl,de
    		ex	af,af'
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    		ldir
    		pop 	hl
    		ld	ixh,b
    		jp 	aploop
    
    ap_r0_gamma:	call 	ap_getgamma2	;and a new gamma code for length
    		push 	hl
    		push 	de
    		ex	de,hl
    
    		ld	d,iyh
    		ld	e,iyl
    		sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    		ldir
    		pop 	hl
    		ld	ixh,b
    		jp 	aploop
    
    ap_getgamma2:	ex	af,af'
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbit
    		rl 	c
    		rl 	b
    		call 	ap_getbit
    		jr 	c,ap_getgammaloop
    		ret
    
    ap_getbit:	add	a,a
    		ret	nz
    		ld	a,(hl)
    		inc	hl
    		rla
    		ret
    

    2 - Size optimized: 156 bytes:
    ; aPPack decompressor
    ; original source by dwedit
    ; very slightly adapted by utopian
    ; optimized by Metalbrain
    
    ;hl = source
    ;de = dest
    
    depack:		ld	ixl,128
    apbranch1:	ldi
    aploop0:	ld	ixh,1		;LWM = 0
    aploop:		call 	ap_getbit
    		jr 	nc,apbranch1
    		call 	ap_getbit2
    		jr 	nc,apbranch2
    		ld 	bc,16
    		call 	ap_getbit2
    		jr 	nc,apbranch3
    apget4bits:	call 	ap_getbit2
    		rl 	c
    		jr	nc,apget4bits
    		ld 	a,b
    		jr 	z,apwritebyte
    		and	a
    		ex 	de,hl 		;write a previous byte (1-15 away from dest)
    		sbc 	hl,bc
    		ld 	a,(hl)
    		add	hl,bc
    		ex 	de,hl
    apwritebyte:	ld 	(de),a		;write a 0
    		inc 	de
    		jr	aploop0
    apbranch3:	ld 	c,(hl)		;use 7 bit offset, length = 2 or 3
    		inc 	hl
    		rr 	c
    		ret 	z		;if a zero is encountered here, it's EOF
    		ld	a,2
    		adc	a,b
    		push 	hl
    		push	bc
    		pop	iy
    		ld 	h,d
    		ld 	l,e
    		sbc 	hl,bc
    		ld 	c,a
    		jr	ap_finishup2
    apbranch2:	call 	ap_getgamma	;use a gamma code * 256 for offset, another gamma code for length
    		dec 	c
    		ld	a,c
    		sub	ixh
    		jr 	z,ap_r0_gamma		;if gamma code is 2, use old r0 offset,
    		dec 	a
    		;do I even need this code?
    		;bc=bc*256+(hl), lazy 16bit way
    		ld 	b,a
    		ld 	c,(hl)
    		inc 	hl
    		push	bc
    		pop	iy
    
    		push 	bc
    		
    		call 	ap_getgamma
    
    		ex 	(sp),hl		;bc = len, hl=offs
    		push 	de
    		ex 	de,hl
    
    		ld	a,4
    		cp	d
    		jr 	nc,apskip2
    		inc 	bc
    		or	a
    apskip2:	ld 	hl,127
    		sbc 	hl,de
    		jr 	c,apskip3
    		inc 	bc
    		inc 	bc
    apskip3:	pop 	hl		;bc = len, de = offs, hl=junk
    		push 	hl
    		or 	a
    ap_finishup:	sbc 	hl,de
    		pop 	de		;hl=dest-offs, bc=len, de = dest
    ap_finishup2:	ldir
    		pop 	hl
    		ld	ixh,b
    		jr 	aploop
    
    ap_r0_gamma:	call 	ap_getgamma		;and a new gamma code for length
    		push 	hl
    		push 	de
    		ex	de,hl
    
    		push	iy
    		pop	de
    		jr 	ap_finishup
    
    ap_getbit:	ld	a,ixl
    ap_getbit2:	add	a,a
    		jr	nz,ap_endbit
    		ld	a,(hl)
    		inc	hl
    		rla
    ap_endbit:	ld	ixl,a
    		ret
    
    ap_getgamma:	ld 	bc,1
    ap_getgammaloop:call 	ap_getbit
    		rl 	c
    		rl 	b
    		call 	ap_getbit2
    		jr 	c,ap_getgammaloop
    		ret
    
  • edited November 2012
    Given the thread title, it would be nice to have some links to compressors which actually run on a Spectrum, as the aplib compression routines don't, and the links herein to other programs are all old & broken. I've done some searches on this topic before, and haven't found anything which uses LZW or similar and includes both pack & unpack and is written in Z80 assembler and actually works (or at least that I could work out how to get it to work). As a start, here's one which I wrote a while ago called WIP-LZ77, based upon LZW, but stra-a-angely different. There's lots of comments in the assembler listing and a separate program notes file which I've just added. The pack & unpack are all in one at the moment, as I never did get around to separating them.
  • edited November 2012
    I've always wanted one that can decompress in stages, or can decompress smaller files from a volume, but I have no skills in compression.
    Joefish
    - IONIAN-GAMES.com -
  • edited November 2012
    Given the thread title, it would be nice to have some links to compressors which actually run on a Spectrum

    If you don't mind using TR-DOS, take your pick from this list: http://vtrdos.ru/system.php#s5 - Hrust and Hrum are the ones I've used to much success in the past (and still do semi-regularly, via my command-line port Chrust... it's been beaten in compression ratio by MegaLZ / Exomizer these days, but it Just Works and saves me from having to remember what sorts of in-place compression you're able to do on the others).

    Way back before internets and emulators, I used Turbo Imploder, but later moved on to a couple of others that had better compression - Tompacker and PKLITE, I think - but those don't appear to be on WOS yet. (I'm now back in possession of all my old +D disks - need to do an intensive archiving session some time soon...)
  • LCDLCD
    edited November 2012
    Given the thread title, it would be nice to have some links to compressors which actually run on a Spectrum, as the aplib compression routines don't, and the links herein to other programs are all old & broken.

    http://tcg.speccy.cz/zoom/soft.html

    I wish there would be a packer similar to PkLite or Turbo Imploder 2 (okay, the last one is dangerous as the unpacked file can be different from original, so I prefer PKLite with its multiple options), but as PC console program, where a program at 25000 can be packed and run from 25000 without any skew. With a unpacking routine that can be put anywhere in memory.
  • edited November 2012
    OK, for a first go I just gave the aPPack executable and Metalbrain's speed-optimised decompressor a go. A simple 'include' to add in his code and an 'incbin' to include the file I'd pre-compressed and it worked great. This could well come in useful in my coding. Though I'd prefer a nice Windows interface to drive the packer, so may have to add one.

    Now for a level-based game, I can see myself unpacking up to 6K of data to a black screen, copying out a particular level's data from that into a buffer, then clearing the screen again. Maybe the same again for the sprite data, just picking off the objects I need from a large set.

    For a scrolling game it would be nice to be able to stream data out of a packed set. I realise this would mean holding on to a buffer of the data as it comes out so that back-references are possible, but I'm guessing this compression method can make back-references all the way back to the beginning of a block. Is there any way to limit that? I also struggle to see (looking at the optimised code) where I could break the routine and save its state to pause / resume after nn bytes have been unpacked.
    Joefish
    - IONIAN-GAMES.com -
  • edited November 2012
    joefish wrote: »
    but I'm guessing this compression method can make back-references all the way back to the beginning of a block. Is there any way to limit that?
    LZ refers to data history. while compressors with best result easily can refer even 32K or so back, simpler compressors (like turbo imploder) refers about 2K - 4K back. for your purposes, custom LZ with quite small maximal offset would be better then general purpose compressors.
    I also struggle to see (looking at the optimised code) where I could break the routine and save its state to pause / resume after nn bytes have been unpacked.
    Generally every time after token (literal / reference) is proceeded but internal state of compresor and history buffer has to be saved.
Sign In or Register to comment.