An appeal to preserve InfoseekID compatibility!

Instead of worrying about competition from other Spectrum sites, Martijn always supported them. He actively worked together with other sites to help improve them and integrate them with WoS, because he knew all sites combined could provide much better results than any individual site alone. If you look at current WoS pages, you will find links to other Spectrum sites like Tipshop, Spectrum 2.0, Preservation Team Shop, Crash Online, YSRnRY and even non-Spectrum sites like Wikipedia, Lost in Translation, Lemon64, Generation MSX, etc. This "spirit" of collaboration is probably one of the main reasons WoS became the center of the strongest, most productive retro comunity.

Nowadays, almost all Spectrum sites adopt the same InfoseekID from WoS, which is a number assigned to each Spectrum program, book, etc. This "standard" has been very important to ensure all these sites remain integrated and can provide links to each other easily. For instance, the InfoseekID for "R-Type" is 0004256, therefore the website address for this game at WoS is:

http://www.worldofspectrum.org/infoseekid.cgi?id=0004256

Tipshop is:

http://www.the-tipshop.co.uk/cgi-bin/info.pl?wosid=0004256

Spectrum 2.0 is:

http://spectrum20.org/games/4256

ZXInfo is:

http://sinclair.kolbeck.dk/#!/details/0004256

Spectrum Computing is:

http://spectrumcomputing.co.uk/index.php?cat=96&id=4256

ZXSR is:

http://zxspectrumreviews.co.uk/search.aspx?wosid=0004256

Speccy Screenshot Maps is:

http://maps.speccy.cz/index.php?wosid=0004256

For about 2 decades, the old WoS archive catalogued almost 27,000 items, assigning an InfoseekID number to each one. The ZXDB project preserved these InfoseekIDs, and we have now even more Spectrum sites sharing these InfoseekIDs than a year ago. However, we all know that the WoS archive stopped receiving updates about 4 years ago, since Martijn left. To fill this void, the ZXDB project has been working for a year to catalogue more titles, assigning new InfoseekIDs to over 2,100 additional items. These new InfoseekIDs include latest releases from recent years, recently discovered old titles not previously catalogued in WoS, and even "lost" items recovered directly from internal WoS files that didn't have an InfoseekID yet. Of course all InfoseekIDs previously assigned by WoS have been preserved in ZXDB too, to ensure full compatibility between them.

It's in the best interest of the Spectrum community to ensure that all sites will keep sharing the same InfoseekID numbers. If websites start breaking compatibility and adopt different InfoseekID numbers for new items, we would be screwing up about a decade of effort to bring Spectrum sites close together...

To prevent this kind of problem in the future, here's my proposal:

1.) From now on, whenever there's a new ZXDB version, I will also provide an updated list of all new InfoseekIDs that didn't exist in the old WoS archive. Moreover, I'm providing this list of new items in exactly the same format as the internal files from old WoS archive (more specifically the format from "maindb.dat"), thus Lee can easily import this list using exactly the same tools he's planning to import the internal WoS archives without any additional work. You can download it right now from here (in directory "ZXDB", file "NewEntries_ZXDB.zip" contains the list and "NewEntries_ZXDB.sql" is the script I wrote to generate it).

2.) All websites are welcome to use this list of InfoseekIDs as reference too, so they can already add new content without breaking compatibility with each other, while waiting for the new WoS site. If anybody would like me to generate this list of new InfoseekIDs in a different, more friendly format please contact me!

3.) Moreover, I would like to invite a technical discussion on the best way to assign new InfoseekIDs, shared by WoS, ZXDB and everybody else, after the new WoS site becomes operational.

How does it sound? :)
Post edited by Einar Saukas on
Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
«13

Comments

  • I understand very little of what you proposed, so I can't comment on that but personally, I am all for the exact same thing Mr WoS-Himself was all about, i.e. including information from other sites, and working with others whenever possible.

    Assuming what you are proposing is compatible with WoS, and vise versa, then yes of course, DOO EEET!!!
    What now?
  • Well it all sounds perfectly reasonable. If we have that consistency, what would be anyone's motivation for breaking it? They all need new ID numbers as they add entries, so I can't see why anyone would favour choosing their own over using the ZXDB one.

    The question appears to be, are the admins of the other sites on board? If they are, this should happen fairly naturally. A webpage where the list of new ID appears in a table for easy lookup might be handy.
  • It's not even an ZXDB system,it's WoS as started by Martijn, anybody would be a fool to change it now.
    I wanna tell you a story 'bout a woman I know...
  • As already stated several times, the original WoS infoseek links will work on the new site. Even the current publishers links on the dev site work.

    However, ZXDB isn't infoseek. The numbering doesn't follow, and therefore it will need to be amended when the new infoseek ID's are assigned.
    My test signature
  • I'm pretty sure he is specifically referring to the NEW stuff, that he is keeping track of, once it's time to integrate it into the WoS database.
    Website: Tardis Remakes / Mostly remakes of Arcade and ZX Spectrum games.
    My games for the Spectrum: Dingo, The Speccies, The Speccies 2, Vallation, SQIJ.
    Twitter: Sokurah
  • His database isn't being integrated into WoS. We had that discussion. ZXDB IS NOT WOS. I'm getting to the stage now whereby I am considering banning ZXDB from WoS. Spectrum Computing are capable of setting up a forum so you can all go discuss as much as you like.
    My test signature
  • What the hell IS ZXDB? I assume its a database and I also assume that its probably incompatible with whatever this site uses. So why the axe grinding? Seems like a no-brainer to stick with the current system but perhaps there are reasons I am ignorant of that make Einar Saukas so persistent in insisting that his way is better?

    Someone enlighten me please, just to satisfy my curiousity and correct my misperceptions?
  • So much has been written about it already - and it's not a matter of one thing being better than the other.

    It's basically got something to do with the WoS database not having been updated for years, and Einar has been maintaining a separate database with the changes that's been happening in the meantime ... with the intent that it could possibly be integrated into the WoS database to bring it up to date in one fell swoop, but for some reason TPTB doesn't want that.
    Website: Tardis Remakes / Mostly remakes of Arcade and ZX Spectrum games.
    My games for the Spectrum: Dingo, The Speccies, The Speccies 2, Vallation, SQIJ.
    Twitter: Sokurah
    Thanked by 1mik3d3nch
  • mik3d3nch wrote: »
    What the hell IS ZXDB? I assume its a database and I also assume that its probably incompatible with whatever this site uses. So why the axe grinding? Seems like a no-brainer to stick with the current system but perhaps there are reasons I am ignorant of that make Einar Saukas so persistent in insisting that his way is better?

    Someone enlighten me please, just to satisfy my curiousity and correct my misperceptions?
    Yes, ZXDB is a database that Einar created about a year ago to try and get some sort of maintained database available that anyone could use. It was started because the new WoS database wasn't available and until Lee releases that into the wild it's the most current and up-to-date Speccy database available. Einar is continually maintaining it - adding tables and new rows. He has also done a huge amount of work fixing things - removing dead/duplicated rows, etc. He's done a magnificent job with it.

    There was talk in a thread where Einar was asking Lee if ZXDB could be incorporated or somehow the updated data used within the new WoS database. Lee declined which is absolutely his prerogative. As a database developer myself I can fully understand this when a huge amount of work has been put into the new WoS database. Personally I've yet to import ZXDB into the ZXSR database and that's only as a read-only database not a fully administrable one that WoS's has to be. But then I'm a lazy sod...

    Personally I hope there's no more talk of banning, albeit ZXDB rather than a user, as it's a useful resource even if I'm not using it (yet, just don't hold your breath) or particularly vocal about it. Eventually I hope, once it's live, the WoS database feeds into ZXDB anyway. That way we have one centrally updated resource and another that's a ready-made database for it rather than fannying around with APIs or export files.
  • Perhaps I didn't make myself clear...

    I'm not proposing to integrate ZXDB into new WoS. I'm just proposing something very simple:

    * Old WoS archive had about 27,000 programs, books, etc. Each one has a number assigned by Martijn. The same numbers are used in several sites, including ZXDB and new WoS. This is obviously not going to change (thus this is not the point I'm worried).

    * In the last year, many people helped to add another 2,100 programs, books, etc. These items didn't exist in old WoS archive. They are new releases, recently recovered titles, etc. Each one has a number assigned in ZXDB. These numbers are already used in at least 4 sites. When it's time to add the same items to new WoS, why not keep the same numbers (to maintain compatibility with sites already using them) instead of assigning new numbers?

    * Technically, new WoS site will import a file called "maindb.dat" from old WoS to obtain the first 27,000 numbers and some information about them. In order to also obtain the new 2,100 numbers, new WoS site will just need to import also a new file called "NewEntries_ZXDB.dat". It has exactly the same format as "maindb.dat", thus no extra work will be needed to use it. Whatever method works for "maindb.dat" will also work for "NewEntries_ZXDB.dat". Right now this file only contains program title, genre, publisher and InfoseekID (thus any relevant information about these games can be added later through new WoS site interface regardless of ZXDB), but I will gladly modify this file as requested.

    Again, why not? There's nothing to gain from assigning new numbers, only disadvantages for everybody involved...
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • mik3d3nch wrote: »
    What the hell IS ZXDB? I assume its a database and I also assume that its probably incompatible with whatever this site uses. So why the axe grinding? Seems like a no-brainer to stick with the current system but perhaps there are reasons I am ignorant of that make Einar Saukas so persistent in insisting that his way is better?

    No, I'm not arguing if my way is better, or even questioning if new WoS site should use ZXDB. I'm just pleading new WoS site to adopt the same already assigned numbers for items that didn't exist in old WoS. And making sure it can be done without any effort, by making these new numbers available in a file format identical to old WoS. That's all!
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • And I've already said no because you've created a different numbering system rather than follow on with increments. Stop trying to push this on me.
    My test signature
  • Sokurah wrote: »
    I'm pretty sure he is specifically referring to the NEW stuff, that he is keeping track of, once it's time to integrate it into the WoS database.

    Exactly.
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • Perhaps I didn't make myself clear...

    * In the last year, many people helped to add another 2,100 programs, books, etc. These items didn't exist in old WoS archive. They are new releases, recently recovered titles, etc. Each one has a number assigned in ZXDB. These numbers are already used in at least 4 sites. When it's time to add the same items to new WoS, why not keep the same numbers (to maintain compatibility with sites already using them) instead of assigning new numbers?
    Well, that IS exactly what I was thinking should happen.

    And I've already said no because you've created a different numbering system rather than follow on with increments. Stop trying to push this on me.
    See, I didn't know this (perhaps I haven't been following the conversation that closely). Now, would you be interrested in importing Einar's data IF he changed his numbering system and all the other sites using ZXDB did the same?
    Website: Tardis Remakes / Mostly remakes of Arcade and ZX Spectrum games.
    My games for the Spectrum: Dingo, The Speccies, The Speccies 2, Vallation, SQIJ.
    Twitter: Sokurah
  • ... I should mention that I have NO stakes in ZXDB and haven't contributed to it, but I just think it would be practical to use an available resource instead of having people have to add all the same data to the WoS database manually.
    Website: Tardis Remakes / Mostly remakes of Arcade and ZX Spectrum games.
    My games for the Spectrum: Dingo, The Speccies, The Speccies 2, Vallation, SQIJ.
    Twitter: Sokurah
  • If anybody would like me to generate this list of new InfoseekIDs in a different, more friendly format please contact me!

    I just looked at the data file. I can't see what the columns are. The first entry is:
    hardware    sinclairresearchltd    Hardware    Sinclair Research Ltd    28201
    

    What are those 5 columns? I'm trying to see why Lee thinks you've created a different numbering system?
  • Sokurah wrote: »
    See, I didn't know this (perhaps I haven't been following the conversation that closely). Now, would you be interrested in importing Einar's data IF he changed his numbering system and all the other sites using ZXDB did the same?

    It can be revisited yes, but it should be pointed out that is one of many differences in the data and database structure that I keep being pushed into agreeing to without any consideration that new WoS has an extensive admin system written to allow multiple admins access to the data without having to manually edit SQL files (as ZXDB is). Nobody has any need to edit the database directly. The scripts also have a lot of checks in place.

    Add to that the new data WoS is adding, language translations, etc and you have a very different database to ZXDB.
    My test signature
  • Add to that the new data WoS is adding, language translations, etc and you have a very different database to ZXDB.

    Will the new site continue to generate new infoseek IDs for things that are added to the database or not? If so it's desirable that WoS and other sites don't assign different IDs, If not the whole discussion seems to be basically irrelevant.

    My rubbish website including the redrawn Amstrad schematics and the new home of the Sinclair FAQ wiki.
  • So this thread is not about preserving infoseek id's, but instead a request to skip it altogether and push another system on Infoseek?

    I haven't thought about it, but having another new numbering system for newer games seems sound. I should start one too.
  • standards.png
    :))
    Website: Tardis Remakes / Mostly remakes of Arcade and ZX Spectrum games.
    My games for the Spectrum: Dingo, The Speccies, The Speccies 2, Vallation, SQIJ.
    Twitter: Sokurah
  • Yes. That is my fear.
    I wanna tell you a story 'bout a woman I know...
  • Timmy wrote: »
    So this thread is not about preserving infoseek id's, but instead a request to skip it altogether and push another system on Infoseek?

    No, it's not!

    This thread is just about preserving InfoseekIDs.

    The old InfoseekIDs assigned in old WoS archive were already preserved in ZXDB. Now I'm requesting new InfoseekIDs assigned in ZXDB to be also preserved in new WoS (and providing a technical solution so it can be done with zero effort).
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • I just looked at the data file. I can't see what the columns are. The first entry is:
    hardware    sinclairresearchltd    Hardware    Sinclair Research Ltd    28201
    

    What are those 5 columns?

    They are:
    1st column: a keyword automatically generated from program/book title.
    2nd column: a keyword automatically generated from publisher(s).
    3rd column: Program/book title.
    4th column: Publisher(s).
    5th column: InfoseekID number.
    

    This is the exact format used in internal file "maindb.dat" (from old WoS site), that Lee plans to import into new WoS site.

    I'm trying to see why Lee thinks you've created a different numbering system?

    I didn't create a different numbering system. I have used exactly the same system.
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • guesser wrote: »

    Will the new site continue to generate new infoseek IDs for things that are added to the database or not? If so it's desirable that WoS and other sites don't assign different IDs, If not the whole discussion seems to be basically irrelevant.

    Yes. We are working hard to ensure all old links work as much as possible.

    worldofspectrum.org/infoseekpub.cgi?regexp=^Ocean+Software+Ltd$

    live.worldofspectrum.org/infoseekpub.cgi?regexp=^Ocean+Software+Ltd$
    The old InfoseekIDs assigned in old WoS archive were already preserved in ZXDB. Now I'm requesting new InfoseekIDs assigned in ZXDB to be also preserved in new WoS (and providing a technical solution so it can be done with zero effort).

    And I've said no because it's not zero effort to then maintain a custom numbering system when the one that's been there 20+yrs is fine and implemented in the dev scripts, and accounted for in the admin.
    My test signature
  • Yes. We are working hard to ensure all old links work as much as possible.
    That's not what I'm asking, and you and Einar are talking past each other. Old links to things that are already in the WoS database are clearly not the issue here.
    My rubbish website including the redrawn Amstrad schematics and the new home of the Sinclair FAQ wiki.
  • And I've already said no

    No, you are mistaken.

    You never "already said no". You just said it was unlikely and gave a technical reason you thought it would he hard to do. You exact words (quoted from here) were "I doubt infoseek will continue the numbering of ZXDB because the table is set to auto-increment ID's so even entering new titles in the same order will give different Id's."

    Now I gave you a technical solution so it can be done without any effort.

    because you've created a different numbering system rather than follow on with increments. Stop trying to push this on me.

    No, you are mistaken.

    I did not create a different numbering system. I have followed exactly the same system as the old WoS archive.

    The old WoS archive used non-consecutive incremental numbers from 1 to 28187. A new WoS site would have to import old numbers from "maindb.dat", then initialize a SEQUENCE to value 28188 for generating new numbers afterwards.

    ZXDB is using non-consecutive incremental numbers from 1 to 31999 (numbers 1 to 28187 exactly the same from old WoS archive, and numbers 28201 to 31999 added later). A new WoS site would have to import old numbers from both "maindb.dat" and "NewEntries_ZXDB.dat", then initialize a SEQUENCE to value 32000 for generating new numbers afterwards.

    If you still believe it's a different numbering system, please indicate how you think it differs so I can clarify it!
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • guesser wrote: »
    Yes. We are working hard to ensure all old links work as much as possible.
    That's not what I'm asking, and you and Einar are talking past each other. Old links to things that are already in the WoS database are clearly not the issue here.

    The question is whether infoseek links will still work - yes they will. What Einar is trying to push is different and not compatible.
    My test signature
  • And I've said no because it's not zero effort to then maintain a custom numbering system when the one that's been there 20+yrs is fine and implemented in the dev scripts, and accounted for in the admin.

    I'm not asking to maintain a custom numbering system. I see no problem keeping the same numbering system that's been there 20+yrs, or any reason to change any of your scripts. Where did I say otherwise?

    I'm just asking to keep the same InfoseekIDs, for items that already have InfoseekIDs.

    You are talking about not changing your scripts that will generate new InfoseekIDs, for items that don't have InfoseekIDs yet.

    These are 2 completely different things.
    Creator of ZXDB, BIFROST/NIRVANA, ZX7/RCS, etc. I don't frequent this forum anymore, please look for me elsewhere.
  • Timmy wrote: »
    So this thread is not about preserving infoseek id's, but instead a request to skip it altogether and push another system on Infoseek?
    No, it's not!
    Haha, I'm 100% sure he said that for fun. It's a joke man. :))

    guesser wrote: »

    Will the new site continue to generate new infoseek IDs for things that are added to the database or not? If so it's desirable that WoS and other sites don't assign different IDs, If not the whole discussion seems to be basically irrelevant.

    Yes. We are working hard to ensure all old links work as much as possible.
    And they still do (for now anyway, but I'm guessing they will continue to) and that is very much appreciated. I've actually mentioned before, that I think it's great that all links - especially the forum ones are still the same after the change to the new (and unfortunately much shittier) forum software.

    So ... when is that often touted new backend up and running, btw? ;)
    Website: Tardis Remakes / Mostly remakes of Arcade and ZX Spectrum games.
    My games for the Spectrum: Dingo, The Speccies, The Speccies 2, Vallation, SQIJ.
    Twitter: Sokurah
This discussion has been closed.