I just read up on the conversation.
Excuse me while I wrap my head around everything

Norbert wrote:What I have in mind is that I would use apoplexy to create and save custom tiles that will then be understood by MINIMIN as its extended tiles. (It might be best not to pick tile+modifier combo 10+7 for new potions, since Secrets of the Citadel's shadow potion already uses that.)

Actually, I have 7-11 all in use right now. Note that some of these potions have rather "implementation-specific" aspects (for my mod, that is...)
For reference, these are the specific types that I implemented:
7: shadow potion
8: bonus potion / extra time potion (trading in 1 max life point)
9: full health potion (with magenta-colored bubbles, inspired by Prince of Persia 3D's Tears of Malak )
10: custom open potion (similar to the normal open potion, but more versatile and if used to open a gate, that gate stays open. It also has a different appearance, so that it won't be confused with poison)
11: strong poison (This is probably similar to MININIM's big poison potion. But in the end I wanted these to have a slightly different appearance for my mod specifically)

oitofelix wrote:Speaking about behavior descriptions, what is that 10+7 "shadow potion" about?

It is a potion that temporarily turns you into the shadowman. While the effect is active, you can survive falls, pass through gates, and loose tiles won't fall.

Norbert wrote:

oitofelix wrote:[...], in case there was an established universal standard for those additional tiles.

Well, who better to lay the first bricks towards such a standard than you and Falcury?
To quote from Prometheus, "big things have small beginnings".
We could, for instance, start with establishing that a 10+8 combo in LEVELS.DAT refers to a potion that kills instantly?
And, should it always be a big potion with blue bubbles? Things like that.

[Edit: Or perhaps it's time to move away from basic editors altogether. Although it's nice to not have to learn a new editor to work with a new PoP1 implementation.]

oitofelix wrote:In particular, if we are going to make this standard happen, I think the best way to validate it besides documenting it, is to implement it in independent reference projects, like MININIM, SDLPoP and apoplexy (if possible or applicable).

As soon as we agree upon any single tile specification, I'll make those available for use from MININIM's legacy level module. By the way, I think we could start listing the proposed new tiles that are already validated by a current implementation, like SDLPoP or MININIM, including their ids, expected behavior and any additional relevant information. I'm particularly interested in new tiles supported by SDLPoP.

Interesting ideas!

Hey, and it would be interesting to add the hidden floor tile to SDLPoP, perhaps? Maybe it could use another modifier, like the fake tiles?

Personally I think that the potions could be a 'convention' and not a 'standard', per se. I.e. there should be a way to opt out if (for instance) you wanted potion 7 to be something generic, or radically different. For achieve this, there might be a setting in apoplexy to either use (all or parts of) the extended tiles convention, or to disable this and only display legacy tiles.

I know that such 'undefinedness' goes a bit against what we already did with SDLPoP's fake tiles earlier... and maybe the hidden floor as well if we decide to implement that.
Let me try to understand my own thoughts...

Perhaps the difference might be this: so far, practically all of SDLPoP's added functionality is more or less 'directly derived' from the original DOS version. Even the fake tiles, replays and the quicksave mechanism (pretty huge changes) can be seen as new and interesting/improved ways of interacting with just the same game content as it was originally designed. Whereas new potion effects could be more described as actually new 'original content'.

Hm, so I guess this means I think that 'derived' content may be standardized, but 'original' content should be either fully implementation-defined or using an optional convention. What do you think?

oitofelix wrote:Perhaps we should not enter too much on style or graphical details. Those might be implementation dependent, and maybe we should standardize only the fundamental behavior? What do you think? (I think there should be an implicit preference for following reference implementations, though)

I suppose having a reference appearance would mainly be an advantage for having a correct preview image in the editor, right?
For me personally, although it would of course be nice to have, having only generic/abstract editor previews would not really matter. My muscle memory for level editing has been primed on RoomShaker, so I am more used to a more abstract representation of the tiles anyway. (It's fine as long as I remember which tile modifier corresponds to which potion effect, of course...)

Hm, come to think of it, perhaps there could be a 1:1 mapping of generic 'implementation-defined' tiles for the potions? If I remember correctly SDLPoP can support 32 unique potions (5 bits degrees of freedom; the remaining 3 bits in the background/modifier byte are needed for the frames of the bubble animation). So legacy potion ids 7-31 could in theory be mapped between the formats?

oitofelix wrote:we could start listing the proposed new tiles that are already validated by a current implementation

Agreed, I'll post what I know later.

oitofelix wrote:Modifier bit should be let alone for a possible future feature that affects most if not all tiles under the floor umbrella.

Yes, plus we may want to combine the modifier bit with the random bits to get as much out of what we can work with as possible.
If we use random and modifier separately, we have 0-3 (00, 01, 10, 11) and 0-1 (0, 1), which totals 6.
While if we use them together, we have 000, 001, 010, 011, 100, 101, 110, 111, which totals 8.
(Am I calculating this right?...)

oitofelix wrote:Therefore, theoretically it's possible that some level files out there may have them set to values other than 0.

That is true, but almost all will be set to 0, and we should be able to modify those that are not to 0 where necessary.
Most mods are either on this forum or on popot.org (that Total Pack also relies on), all places we have access to.

oitofelix wrote:maybe we should standardize only the fundamental behavior?

Yeah, probably, with more abstract miniature previews.

oitofelix wrote:Congratulation on legbop! That's very interesting and appreciated.

oitofelix wrote:That's fine with me! To demonstrate my commitment in validating the standard, commit 2ca31ad extends the legacy level module to implement the big poison potion as combo 10+8 as you suggested [...]

Your display of commitment is commendable.
A mistake on my part was to provide you with faulty information: as it turns out - I learned later, 10+11 is a better candidate to become a strong poison potion.

Falcury, do you have any thoughts on 1+6 and 1+14? These are floors that appear empty (space). MININIM sets the random bits to 1 to achieve similar functionality. Should 1+6 and 1+14 be replaced, remain for legacy compatibility, something else?

Norbert wrote:A mistake on my part was to provide you with faulty information: as it turns out - I learned later, 10+11 is a better candidate to become a strong poison potion.

You do know that I have been adding all these new custom potions basically on a whim, right?
I wouldn't want to 'force' them on anyone... Although if others find a use for them for in their own levelsets that's of course nice

Norbert wrote:Falcury, do you have any thoughts on 1+6 and 1+14? These are floors that appear empty (space). MININIM sets the random bits to 1 to achieve similar functionality. Should 1+6 and 1+14 be replaced, remain for legacy compatibility, something else?

Well, I suppose they would not be quite the same tile, because unlike the hidden floors, the invisible floors would stay invisible if you step on them.
I don't think there is an equivalent of the 'random' field in the legacy code? So for SDLPoP, choosing another modifier value for the hidden floor would be the most practical, probably?

Sorry for the late reply. In the past weeks I've been busy with an endless stream of college exams. Meanwhile, in my free time, I implemented a generalization of SDLPoP's fake tiles: MININIM's fake constructions. The idea is that any construction may be masked to look like any other one, while retaining all of its physical properties (hangability, traversability, colidability, rigidity, etc...). This allows for hundreds of new constructions, and likely thousands of grouping arrangements which greatly enlarges the potential creativity and complexity feasible for a PoP level. Following MININIM's tradition of orthogonal attributes for maximum combinatorial space, besides the usual foreground, background and extension, all constructions have a fourth attribute (called "fake") that can assume any value valid for the foreground attribute. Disregarding extension, the theoretical number of constructions goes up from about 200 to over 5000. Of course, not all combinations are practically distinct or anyhow useful but there is plenty of space for very interesting constructions to emerge, specially when considering grouping of properly chosen adjacent fake constructions. I prepared a video demonstration of some noteworthy fake wall-based constructions.

This video gives an idea of what one might achieve. There is, however, a big universe of fake constructions out there waiting to be explored. It's very interesting how one can program the general principles to pop that universe out into existence, without having realistic hope of inspecting all of its corners by oneself.

The process of implementing fake constructions brought many improvements to MININIM. When I finished, the diff had more than 10 thousand lines (currently one fifth of the engine's size), and I estimate somewhere between 8 and 10% of the engine's code has changed. Core parts like the collision and rendering code have been largely rewritten. The new collision algorithm is the most versatile and robust I could devise to date. It works as well as I could hope for under extreme enclosure circumstances and naturally fix many corner case flaws of the previous one. The new rendering algorithm optimizes and generalizes rendering, moreover adding adequate horizontal and vertical depth perspective (as one might notice in the above video). Many physics restriction originally introduced to prevent the old rendering sub-engine from misbehaving (like a minimum distance the kid kept from construction's right faces, when looking left) were removed, because the rendering techniques are now overall consistent. Also, improvements were made into the level editor and undo routines.

Given that MININIM implements all possible fake constructions (based on the original game's tiles), it's now trivial (one line change) to implement support in the legacy level module for any fake tile SDLPoP might introduce in the future as a result of our standardization efforts.

Hereafter I reply to your comments.

Falcury wrote:Hey, and it would be interesting to add the hidden floor tile to SDLPoP, perhaps? Maybe it could use another modifier, like the fake tiles?

I originally chose to use the random bits field in order to allow for hidden floors to be modified by the free group, like a normal floor or empty space. If we use another modifier we are excluding those combinations.

Falcury wrote:Personally I think that the potions could be a 'convention' and not a 'standard', per se. I.e. there should be a way to opt out if (for instance) you wanted potion 7 to be something generic, or radically different.

I see that we can conceive all kinds of different potions, and we can't account for all minor variations. However, I think that in principle potions that are really unique and abstract in their semantics could be standardized. I'm not sure yet on how to define a criterion for choosing a potion over another. Anyways, we can always reserve a range for implementation-specific potions (or itens, in general).

Falcury wrote:Perhaps the difference might be this: so far, practically all of SDLPoP's added functionality is more or less 'directly derived' from the original DOS version. Even the fake tiles, replays and the quicksave mechanism (pretty huge changes) can be seen as new and interesting/improved ways of interacting with just the same game content as it was originally designed. Whereas new potion effects could be more described as actually new 'original content'.

Although I may find your point somewhat subjective, it's somehow clear to me that MININIM and SDLPoP are implementations with different philosophies. As a reverse engineering project, SDLPoP is technically attached to its legacy origins and people involved in its development seem to be relatively conservative on departures from the original game. On the other hand, MININIM is an implementation written from scratch that have had since its inception a focus on extensibility, and no hard commitment on being a 1:1 replica of the original game when its developers (myself) think they can do better. However, since this is a process to standardize the legacy level file format, that is naturally the main (and only) format used by SDLPoP, I think SDLPoP developers are the ones who have this in their best interest. Therefore, I accept that their conservative vision is the most applicable in this context.

Norbert wrote:Here's an updated overview, hopefully it's correct:...

Currently MININIM's legacy level module in the VCS implements all of them (considering your errata) except for: 10+7, 10+8, 10+9 and 10+10.
I'm wondering if the potion concept can be generalized like the one of fake tiles. In affirmative case, we could have the basis to discriminate between potions that should and shouldn't be standardized.

Norbert wrote:(*) 1+random bits on 1 = floor, looks like: empty, until you step on it

MININIM's semantics for the hidden floor is stronger: "empty, until you press it.". For instance, the kid could press it by hanging.

Falcury wrote:I don't think there is an equivalent of the 'random' field in the legacy code?

The foretable has rrmccccc, where rr are random bits, and m is a modifier.
Not sure what you mean.

Thanks. I didn't know this until you posted this. Maybe I never came across it before... the SDLPoP code doesn't seem to reference the random bits anywhere, as far as I can tell.

oitofelix wrote:

Falcury wrote:Hey, and it would be interesting to add the hidden floor tile to SDLPoP, perhaps? Maybe it could use another modifier, like the fake tiles?

I originally chose to use the random bits field in order to allow for hidden floors to be modified by the free group, like a normal floor or empty space. If we use another modifier we are excluding those combinations.

Yeah, I agree that using the random bits for stuff like this makes sense. Then, my only problem with using the random bits would be that I don't think all level editors can actually edit them (I use RoomShaker and there doesn't seem to be an option)

Norbert wrote:Nice, that must have been a lot of work to implement.
A diff of more than 10,000 lines.

Indeed. Nevertheless worth it. By the way, congratulations on lemdop!

Falcury wrote:The generalized fake tiles look impressive!

Thank you!

Falcury wrote:Then, my only problem with using the random bits would be that I don't think all level editors can actually edit them (I use RoomShaker and there doesn't seem to be an option)

Is RoomShaker free software? It shouldn't be hard to implement support for the random bits field. Submit a bug report, or talk to the maintainer about that. Alternatively, use apoplexy.

Falcury wrote:
7: shadow potion
8: bonus potion / extra time potion (trading in 1 max life point)
9: full health potion (with magenta-colored bubbles, inspired by Prince of Persia 3D's Tears of Malak )
10: custom open potion (similar to the normal open potion, but more versatile and if used to open a gate, that gate stays open. It also has a different appearance, so that it won't be confused with poison)

Falcury wrote:

oitofelix wrote:Speaking about behavior descriptions, what is that 10+7 "shadow potion" about?

It is a potion that temporarily turns you into the shadowman. While the effect is active, you can survive falls, pass through gates, and loose tiles won't fall.

Could you describe in detail everything there is to know about each of those potions? I think I've found a versatile way to generalize potions, and I'm working on its specification, but I need to be aware of every detail in order to make it general enough to cover all existing potions and natural enough to predict the whole potential universe of meaningful PoP potions. With this specification in hands we'll hopefully be able to look to all theoretically admissible potions at once, and then decide on a reasonable criterion for potion discrimination when assigning codes in the standard for the relatively scarce legacy level format potion space.

Falcury, instead of describing in detail what those potions do, as I've asked you to, please, read the specification (checking the "Reference standard item profiles" section) and in case those potions aren't exactly covered by it, tell me why.

I'd very much like to hear what you guys think about the specification. It's meant to be a constructive collective effort. In case you have suggestions or questions to discuss of any kind, they are all welcomed. As soon as we agree on modifications to the specification, after each iteration of refinement, the revision number should be incremented. After we have settled a final version, it should be published as so. If we decide to change anything after that point, we should increment the version number, and restart the process.

Norbert wrote:

oitofelix wrote:Is RoomShaker free software?

No, it's not.
Its author has also specifically stated there will be no source code release.
(Neither are NESPrincEd and the Total Packs, by the way. Almost everything else is though.)

As a matter of personal opinion on the subject, I think RoomShaker's author effectively made (on that post) an attestation of his ignorance about the workings of software licensing and collaborative free software development. Honestly, reading that post gives me moral nausea and disgust. I think that kind of developer deserves no respect from other developers and deserves to have no user. Everybody is entitled to respect by default. People lose that right when being actively disrespectful to others, which in my opinion, is what he is doing to his software users (by not respecting their individual freedom and harming their user community by being so unjustifiably selfish). I won't ever use software written by him, while he sustain that immoral behavior, and I feel sorry for those who does. In my opinion, his software shouldn't be promoted, but boycotted. People should demand the respect they deserve by default, and ridicularize that type of behavior.

With regards to RoomShaker(2). I do understand your resolute stance when it comes to software freedom. However, I would like to suggest a different, let's call it, more diplomatic approach. Instead of "boycotting" the developer's work or denying him respect referencing ignorant and/or immoral behavior, I suggest we explain and show to him the benefits of free software and invite him to join those of us who have chosen to distribute our works as free and open source software. Also, in the end, if end users are well-informed and we've created proper free software, users will make choices that could demonstrate what are and are not wise decisions for developers to make. This takes time, just look at how Microsoft has changed the last decade.

It's important to place his remarks and past choices in perspective and take into account the history and developments that have taken place within this community and the software world in general. It was about fifteen years ago that he created his mods and released the first version of his editor. As you know, this was a different era for most developers. GNU/Linux was more of a niche, and even simple, commonly used programs were primarily proprietary software. Collaboration, such as on GitHub, wasn't commonplace. It takes a shift in one's views to take the 'bold step' to open source previously closed software, perhaps even more so to start distributing it using a free software license.

Brendon, together with poirot, is credited as the reverse engineer of the PoP1 level format; documented by him and others. Documentation that was subsequently used by me and others.

I do understand where Brendon is coming from when he writes it's possible to invest time and energy and then not be credited. I've actually contacted GNU/SFLC in the past to suggest they update their FAQ to supplement the #IWantCredit section with information about what happens to credits if a work is being forked. They did not get back to me, even though my e-mail was solid, detailed, I believe accurate, and I expressed some legitimate concerns while citing from relevant sections of the GPL(3). Anyway, my point is simply that if he has feelings of uncertainty, I get that. I can imagine keeping software closed source might feel 'safer' to him.

I started working on apoplexy because there was no free software level editor for GNU/Linux. It's unlikely I would have if RoomShaker(2) would've been portable free software. This could make the (closed source) Total Pack author scratch his head, if you know what I'm saying. (I don't mean to imply I'm working on something similar.) It would be nice if, in the future, the PoP1 Total Pack could download SDLPoP/MININIM replays from popot.org or another mirror. What if starwindz is MIA or decides it's not going to happen. Brendon's website disappeared quite a while back. He recently made a small change to RoomShaker after more than 6 years of absence. What if both his website and he disappeared, and we want to modify RoomShaker. Game over, man. Or rather, it might be dumped in favor of something else.

Examples of why free software works all around. Even close to home and PoP1 related. David as the main/initial SDLPoP author now primarily maintainer; Falcury who added neat stuff to it. Porting RoomShaker(2) to portable QT/GTK+, could be a crazy but interesting project. I'm not going to, but someone could. But... they can't, because it's closed software. Brendon, join the light side of the Force.

I agree with you overall. Please, don't get me wrong. There is no denying that the diplomatic approach is the best way to go for expanding the reach of the free software movement (and for that matter of most causes, if not all, I know about) from a strategic view point. This said, that was just an expression of how strongly I feel about this issue. It might not seem like this at first, but I have no problem in putting differences aside and working with people I disagree with, for the sake of some common or mutually beneficial goal. Anyways, in the hopefully unlikely, but possible case diplomacy fails miserably, I've got a backup plan.