Hi,
This is a long shot, but is there a Python module to read and write VASSAL module files? I’d like some nice API to be able to easily add some images and counters that use those images from a Python script. I don’t want to spend a lot of time implementing one myself if there already is one I can use. 
(To be exact I’m planning to add a new feature to my Inkscape countersheet generator to allow the user to automatically export all created counters to a choosen VASSAL module.)
No, there is not Python Module to do it, but Vassal version 3.2 will allow you to do this. It includes a new feature to create counters in bulk from a directory full of images.
B.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
OK, thanks! I heard about that feature, and it sounds very useful. I still think I want to add this to my countersheet generator though. Generating everything from one spreadsheet document in one go feels like a good thing to have (at least to me, personally, for my current project). I also intend to have (optional) columns in the spreadsheet to tell which panel to put each counter on and what prototypes each counter should have. Connection between front and back of each counter is already available in the generator, and I hope it will not be too much work to automatically add that connection to the VASSAL XML file for each counter. So it’s more to it than just importing the graphics into the ZIP file, I also want to make use of as much meta data as I can think of to add, that I will not have access to later by just looking at the generated countersheet PNGs (or SVGs).
Besides I can see other uses for a Python (or other scripting language) API. There will always be things that are easier (or at least faster, and more easily done over and over) using a script rather than manually with a GUI. I know I have several times resorted to batch-editing things manually in the VASSAL module XML files in a text editor.
The new Add Multiple Piece feature does more than just import the graphics - You specify a counter template including all of the traits and prototypes you wish to use, plus rules to identify front/back and Layer levels and it builds the entire counters.
However, I understand what you are saying about keeping an external ‘Source’ for your module build.
B
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
Thus spake “pelle”:
I think a Python module for manipulating modulue files is a good idea. (I
wouldn’t use it myself, I’d write a Perl module for it instead, as I prefer
Perl to Python.) But I support this kind of thing, as it would save module
designers who want to build modules programatically from having to mess with
the awful buildFile syntax.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
in the end, it turned out that a generic Python module to handle VASSAL modules was a bit too much work to get into at the moment. Instead I settled for a more limited implemenation that can do exactly the things I need (add counters, optionally with a flip-side, and optionally with a given Prototype added, to a given Panel).
I haven’t even bothered to fully understand the format used to store game pieces. I expected it to be XML, but it turned out to be some sort of hack to get nested string values quoted into an XML string. Even a simple thing like adding a second Prototype turned out to completely alter the way the string must look like. I postponed reading DataSequence.java for now. I guess most things I need to know are in there, but I also noticed some other values that seems to randomly vary even when I create a duplicate of a counter.
Probably because of my bad implementation of the piece data string there is a weird bug now: If I edit a module after importing images, then save it, all the images disappear. They are completely gone from the zip, but the counters remain in the buildFile. If I don’t edit it it works fine though (and that’s fine to me really, since I intended anyway to keep a module without counters for doing all editing, and just batch-add all counters to get a copy that I will not edit anyway; but still it could be nice to fix it).
The first release can be found at that boardgamegeek thread linked to above if someone wants to have a look. It has only been tested on Linux, and I fully expect the bitmap export feature (required by the VASSAL export) to be tricky to use on Windows (and possibly on MacOS), especially if you don’t have inkscape.exe in your PATH.
Everything related to VASSAL is in vassal.py, not depending on anything else in the distribution, so it can be easily reused (but it depends on the lxml library for xml parsing).
Thus spake “pelle”:
One of my long-term goals is to expose the data as XML elements and
attributes, rather than as XML elements containing unintelligible blobs.
Yeah, this doesn’t surprise me all that much. It’s very easy to screw up
a buildFile right now be hand-editing it.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
I think I pretty much got what I deserved for just hacking away without properly understanding the problem first. It would have been REALLY weird if the generated data blobs worked perfectly as intended. At least I have seen no problems with playing the module, as long as I don’t try to edit it first. 
A Python module would be very limited in what it could do. One of the advantages of using the native Java code for that is that it would keep up with the changes in the actual VASSAL code. How the buildFile is created is not that difficult to understand – the only problem being that it’s not well documented. It wouldn’t be a bad idea to add methods to the existing Java code to serve as an API for other applications to manipulate the buildFile. For example, it’s obvious from the Java code that no one really intended for the existing methods to be used for any other purposes.
P.S.: I would also prefer a Perl module, but I don’t know Python at all.
Post generated using Mail2Forum (mail2forum.com)
They’re not that bad. It gets a little weird with nested elements due to the quoting mechanism, but that doesn’t happen really often.
Post generated using Mail2Forum (mail2forum.com)
Thus spake Michael Kiefte:
No, I don’t mean adding any hooks to VASSAL for other apps—what I want
is a file format which is human-readable.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
Thus spake Michael Kiefte:
They are that bad. They make it impossible to leverage all of the XML
slurping code which is already out there, and not easy to do scripting.
That by itself is going set the bar too high for almost everyone to get
useful results.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
I agree with that, but it is another issue entirely.
Post generated using Mail2Forum (mail2forum.com)
Thus spake Michael Kiefte:
Then I don’t understand your original reply. If the file format is either
(1) human-readable, or (2) well-documented, then it should not be hard to
write a Python or Perl module for manipulating it.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
To me it would be much easier to work with XML than a mix of XML and some other format. But I perfectly well see how most VASSAL users and developers have no reason to change the way it is now.
The quoting is quite complex btw, not as simple as suggested above. As an example, here are the data blob for two very simple counters I created using VASSAL for reverse-engineering purposes:
+/null/emb2;Flip;2;F;;0;;;0;;;;1;false;0;0;c_r.png;;true;;;;false;;1 prototype;PPP\ piece;;;e_d.png;FRONT/1;F \ null;0;0;0+/null/prototype;P2 emb2;Flip;2;F;;0;;;0;;;;1;false;0;0;g_f.png;;true;;;;false;;1\ prototype;PPP\\ piece;;;c_f.png;FRONT2/ -1;\ \\ null;5;0;The only difference is that one has two Prototypes (PPP and P2) while whe other has only one (PPP). The second was created by copy-paste from the first from within the VASSAL editor, and nothing was edited except adding the second Prototype. This tiny change makes the quoting all different, in ways that are not easy to guess without further studying the source code I think. For instane look at that last \ that suddenly becomes \. Also it is not obvious why one ends in 0;0;0, while the other ends in 5;0;. There are also a few other minor differences that is in no obvious way connected to the addition of a Prototype.
[/code]
And, yes, an external VASSAL API is probably more useful than a stand-alone implementation in some other language to access a VMOD. Avoiding the problem of keeping up with format updates is a good reason. There is always Jython to be able to make use of an API from python. 
But for the particular use-case I’m having I need to get to the file directly, and calling Java code isn’t an option really in this case. Trying to keep it as small a hack as possible, relying on as little as possible of the file format to stay the same, is probably a good idea.
Hi again,
after adding a feature to the Text Label to handle transparency, I realized that for backwards compatibility I was going to need to support people still using 3.2.17 for some time - so I was going to have to provide a Layer trait with transparent images to replace the Text Label when people weren’t using 3.4…
I ran into a challenge of needing to modify a number of existing modules to use this feature (around 10) that each contain many hundreds of counters. Not something I’m excited about continuing to do manually. Now regexes go a long way to handling the problem but… there are some niggly details about the string encoding for pieces that I’m still not quite clear about.
I’ve dug through the module save code a bit… but it wasn’t giving me what I was looking for. This post gets me closer than I’ve been so far - could any of the developers who understand the process of how GamePieces are serialized point me in the right direction so I can backwards engineer a piece of Perl/Python code to parse buildFiles?
For example, I’ve figured out through trial and error that as traits are added to a piece the number of ''-character bits at the end of the file increases. So, if I’m removing a trait from a piece (a delete trait) that ends with two ‘\’, then I need to remove the section at the end which has two ‘\’ and reduce the number of '' from all the traits that appear after this one… Yuck. But I’ve yet to find the java code that actually implements this.
If you can point me forward, I’d be happy to write this up as a doc on the wiki and publish some buildFile util scripts to help people.
Cheers, jas…
In VASSAL, reading and writing these sequences (that specify Traits in the buildFile) is handled by the SequenceEncoder. Most traits will have a “myGetType()” and “mySetType()” method that will deal with these strings.
When you add a new bit of data to a trait, then it is best to add it at the END of the list (for minimum disruption if files are loaded by an earlier version, although in practice this won’t happen much). And most of the myGetType() methods are designed so that if no data is found for the “new feature” it is automatically initialized to default values (and so then next time it is written, it will be automatically written with a record for the default value – or any new value the user changed it to in the meantime).
SO… if the main problem you’re describing is that you want to be able to easily update a bunch of pieces in the module to use the new default value, all you have to do is load and save the module once with the new version.
BUT… if the main problem is you want to change a bunch of counters to use an updated non-default value, then you would still load and save the module once first, and THEN change all the records. And since you are designing the new feature, you could conceivably give the encoding values for your trait a slightly unique “signature” so that you could easily global-search-and-replace them in your buildFile without accidentally picking up other strings?
Brian
Also: HOLY THREAD NECRO, BATMAN!!!
It’s in GameModule:
public String encode(Command c) {
if (c == null) {
return null;
}
String s = encodeSubCommand(c);
String s2;
Command[] sub = c.getSubCommands();
if (sub.length > 0) {
SequenceEncoder se = new SequenceEncoder(s, COMMAND_SEPARATOR);
for (Command command : sub) {
s2 = encode(command); // <------ note the recursion here
if (s2 != null) {
se.append(s2);
}
}
s = se.getValue();
}
return s;
}The traits of a piece are in a tree instead of a list, each trait is a child of the previous trait, hence the necessary recursion and the increasing number of \ escapes. The innermost trait gets encoded, then encoded again by its containing trait, encoded again by the next trait level, and so on, until 90% of the savegame is .
Glad you noticed…
Right. This given your super description of traits using the Decorator pattern is helpful.
The kinds of edits I want to do are various:
Because the text string for game pieces is not documented anywhere, it’s a challenge to grok the meanings or importance of things like non-printing ascii characters embedded within the string (lots of tabs), or the meaning of combinations of characters (‘emb2’ vs. ‘emb’, or ‘;;;;’), or the increasing pattern of '' characters at the end of the piece.