Improve execution speed

I have a big problem with execution speed of my latest automation of Carrier Battles: Philippine Sea. The number of traits for one type of counters has grown to be very large, and it appears to me that execution times grow proportionally, even for “quick” commands using few traits.

I don’t know anything about how Vassal works internally, but I hypothesise that the traits of a counter are searched through sequentially so that the more traits, the longer execution time.

I can try to reduce the number of traits by moving traits to other counters to be executed there, when possible, and some traits can be combined, for instance trigger traits that call other traits with no trigger conditions.

But I have wondered if trait names are also an issue. Will the search time be noticeably reduced if I shorten all trait names to, say, three characters? I have 400+ traits to rename in that counter alone, so I wouldn’t start doing it if experts here say that it doesn’t matter.

Any other tips for speeding up execution?

When a (key) command hits a piece, I believe all traits are queried to see if they will handle the command. Also, command execution does not stop on the first match, but continues until the traits are exhausted. Also note that the traits are search twice - see Trait Ordering and YOU, possibly with some caching.

In short, the execution time of any command hitting a piece should be order N where N is the number of traits - i.e., take proportionally longer the more traits your piece has.

This could be optimised in the Vassal code by caching acceptable commands of a piece - preferably in a hashed associative container for quick look-ups.

Looking up a trait to see if it will handle a command involves string comparison. String comparison is best-case complexity O(1) (different size strings) and worst case O(n). Or, the time to execute a comparison between two string is constant and independent of the string size n if the two strings compared are of different length. If the strings are of equal length, then n characters need to be compared:

FUNCTION string_compare TAKE S1 A string AND S2 A string DO
  IF length of S1 NOT EQUAL TO length of S2 THEN
     RETURN false
  END IF
  FOR index FROM 0 TO length of S1 - 1 DO 
     IF character at index of S1 NOT EQUAL TO character at index of S2:
        RETURN false
     END IF
  DONE
  RETURN true
END FUNCTION

That is, comparing long strings to other long strings does take longer, especially if the strings are more or less of equal length.

That said, the string comparisons at play probably take far less time than the iteration over the traits container, so there’s probably very little to be gained by shortening the strings.

But, if you make all command the same length, then all string comparisons will be O(n), while if they have variable length you will often hit the O(1) - simply because the strings are of different length.

Sure, string comparisons can be optimised under certain assumptions. One can use use hashes of the strings if one is sure there’s no collisions, which may be at the cost of increased time in calculating the hash-values. One can cache these hash-values - for example in a associative container where the keys are the hash-values.

One also has to weigh up the benefit - for maintenance etc. - of more descriptive strings versus the time spend and the possible gain.

The real killer - in terms of time - is typically BeanShell expression. Evaluating a BeanShell expression involves setting up and tearing down the interpreter environment, which takes non-trivial amount of time. Check that you are not evaluating a lot of BeanShell expression - for example in Calculated Property traits.

Yours,
Christian

1 Like

With 400+ traits, I can’t have them all in different sizes, but maybe I should try to make them shorter. I have tended to use very long names for descriptive purposes.

But you mention that the real killer in terms of time are BeanShell expressions. It appears that I have got it all wrong, because I have changed everything I could into BeanShell, even constants like {1}. Perhaps I should go through everything and remove those curly brackets wherever it is possible.

Without knowing specifics, you’re making the pieces far too complicated.

Make a few general types of units:

Status Markers, Ships, Air units (and because I’m not familiar with this game, possibly ground units).

There’s no reason for air units to have traits which only apply to ships, and vice versa. And status markers should have VERY few traits. (stackable, and degrees of rotation, and hideable, so that when a unit it’s applying to is turned to hidden, the marker can become hidden also).

Also, honestly, is a commercial boardgame (not some department of defense simulation) truly handling 400 traits in the first place?

It REALLY sounds like you’re making your pieces far, far more complicated than they need to be.

1 Like

I would tend to agree with that assessment. It is very hard, even for a highly complicated game, to imagine that a single piece will have 100, let alone 400, traits (directly or indirectly through prototypes).

Some, very few, isolated piece can amass a fair number of traits, but those pieces are typically “hidden” pieces that does calculations in the background. If one has a piece like that that becomes very big, one can often split the piece into distinct tasks and pieces.

The length of the trait names, properties, etc. are likely not the issue - rather the number of traits seem much more likely to cause overhead.

Perhaps you could explain - in some detail - what it is you are trying to achieve and in particular why you believe you would need many traits to handle that. Of course, if you uploaded your module somewhere, then people could take a look for themselves.

Funny thing is, that wargames used in a “professional” setting are often relatively simple. They tend to focus on a specific or few specific aspects of a conflict - for example, logistics, command’n’control. combined arns, drones, etc. Other aspects are abstracted away.

It seems much more prevalent in “hobby” games to want to encompass all aspects of a conflict, ranging from political and strategic considerations to tactical considerations, at the same level of detail. Napoleon couldn’t give a rats arse if the 2nd brigade of I corps has 25% or 50% ammo left - only if the unit is capable of overrunning Hougoumont.

Any case, my 2¢

Yours,
Christian

Thanks for the comments from both of you.

Of course I am also worried about the number of traits but I doubt that it can be avoided in this case. The module is my version of Carrier Battles:Philippine Sea, and the previous version is fast and has gained a lot of positive comments, among other things for the automations I have made.

The game is a solitaire game, so a a lot of things is done by die-roll, particularly for the non-player side. Most of this stuff is tedious, and time-consuming, and one the players wrote to me and asked if I could automate the most frequent, and time-consuming part, which is the Air Raid Procedure where it is determined if a Japanese force is conducting an air raid, or not. Japanese forces come in a number of flavours, depending on whether they have carriers, or not, and what knowledge there is of the force, and how that knowledge influences the possibility of an air raid - in short, a force with more carriers is more likely to send out s large air raid than a smaller force.

At first, I added a command that conducted an air raid to the forces known to have carriers (a so-called “Butai’). This worked OK with a noticeable degradation in performance, but not too much. On playing the game, I realised that it would be beneficial to extend this to the other category of forces, namely those that may or may not have carriers. Unfortunately, this involves situations where a single force might turn out to be two forces, and a lot of chit-pulling, and die-rolling is needed to resolve this.

I think there at something like seven different situations of die-rolling, and a corresponding number of tables, some of them large. Hence the large number of traits to relieve the player of several minutes of playing. The tables have all been placed outside the force counters, but they involve a large number of global properties (usually one for each table column).

I have myself thought of increasing speed by moving more traits outside the force counters, because this is what I have done in other modules. There is a lot of stack manipulation (forces have their intelligence level in a separate counter, and for Butais, this counter is placed on a separate display). I used to do this by moving counters to the top or to the bottom, and placing new counters on top of the stack, and so on. This has not stood the test of time, and I have rewritten most of the code to have the four main counters moved away to holding areas outside the map, and to move them back at the end. I am thinking about putting an invisible counter on top of the stack, and have most of the traits moved to that counter. This will make more mundane functions of the forces execute faster, but the the automation itself might not become faster.

Another optimisation that I have only thought of now, os to move the global properties for tables to special invisible table counters that could be called, and leave the result in a global property, or a dynamic property of the calling counter. That might have a positive effect on performance.

By the way, I did make a version of the module where most BeanShell had been replaced by old-style expressions, but there was too many expressions where this was not possible, and no increase of execution speed that I could notice without actually measuring it, so I stick to BeanShell.

The very best I could do, would probably be to compile the automation in another language, but though I am skilled, my skills do not go that far - yet.

OK, thank you for giving a bit of feed back.

The best way to encode an array with some indexed look-up in Vassal BeanShell, is probably to define properties that are encoded tables - e.g.,

  • Property:
    • name: Foo
    • value: Bar,Baz,Gnus,Gnat

Now you can use the String.split method to get a particular element of the “array” - e.g.,

{Foo.split(",")[2]}

should yield "Gnus". You can “un-roll” a 2-dimensional array into a single dimensional array. Suppose you have

Bar Baz
Gnus Gnat

then it will be stored like above, and you would also need to store the number of columns or rows

  • Property:
    • name: FooNCol
    • value: 2

Then

{Foo.split(",")[col + FooNCol * row]}

will return the element at col,row - e.g., for col=0 and row=1, it would return "Gnus".

If pieces are associated with each other, or they have representations in different locations, then an efficient way of doing that association is via the Attachment trait. That will bind two (or more) pieces together and they can very efficiently pass messages back and forth.

Maybe some of these techniques can help you to streamline the code a bit.

Yours,
Christian

1 Like

Thanks, @cholmcc . I didn’t realise that it could be useful to use array notation in the BeanShell expressions. I have used a similar technique to put two-dimensional arrays in a string. I just used substring() and fixed-length cells instead. But I stopped using it and put each row in a single variable because it seemed that the strings have a rather low limit, and I also found that it was too difficult to debug if I had typed a wrong value somewhere in a huge two-dimensional array.

I am warming to the idea of putting code and tables in invisible counters. Tables because apart from any speed advantage, I might also get rid of a Vassal bug that I have encountered a number of times, where global properties were suddenly without value until I view them without changing anything. I have had to go through a lot of variables with Enter-ESC, Enter-ESC in order to restore the values. I can’t predict when it happens, so I can’t produce a case to reproduce the bug.

The code can also be put into invisible counters, and because there are not loops, and hardly any jumps, I could put the beginning of the procedure in one counter, and the end in another. This could lead to serious speed improvements, I think.

There is one area that has caused a lot of little extra code: the tendency of Vassal to think that LocationNames (hex numbers) are numbers where it removes leading zeros. I have several methods to overcome the problem, but the most effective seems to store all locations appended to a “Z”, like “Z0568”, and then remove the “Z” just before using the location. I have searched here for a better solution, but I haven’t found anything. If you have a solution, I would be grateful.

I don’t think properties have an upper limit on the number of characters that can be stored in them. Even if it did, it will be highly unlikely that you would ever reach that limit.

That is why I create most of the modules I’ve done with pywargame. Using that Python module, I write the entire module in Python code in which it is far easier to encode arrays and the like. After all, a module is really only one XML file (buildFile.xml) - actually two, but moduleData is trivial - and a bunch of images, put together in a ZIP archive.

There’s a relatively important thing to keep in mind with regards to Global Properties and piece properties like Dynamic Property and Marker trait properties. The values stored by Global Properties are only minimally encoded to be stored in the XML buildFile.xml and when in memory. Properties stored in pieces have to be encoded for them to be stored in the rather obscure trait encoding. That means, that certain characters can be a problem in the trait property values.

One way I’ve encoded “tables” like Combat Results/Resolution Tables (CRTs) is that I have markers for each column, and that marker then contains that column of the table only. You can see how that works in most of the modules that I’ve done. Take for example Anzio Beachhead.

  • A user selects the belligerents of a combat and then presses Ctrl-X.
    • First, a battle marker is placed on the belligerents, so as to easily identify the combat participants.
    • The module then calculates the total Attacking Combat Factors (ACF) and total Defending Combat Factors (DCF), possibly taking terrain or features into account.
      • Terrain and features are stored in global properties. For example, all woods hexes are stored in a global property like

        • Property: WoodsHexes
          • Value: "|01@23|02@23|05@43|...|10@50|"
            (assuming grid number separator is set to @).

        To see if a piece is in a woods hex, the code will do

        • Calculated Property: InWoods
          Expression: {WoodsHexes.contains("|"+LocationName+"|")}

        This can then be used like

        • Calculated Property: EffectiveDF
          • Expression: {(InWoods ? 2 : 1) * DF}

        where DF is a for example a Marker trait property.

    • It then calculates the odds, again taking possible odd shifts into account.
    • An odds marker is then placed on the first defending unit.
      • This odds marker knows the CRT column of the odds it represents - no more.
  • The user can then select the odds marker and press Ctrl-Y
    • The odds marker then rolls the dice
    • It then looks up the result of the combat in the column of the CRT
    • It replaces itself with a results marker.
  • Now the user(s) need to implement the result by manually applying step losses, eliminations, retreats, and so on.
  • Finally, the user may clear the combat markers by selecting one of them and pressing Ctrl-C, or they can wait and let the module automatically clean them up at the end of the current phase.

Although this requires a fair bunch of traits, it is no where near 100 - more like 10 or so.

The setting Preserve leading zeros in Integers in the module Global Options in some sense circumvents the need for this. If this option is enabled, then all values are treated as strings, which means a LocationName like 0123 will have the value "0123". You can of course always get back integer values if you like with something like

{Integer.parseInt(LocationName.substring(0,2).replaceAll("^0*",""))}
{Integer.parseInt(LocationName.substring(2,4).replaceAll("^0*",""))}

See also this thread.

Another option is to define the column-row separator in the Grid Numbering. For example, if you set that to @ (do not use : because that can cause problems with the trait encoding), then LocationName will be something like 01@23 which still needs to be parsed out to get the column and row numbers

{Integer.parseInt(LocationName.substring(0,2).replaceAll("^0*",""))}
{Integer.parseInt(LocationName.substring(3,5).replaceAll("^0*",""))}

but other properties that hold numbers will still be treated as numbers.

All in all, it sounds to me as if your strategy for the module implementation is a bit off. I think that you are perhaps choosing some complicated techniques which end up hurting performance and maintainability. If you can post a draft of your module somewhere - perhaps even make an entry in the Game Libary, making sure to note that this is work-in-progress - for example with a release number like 0.0.1-draft, then others may take a look and give suggestions on better techniques to achieve what you want to achieve. If I have the time - no guaranties - then I might. Of course, if you use third-party copyrighted materials, you should make sure you have license to share those materials.

Yours,
Christian

1 Like

I think you would be surprised if you were to see some of the computer-assisted wargaming simulations I participated in when I was in the US Army.

Like an entire Stinger Air Defense Artillery being modelled down to each humvee-equipped two-man Stinger team, platoon and company leadership elements (platoon sergeant/first sergeant and platoon leader, each with a with radio-operator/driver) and the number of rounds remaining on their vehicle. Even the repair and fighting capabilities of the mechanics in the battalion maintenance platoon.

@cholmcc, thanks for your lengthy comments and suggestions.

It is very possible that was wrong in my assessment of the problem I encountered. At the time I still had not encountered the problem that global properties could lose their value spontaneously, and when I got an expression evaluation error, I thought it was the length of the string (which was more than 512 characters). Perhaps the error would have disappeared if I had viewed the string, as I later found out. In the event I recoded the table.

Interesting information that I have not seen elsewhere, thanks! I have myself coded tables in markers in the same way, but I moved some of them out to global properties precisely to limit the number of traits in the counter. Some of the tables here have more than thirty columns, although they look smaller in the rules, because when a column is called “21-28” I turned it into eight identical columns to make the algorithm for finding a value in the table easier.

The setting Preserve leading zeros in Integers sounds interesting. Will arithmetic still work normally? I think I read somewhere that numbers are always stored as strings. I have to mull the consequences for all the messages that include numbers.

The column/row in the grid numbering suggestion is also interesting, but I wonder if the trouble of doing is really less intensive than adding and removing the “Z” in front of the grid number.

Thanks for your offer, I might take you up on it later if my efforts in optimising the code does not bear fruit. For the moment, I find it too complicated to explain how to even get to the point where these functions can be tested, because you have to know the game.

Just for the fun of it, and in case you might be interested, I present the chat log from one call to the “Air Raid Procedure”:

As you see, this is not a question of detailed simulation (the game is actually rather simplified).

And of course; I’ll be very interested if somebody can code this in about 10 lines :wink:

Expanding equal columns or rows is probably a bad idea. Perhaps a better way is to have a calculated index. For example, if each column spans 8 numbers, then the column number can be calculated by The column/row in the grid numbering suggestion is also interesting, but I wonder if the trouble of doing is really less intensive than adding and removing the “Z” in front of the grid number.

{(int)(number/8)}

E.g., values 0 to 7 will map to 0, 8 to 15 map to 1, and so on. If the mapping is more complicated, it can probably be fairly easily done with ternary expression

{(number < 2 ? 0 : (number < 6 ? 1 : (number < 10 ? : 2 (number < 15 ? 3 : (number < 20 ? 4 ...)))))}

Of course, if you are thinking about something like the Known air strength modifier table on page 23 of the rules, then it is more easily and effectively implemented as

{(AS <= 5 ? -3 : 
   (AS <= 7 ? -2 :
    (AS <= 9 ? -1 :
     (AS <= 12 ? 0 : 
      (AS <= 14 ? 1 : 
       (AS <= 16 ? 2 : 
        (AS <= 18 ? 3 :
         (AS <= 20 ? 4 : 5))))))))}

I’m not entirely sure. I tend to avoid that setting, exactly because I’m concerned about whether asthmatic will work as expected. Internally, Vassal stores everything as strings. When a property is referenced in a BeanShell script, the interpreter will check to see if is really a number (integer or real) and if so return a java.lang.Integer or java.lang.Double object to the interpreter. I believe that check is disabled when Preserve leading zeros in Integers is enabled, and so the interpreter will always get java.lang.String objects. If that is indeed the case, then if property A has the value 1 and property B the value 2, then the expression

{A+B}

will give the result "12" because the + will be the string concatenation operation - not the arithmetic addition.

Where exactly would you prefix the grid coordinate with a Z? In the Location format of Grid Numbering? I guess that can be done too - only worried that it will have some overhead, and I’m not sure it prevent Vassal from stripping leading 0’s.

If you add the Z elsewhere, I think it will not prevent LocationName from having leading zeros stripped, as it will likely happen after the above mentioned logic to recognise numbers.

BTW, is it the same game as this one? Current module

Yours,
Christian

You do know that you can right-click the chat interface and say Save chat log as HTML, right? Then you can essentially paste or upload the relevant portions.

When I talked about detail simulation, it was not in the context of this game - but in the context of needing many traits. What I said was

However, it is also clear that to implement a bot in Vassal is not uncomplicated. Even a relatively simple bot as in Hitler’s Reich is a bit of a challenge.

Perhaps a strategy would be to Creating Custom Classes in Java, so that complicated decision trees can be more effectively implemented. A fundamental problem, however, is that you will need to get information about the game state into that code, and Vassal has rather poor support for that. Of course, the promise is that Vassal 4 will remedy all that :slight_smile:

Yours,
Christian

Well, I did implement it as a single line table, but I’ll consider changing it.

It is of the more complicated sort, but I am not sure if it really is better to use lots of nested ternary expressions rather than a table and a simple method to extract the value.

I prefix it only when I need to store the LocationName in a variable. In fact, I have used a number of methods to preserve the leading zeros - or to create them when needed. No method seems entirely satisfactory.

It is indeed the second one. I had no further plans for the module, but one of the players wrote me, and asked for this function, and some more. I took it as a challenge, and here I am!

Thanks for that link. I haven’t done anything in Java for more than 20 years, and it is as if I have forgotten all of it, but it could be interesting to brush it up again.

When I have read about Vassal 4, it seems that it will be a complete rewrite, but I am not sure if it will support the old style Vassal, so perhaps all the old modules will be stuck with Vassal 3?

Anyway, I am eager to get on with the ideas I have got from this thread, although real life is threatening to cause me to take a break. Before I do anything else, I want to create an invisible “action” counter to be placed where the force or Butai is located, and this counter will contain the code of the present forces. Or possibly a series of action counters containing the first part, and the second part, and so on. I am also interested in trying to create table counters that you call, and get the result back. Theoretically, that should give the most speed improvement. After that, I can try rewriting the table codes, and other parts of the code.

As I understand things

  • Vassal 4 will be a complete rewrite, probably in Rust - all the rave these days.
  • Vassal 4 will be able to read Vassal 3 modules, although there will be several incompatibilities. Specifically, I think it will be hard to make a new interpreter that can work with the embedded BeanShell code in modules.
    • So what I see as the route, is that one can import Vassal 3 modules into Vassal 4, and then edit it to make a new Vassal 4 module.
    • However, the promise is that the new module format will be much easier to handle. For example, the traits encoding will be some standard format like JSON, YAML, or what not.

One thing about encoded arrays: I found that you may need to use GetString to get the property value as a java.lang.String so that you can call .split on it.

{GetString("Array").split(":")[calculatedIndex]}

Yours,
Christian

1 Like

Thanks! I don’t think I would have thought of using GetString myself. I thought the value was already a string. But it works! :+1:

Success!

I split up the code in three “action markers” that call each other sequentially, and kill themselves after use, and that did the trick. The code is now executing at the speed I hoped for.

Actually, there is one stage where there is a slight wait time while the code calculates the facing of a couple of units based on the direction to a target. The rest of the code in the same action counter executes fast, so I can’t speed it up further by introducing a fourth action counter. This code does some simple arithmetic, and I might be able to speed it up by trying to eliminate BeanShell, by I don’t know if it is worth it.

I have not transformed the tables because the speed is already fast.

Instead of making the action counters invisible, I gave them some red graphics that flashes like a little explosion in the hex, and looks nice in my opinion.

Thanks for the thoughts and inspiration.

You’re BOTH wrong.

The proper, most efficient way to solve that sort of thing is with a case statement, or possibly 2 of them. The case statement can use programming tricks more sophisticated than a succession of if-then statements (which is all ternery expressions are, with a return value). For example, most languages, (and i’m sure Java does, too, will implement a “Computed GOTO” which is implemented itself by a jump table. it’s FAR more efficient than a litany of comparisons.

Personally, I don’t trust the rust cult. People who perpetually have problems with losing track of memory are the types of people who either didn’t take, or failed Data Structures and Agorithms. So they’re attracted to the “training wheels” version of C. But there’s a 20% penalty for all of the checking.

There ARE some legit reasons to use Rust…

But the “let’s rewrite standard utilities which have been working perfectly for 50+ years, just so that we can say they’re “memory safe” when the fact that they have ALREADY BEEN MEMORY SAFE FOR DECADES… is putting the cart before the horse. So, I don’t trust the rust cultists. Plus it’s being pushed by the three most suspicious entities in the software industry – IBM, Microsoft, and RedHat.