My huge Case Blue map is maxing out Vassal. In fact the only way I can run this module is by setting -Xmx1024M and using one of Joel’s beta releases!
Any idea how I can reduce the memory being taken up by the map? Right now it’s a 11403 x 8491 jpg. I’ve already made it as small as I can dimensionally, just wondering if there’s anything else which can be done related to color depth etc.
cheers.
Thus spake “bsmith”:
This is kind of long, because I’m going to explain the cause of the
problem, and give a history of how it came to be this way.
In the test builds, peak memory usage occurs when a map image is being
written out to a memory-mapped file. The reason for this is a technical
one: ImageIO, which loads images for us, often returns BufferedImages
which are TYPE_CUSTOM. Everything in out image processing chain wants
TYPE_INT_ARGB, including the subclass of BufferedImage which is backed
by a memory-mapped file.
So there are two approaches to take:
Read the whole image into memory with ImageIO, and then write it back
(doing color conversion as we go) to the real destination image.
Set the image which ImageIO will write into, using
ImageReadParam.setDestination().
The advantage of #2 is that we don’t spend time copying one image to
another on load, which means that we don’t have to have two copies
of the same image around at once. This is especially advantageous
for enormous images like yours, since they would be written directly
to a memory-mapped file and never occupy any RAM (other than what
ImageIO needs for decoding) in the first place.
The disadvantage of #2 is that I’ve never been able to get the damn
thing to work like the docs say it should. The problem appears to be
that ImageIO really wants to load the image as TYPE_CUSTOM, and
crashes and burns on you if you give it some other kind of image type.
I would appreciate advice from anyone who has some insight on how to
get this working, since map image loading is the last thing in VASSAL
which requires a huge block of memory to be available all at once.
So, #1 is what my test builds do. What this means is that if you want
to load an image which is L x W, then during loading it will need
4LW bytes of RAM. But this is only during loading—as soon as the image
is copied to the one backed by a memory-mapped file, the original image
will be releaed and (probably) garbage collected very quickly.
Loading your Case Blue map will take 387291492 bytes, which is about 369MB.
That means in order to load the map, you have to have at least 369MB of
heap left. Once the map is loaded, that heap space will be given back,
but I’m not surprised that you have to have a 1GB of heap to get this to
load.
Right now, if you want to load this map with less heap, you need to slice
the map into several smaller images. My copy of Case Blue is at my in-laws
right now, and I won’t see it until June. Am I correct that the paper
maps are not all the same size, and don’t line up in a grid? If they do
line up nicely, then you could cut the full map along the same lines as
the paper maps. If they don’t, then I’d suggest slicing the full map into
an array of smaller maps (maybe 3x3, or 3x4?). This will result in
dramatically less memory usage when loading maps, and it should also
decrease overall load time pretty subatantially.
Reducing color depth will only make the map look worse.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
In my experiments with my mod - I don’t have a map that size but I have a lot of big maps - I’ll get these errors when I’m using the .JPG format. In fact, I had 20 large character player sheets in JPG as I was trying to keep the file size down but I kept getting memory errors.
So, I converted them to PNG-8, which actually increased the size of the files but I no longer had memory crashes. So, for me, I’ve stuck with PNG for all my graphics and I haven’t had a memory crash since.
Thanks Joel, this new build is awesome. I ended up splitting it into 2 x 4 slices which did help reduce the initial memory splurge considerably; It works at -Xmx512M now which means I should be able to add the GBII maps to the top eventually, making it even bigger.
Unfortunately, anyone using the current Vassal release will need to set -Xmx1300M!
Thus spake Joel Uckelman:
I had something of a breakthrough this evening and got #2 to work,
so now we can load large images directly to memory-mapped files.
As a test, I pasted all five maps from DAK II together (10660x3400)
to see what memory usage was like when loading those. Peak was 422MB,
but when I triggered the garbage collector after the map was done
loading, memory usage went down to 21MB.
Try this:
nomic.net/~uckelman/tmp/vass … vn2820.zip
I’ll bet you’ll be able to load the Case Blue map now with only 512MB
heap. I still don’t recommend making single maps that size, because
the entire map has to load before you start getting tiles to display.
If you divide the map into 10 or so smaller pieces, you won’t have to
wait as long for the map to start displaying.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
I’ll give that a try in a tic.
I’d prefer to divide it into as few slices as possible because I need to create a bunch of zones on each slice for the grid numbering to work. For example on slice 1 of 8 I need zones for Map A, B, F. On slice 2 of 8 I need zones for Map A, B, C, D, F, G. You get the idea, it’s becoming a job!
Thus spake “bsmith”:
Hmm, I wonder if we should change the way zones work, so that they are
a subcomponent of Map instead of Board.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
You’re a genius Joel! This is a big breakthrough, it will enable grogs to use Vassal to play wargames which would be practically impossible to play in real life due to their physical size.
It will be a good sales pitch on CSW when the time comes 
I think to many existing mods would get affected if you make Zone a
subcomponent of Map instead of Board. Some zones might be board dependent
etc… and moving zone to map level would no longer work in those cases
Post generated using Mail2Forum (mail2forum.com)
Thus spake “Tim McCaron”:
Maybe we could let Zones be children of Maps in addition to Boards?
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
There are two 60x60 divisions somewhere on the map, see if you can spot them; It’s the grog edition of Where’s Wally 
My main concern is you lose a general use ability to Zone when its tied only
to Map.
For example a game might have 2 boards one square and one triangle, but you
only use one of the boards in certain game or scenario setups. If the zone
was only able to be defined in Map level and its made rectangular in nature,
it wouldn’t work with the triangle board in most likely hood.
If it were to be tied to both, I would recommend instead, to just copy Zone
and all its functionality and make a new trait (say "MapZone’) so one can
distinguish between the two types of Zones. This would also add another
module property that could be exposed: CurrentMapZone = blah as opposed to
CurrentZone = blah
Post generated using Mail2Forum (mail2forum.com)
They are just above the city/town (cant see name) a few spaces that is at
the mouth of the gulf of ‘something’
Post generated using Mail2Forum (mail2forum.com)
Yep that’s it, right near Rostov.
When I originally started implementing Zones, I had planned to have them as children of the Map, but is soon became obvious that it just would not work.
A Board is a fixed entity that can contain only one image, so Zones make sense - A Zone is completely contained on one Board and can contain a sub-grid.
A Map can contain different maps in different scenarios, so a ‘Map Zone’ will only make sense for games with a single fixed map setup.
Stuff like setting up extra zones on boards that have to be cut up is a one-time cost.
Cheers,
Brent.
This got me thinking.
Counter art shows a lot of information in a small space. But for the map, the same amount of detail is not needed. For example, have a look at this screenshot. The map is zoomed to 44%. Some counter information is lost but the map detail is fine. Now if I hover over a stack, I can still see the full counter details if needed.
Mention was made a while ago about having different levels of scaling for maps and counters. Now this would suit Case Blue really well. The theory being that the full sized map could be physically much smaller and shown at 100% magnitude, whereas the counters stay large and could be shown at say 30% magnitude. The players could then rely on the Mouse Over Stack viewer to see the finer details of each stack. The reduction in map size would probably be worth the trade-off.
Thus spake “bsmith”:
The Eastern Front did tend to swallow troops…
I think they’re north of Rostov.
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
This would be possible with limited functionality. You would not be able to have subgrids in a MapZone. I really don’t see this as a high prioity as it’s only purpose is to save the odd Module Designer a half hour of work once when setting up a module. Your probably looking at 10’s of hours of programming to get it right.
Logically, Zones do not belong on Maps, it would be a horrible hack.
Regards,
Brent.
Brent Easton
Analyst/Programmer
University of Western Sydney
Email: b.easton@uws.edu.au
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
Thus spake “Brent Easton”:
Would it be useful to have some container between Maps and Boards, then,
which doesn’t have the context-change issue that Maps have?
–
J.
Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org
Post generated using Mail2Forum (mail2forum.com)
Yeah I don’t think it’s worth worrying about, too few people need it and it only saves a small amount of time. I think there are many other more needed time saving features which should take higher priority.