We get a lot of questions from users about heap sizing and the options which
affect memory management. Few users understand the issue, and what we
recommend to them must seem like magic. E.g., in the past few weeks, I’ve
recommended both increasing and decreasing the max heap to various people.
(Needing to decrease the max heap due to an OOM exception is
counterintuitive.)
There is no other desktop application I regularly encounter where this kind
of thing is an issue. We should think hard about whether there is a way to
set better defaults than the ones we have now. Since the Player is a
separate process launched from the Module Manager, we have the opportunity
to do some calculations before setting the max heap size for any Player we
launch. Is there anything adaptive we can do here?
While there isn’t any way to get the amount of RAM in a machine in a
platform-independent way, it would be a tiny amount of code to get this
information on each platform we support. Knowing how much RAM there is could
help with determining an appropriate max heap, because the amount of RAM is
an upper bound. We also know that some amount of RAM should be left for the
OS to prevent thrashing. Is there some rule we can extract from this?
Knowing when to increase max heap size is easy. Knowing when to decrease it looks quite difficult.
OOM errors should be caught separately and a very instructive message should be provided (instead of a bug report…)
I’m not sure how we can catch swap problems. We could have a failsafe mechanism whereby if the module takes much too long to do something, then it offers a warning. I would have absolutely no idea where to put that catch though. We saw a recent issue with an imported module which is likely caused by swapping and I suspect the problem in that case is just the display.
This brings up another issue. Whatever you do for heap size could also be done for memory mapping of images.
It seems like witchcraft at the moment, but it hadn’t really bothered me. Do you think this could drive people away?
M.
P.S.: no one on Windows should even use a swap partition. I’ve never seen any good come of it. It’s not even a great idea under Linux, although the OS appears to use it sensibly while there are no obvious problems, but even then, if the hard-drive starts thrashing, it’ll take a minute to stop the process. That’s just my opinion, however.
I’ve identified a few conditions when decreasing the max heap is what’s
needed:
The max heap is too close to the amount of physical RAM. This will
cause thrashing, since the OS won’t have enough RAM for itself.
There is no contiguous block of memory large enough for the heap.
This seems to happen on 32-bit Windows machines a lot—people have 4GB
of RAM which is mostly empty, but Windows has decided to allocate the
address space so that there’s no contiguous, unallocated 1GB chunk, and
the JRE needs a contiguous chunk of memory for the heap.
The heap is too small for holding all of the maps, but large enough
to prevent allocation of a contiguous chunk of address space for memory-
mapped files.
There might be more, but these are the ones which come to mind right now.
We can detect at least some cases of #1, e.g., when the max heap is set
to more than the available RAM.
For increasing max heap:
Everything on the heap with a size worth worrying about is going to be
an image. When the heap gets full, that will cause the image cache to
drop some images. I think that poor performance (rather than OOM errors)
due to lack of heap space is going to correlate very strongly with images
being droped from the cache frequently. I’ve never measured this, but it
might be possible to detect a difference between normal and excessive
cache clearances.
They are supposed to be handled separately right now. If you see a bug
dialog due to an OOM condition, that that is itself a bug.
I don’t follow. What’s the display problem?
Yes, I’d been thinking about that in conjunction with this.
I think it’s the thing we get asked most about by users these days, and it
involves some esoteric settings. I imagine that a lot of users will wonder
“Why can’t they fix it so I don’t have to worry about this? No other program
is this way.” Now, we both know that’s not a fair complaint, since most
programs will happily try to use as much memory as they want, consequences
be damned. (E.g., try loading an image which can’t fit into RAM in any bitmap
editor. It will gamely try, and in the process cause your machine to thrash
until you reboot.) Nonetheless, this is what they’ll be thinking, and so
we could benefit from heading this off.
You can’t, using Java only. There are other (platform-specific) ways, though.
Yes, Windows is dumb.
No, it’s the same problem as #2, but for memory-mapped files. After
allocating the heap, the address space is too fragmented to allocate a
memory-mapped file of the requested size. This also seem to happen on
Windows only.
Your fix doesn’t fix the bug, because the thing which is broken is the
error handling code. There’s only one place in the whole codebase right now
(in ImageSaver, where it’s trying to determine how large a BufferedImage it
can allocate) where we should be catching an OOME.
Well, it’s also a feature of Java programs, since they are (by
sandbox design?) required to declare how much memory they need at
start up time. They can’t dynamically request more memory from the
operating system. That differs from how most other non-Java
applications work.
To someone who’s not in the know, it’s pretty obscure sounding. It occurs to me that all of this is pretty darned mysterious if you don’t know how dynamic memory is allocated in general. Then there’s the added layer of Java memory management.
Detecting #2 and #3 seems overly heroic, but it’s certainly reasonable to query the system for total RAM and do some reasonable heuristic, such as setting the initial heap to use 25% and max heap to 75%
Does the amount of RAM used by your OS increase as you add more RAM? If
not, then percentages aren’t going to track the maximum feasible heap very
well. E.g., I suspect that the same OS running on systems with 512MB and
16GB RAM, respectively, will not require 128MB on the former and 4GB on the
latter—though in each case, that’s 25%.
I wonder if there’s a fixed amount which would safely work, e.g., 128MB?
I would think more like 256mb would be a safer bet at minimum as a fixed amount
Windows will use somewhere between 120 - 200mb for OS (a general consensus Ive noticed on tech forums), plus throw in background stuff like video drivers, sound drivers, printer drivers, whatever etc… and and your avg user is going to be sucking around 256mb right on start up, and we havent even launched any typical programs yet that users might generally always have running such as email client, av program (which seem to like to gobble mem) and so on…
From: Joel Uckelman uckelman@nomic.net To: VASSAL Engine Forums Mailing List messages@forums.vassalengine.org Sent: Wednesday, March 25, 2009 11:47:45 AM Subject: Re: [Developers]automatic heap sizing
Thus spake Rodney Kinney:
Does the amount of RAM used by your OS increase as you add more RAM? If
not, then percentages aren’t going to track the maximum feasible heap very
well. E.g., I suspect that the same OS running on systems with 512MB and
16GB RAM, respectively, will not require 128MB on the former and 4GB on the
latter—though in each case, that’s 25%.
I wonder if there’s a fixed amount which would safely work, e.g., 128MB?
True,
I may not have been too clear on what I was thinking (normal )
Essentially once you query RAM we categorize and populate values against it
i.e
physical/initial heap/max heap
Basically we set the machine in 4 tiers. Those w/ 2GB or more wont care and will see optimal settings / jvm performance w/ vassal. Those w/ less than 512 - time to upgrade. That could even stretch into those w/ 512 because w/ bigger modules they are going to see h.d paging eventually at some point and that is the slow death on our way to an eventual OOM
From: Joel Uckelman uckelman@nomic.net To: VASSAL Engine Forums Mailing List messages@forums.vassalengine.org Sent: Wednesday, March 25, 2009 12:45:06 PM Subject: Re: [Developers]automatic heap sizing
Thus spake Timothy Mccarron:
If we reserve 256MB of the physical RAM, that’s going to make it difficult
for users who have only 512MB RAM to start with.
)