I’m working on changing the preferences file format to something human-
readable (this was something Michael and I discussed some time ago, IIRC).
Here’s my proposal, in the form of an XML DTD:
<!ELEMENT preferences (manager,module*)>
<!ATTLIST preferences version CDATA #REQUIRED>
<!ELEMENT manager (node*)>
<!ELEMENT module-global (node*)>
<!ELEMENT module (node*)>
<!ATTLIST module name CDATA #REQUIRED
version CDATA #IMPLIED>
<!ELEMENT node (node*)>
<!ATTLIST node name CDATA #REQUIRED
type (boolean|byte|char|short|int|long|
float|double|string|color|rect|file) #IMPLIED>
value CDATA #IMPLIED>
Further, note that it’s possible to nest nodes. We don’t have any
hierarchical preferences right now, but in the event that we wanted them,
it would just be a matter of doing this:
I’d like comments on this. Does it seem like a sane way of storing prefs
to everybody? Is there anything missing here which anyone wants?
(I already have the code written to convert old preferences files to this format, and to load preferences from this format, so the work is substantially done.)
The format is fine. Are you sure it’s a good idea to put the preferences for all modules under the same XML root? You’d be locking yourself forevermore into needing a separate service to manage simultaneous changes to preferences, even for different modules.
rk
I was expecting Preferences to only be written back to file when the Module
Manager, or a Player or Editor is closed. File locking is sufficient to
prevent two of those from trying to write to the preferences file at once.
Is that what you’re worried about?
We don’t even have that kind of protection right now—it think it’s
possible to corrupt the current preferences file by closing two Players
at once.
You’re right, we do have this problem now. I was just thinking that if we’re going to change the format of the Preferences file, maybe we could change it to something that would help with the problem. For example, you could make a Preferences directory with a separate file for each module, so you’d only need locking in the rare case of simultaneous edits to the global preferences.
Having one file per module wouldn’t remove the need for locking module
preferences. If you have two Players open with the same module, then
closing them at the same time could potentially corrupt the preferences
file for that module if it isn’t locked for writing.
There’s also a serious issue with file naming if you do that. If you
name module preference files after the modules, then you can end up
with all sorts of weird characters in filenames, but maybe you’ll be
on a filesystem which simply doesn’t support having those characters
in filename, or is case-insensitive. Otherwise, if you don’t name prefs
files after the modules, then you need some other way of finding them.
Reading this again, I’m confused by what you’re saying here. I think what
I’m proposing eliminates this problem. Are you saying that you think it
doesn’t? If not, how not?
It definitely doesn’t eliminate the problem of simultaneous edits to the Preferences. It does include type information with the data, which is good. But I’d argue that it makes the simultaneous-edit problem a little harder to handle, since changes to different modules would have to be merged into a single XML document, whereas today they just change different entries in the Zip archive.
Could you explain why you think it doesn’t? If you have to get a lock to
write to the preferences file, and all writers wait for locks, under what
circumstances can you get a simultaneous edit?
It’s simple to handle changes in XML: Read the XML in as a DOM Document,
apply the changes, and write back.
Sure, a global lock will solve the problem. I just meant that the new format in itself didn’t address the problem. By creating a single document you are forcing yourself into a single lock, whereas a different solution could give you the option of having multiple finer-grain locks. (Multiple locks might be an advantage if, for example, rogue processes holding onto the lock turn into a problem.)
The new format was never intended to address the problem of simultaneous
writes—that’s what file locking was intended to do. The point of storing
prefs as XML was to:
Put the prefs in a human-readable format.
Make it possible to have arbitrary characters in module names, and still
have the prefs work properly.
If VASSAL had an exception while it had a lock, the lock would be released
by a finally block. If the JVM crashed and its process ended, the lock would
be released by the OS. The only way I can see that the lock would continue
to be held is if the JVM became a zombie while holding the lock. So we’re
looking at a window of less than a second for an event which would have to
be an OS or JVM bug if it occurred. I’m not too worried about that, as we
don’t often have reports of the JVM going zombie.
First of all, I think the idea of an XML or other human readable
preferences format is good.
In addition to XML, one could consider just using a properties file.
It has the benefit of being a more easily human readable. It has
support in the standard Java distribution rather than requiring an
additional library. The format and use of the file is simpler. But
it is a flat format rather than a structured one. That would mean
either adopting a hierarchical naming convention for properties or
else using separate property files for each module. Either of those
may run into some of the module name issues Joel is concerned about.
It also seems that there are some tradeoffs between a single,
comprehensive preferences file and having separate ones for each module.
Single File:
Easier to manage at OS level.
No proliferation of files.
No pressure for a separate preferences directory.
Allows arbitrary naming, since the names are all internal to the
preferences file.
Multiple Files:
More robust. Problem writing a file (corruption) only affects a
single module.
Each file is smaller, perhaps an issue if a user has lots of
Vassal modules
Less contention for write locks (not likely significant in any
case)
Need to link preference file with module, requiring OS-compatible
naming
Better portability, in that one could easily transfer preferences
for a single module to another machine
I’m not really sure how important the issue with file naming is likely
to be, since the module itself already has to have a name that is file-
system compatible. So the filename for the module could also be used
as the base for any preferences file.
Here you’re conflating two things, module name, and module filename,
which are not necessarily the same. Currently, module preferences are
associated with the module name, not the module filename. This has several
consequences:
There can be preference files inside the preferences ZIP archive
which are not compatible with the user’s filesystem. (This can happen,
e.g., if you have modules where the names contain slashes. I have
several of these.) The upshot of this is that you can get a preferences
ZIP archive which is not unzippable by normal ZIP tools.
There can be many modules which have the same filename, but are not
in fact the same module. (I have several of these, usually named
test.vmod.) The preferences for these will not clobber each other, so
long as the modules have different names.
Two modules which have the same name but different filenames will
share preferences. Chances are this is the desired behavior. This way,
preferences will persist across module file renaming.
Hence, in response to what you’re proposing: I think basing preferences
on module filename will result in people’s preferences becoming unglued
from the modules they belong to.
No additional library is needed for parsing and writing XML. The classes
needed for this are in javax.xml.* and org.w3c.*, all of which are part
of the standard Java API.
Theese three I don’t give much weight to. If someone had every VASSAL
module there is, that would amount to under 1000 prefs files in one
directory.
This one matters a lot, unless we want to attach prefs to module filenames
instead of module names.
I have about 100 modules sitting around. As XML, the combined preferences
for all of them is 131kB.
I would file this under “no one will ever notice”. Contention does not
matter for something which will happen twice over the lifetime of each
instance of an app.
This one I see as a big PITA, for reasons already stated.
Granted, but is that something that many people try to do? I would think
that wanting to transfer all of one’s prefs would be a more common thing,
and in that case having a single file has the advantage.