VASSAL 4 Design, Broad Overview

[uckelman]

Thus spake george973:

I’ve read the API and have a few questions.

1. I assume that GET /games///… will if it hits a subtree
   (eg /games///state/pieces) will send the whole subtree in
   response. If this is not the case then bin the design:

My understanding is that one generally expects to get a collection
of the children of a node when you request a node in a REST design.

a) the time to load over the internet will be enormous - Case Blue with
2000 pieces each having at least eight properties (with a value) would
require 34,000 request to get piece data = 68,000 to do in twice with 10
msecs response (which is fantasticly fast) = 680 seconds = 11 minutes.
and that allowing zero time for data transmission just turnaround time.

Any client fetching all the properties of all the pieces one at a time
would be terminally stupid. I wasn’t intending for clients to keep
current by sending requests to the state part of tree. Clients should
be doing that by processing the changes the server sends out.

b) It would be impossible to fetch a guaranteed consistent state of the
game. Since the state could in theory change between each fetch you do.
So you could fetch the list of pieces and before you fetch a piece’s
data it could have been deleted.

There should be a way to request the full state at once—I didn’t
include it in the draft. It’s unclear to me that GET on
/games///state is really the right way to do that.

2)The only useful fetch is GET /games// since it’s the only
one that gives you a complete consistent game state together with the
hash value which lets you make changes to it.

No, that would just get you “state”, “players”, and “history” back,
following the usual REST paradigm.

Unless other GETs are
intended to be used to get changes to the state.

No, GETs are nullipotent. A GET having a side-effect violates RFC 2616.

3. POST / games / < gid >/ < pid >/ history / current - this changes the
   hash value. What use is that? Unless the server is supposed to deduce
   the required change from the change to the hash value?

Nothing about the input should be deducible from the hash value if we
are using a cryptographically strong hash.

The other thing is why does the
client calculate the new hash? Does the server validate the new hash?

Yes, the server calculates the new hash from the old hash and the change
the client sent. If the new hash the server computed does not match the
new hash the client computed, then either the client and server started
with different old hashes (likely) or the change sent by the client got
corrupted in transit. In either case, the change should not be applied.

There’s a lot of requests going on here. Each request in itself may lead
to an invalid state of the game (eg consider a stack of two pieces one
in position zero and the other position one) changing the their order in
the stack means changing the first to position one - two pieces in
position one now and none in position zero - and then the second to
position zero - state valid again.

This would be one change, not three.

The server must calculate new hashes
for all the other players if the changes are visible to them. How do the
other clients get told the has has changed?

There needs to be a URL (not in the draft) which clients can long-poll,
over which they’d receive changes from the server.

4. GET / games / < gid >/ < pid >/ history / < hash > -this returns the
   change at hash visible to the player. How is the change packaged?

The payload of the reply would be some XML representing what changed.
It makes sense for this to be the same XML which would appear a game
log (for player ).

By the way using pid for player id and pid for piece id is a
recipe for confusion.

Yes, I’m aware of this, as is reusing n as an index. At this point,
being scrupulous about that would not not make the diagram clearer.

I assume the change sent is the change from the
previous state.

No, the change is the change named by . It need not be the case
that == the value of current.

5. POST / games / < gid >/ < pid >/ history / < hash >
   [snip]
   I’m not sure if I understand this. If the hash an existing value in
   history in which case this is an undo or is it a new value in which case
   this is a new move and the has will become the current value - in which
   case why was there a post to change the current hash?

If is already in the history which the game server has, then
the request will fail. must be new.

LIGHT HAS DAWNED - current does not represent the latest game state but
where a client is stepping through it.

“current” is just that—it’s a pointer to which state is the current
state.

In which case the design allows
players to go back to a previous point and change history. In an email
game it’s a CHEATER’S CHARTER since you can do a die roll and if you
don’t like the result you can step back, delete it and do it again until
you like the result.

You can do the same now in an email game. I don’t see any way to
prevent such a thing in a game where the turns are conducted without
the assistance of a trusted third party, regardless of the design.

Also it’s dangerous. If I’m stepping through a log
file and accidentally do something in the middle then the rest of the
log is deleted!!!

I’m not convinced that it should be possible to make changes in the
middle of log playback, at least not without something warning you that
you’re about to truncate your log. Anyway, you could always reload your
log.

I would say as a design principle: no history is deleted. If a player
takes back a move then the move and taking it back are both recorded as
events.

This will make for intollerable logs, I’m afraid.

Also if you are not deleting things you don’t need hashes. A simple
incrementing state number will do.

A state id which is not a hash can’t be used for checking history
integrity.

7. Many requests to the server are not changes but requests to run
   scripts (eg if a piece has a menu item “Flip” then client does not know
   what to do -it just asks the server to execute the attached script). The
   changes will all be made at the server so the client cannot calculate a
   hash as the result is not known to it.

This can be accommodated as follows: The hash sent by the client has the
script identifier as part of its input. The server can still verify that
the client had the current state. However, the server need not add
anything with that hash to the log—the server can update the state
using the actual change produced by the script, store that change with
its hash in the log, and both the change and its hash to the client.

8. The server understands the game structure. If the structure changes
   in 4.1 you’ll need a different server.

I don’t see why you think this. Can you explain further?

–
J.

[Ainulindale]

Joel,

Here’s an (incomplete) draft game server API. This is a RESTful API,

intended to be accessed over HTTP. (Ainulindale has been trying to
convince me that it should be done with SOAP instead of REST. I have not
yet grasped what advantage he thinks that has, though I could still be
convinced.) In particular, the hash mechanism is described. Hopefully
this makes it clearer.

Did you try the equivalent SOAP design?
I still dislike a lot the idea to use a system which was not designed for
RPC (that is HTTP protocol) for that purpose, as well as not using an
implicit grammar validation system (XML grammar that is) to ensure that a
given request will be syntactically exact.
Plus, defining a proper WSDL ensures that the way a given developer can
interact with the system is self-documented and can even be introspected
which isn’t the case with REST. IMHO, it allows us to have the grammar
evolve if need be in kind of a smoother way.

Anyways, following our discussions ad nauseam on IRC, let me reverse the
roles a bit there: why do you think REST is better than SOAP?

–
Julien Cassignol
ainulindale.net

[Ainulindale]

Georges,

On Wed, Jun 5, 2013 at 1:24 PM, george973 george973@btinternet.com wrote:

I’ve read the API and have a few questions.

1. I assume that GET /games///… will if it hits a subtree
   (eg /games///state/**pieces) will send the whole subtree in
   response. If this is not the case then bin the design:

a) the time to load over the internet will be enormous - Case Blue with
2000 pieces each having at least eight properties (with a value) would
require 34,000 request to get piece data = 68,000 to do in twice with 10
msecs response (which is fantasticly fast) = 680 seconds = 11 minutes.
and that allowing zero time for data transmission just turnaround time.

b) It would be impossible to fetch a guaranteed consistent state of the
game. Since the state could in theory change between each fetch you do.
So you could fetch the list of pieces and before you fetch a piece’s
data it could have been deleted.

As told by Joel later on, the design envisioned there (and talked through
in front of beers) implied that you start with a shared state (i.e. a
scenario), so you have no need to get all the pieces at once for starters.
On that, you apply state changes which are hashed and ordered to ensure
continuity. So at a given time, what you actually may need to request in
the worst case is initial state reference (stored in the game module) plus
N state changes where N is the number of actions taken by the players.
That’s if you want to “replay” everything from the start.

2)The only useful fetch is GET /games// since it’s the only

one that gives you a complete consistent game state together with the
hash value which lets you make changes to it. Unless other GETs are
intended to be used to get changes to the state. Missing is how the
client knows the state has changed and what changed in it and the new
hash value.

As to how to know what has changed: as always we have two techniques, PULL
or PUSH.
I think it is quite obvious that PUSH (from the game server perspective)
will be hard due to network considerations (firewalls, NAT and such).
So IMHO, the client should pull at regular intervals and/or before any
action is undertaken by the user.

3. snip
   for all the other players if the changes are visible to them. How do the
   other clients get told the has has changed?

See before.

4. GET / games / < gid >/ < pid >/ history / < hash > -this returns the
   change at hash visible to the player. How is the change packaged?
   Something like “state/pieces//property/visible
   face/1,…/…/…//property/visible face/1” to flip two pieces in a
   stack? Will the piece changes be sent twice (as they occur twice in URL
   tree)? By the way using pid for player id and pid for piece id is a
   recipe for confusion. I assume the change sent is the change from the
   previous state.

Yes, the change is the change from the previous state so it’s, in effect,
structured data (the model) explicitely saying the value which changed due
to the actions undertaken for the state, visible for the player requesting
it (so for instance, drawing a card would imply saying there’s one more
card in the hand of the opposite player, as opposed to one given card in my
hand).

5. POST / games / < gid >/ < pid >/ history / < hash > - Deletes all
   changes after current which are visible to player pid in game gid , then
   sets the change at hash to the request payload. Fails if
   hash(current+payload) = hash. On success, current is set to hash. This
   request may modify the state for pid and the trees of other players. -
   I’m not sure if I understand this. If the hash an existing value in
   history in which case this is an undo or is it a new value in which case
   this is a new move and the has will become the current value - in which
   case why was there a post to change the current hash?

LIGHT HAS DAWNED - current does not represent the latest game state but
where a client is stepping through it. In which case the design allows
players to go back to a previous point and change history. In an email
game it’s a CHEATER’S CHARTER since you can do a die roll and if you
don’t like the result you can step back, delete it and do it again until
you like the result. Also it’s dangerous. If I’m stepping through a log
file and accidentally do something in the middle then the rest of the
log is deleted!!!

This may be possible but it doesn’t mean that it should be doable
everywhere. A given state may not be cancelable/undoable by any other
player than the actual owner of the state (the player who did it), at the
discretion of the module developer.

I would say as a design principle: no history is deleted. If a player
takes back a move then the move and taking it back are both recorded as
events. This isn’t GIT: we don’t want branches.

I think this sounds sensible.
And to answer to Joel as well: I don’t think logs would be intolerable.
Actions aren’t undone that often. It’s also “part of the game” and shows
the fair-play, IMHO.

Also if you are not deleting things you don’t need hashes. A simple

incrementing state number will do.

See what Joel said.

6. Why are pieces in the URL tree twice. Anything which duplicates data

is bad.

Because pieces aren’t necessarily tied to surfaces (turn counter for
instance, or reserve). This is why I don’t like this REST model: it’s not
structured, it’s an arbitrary structured data for a non structured system.
Bleh!

8. The server understands the game structure. If the structure changes
   in 4.1 you’ll need a different server. So you could end up running
   servers for 4.0, 4.1, 4.2, … unless you’re clever and can distinguish
   the type of client. Remember your users are not going to update to the
   latest version very quickly (they may be playing an ongoing game which
   could take several years to complete).

This is also a reason for me not to use an implicit way to communicate with
the server (REST) as opposed to an explicit way (SOAP).

My 2 cents!

–
Julien Cassignol
ainulindale.net

[Ainulindale]

Joel,

They are different ways of accessing the same data, yes. Any reasonable

game server implementation would store the pieces once internally.

The important point here is that state/surfaces//pieces is the
collection of pieces on the surface identified by , while
state/pieces is the collection of all pieces.

Which is useless IMHO as this is in the game module.
If this is actually a way to access non-surfacic pieces, then I’m ok with
it. If it’s a way to get the definition of pieces, I can’t see the point as
they are in the module.

–
Julien Cassignol
ainulindale.net

[uckelman]

Thus spake Julien Cassignol:

Did you try the equivalent SOAP design?

I haven’t been able to convince myself that it would not simply be
a great deal of typing to arrive at something with the same
functionality in a more opaque way.

I still dislike a lot the idea to use a system which was not designed for
RPC (that is HTTP protocol) for that purpose, as well as not using an
implicit grammar validation system (XML grammar that is) to ensure that a
given request will be syntactically exact.

I do not believe that malformed requests will be a problem, for several
reasons: (1) If the request paload uses the the game rep langauge, we can
already validate the payload. (2) Whether it’s REST or SOAP, the part of
the request which is not the payload doesn’t amount to much. The HTTP
parts should be handled by the HTTP lib we use. what’s left? (3) We
should not build clients which send malformed requests.

Plus, defining a proper WSDL ensures that the way a given developer can
interact with the system is self-documented and can even be introspected
which isn’t the case with REST. IMHO, it allows us to have the grammar
evolve if need be in kind of a smoother way.

After having looked at WSDL, I simply don’t believe that it is self-
documenting.

Anyways, following our discussions ad nauseam on IRC, let me reverse the
roles a bit there: why do you think REST is better than SOAP?

The number of operations a game server needs to perform is small: Get
and set the log as a whole, get the current state, apply or undo
changes. I don’t forsee that changing, because the operations we need
are driven by the logical features of games—which I also don’t forsee
changing, and which at this point, we understand well.

It seems to me that the value of doing this with REST rather than SOAP
is:

• The definition of the interface can be similar in complexity to the
  use of the interface.
• We don’t draw in any addtional libraries with REST.
• We will need to document the server interface either way, so with
  REST we don’t have to do that twice.

–
J.

[uckelman]

Thus spake Julien Cassignol:

Joel,

They are different ways of accessing the same data, yes. Any reasonable

game server implementation would store the pieces once internally.

The important point here is that state/surfaces//pieces is the
collection of pieces on the surface identified by , while
state/pieces is the collection of all pieces.

Which is useless IMHO as this is in the game module.

I don’t mean all piece definitions here, I mean all pieces existing
the current state. The latter are not part of the module.

If this is actually a way to access non-surfacic pieces, then I’m ok with
it. If it’s a way to get the definition of pieces, I can’t see the point as
they are in the module.

Now that you mention it, there might be no reason to have the state
tree at all—or at least that it would not be of any use to the kind
of client we’re aiming to build.

–
J.

[Ainulindale]

On Thu, Jun 6, 2013 at 11:31 AM, Joel Uckelman uckelman@nomic.net wrote:

Thus spake Julien Cassignol:

Did you try the equivalent SOAP design?

I haven’t been able to convince myself that it would not simply be
a great deal of typing to arrive at something with the same
functionality in a more opaque way.

Can you honestly say that doing GET /gameserver?id=X&id2=Y is clearer?

I do not believe that malformed requests will be a problem, for several

reasons: (1) If the request paload uses the the game rep langauge, we can
already validate the payload. (2) Whether it’s REST or SOAP, the part of
the request which is not the payload doesn’t amount to much. The HTTP
parts should be handled by the HTTP lib we use. what’s left? (3) We
should not build clients which send malformed requests.

For 1) agreed.
For 2) it depends, particularly when it comes to authentication (cf.
WS-Security, more on that later).
For 3) that’s not counting on people developing 3rd party clients

After having looked at WSDL, I simply don’t believe that it is self-
documenting.

Look again!
My initial XSD draft: ainulindale.net/vassal/vassalengine.xsd
My WSDL draft: ainulindale.net/vassal/gameserver.wsdl
Sample request for get state:
ainulindale.net/vassal/getrequest.xml
Sample response for get state:
ainulindale.net/vassal/getresponse.xml

Bear in mind that the only that I wrote myself (other than the XSD I wrote
ages ago) is the WSDL. Everything else was generated as is through SOAPUI.

The number of operations a game server needs to perform is small: Get

and set the log as a whole, get the current state, apply or undo
changes. I don’t forsee that changing, because the operations we need
are driven by the logical features of games—which I also don’t forsee
changing, and which at this point, we understand well.

Point taken, but I’d rather be sure.

It seems to me that the value of doing this with REST rather than SOAP
is:

• The definition of the interface can be similar in complexity to the
  use of the interface.

Can’t see your point there.

• We don’t draw in any addtional libraries with REST.

Nor with SOAP. Or let me put it that way: if you don’t reuse anything, you
will end up building a library for either solution.

• We will need to document the server interface either way, so with
  REST we don’t have to do that twice.

?! Why twice?

Joel, you’re doing the devil’s advocate again, aren’t you?

–
Julien Cassignol
ainulindale.net

[Ainulindale]

On Thu, Jun 6, 2013 at 11:44 AM, Joel Uckelman uckelman@nomic.net wrote:

Which is useless IMHO as this is in the game module.

I don’t mean all piece definitions here, I mean all pieces existing
the current state. The latter are not part of the module.

I disagree on that: “all pieces of the current state” seems wrong to me.
You should only have the changes in a state. A state is basically a diff
to me.

If this is actually a way to access non-surfacic pieces, then I’m ok
with
it. If it’s a way to get the definition of pieces, I can’t see the point
as
they are in the module.

Now that you mention it, there might be no reason to have the state
tree at all—or at least that it would not be of any use to the kind
of client we’re aiming to build.

Well to me describing a state is a way to say how to go from point A to B.
So a state says “here are the values which changed”. If you actually want
to go from A to Z, you’ll apply all the changes as a whole, from A to Z.
So in this sense, a state is kind of a linear set of differences.

–
Julien Cassignol
ainulindale.net

[Ainulindale]

Some additional info:

On Thu, Jun 6, 2013 at 12:24 PM, Julien Cassignol ainulindale@gmail.comwrote:

For 1) agreed.
For 2) it depends, particularly when it comes to authentication (cf.
WS-Security, more on that later).
For 3) that’s not counting on people developing 3rd party clients

Here is a list of standards (emphasis on standards even though I’m sure
that Joel will complain ) which might be of use to us through SOAP:

en.wikipedia.org/wiki/WS-ReliableMessaging - reliable messaging
obviously
en.wikipedia.org/wiki/WS-Security - security and authentication
(this might change depending on lots of things)
en.wikipedia.org/wiki/WS-Transaction - transactional purposes
although not sure this would apply.

Being a security consultant, I’d like to emphasize on the security part.
Yesterday we used username tokens and such. Today we’re using a mix of SAML
or OAuth tokens. This may change, the technoly may evolve and the needs
we’d have as well. Using such a thing would abstract us from handling it on
the protocol level (up to a certain limit).

–
Julien Cassignol
ainulindale.net

[george973]

I’ve read the discussion betwenn Uckelman and Ainulindale with interest and I’ll come back later with a more general view on it but here I’ll just touch on some points briefly

#1

The server understands the game structure. If the structure changes

in 4.1 you’ll need a different server.

I don’t see why you think this. Can you explain further?

Let us say that in 4.1 you introduce “surfaces” with “globules” as well as “maps”. If you send “globules” to a 4.0 client it will choke so you need either a separate server or some intelligence to tailor its responses to the level of the client. Or to take a practical example when I did by new map stuff I added a property to maps “canRotate” and default it to true. If this was the 4.1 change then the server would have to add this property to all maps in all modules so a 4.1 client could deal with it properly but 4.0 client wouldn’t understand it. There needs to be some mechanism to accomodate this which could be server end or client end or both.

ASIDE: At the moment a piece exists on a map and its position on the map is a point (x,y). But the position could be something other than a point depending on the type of the map. For instance in a setup map the only thing that matters is its position in a sequence of pieces and bits of text like “set up within 3 hexes of 31.17” as its actual position would depend on how the client laid out the map according to the amount of space available (like a browser laying out an html page). Or in a hypothetical 3D game it could be (x,y,z) - VASL could use this already as it has multilevels in a hex at present inidicated by special markers, with the information in the position the markers could be generated automatically. Or considering a piece palette as a map the position is something like (“german”,“infantry”,“division”,6th place). So although position is a property of a piece its possible contents are a property of the map it is on and can only be interpreted by the map).

#2a

In which case the design allows
players to go back to a previous point and change history. In an email
game it’s a CHEATER’S CHARTER since you can do a die roll and if you
don’t like the result you can step back, delete it and do it again until
you like the result.

You can do the same now in an email game. I don’t see any way to
prevent such a thing in a game where the turns are conducted without
the assistance of a trusted third party, regardless of the design.

You can’t since the system current logs undos. So it would log the dice roll and the undo. You can’t make it impossible for some one to cheat (they can always run under a debugger ) but if you make it easy then people may be tempted.

#2b

Also it’s dangerous. If I’m stepping through a log
file and accidentally do something in the middle then the rest of the
log is deleted!!!

I’m not convinced that it should be possible to make changes in the
middle of log playback, at least not without something warning you that
you’re about to truncate your log. Anyway, you could always reload your
log.

At present you can flip your pieces to see the back, move them to see what’s underneath or do anything you like between steps of playing the logfile back. Suddenly having these changes wipe the rest of the log (which is what your API sepcified) is not a good idea. Currently anything you do in the middle of a log file will stand unless it is affected by something later in the playback. Perhaps a better solution would be to allow changes locally but undo them when the player continues to step the log file.

#2c

I would say as a design principle: no history is deleted. If a player
takes back a move then the move and taking it back are both recorded as
events.

This will make for intollerable logs, I’m afraid.

It doesn’t now. Indeed my experience is that is very little taking back of moves etc.

#2d

LIGHT HAS DAWNED - current does not represent the latest game state but
where a client is stepping through it. In which case the design allows
players to go back to a previous point and change history. In an email
game it’s a CHEATER’S CHARTER since you can do a die roll and if you
don’t like the result you can step back, delete it and do it again until
you like the result. Also it’s dangerous. If I’m stepping through a log
file and accidentally do something in the middle then the rest of the
log is deleted!!!

This may be possible but it doesn’t mean that it should be doable
everywhere. A given state may not be cancelable/undoable by any other
player than the actual owner of the state (the player who did it), at the
discretion of the module developer.

What I worrying about is the player who did it undoing it!! (a unfavourable die roll, a piece move which revealed an opponent’s position)

#3

As to how to know what has changed: as always we have two techniques, PULL
or PUSH.
I think it is quite obvious that PUSH (from the game server perspective)
will be hard due to network considerations (firewalls, NAT and such).
So IMHO, the client should pull at regular intervals and/or before any
action is undertaken by the user.

This assumes that there is no connection maintained between cleint and server. With an open connection PUSH is trivial. Also opening a connection has a large overhead, so not to be done unless required.

#4

As told by Joel later on, the design envisioned there (and talked through
in front of beers) implied that you start with a shared state (i.e. a
scenario), so you have no need to get all the pieces at once for starters.
On that, you apply state changes which are hashed and ordered to ensure
continuity. So at a given time, what you actually may need to request in
the worst case is initial state reference (stored in the game module) plus
N state changes where N is the number of actions taken by the players.
That’s if you want to “replay” everything from the start.

N could be very large. There are games such as Case Blue and Der Weltkrieg Campaign Game with thousands of pieces and hundred of moves. N could exceed a million and given that a state change would take quite a few bytes (especially if it’s in human readable text) even those of us with fibre optic connections are going to wait (assuming Vassal server is very high bandwidth, high end machine)

[Rindis]

@george973

N could be very large. There are games such as Case Blue and Der Weltkrieg Campaign Game with thousands of pieces and hundred of moves. N could exceed a million and given that a state change would take quite a few bytes (especially if it’s in human readable text) even those of us with fibre optic connections are going to wait (assuming Vassal server is very high bandwidth, high end machine)

I have been trying (and largely failing) to follow this conversation, but the same thought hit me here. The F&E module will easily generate huge logs of this type (lots of pieces, lots of turns, and lots of shuffling pieces to sub-maps such as the battle board to inflate the log). More to the point, this starts sounding like how Cyberboard works, and while the ability to be able to step through an entire game is really neat, the F&E gamebox is capable of bogging down performance, requiring periodic ‘refreshes’ to a new start state to clear out the log (as I understand it, I’ve never really used Cyberboard).

Now, in all of this that I don’t understand, just what is the proposed handling of inserting new actions? Most PBeM games rely on inserting reactions to an existing log, and then going forward with more of the log, unless the reaction obviously invalidated everything. The talk of changing things wiping later portions of the log would break PBeM, but I haven’t gone back to check (and try to parse) the original statements that cover that sort of thing.

[george973]

SOAP and REST are protocols for what I call one shot interactions:

Client connects, sends request, gets replay and disconnects. Repeat until done. This is suitable for browser interface where the client is likely to take its time, have a cup of coffee, etc

A better model for what we are after could be the X Protocol where the client connects, send requests and receives replies until done and then disconnects. After all, the Vassal server will be more like a Chess or Go server.

Perhaps it would be an idea to step back and list what we need to achieve, what we would like to achieve and let that drive our choice of implementation.

(a) Email:

(i) Need: To receive a log file of opponent’s moves, to step through this one by one or jump to end, to do our own moves creating alog file which when done we can send to our opponent.

(ii) Like: Match the starting state of received log file with end state of last sent log file to ensure they match.

(b) Internet:

(i) Need: Several players link together so that they can all see each others’ moves and play the game like they were all ftf.

(ii) Like: Some resilience so an individual player can drop out and reconnect without data loss other than local for the player who drops out.

(c) Security

(i) Need: Enough to prevent John Doe joining a game as himself or masquerading as another player and moving pieces at random or otherwise ruining the game for the other players. We don’t need to keep out the CIA, Russian Mafia, Hackers Anonymous who probably won’t want to get in anyway.

(ii) Need: To prevent malicious third parties placing/executing files on any of the player’s machines.

(iii) Like: If a player drops out of a ongoing big game some mechanism to allow his part to be taken over by some one else with the consent of all the remaining players without providing too easy a loophole for cheating.

(d) Platform

(i) Need: To be able to run clients on the current platforms: Windows, Mac OS X and Linux.

(ii) Like: To be able run clients on Android Tablets, IPads, smartphones etc

(e) Legacy

(i) Need: To run 95% of existing modules

(ii) Need: To run VASL for our ASL friends.

These I think are our targets. Is there anything I’ve missed?

Additionally we should heed the principles of minimum overhead and minimal bandwidth;

(i) Protocol should require the minimum number of connects. Disconnection should be forced only where there is danger of hogging a resource needed for other users.

(ii) Data transmissions for requests and replies should be as short as possible - this means binary data rather than text, compression of very long messages - CPU is much cheaper then internet bandwidth in response time.

[uckelman]

Thus spake george973:

8. The server understands the game structure. If the structure
   changes
   in 4.1 you’ll need a different server.

I don’t see why you think this. Can you explain further?

Let us say that in 4.1 you introduce “surfaces” with “globules” as well
as “maps”.

I don’t believe we will add any object types beyond surfaces, maps
pieces, faces, and possibly something for handling grids and locations,
so I think the rest is moot. (And I’m not even sure that we need maps
for anything other than syntactic sugar, as maps could simply be pieces
which are fixed to their surfaces and have but one face.)

If you send “globules” to a 4.0 client it will choke so you
need either a separate server or some intelligence to tailor its
responses to the level of the client. Or to take a practical example
when I did by new map stuff I added a property to maps “canRotate” and
default it to true. If this was the 4.1 change then the server would
have to add this property to all maps in all modules so a 4.1 client
could deal with it properly but 4.0 client wouldn’t understand it. There
needs to be some mechanism to accomodate this which could be server end
or client end or both.

This is not how properties are intended to work. There will be a few
(which I called “designated” properties the last time I wrote about
this) which clients will be expected to understand, among them
properties indicating coordinates, angles, and which faces are showing.
Other properties hold data to be used by scripts.

ASIDE: At the moment a piece exists on a map and its position on the map
is a point (x,y). But the position could be something other than a point
depending on the type of the map.

I think we should distingush between map locations and plane coordinates.
The latter should only be used in the game domain for situating pieces
which are not on maps.

#2a

In which case the design allows
players to go back to a previous point and change history. In an
email
game it’s a CHEATER’S CHARTER since you can do a die roll and if you
don’t like the result you can step back, delete it and do it again
until
you like the result.

You can do the same now in an email game. I don’t see any way to
prevent such a thing in a game where the turns are conducted without
the assistance of a trusted third party, regardless of the design.

You can’t since the system current logs undos. So it would log the dice
roll and the undo.

It’s irrelevant that the current system logs undos. One can save at the
point before the roll and reload there.

#2c

I would say as a design principle: no history is deleted. If a
player
takes back a move then the move and taking it back are both recorded
as
events.

This will make for intollerable logs, I’m afraid.

It doesn’t now. Indeed my experience is that is very little taking back
of moves etc.

We’ve not had the same experience. In some games I’ve played, I’d say
that moves undone ran at maybe 10x the number of kept moves.

This may be possible but it doesn’t mean that it should be doable
everywhere. A given state may not be cancelable/undoable by any other
player than the actual owner of the state (the player who did it), at
the
discretion of the module developer.

What I worrying about is the player who did it undoing it!! (a
unfavourable die roll, a piece move which revealed an opponent’s
position)

I misread what Ainulindale wrote. What I would say is that a player
should not normally have permission to truncate the log in a way which
would remove changes which revealed information to that player. This
would include things like card flips, dice rolls, etc.

#3

As to how to know what has changed: as always we have two techniques,
PULL
or PUSH.
I think it is quite obvious that PUSH (from the game server
perspective)
will be hard due to network considerations (firewalls, NAT and such).
So IMHO, the client should pull at regular intervals and/or before any
action is undertaken by the user.

This assumes that there is no connection maintained between cleint and
server. With an open connection PUSH is trivial. Also opening a
connection has a large overhead, so not to be done unless required.

So you’re wanting this to be done over a custom protocol instead of
HTTP, then?

#4

As told by Joel later on, the design envisioned there (and talked
through
in front of beers) implied that you start with a shared state (i.e. a
scenario), so you have no need to get all the pieces at once for
starters.
On that, you apply state changes which are hashed and ordered to
ensure
continuity. So at a given time, what you actually may need to request
in
the worst case is initial state reference (stored in the game module)
plus
N state changes where N is the number of actions taken by the players.
That’s if you want to “replay” everything from the start.

N could be very large. There are games such as Case Blue and Der
Weltkrieg Campaign Game with thousands of pieces and hundred of moves. N
could exceed a million and given that a state change would take quite a
few bytes (especially if it’s in human readable text) even those of us
with fibre optic connections are going to wait (assuming Vassal server
is very high bandwidth, high end machine)

I am assuming that it will be possible to request the complete current
game state. This should not exceed more than a few hundred kB
uncompressed. I am saying this on the basis of the little demo I worked
up for Afrika II, which has a state file containing approximately 200
pieces and is 14kB. You could increase the number of pieces by an order
of magnitude in addition to adding a half-dozen properties per piece
beyond location, angle, and showing face, and still be under 1MB. It’s
also the case the XML compresses exceedingly well. The Afrika II example
gzipped 7:1, so it looks like you can get about 100 pieces per 1.6kB.

Lots of commands relayed by our game server currently are hugely
duplicative (in the sense that they contain information that the clients
could already get from the module if they knew to look there), yet we’re
only averaging 186.1kb/s (that’s kiloBITs, not kiloBYTEs) outgoing over
the past month. This puts no noticeable strain on our connection—over
the same period, our web server averaged 2.5Mb/s outgoing. Virtually
nothing is happening on the game server, ever (mean 1-, 5-, and 15-
minute load averages for the past month: 0.03, 0.03, 0.05). So, there’s
no reason to assume a high-end, high-bandwith machine.

Regarding history: If a client already has the game history due to
having a local copy of the log, it won’t need to request it. If a client
never backs up in the history, it won’t need the history at all, so won’t
need to request it. The only case where a client will need to request the
full history would be when it doesn’t have the history and tries to back
up to the initial state. If you undo to the beginning of a log in V3.2
in an online game, you will send data through the game server equivalent
in size to the whole game log—which, as I said, is far larger per
action than is necessary.

So, I don’t see a problem here as we already handle large N in a very
inefficient way without any difficulty.

–
J.

[uckelman]

Thus spake Rindis:

I have been trying (and largely failing) to follow this conversation,
but the same thought hit me here. The F&E module will easily generate
huge logs of this type (lots of pieces, lots of turns, and lots of
shuffling pieces to sub-maps such as the battle board to inflate the
log). More to the point, this starts sounding like how Cyberboard works,
and while the ability to be able to step through an entire game is
really neat, the F&E gamebox is capable of bogging down performance,
requiring periodic ‘refreshes’ to a new start state to clear out the log
(as I understand it, I’ve never really used Cyberboard).

What you’re seeing is due not to the number of actions in the logs,
but rather the duplicative nature of the logs in V3.2. The preponderance
of what’s in a V3.2 log is redundant with information earlier in the log
or in the module.

Now, in all of this that I don’t understand, just what is the proposed
handling of inserting new actions? Most PBeM games rely on inserting
reactions to an existing log, and then going forward with more of the
log

This is entirely new to me. People really do this? Inserting new actions
into the middles of logs sounds like a disaster waiting to happen.

–
J.

[Tim_M]

@uckelman

This is entirely new to me. People really do this? Inserting new actions
into the middles of logs sounds like a disaster waiting to happen.

–
J.

Yes, and probably also yes but nothing prevents it from happening and those that do this seem to manage fine or we’d hear a lot more about it - however we have no data though to see whats going on in the pbem world either even if we were to investigate

[uckelman]

Thus spake Tim M:

“uckelman” wrote:

This is entirely new to me. People really do this? Inserting new
actions
into the middles of logs sounds like a disaster waiting to happen.

–
J.

Yes, and probably also yes but nothing prevents it from happening and
those that do this seem to manage fine or we’d hear a lot more about it

• however we have no data though to see whats going on in the pbem world
  either even if we were to investigate

I have no idea what will even happen in 3.2 if you take an action in the
middle of replaying a log which violates the preconditions for some
other action which comes up later in the log.

It wouldn’t be that hard to test whether new changes can be added
without conflict—this is just adding the new change to the history
before the insertion point and then rebasing the history after the
insertion point onto that. I’m just not sure that it’s a good idea.

–
J.

[Tim_M]

If the action would violate the future logs content the log will override (just tested)

[uckelman]

Thus spake Tim M:

If the action would violate the future logs content the log will
override (just tested)

Does it tell you that’s happening, or does it just happen?

–
J.

[Tim_M]

It just happened. I didn’t look in the error log to see if it generates
anything though

-----Original Message-----
From: messages [mailto:messages-bounces@vassalengine.org] On Behalf Of Joel
Uckelman
Sent: Thursday, June 06, 2013 4:16 PM
To: The mailing list for VASSAL
Subject: Re: [messages] [Developers] VASSAL 4 Design, Broad Overview

Thus spake Tim M:

If the action would violate the future logs content the log will
override (just tested)

Does it tell you that’s happening, or does it just happen?

–
J.

[uckelman]

Thus spake Julien Cassignol:

Well to me describing a state is a way to say how to go from point A to B.
So a state says “here are the values which changed”. If you actually want
to go from A to Z, you’ll apply all the changes as a whole, from A to Z.
So in this sense, a state is kind of a linear set of differences.

What you’re calling “state” here I would call a “transition”. State is
what you have between transitions.

I think it’s useful for a client to be able to get the current state
rather than have to calculate it, as the history could be very long
and the client might not care about having the history.

–
J.