VASSAL 4 Design, Broad Overview

Development
1

I’m posting here a broad overview of my current thinking on the VASSAL 4 architecture. I realize this is vague in parts. I’m trying to avoid implementation details as much as possible, as those tend to derail design discussions. I also hate writing this kind of thing, as (a) I know that it does not explain well what I have in mind, but (b) I do not know in what ways it fails to explain what I have in mind. In order to clarify what I’m proposing and to find potential flaws, I ask that everyone read this document and comment on it. Think about how the architecture would affect you, about whether you have use cases I have not considered, about places where malicious users could cause trouble, etc.

The document is posted below, or available as a PDF if you prefer.

1 Design overview

The description here is purposely abstract, in order to give an idea of the shape of the plan without wandering off into the weeds of implementation details.

We define a game representation language for specifying game components, game history, game state, and changes to game state. The game engine API provides functions for loading game components, history, and state; for applying state changes and reporting their results; and for inspecting the current state. A game server implements the game engine API, and talks to clients using it.

Clients wanting to play in real-time find each other by registering themselves with one or more meetup servers. Clients wishing to store game history or state online for real-time or asynchronous play can do so using a storage server.

We define a game presentation language for associating game components with their visual representations. Graphical clients may use this information for displaying a graphical representation of game state with which users may interact. A complete game module would consist of a description of the rules-relevant features of the game components using the game representation language and a description of the visual features of the game components using the game presentation language.

There will be converters for producing VASSAL 4 game modules from game modules in other formats.

2 Game representation language

2.1 Description

The game description language provides a method for defining all rules-relevant game components—pieces and their properties, piece faces and their properties, maps and locations and features on them—and their visibility to the players, as well as the game state and changes to it, which together comprise the game history.

2.2 Discussion

∙ Scripting: Scripts can be attached to game objects to automate a series of changes.

3 Game engine API

3.1 Description

The game engine API specifies methods for interacting with the game state and game history. Provided will be methods for querying the properties of pieces and maps visible to each player; for getting and setting where in the history the next change will be inserted; for applying new changes to the history.

3.2 Discussion

∙ Keeping the game engine API as simple as possible will make implementing it (as a game server) and clients which use it easier.

4 Game server

4.1 Description

A game server implements the game engine API and fulfills requests for clients. Clients will be encouraged to use locally-hosted game servers if possible, but we will maintain a public game server for situations when there is no client capable of hosting a game server.

4.2 Discussion

∙ In order to avoid our public game server being a singe point of failure, we could provide clients with an updatable list of public game servers, similar to the list of meetup servers described below.

∙ Clients can coordinate with each other to determine what game server should be used, as follows: Each client joining a game attempts to connect to each other client. Any client which is capable of having an open connection with all other clients is capable of hosting the game server for that game. The clients can then elect one such to run the game server.

∙ Should the client hosting the game server drop out, how do the remaining clients recover, in the case where the game server was holding data not visible to all players? There seems to be a tension here between robustness and data visibility.

∙ A game server may additionally need to track ephemera not part of the game state and notify interested clients when these things change. A possible example of something which should not be logged but might be of interest to clients is cursor location. This should not be considered as part of the game engine API, but possibly as part of some simple client data API to which clients may subscribe (or ignore, as it has no effect on state or history).

∙ Since a client will not have data not visible to it, how will we handle recovery of game history if a player becomes unavailable?

∙ Similarly, we need some way to ensure that full game history can become available when games end.

5 Meetup server

5.1 Description

A meetup server maintains a list of username-IP address-game name-module hash tuples which clients may query or add to. A meetup server may periodically expire these tuples.

5.2 Discussion

∙ When a client connects to a meetup server, what it’s doing is saying ”hey, I’m here”: The client tells the meetup server its IP address, along with a game name and a module hash. When a client asks a meetup server who is connected, these records are returned.

∙ Clients will be expected to have a built-in default list of meetup servers, which the user can modify. This way, all a client needs to do to find other clients is ask each meetup server it knows about for the list of player it has.

∙ If one meetup server is down, there’s no problem, since presumably there would be more than one in operation.

∙ We could offer clients the opportunity to update their default list of meetup servers from our own web server—that would be a single point of failure, but only for updating the list, not for connecting to a meetup server.

∙ Each meetup server could have a group: Either it would be part of the official public ones, or part of some other public group, or part of a private group which would require a username and password for connections.

∙ Should meetup servers also facilitate chat, or should that be done through some other mechanism?

∙ Should meetup servers also facilitate posting of persistent notices? Server-wide? Module-specific? For finding opponents? Directed at particular users? Should this be done through a different mechanism or not at all?

6 Storage server

6.1 Description

A storage server will store game history for clients and make it accessible to them over the internet. Use cases are games with more than two players, and games played by the same user on different devices—situations where distributing logs or save files by email could be onerous. A storage server needs to provide a methods for setting and retrieving data, and possibly also deleting it.

6.2 Discussion

∙ No authentication is necessary if data is identified with keys of sufficient length to make them unguessable—however, we might still want user accounts in order to set per-user storage limits.

∙ We would want to temporarily block IPs which make several consecutive requests for non-existent keys, to thwart brute-force searches of the key space.

∙ Incoming data should be validated to ensure that we don’t end up storing arbitrary junk.

∙ Should stored data expire? If so, how to notify the user of this in order to prevent data loss?

∙ Could we increase robustness by having storage serves in multiple locations, all of which use a distributed hash table for storage?

7 Clients

7.1 Description

A game client will send changes to and receive changes from a game server. A client may use definitions from the game presentation language to provide a graphical representation of the game state.

7.2 Discussion

∙ The game server handles calculating the effects of changes on the game state. Clients need not do this themselves, which should ease implementation of clients.

∙ Clients will not have any hidden information not available to the player using them.

8 Game presentation language

8.1 Description

The game presentation language provides a method for mapping objects defined in the game representation language to graphical assets.

8.2 Discussion

∙ Clients can use the game state and definitions expressed in the game presentation language to display game state graphically.

9 Converters

9.1 Description

There are four file formats which we should aim to convert to the V4 file format:

VASSAL 3.2 The proposed module format is new, not an extension of the existing V3.2 module format. In order to make use of the large library of modules VASSAL has now, we must provide a module converter. It should be possible to programmatically convert any module which does not contain custom code so long as we ensure that each V3.2 module component maps to some structure in the new game language format. Modules containing custom Java classes will not be manually convertible, at least not in their entirety, as we have no way of determining programmatically what the custom classes do. This should still allow us to convert the vast majority of modules automatically.

Aide de Camp 2 Michael Kiefte reverse-engineered the Aide de Camp 2 format and wrote converter for which is in V3.2. Presently the converter code is the only available documentation of the ADC2 format. We (probably Michael) need to document the ADC2 format, and use that to write a converter which produces V4 output instead of V3.2 output.

ZunTzu ZunTzu stores its game modules in a simple XML format. Converting this to V4 should be trivial.

CyberBoard Michael Kiefte is reverse-engineering and documenting the CybyerBoard gamebox format. We need to write a converter using this information.

9.2 Discussion

∙ Converters can be standalone programs, and need not be part of any particular client.

∙ For any given module, conversion is a one-time task. Converters need not run everywhere clients do. E.g., if we don’t provide converters for use on tablets, that’s fine. Users can convert on a real computer.

∙ The ADC2 and CyberBoard converters will involve reading and manipulating binary data into something structured, as well as writing images as PNGs, and as such are best suited for C++.

∙ It might be possible to do the ZunTzu converter largely, or even entirely, using XSLT, since we would be converting one XML format to another.

∙ The ZunTzu converter is the one I intend to complete first, as soon as we have an initial draft of the game representation language, as it will provide an easy source of ready-made modules for testing purposes.

∙ We need to create some simple test modules for each converter in order that we can create acceptance tests for the converters.

2

2 Game representation language

2.1 Description

The game description language provides a method for defining all
rules-relevant game components–pieces and their properties, piece
faces and their properties, maps and locations and features on
them–and their visibility to the players, as well as the game state
and changes to it, which together comprise the game history.

2.2 Discussion

  • Scripting: Scripts can be attached to game objects to automate a
    series of changes.

I understand where this is going, but I see a potential problem. Right now
VASSAL supports doing a number of things in order to automate tedious tasks
such as discarding cards, flipping counters in response to an event, move
all counters of a certain type to box, etc. These things are not
considered tedious in a FtF game but can be time consuming in a computer
application.

Although we could relegate all these activities to a script, there are a
couple of issues:

  • the point of entry for new module developers is necessarily higher.
    Graphical programming is more approachable to non programmers than others
    and will encourage more module developers.

  • the part that I’m really concerned about is portability: Right now
    getting anything with scripting through the Apple Store is painful. A lot
    of people are going to say “well, who cares: The people at Apple are
    demagogues and should be ignored until they vanish into obscurity in 40 or
    50 years.” Part of that is true: The Apple people are difficult to deal
    with at best. However, there’s a reason that they have that restriction
    there and it’s not entirely a bad one. That being said, they do have a
    Python interpreter and I don’t know how those developers got away with that
    – I’ll have to find out.

  • the other part of portability is simply porting an interpreter. That may
    or may not be a concern with newer devices.

Overall, I think simplicity is good, but there are a lot of features that
VASSAL currently uses that shouldn’t be tossed in the direction of an
interpreter. I have to admit that I’m a bit biased right now as I’ve been
thinking about the iOS port for some time (and even playing around with
some possibilities regarding the interface). Apple’s policies may evolve in
the meantime.

Also, the choice of a specific interpreter language may prove contentious.
I would suggest a balance between portability, ease of use for new
developers and small footprint.

On a positive note, I strongly agree with everything else!

  • M.
3

Thus spake Michael Kiefte:

  • Scripting: Scripts can be attached to game objects to automate a
    series of changes.

I understand where this is going, but I see a potential problem. Right now
VASSAL supports doing a number of things in order to automate tedious tasks
such as discarding cards, flipping counters in response to an event, move
all counters of a certain type to box, etc. These things are not
considered tedious in a FtF game but can be time consuming in a computer
application.

Although we could relegate all these activities to a script, there are a
couple of issues:

I see from what you’ve said that I need to clarify two points about
scripting.

  • Scripting is server-side, not client-side.

  • Traits in V3.2 will correspond to simple scripts which can be
    automaticaly produced by a module editor.

  • the point of entry for new module developers is necessarily higher.
    Graphical programming is more approachable to non programmers than others
    and will encourage more module developers.

Module developers won’t need to know anything about scripting in order
to make modules. What I’m expecting to see in a graphical module editor
is lists of affordances, similar to the traits we have now. Selecting
“Can Rotate” for a piece will produce a script (most likely a one-liner)
for rotating the piece. The script here has the same status as the
sequence in the V3.2 buildFile which defines a trait. Only if you want
something differing from the behaviors configurable from the editor
would you need to edit the script yourself.

  • the part that I’m really concerned about is portability: Right now
    getting anything with scripting through the Apple Store is painful. A lot
    of people are going to say “well, who cares: The people at Apple are
    demagogues and should be ignored until they vanish into obscurity in 40 or
    50 years.” Part of that is true: The Apple people are difficult to deal
    with at best. However, there’s a reason that they have that restriction
    there and it’s not entirely a bad one. That being said, they do have a
    Python interpreter and I don’t know how those developers got away with that
    – I’ll have to find out.

Clients don’t need to execute scripts, only game servers. Being unable
to put in Apple’s store a MacOS or iOS client which can also host a game
server would be annoying, but ultimately not a huge problem.

  • the other part of portability is simply porting an interpreter. That may
    or may not be a concern with newer devices.

I’d like to have a game server able to run on every architecture where
we have a native client, but it’s not a hard requrement.

Overall, I think simplicity is good, but there are a lot of features that
VASSAL currently uses that shouldn’t be tossed in the direction of an
interpreter. I have to admit that I’m a bit biased right now as I’ve been
thinking about the iOS port for some time (and even playing around with
some possibilities regarding the interface). Apple’s policies may evolve in
the meantime.

Would you mind describing what you’ve found so far?

Also, the choice of a specific interpreter language may prove contentious.
I would suggest a balance between portability, ease of use for new
developers and small footprint.

There’s a consideration here of which I became aware a few weeks ago but
haven’t had time to run to ground yet, viz.: Python is apparently
devilishly hard to sandbox.

What I want is to give scripts access to some objects from the game
model and nothing else. No access to the filesystem. No access to the
network. Bounded CPU time and memory. This way, broken or evil scripts
can’t do anything bad. You’d think Python would the perfect language
for this kind of thing, but from what I’ve read, several official
attempts to produce sandboxed Python have failed. (The difficulties
seem to come down to late binding and introspection.) This might narrow
our choices to Lua and JavaScript. If anyone can turn up some useful
information about how to do this properly with Python, I’d be grateful,
as I’d much prefer we use Python to Lua or JavaScript.

–
J.

4

First thing, I don’t think you can consider design without some thoughts on implementation as any particular implementation will limit what is possible to achieve in the design given finite CPU, finite memory and finite bandwidth. The design process has to be an iteration with implementation with a readiness in the early stages to bin it all and start again with what you learned going the wrong way.

But let’s see if I understand the document right. The current system will be broken into two parts: the game server with the game state and one or more clients with the GUI. The meetings servers and storage servers I’ll skip since I think they’re just adding helpful features which are not central to Vassal. I see the minimum requirements of a new system is to provide the same capabilities to the players as the current system and to be able to run all existing (except for a limited group with their own code) modules without problem.

Question 1) Does the client has access to its own definition of the module or does it have to retrieve everything from the server? I’m assuming its the latter at the moment. Which brings us to bandwidth immediately. If a client is displaying a section of a map does it obtain the full bitmap of the map (or the image file) from the server and scale it as required or a sub bitmap scaled appropriately? Or does it obtain individual tiles and build the image from that? What about map overlays? Does the server pre apply them or does the client get both images and merge them? Does it get piece images from the server or the fragments from which images are built? In the latter case is part or all of the scripts/properties sent to the client so it can assemble them correctly? In the former case does the client query the server every time it displays a piece to check its image is still valid or does it just download the latest image?

Question 2) How dumb is the client? This relates closely to the previous question. If the user right clicks on the window will the client know that this click is on a piece? Will it know what menu is appropriate for the piece and be able to display it? Or will it obtain it from the server as required? In the former case there is the problem of the client information being stale due to changes at the server and in the latter case you’ve the responsiveness problems if the link to the server is over the internet.

Question 3) When the user selects an option from the piece menu presumably a request is sent from the client to the server requesting the appropriate action. Will this just be a request to run a script? How will the server verify that the client state matches its own so the request is still valid? Will the server execute each incoming request in turn sequentially or will it execute them in parallel? How will it report to the clients that a change has occurred? Will it just flag the change and the clients rebuild their data from the server? Or will it track what data each client has and just tell them individually?

Question 4) Will the server log the ongoing game automatically (like an SQL server)? Will a server support multiple simultaneous games (presumably yes for the public server)? This has implications for how much load a game can put on the server.

I’ve run out of time to ask more at the moment. I’ll come back later with some more questions.

5

At one stage we discussed using Public Key Cryptography so that ‘clients’ could have data that was not visible to them, but not be able to do anything useful with it.

I think this is a very important issue, it is vital that all remaining clients be able to recover and continue should the host game server become unavailable.

Closely related to this issue is how you handle offline play (i.e. PBEM)? PBEM is an essential feature of Vassal.

6

Thus spake george973:

Question 1) Does the client has access to its own definition of the
module or does it have to retrieve everything from the server?

Clients will need to have modules locally. Game servers won’t be
handling graphics at all.

Question 2) How dumb is the client?

My intent was to impose as few requirements on clients as possible.
A client is something which knows how to talk to a game server. I was
intending for the reference clients we build to provide good examples of
how the data can be displayed and interacted with—but what a client
does as a result of talking to a game server is up to the implementers
of that client.

This relates closely to the previous
question. If the user right clicks on the window will the client know
that this click is on a piece? Will it know what menu is appropriate for
the piece and be able to display it? Or will it obtain it from the
server as required? In the former case there is the problem of the
client information being stale due to changes at the server and in the
latter case you’ve the responsiveness problems if the link to the server
is over the internet.

Anthing having to do with user interaction with a GUI is outside the
scope of game servers. Game servers won’t know anything about clicks
or windows, etc. They’re concerned only with game state.

I see that I neglected to mention an important part of the game
representation language, namely that it should provide a way of
specifying possible actions for pieces, which can be displayed by
clients on context menus, e.g., if they so choose.

Question 3) When the user selects an option from the piece menu
presumably a request is sent from the client to the server requesting
the appropriate action. Will this just be a request to run a script?

Yes.

How will the server verify that the client state matches its own so the
request is still valid?

By matching hash values. Every change in the change log will be named by
a hash value, the result of hashing the hash value of the previous
change and the body of the change. (This is similar to git.) In order to
post a change to the game server, a client sends the change and the
expected hash following that change. If the state the client had when it
sent the change is the same as the state when the server receives the
change, the hashes will match. If the hashes don’t match, then the
client’s state was stale and the change is rejected.

Will the server execute each incoming request in
turn sequentially or will it execute them in parallel?

Sequentially, for any single game.

How will it
report to the clients that a change has occurred?

This depends on the implementation of the game server API. I don’t want
to get too far into the weeds with this right now, but I was expecting
that we’d do this over HTTP, so clients would be notified using long
polling. I have a design document written for this already, but I’d like
not to get into that yet.

Will it just flag the
change and the clients rebuild their data from the server? Or will it
track what data each client has and just tell them individually?

The game server won’t track client state at all. Whatever information
about the game state clients need, they can ask for.

Question 4) Will the server log the ongoing game automatically (like an
SQL server)?

Yes, except that I’m not sure how this is like an SQL server.

Will a server support multiple simultaneous games
(presumably yes for the public server)? This has implications for how
much load a game can put on the server.

It depends on the game server implementation. The game server API I have
in mind doesn’t make any assumptions at all about whether multiple games
are running at the same time.

–
J.

7

I see from what you’ve said that I need to clarify two points about
scripting.

  • Scripting is server-side, not client-side.

Oh, okay. That’s interesting. I’m not opposed to that. I can see that
bringing up other issues but I’ll have to think about that.

  • Traits in V3.2 will correspond to simple scripts which can be
    automaticaly produced by a module editor.

Which are then passed on to the server. Interesting idea. Obviously this
addresses all my concerns.

Module developers won’t need to know anything about scripting in order

to make modules. What I’m expecting to see in a graphical module editor
is lists of affordances, similar to the traits we have now. Selecting
“Can Rotate” for a piece will produce a script (most likely a one-liner)
for rotating the piece. The script here has the same status as the
sequence in the V3.2 buildFile which defines a trait. Only if you want
something differing from the behaviors configurable from the editor
would you need to edit the script yourself.

Okay, so now I understand that what you’re talking about with respect to
scripting is implementation not interface. That’s great.

Clients don’t need to execute scripts, only game servers. Being unable
to put in Apple’s store a MacOS or iOS client which can also host a game
server would be annoying, but ultimately not a huge problem.

I agree. That’s not to say that Apple won’t go along with it though. It’s
just an unknown.

Overall, I think simplicity is good, but there are a lot of features that
VASSAL currently uses that shouldn’t be tossed in the direction of an
interpreter. I have to admit that I’m a bit biased right now as I’ve
been
thinking about the iOS port for some time (and even playing around with
some possibilities regarding the interface). Apple’s policies may evolve
in
the meantime.

Would you mind describing what you’ve found so far?

Mostly, I’ve been working on interface stuff. iOS is severely limited in a
number of respects with respect to interface, while it enables you to do a
number of things that you can’t do on a desktop with respect to gestures.
It’s going to require some experimentation to get the right translation of
interaction to match the capabilities of the desktop version.

I assume you’re going to have a clear doc/model/controller separation, so
it shouldn’t be do difficult. However, the iOS model code is going to look
very different from anything else.

What else I’ve found is a lot of bureaucracy. Apple is infuriating. But
I’ve lowered my standards and have adapted to it.

There’s a consideration here of which I became aware a few weeks ago but

haven’t had time to run to ground yet, viz.: Python is apparently
devilishly hard to sandbox.

This is a stupid question, but…

Have you considered just developing a new stripped-down language that is
very focused on this particularly application? It’s pretty easy to put
something together with lex and yacc. How sophisticated do you want this
to be?

  • M
8

Thus spake Brent Easton:

∙ Should the client hosting the game server drop out, how do the
remaining clients recover, in the case where the game server was
holding data not visible to all players? There seems to be a tension
here between robustness and data visibility.

At one stage we discussed using Public Key Cryptography so that
‘clients’ could have data that was not visible to them, but not be able
to do anything useful with it.

This came up yesterday in the IRC channel. There are several things we
could do, not all of which cover all situations.

For games occuring on an internet game server, we could give clients the
full game state if they request it, but then notify the other players
that this has happend. In the case where you have to recover state, this
notification is expected; in the case where someone is trying to cheat,
everyone is alerted.

For games not occcuring on an internet game server, the problem is
different, as then we don’t have a trusted third party holding the full
game state—in that case, there’s nothing we can do to keep hidden
state secure anyway.

Closely related to this issue is how you handle offline play (i.e.
PBEM)? PBEM is an essential feature of Vassal.

Offline play is the same as online play, the only difference being that
your client is talking to a game server which is local.

–
J.

9

Thus spake Michael Kiefte:

Clients don’t need to execute scripts, only game servers. Being unable
to put in Apple’s store a MacOS or iOS client which can also host a game
server would be annoying, but ultimately not a huge problem.

I agree. That’s not to say that Apple won’t go along with it though. It’s
just an unknown.

Yes. All I mean is that if we can’t put clients which are able to spawn
local servers into Apple’s store, then we have the option to put clients
which have no local servers there.

There’s a consideration here of which I became aware a few weeks ago but

haven’t had time to run to ground yet, viz.: Python is apparently
devilishly hard to sandbox.

This is a stupid question, but…

Have you considered just developing a new stripped-down language that is
very focused on this particularly application?

Yes. I’ve also considered eating broken glass. :slight_smile:

I don’t want to design a DSL for scripting, for reasons I give below.

It’s pretty easy to put
something together with lex and yacc. How sophisticated do you want this
to be?

I’ve used lex and yacc a lot, so I’m quite familiar with what would be
needed to do this. It’s not the parsing which is the hard part. What
is hard is devising a language which is sufficiently expressive and has
consistent semantics. This is hard even for people who are language
designers. (Have a look at PHP if you an example of an epic failure on
this point.)

By using an existing language, we don’t have to spend time getting the
semantics right, debugging the interpreter, designing and implementing
a standard library, etc. We will also not be the sole experts on the
scripting language, so if someone has a question about how to write
something in it (or why they’re getting a syntax error, etc.), there is
a chance that they can find an answer somewhere else.

–
J.

10

Hmm, let’s see if I’m understanding this right.

The client gets the state from the game server with ah hash code and displays the game however. The user selects an action for a piece by some means. The client sends the action plus has code to server. If hash code matches the server executes the action and completely recomputes the state. The client now retrieves the new state and hash code and updates its display.

11

Thus spake george973:

Hmm, let’s see if I’m understanding this right.

The client gets the state from the game server with ah hash code and
displays the game however. The user selects an action for a piece by
some means. The client sends the action plus has code to server.

Yes.

If hash
code matches the server executes the action

Yes.

and completely recomputes the state.

No. There’s no need for the game server to completely recompute the
state. The game server needs only to apply the change sent by the
client.

The client now retrieves the new state and hash code and
updates its display.

The client won’t need to retrieve the new state in its entirety if
it is listening for changes.

–
J.

12

If the state change is a piece moving then the visibility/concealment status of many pieces might change if range effects are in use. Removing moved markers could affect all the pieces on the map. Running a script could fire off other scripts. properties of pieces could change everywhere. What happens is only limited by the module designer’s imagination.

Applying the change is not just logging something.

If lots of changes are fired off by a single application will the clients get these as one single update with a new hash tag?

Will the server verify the action requested by the client is valid in the current state? - The client could have bugs relating to accepting commands from the user, updating from the server and then sending a command which is no longer appropriate but with the latest hash tag.

13

Thus spake george973:

Thus spake george973:

Hmm, let’s see if I’m understanding this right.

The client gets the state from the game server with ah hash code and
displays the game however. The user selects an action for a piece by
some means. The client sends the action plus has code to server.

Yes.

If hash
code matches the server executes the action

Yes.

and completely recomputes the state.

No. There’s no need for the game server to completely recompute the
state. The game server needs only to apply the change sent by the
client.

If the state change is a piece moving then the visibility/concealment
status of many pieces might change if range effects are in use. Removing
moved markers could affect all the pieces on the map. Running a script
could fire off other scripts. properties of pieces could change
everywhere. What happens is only limited by the module designer’s
imagination.

Applying the change is not just logging something.

What I’m disputing is “recompute”. Nothing already computed needs to
be computed again.

The client now retrieves the new state and hash code and
updates its display.

The client won’t need to retrieve the new state in its entirety if
it is listening for changes.

If lots of changes are fired off by a single application will the
clients get these as one single update with a new hash tag?

No, for two reasons:

  1. Clients won’t get changes not visible to them. It won’t always be
    the case that when client A makes a state change that client B will
    receive it. (E.g., consider a game where player A has a secret display
    on which he can make moves, but where player B isn’t even allowed to
    know that a move has been made there.)

  2. Each visible-to-you change has to have it’s own hash in order to
    support stepping through the history.

Will the server verify the action requested by the client is valid in
the current state?

Yes.

–
J.

14

What do you mean by “listening for changes”? Won’t any changes to the state be sent to all connected clients automatically?

Do you mean this literally? No dynamic properties (ie calculate when required)?

I hope not. Taking this literally it means that when I click to clear “moved” markers and then replay it every single piece having its marker cleared is a separate step in the log file. With 1,000+ pieces each side in OCS Case Blue - definitely not!!

Besides clients don’t work in terms of state changes. They can’t if they have no access to scripts. They just send to the server a request from the player to do something which might be moving a piece or executing the script attached to some menu item. A state change will occur because of this but it is entirely in the remit of the server. The clients only receive the visible effects to display them.

15

Thus spake george973:

The client won’t need to retrieve the new state in its entirety if
it is listening for changes.

What do you mean by “listening for changes”? Won’t any changes to the
state be sent to all connected clients automatically?

I mean that a client will have to have an open connection to receive
changes.

What I’m disputing is “recompute”. Nothing already computed needs to
be computed again.

Do you mean this literally? No dynamic properties (ie calculate when
required)?

Yes.

  1. Clients won’t get changes not visible to them. It won’t always be
    the case that when client A makes a state change that client B will
    receive it. (E.g., consider a game where player A has a secret display
    on which he can make moves, but where player B isn’t even allowed to
    know that a move has been made there.)

You will still need to send a hash code to B otherwise he’ll not be
able to do anything as his client’s has code will not be correct.

No. If B’s history has not changed, then he’ll receive no new hash.
The hashes are intended to be per-player.

Why are we calling it a hash code? A simple state number would do
incremented for every command executed.

A state number won’t work for maintaining consistency. Suppose that
client A tries to make a change to state 4. Before client A’s request
arrives at the server, the server receives a request from client B to
undo change 4 (rolling us back to state 3) and then receives a request
from client B to make a change to state 3—which increments the state
number back to state 4. When client A’s request arrives at the server,
the server now has no way to tell if the state client A was intending
to change was the original state having number 4, or the current state
having number 4.

The clients can’t calculate a
state hash since they don’t know the full state.

Clients don’t calculate a hash on the full state. A client calculates
a hash from the previous hash and its view of the current change.

  1. Each visible-to-you change has to have it’s own hash in order to
    support stepping through the history.

I hope not. Taking this literally it means that when I click to clear
“moved” markers and then replay it every single piece having its marker
cleared is a separate step in the log file. With 1,000+ pieces each side
in OCS Case Blue - definitely not!!

No, that’s not what I mean. One kind of change (which I expect to be
rather common) will consist of a list of other changes. Your example
of clearing moved markers would be one change containing a change
for each peice having its moved marker cleared.

I think I should post the game server API now so you can see in more
detail what I’m talking about.

–
J.

16

Here’s an (incomplete) draft game server API. This is a RESTful API, intended to be accessed over HTTP. (Ainulindale has been trying to convince me that it should be done with SOAP instead of REST. I have not yet grasped what advantage he thinks that has, though I could still be convinced.) In particular, the hash mechanism is described. Hopefully this makes it clearer.

17

I’ve read the API and have a few questions.

  1. I assume that GET /games///… will if it hits a subtree (eg /games///state/pieces) will send the whole subtree in response. If this is not the case then bin the design:

a) the time to load over the internet will be enormous - Case Blue with 2000 pieces each having at least eight properties (with a value) would require 34,000 request to get piece data = 68,000 to do in twice with 10 msecs response (which is fantasticly fast) = 680 seconds = 11 minutes. and that allowing zero time for data transmission just turnaround time.

b) It would be impossible to fetch a guaranteed consistent state of the game. Since the state could in theory change between each fetch you do. So you could fetch the list of pieces and before you fetch a piece’s data it could have been deleted.

2)The only useful fetch is GET /games// since it’s the only one that gives you a complete consistent game state together with the hash value which lets you make changes to it. Unless other GETs are intended to be used to get changes to the state. Missing is how the client knows the state has changed and what changed in it and the new hash value.

  1. POST / games / < gid >/ < pid >/ history / current - this changes the hash value. What use is that? Unless the server is supposed to deduce the required change from the change to the hash value? Or you’ve posted the changes earlier and this is just to mark the changes as done. In which case presumably the server must lock other clients out until your changes are done (with the possibility of it all hanging if the client goes away) or keep all the changes you’ve posted (using them to reply to any get requests you make) until the hash change comes in before doing them all as one thing - or rejecting them because another client got in in the meantime and changed the state. The other thing is why does the client calculate the new hash? Does the server validate the new hash? There’s a lot of requests going on here. Each request in itself may lead to an invalid state of the game (eg consider a stack of two pieces one in position zero and the other position one) changing the their order in the stack means changing the first to position one - two pieces in position one now and none in position zero - and then the second to position zero - state valid again. The server must calculate new hashes for all the other players if the changes are visible to them. How do the other clients get told the has has changed?

  2. GET / games / < gid >/ < pid >/ history / < hash > -this returns the change at hash visible to the player. How is the change packaged? Something like “state/pieces//property/visible face/1,…/…/…//property/visible face/1” to flip two pieces in a stack? Will the piece changes be sent twice (as they occur twice in URL tree)? By the way using pid for player id and pid for piece id is a recipe for confusion. I assume the change sent is the change from the previous state.

  3. POST / games / < gid >/ < pid >/ history / < hash > - Deletes all changes after current which are visible to player pid in game gid , then sets the change at hash to the request payload. Fails if hash(current+payload) = hash. On success, current is set to hash. This request may modify the state for pid and the trees of other players. - I’m not sure if I understand this. If the hash an existing value in history in which case this is an undo or is it a new value in which case this is a new move and the has will become the current value - in which case why was there a post to change the current hash?

LIGHT HAS DAWNED - current does not represent the latest game state but where a client is stepping through it. In which case the design allows players to go back to a previous point and change history. In an email game it’s a CHEATER’S CHARTER since you can do a die roll and if you don’t like the result you can step back, delete it and do it again until you like the result. Also it’s dangerous. If I’m stepping through a log file and accidentally do something in the middle then the rest of the log is deleted!!!

I would say as a design principle: no history is deleted. If a player takes back a move then the move and taking it back are both recorded as events. This isn’t GIT: we don’t want branches.

Also if you are not deleting things you don’t need hashes. A simple incrementing state number will do.

  1. Why are pieces in the URL tree twice. Anything which duplicates data is bad.

  2. Many requests to the server are not changes but requests to run scripts (eg if a piece has a menu item “Flip” then client does not know what to do -it just asks the server to execute the attached script). The changes will all be made at the server so the client cannot calculate a hash as the result is not known to it.

8) The server understands the game structure. If the structure changes in 4.1 you'll need a different server. So you could end up running servers for 4.0, 4.1, 4.2, ... unless you're clever and can distinguish the type of client. Remember your users are not going to update to the latest version very quickly (they may be playing an ongoing game which could take several years to complete).
18
8)

in the last post is really )

Damn software is too clever

19
  1. Why are pieces in the URL tree twice. Anything which duplicates data
    is bad.

I was wondering this myself. I assume that these are pointers to the same
structures.

  • M.
20

Thus spake Michael Kiefte:

  1. Why are pieces in the URL tree twice. Anything which duplicates data
    is bad.

I was wondering this myself. I assume that these are pointers to the same
structures.

They are different ways of accessing the same data, yes. Any reasonable
game server implementation would store the pieces once internally.

The important point here is that state/surfaces//pieces is the
collection of pieces on the surface identified by , while
state/pieces is the collection of all pieces.

–
J.