Thus spake “Tim McCarron”:
It just happened. I didn’t look in the error log to see if it generates
anything though
That’s bad. I wonder if this explains some of the “my log did something
weird” bug reports we’ve had over the years but for which we never found
causes?
–
J.
This probably happens in every single PBeM ASL game ever played. You move, I declare fire, I roll, nothing happens, I put down the residual counter, and keep going. That’s just a simple example. “Don’t stop unless I break.” “Don’t stop unless you reveal HIP.” “Don’t stop for anything.” are not uncommon instructions for going through movement logs.
Or, playing F&E PBeM: Both sides set up battle lines in secret from each other. So, I log the set up of my battle line, and then give the instructions for BIR and the like. My opponent hits the point where I start setting up my line, he sets up his line (which means my was effectively hidden from him), continues, picks his BIR, resolves the amount of damage done, declares if he’s doing directed damage, and possibly gives conditional orders on his damage, then sends back for me to decide if I’m doing directed damage, ect.
Frankly, I understand being able to insert actions was one of the design goals of the original VASL. Playing through the MPh without it is just too painful. There’s some problems with it: occasionally, a residual counter will want to go along for the ride as the unit keeps moving, and if someone broke in the middle of a move, you’ll have to go fetch him back afterward if you continue the log, but it’s all minor, easily understandable stuff.
On Thu, Jun 6, 2013 at 6:04 PM, george973 george973@btinternet.com wrote:
This may be possible but it doesn’t mean that it should be doable
everywhere. A given state may not be cancelable/undoable by any other
player than the actual owner of the state (the player who did it), at
the
discretion of the module developer.What I worrying about is the player who did it undoing it!! (a
unfavourable die roll, a piece move which revealed an opponent’s
position)
Well if a player wants to undo it, as long as it’s logged, it’s not a
problem to me.
As to how to know what has changed: as always we have two techniques,
PULL
or PUSH.
I think it is quite obvious that PUSH (from the game server
perspective)
will be hard due to network considerations (firewalls, NAT and such).
So IMHO, the client should pull at regular intervals and/or before any
action is undertaken by the user.This assumes that there is no connection maintained between cleint and
server. With an open connection PUSH is trivial. Also opening a
connection has a large overhead, so not to be done unless required.
Indeed, it assumes there is no connection maintained, which is kind of the
choice we went with in order to be able to treat all types of games (mail,
hot seat or online web/mobile/thick client) the same.
Plus, it simplies things when it comes to network issues yet again.
N could be very large. There are games such as Case Blue and Der
Weltkrieg Campaign Game with thousands of pieces and hundred of moves. N
could exceed a million and given that a state change would take quite a
few bytes (especially if it’s in human readable text) even those of us
with fibre optic connections are going to wait (assuming Vassal server
is very high bandwidth, high end machine)
Yes, N could be very large. This doesn’t change the fact that when you
actually play, you already have parts of N (the log). You won’t ever get
all changes at once except if you change computers for instance.
Let us assume that a given state is a modification of, say, 10 Pieces
(which is a large estimation).
Let us assume that one piece change is around 1K of text.
Let us assume I want to reload 100 state changes. Well it’s only 1000K. Not
that much considering the fact that this is a “worst case”. And that’s not
considering compression on the HTTP level nor message level.
–
Julien Cassignol
ainulindale.net
George,
On Thu, Jun 6, 2013 at 8:14 PM, george973 george973@btinternet.com wrote:
SOAP and REST are protocols for what I call one shot interactions:
big snip
Let me be clear on that: if the protocol isn’t something interoperable, non
binary, or even non XML, it’s for me a huge mistake.
–
Julien Cassignol
ainulindale.net
On Thu, Jun 6, 2013 at 8:37 PM, Joel Uckelman uckelman@nomic.net wrote:
ASIDE: At the moment a piece exists on a map and its position on the map
is a point (x,y). But the position could be something other than a point
depending on the type of the map.I think we should distingush between map locations and plane coordinates.
The latter should only be used in the game domain for situating pieces
which are not on maps.
In any case the XML grammar can be adapted for that. In my XSD, there is a
positionnable “class” which could be extended for that.
I misread what Ainulindale wrote. What I would say is that a player
should not normally have permission to truncate the log in a way which
would remove changes which revealed information to that player. This
would include things like card flips, dice rolls, etc.
Good point. This should be an attribute of a given event (“isUndoable”?).
That way the module designer can set it as he wishes.
So you’re wanting this to be done over a custom protocol instead of
HTTP, then?
Darth Vader sounding voice: NOOOOOOOO!
–
Julien Cassignol
ainulindale.net
Joel,
On Fri, Jun 7, 2013 at 12:11 AM, Joel Uckelman uckelman@nomic.net wrote:
Thus spake Julien Cassignol:
Well to me describing a state is a way to say how to go from point A to
B.
So a state says “here are the values which changed”. If you actually want
to go from A to Z, you’ll apply all the changes as a whole, from A to Z.
So in this sense, a state is kind of a linear set of differences.What you’re calling “state” here I would call a “transition”. State is
what you have between transitions.I think it’s useful for a client to be able to get the current state
rather than have to calculate it, as the history could be very long
and the client might not care about having the history.
Thanks for the word “transition”, I was trying to have someone coin it for
me.
As to your comment, this is what I implied especially with my SOAP draft.
You actually ask the current state from a starting point, and the server
delivers a cumulated change.
–
Julien Cassignol
ainulindale.net
Why would you want to use HTTP??? Or any other protocol designed for other purposes??
What you want to do is fairly simple. Do your own protocol.
KISS!! KISS!! KISS!!
interoperable!! What’s that buzzword mean? XML!! it’s just padding!!
BUT a big game could require a million step to get there in the worst case That’'s a gigabyte of data.
Think I go on holiday but want to use my laptop to continue gaming. I need to copy the relevant save files or face ages downloading. An given the speed of most hotel links, it would have to be a long holiday.
Ypu MUST be able to load the current state directly. Indeed the server should send the shortest message: incremental change or complete state
We were at cross purposes here. I was talking about doing by just a change log from the start.
Have you a sample of your XML. I’ll try generating one for an existing scenario so we can see the likely size for an existing module rather than just hypothesising
Warning: highly useless message.
On Fri, Jun 7, 2013 at 11:03 AM, george973 george973@btinternet.com wrote:
Why would you want to use HTTP??? Or any other protocol designed for
other purposes??What you want to do is fairly simple. Do your own protocol.
KISS!! KISS!! KISS!!
Because implementing a new protocol is simple? Come on 
Also, I want to rock’n’roll all night.
interoperable!! What’s that buzzword mean? XML!! it’s just padding!!
If we disagree on that, we’ll probably disagree on everything else, so I’d
rather stop there
–
Julien Cassignol
ainulindale.net
I actually think we are getting a bit ahead of ourselves here.
At this stage, the actual protocol used to send data between Server and
client is pretty much irrelevent. What we need to be discussing is what
is stored where and what is sent accross the links, i.e. high level
design, not bits and bites.
One thing that concerned me was the statement that clients will not have
the full state. At the moment they do and any client can make a full
save game containing the hidden state. Who has the full state? How are
full save games created and where? Will the ‘Server’ for a session
become a single point of failure? At the moment, things are quite robust
because any client can make a full save game.
Thus spake george973:
So you’re wanting this to be done over a custom protocol instead of
HTTP, then?Why would you want to use HTTP???
HTTP is well-understood and well-tested.
HTTP supports things like compressed payloads, authentication, and
redirects.
There is no language worth using which lacks a library for speaking
HTTP. This relieves anyone wanting to implement a client or server of
the burden of designing, coding, and testing a custom protocol.
HTTP lowers the barrier for entry. It is far easier for anyone who
wants to do interesting things with game data to do that if they don’t
have to write something for speaking with the game server first.
HTTP is far more likely to pass through the all the weirdo network
configurations found in the wild than a custom protocol is. I’ve been
in many situations where the only outbound connections I can make are
using HTTP to port 80 on the receiving end. (*Excluding port 53. Often
wireless access points will let through traffic bound for port 53
without checking whether it’s actually DNS requests. Several times I’ve
considered setting up SSH on port 53 on a VM for this reason… Anyway,
I’m not suggesting that we push our data throug DNS requests–that
really would be a misuse of the protocol.)
What you want to do is fairly simple. Do your own protocol.
Designing our own transmission protocol is the antithesis of simple.
Let me be clear on that: if the protocol isn’t something
interoperable, non
binary, or even non XML, it’s for me a huge mistake.interoperable!! What’s that buzzword mean? XML!! it’s just padding!!
“Interoperable” means that various kinds of clients can consume the
data, because it’s in a format they already know how to read. This is
also a reason for using XML.
A second reason for using XML is to ensure that piece definitions
(what is now in a module buildFile, basically), saves, and logs are
human-readable. Interoperability is a consideration here as well, since
there are many tools for manipulating XML, but there will be none except
the ones we write for manipulating a custom format. (If you don’t like
XML, consider JSON. The thing I don’t like about JSON is that there is
no syntax for comments.) I’ve already seen once what happens when we
don’t have human-readable file formats. I have no desire to repeat that.
Let us assume that one piece change is around 1K of text.
Let us assume I want to reload 100 state changes. Well it’s only
1000K. Not
that much considering the fact that this is a “worst case”. And that’s
not
considering compression on the HTTP level nor message level.BUT a big game could require a million step to get there in the worst
case That’'s a gigabyte of data.
Changs are small. E.g., a change which moves a piece:
This is under 50 bytes. If we assume that changes average 50 bytes, and
you move each of 2000 pieces independently for 100 turns, this only
amounts to 9.5MB uncompressed. You’ll get compression upwards of 7:1
for this, so we’re looking at a log under 1.5MB.
Think I go on holiday but want to use my laptop to continue gaming. I
need to copy the relevant save files or face ages downloading. An given
the speed of most hotel links, it would have to be a long holiday.
If you can’t download a 1.5MB file, then your connection isn’t going to
be useful for much.
Ypu MUST be able to load the current state directly.
Agreed.
I am assuming that it will be possible to request the complete current
game state. This should not exceed more than a few hundred kB
uncompressed. I am saying this on the basis of the little demo I
worked
up for Afrika II, which has a state file containing approximately 200
pieces and is 14kB. You could increase the number of pieces by an
order
of magnitude in addition to adding a half-dozen properties per piece
beyond location, angle, and showing face, and still be under 1MB. It’s
also the case the XML compresses exceedingly well. The Afrika II
example
gzipped 7:1, so it looks like you can get about 100 pieces per 1.6kB.We were at cross purposes here. I was talking about doing by just a
change log from the start.Have you a sample of your XML. I’ll try generating one for an existing
scenario so we can see the likely size for an existing module rather
than just hypothesising
See above.
–
J.
Brent,
On Fri, Jun 7, 2013 at 1:52 PM, Brent Easton b.easton@exemail.com.auwrote:
I actually think we are getting a bit ahead of ourselves here.
At this stage, the actual protocol used to send data between Server and
client is pretty much irrelevent. What we need to be discussing is what is
stored where and what is sent accross the links, i.e. high level design,
not bits and bites.
Good point. Sadly we’re cheating there, we talked a lot about that on IRC.
One thing that concerned me was the statement that clients will not have
the full state. At the moment they do and any client can make a full save
game containing the hidden state. Who has the full state? How are full save
games created and where? Will the ‘Server’ for a session become a single
point of failure? At the moment, things are quite robust because any client
can make a full save game.
The point was not to send anything the client shouldn’t know about. The
server has the full state. Getting the full state from the client side is
possible but the other side is informed and the action to ask for it will
be preceeded by strong warnings for the user (e.g. “This is often
considered as the first step for cheaters, mate!”).
The server will kind of become a SPOF, which is why we debated the use of a
kind of circle of trust to send encrypted information back and forth the
links. We could also envision a “merge” function between all opponents, but
this means that one single failure on the players after the server fails
and everything is lost (which is frankly quite improbable).
We could have the server generate keypairs with the user passwords as the
seed in order to produce an encrypted savefile which could be automatically
written to disk for backup purposes (http accessible) for instance, in
order to solve that and allow user to get the file without being able to
read private information.
–
Julien Cassignol
ainulindale.net
Thus spake Brent Easton:
One thing that concerned me was the statement that clients will not have
the full state. At the moment they do and any client can make a full
save game containing the hidden state. Who has the full state?
If you want to have hidden information, then only the game server can
have the full state unencrypted. If there’s no hidden state, then
everyone—the game server and all clients—can have the full state.
(This is because when there is no hidden information, the full state is
equivalent to the state seen by each player.)
How are full save games created and where?
It should be possible for clients to request from the server a save file
containing the total state, with the parts not visible to that player
encrypted. The problem which arises from this is how to manage player
consent and encryption keys: What should happen, e.g., if one of the
players disappears and either needs to be replaced or the remaining
players decide to end the game but would like to see the full state?
So, there are a few possibllites for handling this:
Encrypt with a key that can be recovered by any N of the players, and
set N appropriately (perhaps letting the players set N as they wish).
Let any player request the full state from the server, but notify all
the other players the they have done so.
The idea behind the latter is that cheating only succeeds if the other
players don’t find out—hence, any potential cheater will realize that
he cannot successfully cheat. The latter is not compatible with players
have a complete local copy of the game state. The former is a lot more
complicated, but still feasible. Neither will work for keeping hidden
information secret for offline play, but neither will anything else.
We might want to make both of these things an option.
Will the ‘Server’ for a session become a single point of failure?
You mean in terms of data loss, right? No, not if we adopt the first
alternative I mentioned for securing hidden state.
–
J.
What you want to do is fairly simple. Do your own protocol.
Designing our own transmission protocol is the antithesis of simple.
For what we need it’s trivial. Let’s see you like XML so how about this:
When connected you receive text which is XML with
... more XMLwhenever you finish reading a command you execute it. If you change your data other than via a command you send the appropriate command to the other end.
Note the contents of the command you need to devise anyway as it’s what you’ll send as your step changes.
There’s a complete protocol which will work for us. You could compress the transmissions to save bandwidth so would have to put some fill after each command to ensure its transmission and decoding at the far end.
I’m sure we could devise something more streamlined and efficient without getting too compicated
- HTTP is well-understood and well-tested.
Yes, but so are jet aircraft but you wouldn’t use one to get your granny across the road. It’s overkill, a sledgehammer to crack nuts.
- HTTP supports things like compressed payloads, authentication, and
redirects.
But we don’t need these things. Except possibly compression.
- There is no language worth using which lacks a library for speaking
HTTP. This relieves anyone wanting to implement a client or server of
the burden of designing, coding, and testing a custom protocol.
See above. The protocol we need is trivial.
- HTTP lowers the barrier for entry. It is far easier for anyone who
wants to do interesting things with game data to do that if they don’t
have to write something for speaking with the game server first.
If some one wants to write a client then coding our simple protocol is not going to phase them and will be the least of their problems.
- HTTP is far more likely to pass through the all the weirdo network
configurations found in the wild than a custom protocol is. I’ve been
in many situations where the only outbound connections I can make are
using HTTP to port 80 on the receiving end. (*Excluding port 53. Often
wireless access points will let through traffic bound for port 53
without checking whether it’s actually DNS requests. Several times I’ve
considered setting up SSH on port 53 on a VM for this reason… Anyway,
I’m not suggesting that we push our data throug DNS requests–that
really would be a misuse of the protocol.)
If any would be Vassal player is somewhere that only lets out Http requests then he’s probably somewhere that would frown on him playing games on their computer.
Also the protocol is transparent to routing software. All it knows is the destination port and whether it is TCP or UDP. So we could run our server using port 53 or port 80 regardless of what protocol we were running (though it would be a good idea to make sure we didn’t choke on real DNS or HTTP requests)
“Interoperable” means that various kinds of clients can consume the
data, because it’s in a format they already know how to read. This is
also a reason for using XML.
My browser can read XML. It will put a pretty tree listing showing the XML. But it can’t do anything useful with it. Unless a program has been coded to understand your log format it can’t do anything useful with it so “interoperability” amounts to nothing. I suppose you could have a program which read arbitrary XML and translated all the strings into French in which case it would translate our log files,
A second reason for using XML is to ensure that piece definitions
(what is now in a module buildFile, basically), saves, and logs are
human-readable. Interoperability is a consideration here as well, since
there are many tools for manipulating XML, but there will be none except
the ones we write for manipulating a custom format. (If you don’t like
XML, consider JSON. The thing I don’t like about JSON is that there is
no syntax for comments.) I’ve already seen once what happens when we
don’t have human-readable file formats. I have no desire to repeat that.
Actually, I’d say the opposite. The data should not be human readable. This encourages people to apply a text editor to it which might leave as well formed XML but still crash your program when it reads it because it violates assumptions about how things fit together. Or your program has to accept arbitrary XML files and try and make sense to them.
Also if it is not human readable then there is no reason not to keep the full data on each client. The player can’t read it without a debugger which eliminates casual cheating and we are not trying to make it impossible for anyone however determined to cheat. There no money riding on Vassal Games (that I know of).
I think the current design is far too clever.
What is wanted is a MINIMAL design to fit our NEEDS which is EXTENSIBLE so we can those extras which we haven’t thought of yet.
Thus spake george973:
What you want to do is fairly simple. Do your own protocol.
Designing our own transmission protocol is the antithesis of simple.
For what we need it’s trivial. Let’s see you like XML so how about this:
When connected you receive text which is XML with
... more XMLwhenever you finish reading a command you execute it. If you change your
data other than via a command you send the appropriate command to the
other end.
So we then have to handle multiplexing over sockets, ordering the
commands as they come in, determining where the boundaries are between
commands, sorting out the situation after disconnects, and assorted
other tedious, error-prone things (endianness!) ourselves. We could do
all of this, sure, but why should we when we could use something which
already exists to let us move on rapidly to creating the parts which
don’t already exist?
- HTTP supports things like compressed payloads, authentication, and
redirects.But we don’t need these things. Except possibly compression.
So you’re worried about server load and bandwidth but you don’t see the
usefulness of redirects (e.g., for balancing load)?
- There is no language worth using which lacks a library for speaking
HTTP. This relieves anyone wanting to implement a client or server of
the burden of designing, coding, and testing a custom protocol.See above. The protocol we need is trivial.
- HTTP lowers the barrier for entry. It is far easier for anyone who
wants to do interesting things with game data to do that if they don’t
have to write something for speaking with the game server first.If some one wants to write a client then coding our simple protocol is
not going to phase them and will be the least of their problems.
- HTTP is far more likely to pass through the all the weirdo network
configurations found in the wild than a custom protocol is. I’ve been
in many situations where the only outbound connections I can make are
using HTTP to port 80 on the receiving end. (*Excluding port 53. Often
wireless access points will let through traffic bound for port 53
without checking whether it’s actually DNS requests. Several times I’ve
considered setting up SSH on port 53 on a VM for this reason… Anyway,
I’m not suggesting that we push our data throug DNS requests–that
really would be a misuse of the protocol.)If any would be Vassal player is somewhere that only lets out Http
requests then he’s probably somewhere that would frown on him playing
games on their computer.
What, like an airport? a coffee shop? Both are places I’ve been with
my own laptop where one would think I have every right to send and
receive data so long as I’m not interfering with others.
Also the protocol is transparent to routing software. All it knows is
the destination port and whether it is TCP or UDP. So we could run our
server using port 53 or port 80 regardless of what protocol we were
running (though it would be a good idea to make sure we didn’t choke on
real DNS or HTTP requests)
This is part of why building our own protocol is not so simple as you
make it out to be. Using HTTP means that an existing library handles
for us many of the ways that requests can be bad.
“Interoperable” means that various kinds of clients can consume the
data, because it’s in a format they already know how to read. This is
also a reason for using XML.My browser can read XML. It will put a pretty tree listing showing the
XML. But it can’t do anything useful with it. Unless a program has been
coded to understand your log format it can’t do anything useful with it
so “interoperability” amounts to nothing. I suppose you could have a
program which read arbitrary XML and translated all the strings into
French in which case it would translate our log files,
You’re not being creative enough. Simple programs can transform XML
usefully in many ways. With a little JavaScript, you could create a
page showing a summary of an ongoing game, for example. Want to munge
a game log into some statistics? Load your language’s XML library and
you’re already on your way. No one is going to do things like this if
they have to build something to handle a custom transmission protocol
or a binary file format first.
A second reason for using XML is to ensure that piece definitions
(what is now in a module buildFile, basically), saves, and logs are
human-readable. Interoperability is a consideration here as well, since
there are many tools for manipulating XML, but there will be none
except
the ones we write for manipulating a custom format. (If you don’t like
XML, consider JSON. The thing I don’t like about JSON is that there is
no syntax for comments.) I’ve already seen once what happens when we
don’t have human-readable file formats. I have no desire to repeat
that.Actually, I’d say the opposite. The data should not be human readable.
This encourages people to apply a text editor to it which might leave as
well formed XML but still crash your program when it reads it because it
violates assumptions about how things fit together. Or your program has
to accept arbitrary XML files and try and make sense to them.
No, none of this.
(1) There is absolutely no excuse for a well-designed world-facing program
to crash on bad input. None. If the input is bad, return an error.
(2) XML can be validated against a schema to ensure that assumptions
about how things fit together are not violated. If the XML does not
validate, see #1.
Also if it is not human readable then there is no reason not to keep the
full data on each client. The player can’t read it without a debugger
which eliminates casual cheating and we are not trying to make it
impossible for anyone however determined to cheat.
This is no security at all, as all it would take to enable casual
cheating is for one person to look at the source code and modify
a client to dump the hidden data.
–
J.
Thus spake george973:
Perhaps it would be an idea to step back and list what we need to
achieve, what we would like to achieve and let that drive our choice of
implementation.(a) Email:
(i) Need: To receive a log file of opponent’s moves, to step through
this one by one or jump to end, to do our own moves creating alog file
which when done we can send to our opponent.(ii) Like: Match the starting state of received log file with end state
of last sent log file to ensure they match.
I would call maintaining log consistency a requirement, not one of the
desiderata.
(c) Security
(i) Need: Enough to prevent John Doe joining a game as himself or
masquerading as another player and moving pieces at random or otherwise
ruining the game for the other players.
I concur. I don’t see how this can be done without some form of
authentication.
We don’t need to keep out the
CIA, Russian Mafia, Hackers Anonymous who probably won’t want to get in
anyway.
With something like this, you’re either secure, or you’re waiting for
someone to make exploiting the flaws trivial.
(iii) Like: If a player drops out of a ongoing big game some mechanism
to allow his part to be taken over by some one else with the consent of
all the remaining players without providing too easy a loophole for
cheating.
I see this also as a requirement.
(d) Platform
(i) Need: To be able to run clients on the current platforms: Windows,
Mac OS X and Linux.(ii) Like: To be able run clients on Android Tablets, IPads, smartphones
etc
Similarly, here, I see this as a requirement, not simply as something
nice to have if we can manage it: If we’re not able to run on tablets,
then someone else will create something which will. That thing might not
be open, run on what we want, or have a host of other flaws which make
it unsuitable for our purposes. If we don’t do this, we’re giving up
much of the position we’ve achieved in the virtual tabletop space for
nothing.
You’ve also not mentioned a web-based client.
(i) Protocol should require the minimum number of connects.
Disconnection should be forced only where there is danger of hogging a
resource needed for other users.
Substantiate the claim that establishing connections is expensive.
(If this is so, then how do web servers handle it?)
(ii) Data transmissions for requests and replies should be as short as
possible - this means binary data rather than text, compression of very
long messages - CPU is much cheaper then internet bandwidth in response
time.
Not using significantly more bandwitdh than the current game server is
a goal of mine—but I don’t see the point of putting hard work into
being significantly more efficient than the current game server when it
barely makes a dent in the available bandwidth is has now. We could have
20x the users we have now and still not fill the pipe we’re on with
game server traffic—and were we to get 20x the users we have now, we
could well afford to put the game server on a fatter pipe.
–
J.
I would have but the use VASL players are making of logs means we have to be relaxed about it. Also a design issue so players can do things whilst playing back a log file.
But a simple password will do, simply encrypted over the network if you want a little more security.
We can’t achieve maximum security to keep out anyone with enough resources. We just need to keep out the casual hacker and to ensure that neither the server or the client will do anything outside their domain due to network requests (eg network indicued file saves all go to designated directory)
That’s OK with me.
If it’s a requirement then the server protocol must be done with this in mind. Tablets have severe restrictions on memory usage, CPU usage and bandwidth. Memory sizes of less then 512MBytes or even for a bigger tablet 768MBytes which are shared between all running processes. If the system runs low on memory it will kill off any background processes (ie what’s not on the screen). The bigger the memory footprint the more likely a process is to be chopped if the users answers a phone call or brings up some other activity and the fewer devices it will fit. Whilst the images will be the greatest part of our memory footprint will be the map images we don’t want blow megabytes on XML text if it’s not necessary. Also our protocol needs to allow the input to be consumed as it is read. (Eg the whole command does not have to be read before executing it)
The CPU power of smartphones is not that great. I developed an application which read an XML file form the web, parsed it and processed it. From the log you could see a distinct hiccup when it did the parse and this was a much smaller and simpler file than would be needed for a game state. (Which is another reason to parse whilst downloading - you can overlap CPU and I/O intensive activity). I won’t say it wouldn’t work just that it would limit the devices which could run the App successfully to the most powerful ones.
Bandwidth is important since if it’s going through the simcard the user is paying for every byte - he may be paying anyway if it’s via his broadband connection. Also 3G and even 4G connections are a lot slower than fibre optics or even good copper connections.
No, and I won’t.
To make a TCP connection requires a message to the far end and a response returned (see TCP Protocol RFC). This takes 1 PING time (ie the turnaround time ping reports) at least. To send a message and get the response takes another PING time. So if you reconnect for each message then it’s at least 2 PING times per message. If you connect once then use the link for each message, it’s 2 PING times for the first message and then 1 PING time per message thereafter or twice as fast as the single message per connect design.
The HTTP Protocol has the ability to specify the link is to be kept open for re-use just because of this.
Looking at the largest scenario in Case Blue (which will be near the top end of what the protocol needs to handle) the current size is 5.1MBytes which is compressed to 175KBytes. Now this is so much larger than your estimates because the current system preserves for each piece not just the properties which the player can change but also the complete definition from the module. This is because creating a piece clones the module definition. This means that changing pieces the module does not affect saved games as they have their own piece definitions. Lots of data could be saved by always linking the piece in the save file to the module definition but means that a mechanism is needed to support older save files as the modules changes. This could be dynamically adjusting the save data or keeping multiple versions of the piece definition.
Your quote of me was missing the xml I put there. I don’t know if it was trimmed when you quoted or earlier. Basically the more XML was wrapped in XML start/end pair for a command.
Those tedious error prone things you mention are basic network programming and should be a couple of weeks work for a competent programmer once the design is done. Using something that already exists is fine if it fits your need. If it has to be warped and twisted to get where you want it can take longer then writing what you wanted and distort your whole design, leaving a legacy which can only be solved by a complete re-write.
Simplest redirect is to reject the connection and the client tries the next server on its list. Could also be done when joining a game after chat rooms by specifying server running the game.
If that’s the case we have to put our server on port 80. But we don’t need to run HTTP.
Only the http part. It won’t tell us anything about the payload. So we still have to parse and validate that. Since the http overhead wouldn’t be there if we weren’t using it we’ve saved very little.
Hmm, I’m not convinced that a little javascript is going to change a many MByte XML file to a game summary. I’m not sure there’s the demand to do these things and even if there is we could satisfy it by having an alternative protocol. Hmm, perhaps that’s the way to go. A lean protocol for client with constraints on memory etc and a fat one for the others.
That’s an academic ideal. No one succeeds except perhaps NASA.
Yes but it’s all overhead. Also it prevents parallel parsing and execution which is something we should try and achieve.
Any one who could do this is more than a casual cheat. Most of the players are not technically competent to do that. 99% wouldn’t want to do it. It would ruin their games. If it was likely someone would have done it already for VASL since with their tournaments they’re the most competitive.
Also dumping the data would not be enough. If you’ve got to search a 5MByte XML file for the data you want you’re going to have to have a lot of inside knowledge to find it.
Thus spake george973:
(ii) Like: Match the starting state of received log file with end
state
of last sent log file to ensure they match.I would call maintaining log consistency a requirement, not one of the
desiderata.I would have but the use VASL players are making of logs means we have
to be relaxed about it. Also a design issue so players can do things
whilst playing back a log file.
I’ve thought about this some and now believe we can maintain log
consistency and permit actions to be inserted—but there’s no way
other than manually to resolve conflicts.
(ii) Like: To be able run clients on Android Tablets, IPads,
smartphones
etcSimilarly, here, I see this as a requirement, not simply as something
nice to have if we can manage it: If we’re not able to run on tablets,
then someone else will create something which will. That thing might
not
be open, run on what we want, or have a host of other flaws which make
it unsuitable for our purposes. If we don’t do this, we’re giving up
much of the position we’ve achieved in the virtual tabletop space for
nothing.If it’s a requirement then the server protocol must be done with this in
mind. Tablets have severe restrictions on memory usage, CPU usage and
bandwidth. Memory sizes of less then 512MBytes or even for a bigger
tablet 768MBytes which are shared between all running processes. If the
system runs low on memory it will kill off any background processes (ie
what’s not on the screen). The bigger the memory footprint the more
likely a process is to be chopped if the users answers a phone call or
brings up some other activity and the fewer devices it will fit. Whilst
the images will be the greatest part of our memory footprint will be the
map images we don’t want blow megabytes on XML text if it’s not
necessary. Also our protocol needs to allow the input to be consumed as
it is read. (Eg the whole command does not have to be read before
executing it)The CPU power of smartphones is not that great. I developed an
application which read an XML file form the web, parsed it and processed
it. From the log you could see a distinct hiccup when it did the parse
and this was a much smaller and simpler file than would be needed for a
game state. (Which is another reason to parse whilst downloading - you
can overlap CPU and I/O intensive activity). I won’t say it wouldn’t
work just that it would limit the devices which could run the App
successfully to the most powerful ones.Bandwidth is important since if it’s going through the simcard the user
is paying for every byte - he may be paying anyway if it’s via his
broadband connection. Also 3G and even 4G connections are a lot slower
than fibre optics or even good copper connections.You’ve also not mentioned a web-based client.
No, and I won’t.
I see this as a requirement also.
The HTTP Protocol has the ability to specify the link is to be kept open
for re-use just because of this.
Yes, and this is exactly the way in which we should use it.
Looking at the largest scenario in Case Blue (which will be near the top
end of what the protocol needs to handle) the current size is 5.1MBytes
which is compressed to 175KBytes. Now this is so much larger than your
estimates because the current system preserves for each piece not just
the properties which the player can change but also the complete
definition from the module. This is because creating a piece clones the
module definition.
This is a large part of it, yes.
This means that changing pieces the module does not
affect saved games as they have their own piece definitions.
Yes, and this is one of the major design flaws in VASSAL currently. It’s
caused endless messes with saves getting out of sync with modules. This
can be avoided by properly versioning everytying.
Those tedious error prone things you mention are basic network
programming and should be a couple of weeks work for a competent
programmer once the design is done.
This is precisely my point.
Using something that already exists is fine if it fits your need.
If it has to be warped and twisted to get where you want
This appears to be largely our point of disagreement. I don’t see that
what I proposed would be misusing HTTP at all, in any way whatsoever.
So you’re worried about server load and bandwidth but you don’t see
the
usefulness of redirects (e.g., for balancing load)?Simplest redirect is to reject the connection and the client tries the
next server on its list. Could also be done when joining a game after
chat rooms by specifying server running the game.
… which requires additional code which we must write, test, and
maintain.
What, like an airport? a coffee shop? Both are places I’ve been with
my own laptop where one would think I have every right to send and
receive data so long as I’m not interfering with others.If that’s the case we have to put our server on port 80. But we don’t
need to run HTTP.
Running on port 80 alone won’t get you by a firewall which is doing
packet inspection.
(1) There is absolutely no excuse for a well-designed world-facing
program
to crash on bad input. None. If the input is bad, return an error.That’s an academic ideal. No one succeeds except perhaps NASA.
This is false. There are many, many UNIX programs which you cannot crash
with bad input. I’ve written programs you can’t crash with bad input.
It’s simply a matter of handling input properly.
(2) XML can be validated against a schema to ensure that assumptions
about how things fit together are not violated. If the XML does not
validate, see #1.Yes but it’s all overhead. Also it prevents parallel parsing and
execution which is something we should try and achieve.
I don’t follow this. How do you expect to execute a change before you’ve
determined what it is?
Also if it is not human readable then there is no reason not to keep
the
full data on each client. The player can’t read it without a
debugger
which eliminates casual cheating and we are not trying to make it
impossible for anyone however determined to cheat.This is no security at all, as all it would take to enable casual
cheating is for one person to look at the source code and modify
a client to dump the hidden data.Any one who could do this is more than a casual cheat. Most of the
players are not technically competent to do that. 99% wouldn’t want to
do it. It would ruin their games. If it was likely someone would have
done it already for VASL since with their tournaments they’re the most
competitive.Also dumping the data would not be enough. If you’ve got to search a
5MByte XML file for the data you want you’re going to have to have a lot
of inside knowledge to find it.
I did not say that a casual cheat would design such an exploit.
Dumping the data is not the best example I can give. Here’s a better
one: If the client has all of the data avilable to it, then the client
can be modified to show that data to the user. Other users would have no
way of telling that your client is showing you their cards face up.
Modifying a client to do that would obviously reqiure some understanding
of the code—but using such a client would require no specialized
knowledge whatsoever.
–
J.
Thus spake george973:
If it’s a requirement then the server protocol must be done with this in
mind. Tablets have severe restrictions on memory usage, CPU usage and
bandwidth. Memory sizes of less then 512MBytes or even for a bigger
tablet 768MBytes which are shared between all running processes. If the
system runs low on memory it will kill off any background processes (ie
what’s not on the screen). The bigger the memory footprint the more
likely a process is to be chopped if the users answers a phone call or
brings up some other activity and the fewer devices it will fit. Whilst
the images will be the greatest part of our memory footprint will be the
map images we don’t want blow megabytes on XML text if it’s not
necessary. Also our protocol needs to allow the input to be consumed as
it is read. (Eg the whole command does not have to be read before
executing it)
I don’t see a problem here. 512MB is a huge amount of memory.
The CPU power of smartphones is not that great. I developed an
application which read an XML file form the web, parsed it and processed
it. From the log you could see a distinct hiccup when it did the parse
and this was a much smaller and simpler file than would be needed for a
game state. (Which is another reason to parse whilst downloading - you
can overlap CPU and I/O intensive activity). I won’t say it wouldn’t
work just that it would limit the devices which could run the App
successfully to the most powerful ones.
So, you parse using a validating SAX parser. SAX is for streaming
parsing.
Bandwidth is important since if it’s going through the simcard the user
is paying for every byte - he may be paying anyway if it’s via his
broadband connection. Also 3G and even 4G connections are a lot slower
than fibre optics or even good copper connections.
I was expecting that most people using tablets would be doing so over
wifi if they were playing an online game.
–
J.
I’m curious about this. Why not let the client have the full game state and filter the client view appropriately? Pieces and messages get displayed, or not, per simple properties in the game state as today.
I think you may be over-engineering the security protocol. Is cheating really that prevalent on VASSAL? Someone will have the technical chops to break the protocol and create “cracked” clients. Every game server on the internet has cheaters despite the HUGE amount of time and effort expended to prevent it. And they don’t make their source open for inspection.
I’d vote for just enough security to keep honest players honest.
-David
Thus spake djsullivan:
I think you may be over-engineering the security protocol. Is cheating
really that prevalent on VASSAL?
Not right now, that we know of. That doesn’t mean it couldn’t become
a catastrophic problem tomorrow.
Someone will have the technical chops to break the protocol and create
“cracked” clients.
No, they won’t—not if the only data sent to clients is the data
they’re entitled to have. There is nothing to break or crack in that
case.
Every game server on the internet has cheaters despite the HUGE amount
of time and effort expended to prevent it. And they don’t make their
source open for inspection.
Realtime games can’t do what I’m proposing due to the volume of data
which would have to travel back and forth. Can you show me example of
some game server which is specifically not for RTS or FPS games which
has had a great deal of effort put into security but nontheless still
has a problem with cheaters?
–
J.