JOGL Mipmaps

Thus spake Joel Uckelman:

Heh, the screencast program I was thinking of is called Istanbul
(not Constantinople).


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Why did Constantinople get the works?
That’s nobody’s business but the Turks.

Post generated using Mail2Forum (mail2forum.com)

The same on YouTube … :wink:

fr.youtube.com/watch?v=Czwc-E0_dPE

That’s pretty damn cool all right.

rk

Post generated using Mail2Forum (mail2forum.com)

What are the constraints on texture size we’re working under? Are we
going to run into problems if we throw something at JOGL the size
of, say, the Case Blue map?

Don’t know. We can only try. Once I implement multiple textures across the whole map, I’ll let you know.

How big would Case Blue be in pixels?

  • M.

2008/5/9 uckelman <messages@forums.vassalengine.org (messages@forums.vassalengine.org)>:

Post generated using Mail2Forum (mail2forum.com)

Thus spake “Michael Kiefte”:

It has the largest map that I know of: 11404x8290 when you put all of
the boards together.


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Is that the one that took forever in ZunTzu?

  • M.

2008/5/10 Joel Uckelman <uckelman@nomic.net (uckelman@nomic.net)>:

Post generated using Mail2Forum (mail2forum.com)

Thus spake “Michael Kiefte”:

That was DAK II. There’s no Case Blue module for ZunTzu, but if there
were, I suspect that it would take at least n * forever, where n
is the proportion by which the Case Blue map is larger (n is approximately
2, I think).


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Gosh, golly. Speeding up mipmap generation may be critical then. Although I will be surrounded by free wireless Internet coverage while I am in Alberta, the other resource that is critical – time – will be in short supply. My schedule looks blocked from wake to sleep. I’ll see how much I can do this evening, but after that, I may not be able to do much for a couple of weeks.

I’m also anticipating a flood of reports that the importer doesn’t work for module “X”, so I could easily get bogged down in that too. Nice to see a bunch of ADC2 players move over to VASSAL though. Maybe we will have to do Cyberboard.

  • M.

2008/5/12 Joel Uckelman <uckelman@nomic.net (uckelman@nomic.net)>:

Post generated using Mail2Forum (mail2forum.com)

Thus spake “Michael Kiefte”:

Originally, I was hoping that we’d be releasing 3.2 in June sometime, but
that doesn’t look realistic now, since most of the work has yet to be done.
If you can get the gross structure done, I can hack away at it with a
profiler, though it’s best not to do that prior to that point. (Though
I also will be operating a reduced capacity for the next few weeks—I’m
typing this during a break between talks, and once this conference is over
next week I’ll be heading to Toulouse to work with some colleagues there
for three weeks.)

Virtually all of the complements I’ve seen so far hnave been for the
importer. :slight_smile:


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

3.1beta may not even have settled down by then.

I just learned three important things:
a) I don’t know a lot about image processing
b) What I know about sound processing doesn’t help much
and
c) You really can resize the previously resized image and it makes no difference (as long as they’re getting smaller by factors of 2). I checked.

I’ve implemented c) already and it’s quite a speed up from the previous version, but I find a) and b) depressing. I’ve got a bitchin’ implementation of FFT in Java though. Doesn’t help much, but I’ll put it on my website just to show off to all my colleagues for geek points. I won’t get into the technical details, but what I was doing would have worked beautifully for a 2-d sound file.

On CSW? I think most of them are ADC2 users, so this will be their first foray into VASSAL. Bloody good thing, I say. VASSAL doesn’t get a lot of mention on CSW and interest in the importer will only last as long as there are new ADC2 modules being made. Nice to hear some praise.

I saw one comment along the lines of “They chose to support ADC2 before Cyberboard – that was short sighted.” He went on to say that Cyberboard was 20 times more popular than ADC2. Is this remotely true? Add to this the fact that ZunTzu is apparently popular among the Europeans. If so, we have a few big jobs on our hands:

  1. JOGL graphics
  2. Cyberboard importer
  3. ZunTzu importer
  4. RFEs out the wazoo.

#1 is a biggie and I’ll keep slogging away at the demo until we have something which demonstrates that the plan to implement JOGL in VASSAL is immediately doable. I’ve only got a couple more steps to accomplish.
#2 is going to be a royal pain in the ass if they don’t open source the code. If I knew how hard it was going to be to do the ADC2 importer before I started, I may not have done it. I got something working in two weeks, but it took me several months to get it perfect. It occurs to me that Cyberboard may now balk at OSing their code if they know what we’re up to (or not).
#3 appears to be trivial, but probably depends on #1. Who wants to play a DivX module with Java2D?
#4 is still really, really important. I came across a lot of things that I wished I could easily do. Most of these are RFEs.

I’m really interested in Brent’s terrain editor as ADC2 was doing terrain in a similar way. If elevation gets implemented, we can do something very cool with JOGL (I’ll let you imagine what I mean there).

  • M.

Post generated using Mail2Forum (mail2forum.com)

I would place #3 ahead of #2 first for being trivial and #2 is supposedly going through its own upgrade (although there is very little word on its progress) so there would be no reason to attempt #2 till the dust settles on its own final status, in meantime I would think the GUI could supplant #2 instead as one of the next major components for 3.2

#4 - me too, I’d be able to tackle simple things right now (with trial and error) if everything that had to change for an RFE was contained in a single source with no external refs to other stuff in the engine (Hello World! :slight_smile: )

I too am looking forward to Brent’s final terrain editor - access to these attributes and properties with components will bring a whole new level of possibilities to module design

Thus spake “Michael Kiefte”:

It really doesn’t help? I would like to hear the details—perhaps I
could help beat it into submission.

Yes, on CSW, and to meb by email. I’d disagree that VASSAL doesn’t get much
mention on CSW—the ADC2 forum is basically dead, while the VASSAL forum
is still quite alive.

Where’d you see that? (It’s not a comment I read myself.) While the
commenter is almost certainly correct about the relative popularity of
Cyberboard and ADC2, he’s not considering the probability that we could
have avoided months of reverse-engineering work by by waiting for the
file formats to be opened: For Cyberboard, the probability remains high,
for ADC2 it was zero. The commenter also seems not to get how open-source
development works. We have an ADC2 converter not becuase we all got
together and decided that was what should be done next, but because you
had an itch you wanted to scratch, and so you wrote one. You can’t just
mandate to volunteers that they write certain code. That’s one of the
perks of being a developer.

Moreover, having an ADC2 converter means that we capture a some ADC users
at a time when they have few options and we’re rapidly improving. Tha
makes it likely that they’ll stick with us for the long-haul.

No, I doubt that they’ll change course due to our plans for a converter.
What I don’t know is how far along they are at this point, though. I have
a feeling that the Cyberboard format is both less crappy and simpler than
the ADC2 format, so we clear away all of this other stuff and there’s no
sign that they’re making progress, maybe we should go ahead with it then.

There’s nothing in #3 which depends on #1. Every single object in a ZunTzu
build file corresponds directly to some object in VASSAL.

Yes, there are hundreds of RFEs. I wish I had more time to work on them.
We could easily double the number of developers we have right now and not
run out of things to do. (In fact, I would love to run out of things to
do, as then I could spend more time playing games.)


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Thus spake “Michael Kiefte”:

I had some thoughts about this during a particularly boring talk today:

  1. We could speed up scaling by starting scaling on more than one thread.
    This would help us by nearly a factor of n on an n-core chip, and will
    often (and rather counterintuitively) yield a speed improvement on even
    a single core.

  2. However we do the scaling to produce the mipmaps, it’s going to be a
    CPU-bound process, but it’s also deterministic, and so the resulting
    mipmaps are something we could cache. It’s possible that reading stored
    mipmaps from a disk cache might be faster than recomputing them. I’ve
    already designed a disk cache we could use for this (based on checksums
    so that stale mipmaps are purged if the image in the module changes).
    Since computing the mipmas is CPU-bound, but reading them from the disk
    is IO-bound, maybe we could achieve even better performance by reading
    some of the mipmaps from disk while we use idle CPU-cycles to compute
    others.


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Thus spake “Michael Kiefte”:

Aww! But I went to every single talk during the workshop today!

Sure, but that could stil be a huge wing.

Every time a module is loaded, if we didn’t cache them to disk.


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Thus spake Joel Uckelman:

I ran the last version of the MipmapDemo which I have through the
profiler just now, and found that 99% of CPU time is used by makeImages()
prior to first displaying the first image. No big surprise there.

Within makeImages(), the breakdown is as follows:

32% java.nio.DirectByteBuffer.put()
24% own time
23% BufferedImage.getRGB()
15% Graphics2D.drawImage()
5% TextureDemo.zoom()
1% ImageIO.read()
<1% everything else

I was rather surprised by this, as it means that the two things which
I expected to be slowest—reading the image and scaling—are so fast
that if we replaced them by instaneous calls to an oracle, we wouldn’t
even perceive the difference.

What this means for optimization is that we need to find a way to get
the image data into the mipmap without doing so much copying of pixels
back and forth. The zoomer could be modified pretty easily to write
directly to a ByteBuffer, which would avoid all of the calls to getRGB(),
and maybe we could create the ByteBuffer directly from a byte by
calling wrap(), which would then let us avoid all of the calls to put().

Is it posible to call glTexImage2D() with an int instead of a ByteBuffer?
The zoomer already has the data as an int, which might make things even
faster.


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Thus spake “Michael Kiefte”:

I think the fastest way (for loading purposes) will be to write the mipmaps
out as raw bytes, but do it via a GzipOutputStream. That will minimize both
the amount of disk IO required as well as the amount of image conversion
(none!). The mipmaps aren’t intended to be edited or anything, so don’t
need to be readable by anything else.

Is it better to make the module designer wait for mipmap creation, or
the end user wait the first time he runs the module?

J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Ok, Ok :slight_smile:

I’d better get out the GUI polish and see what I can do for 3.2. Most of the work is done and it will be very easy to integrate.

Michael, I don’t plan to implement Elevation as a special attribute as such, but you will be able to define a Tag called ‘Elevation’ and be able to set a specific value for it for each hex.

B.

On 12/05/2008 at 9:57 AM Tim M wrote:


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)

Thus spake “Michael Kiefte”:

I went ahead and did it, and here’s what I found:

makeImages(), before: 83367ms
makeImages(), after: 17983ms

So that’s 4.6 times faster than before.

Within makeImages(), before:

32% java.nio.DirectByteBuffer.put()
24% own time
23% BufferedImage.getRGB()
15% Graphics2D.drawImage()
5% TextureDemo.zoom()
1% ImageIO.read()
<1% everything else

Within makeImages(), after:

74% Graphics2D.drawImage()
22% TextureDemo.zoom()
3% ImageIO.read()
<1% everything else

drawImage(), zoom(), and read() all take the same amount of time as before.

The call to drawImage() is pretty much unavoidable, since we need the
image to be INT_ARGB in order to scale it, so now the only place where
we can shave off further cycles (without doing something algorithmically
different) is in zoom(). In particular 55% of the time in zoom() is spent
in apply_horizontal(). I’ve already optimized that pretty heavily, so I’m
not optimistic that we can make that any tighter.

I’ll send along the modified file separately.


J.


Messages mailing list
Messages@forums.vassalengine.org
forums.vassalengine.org/mailman/ … engine.org

Post generated using Mail2Forum (mail2forum.com)