Switching to the New Module Library

[woody8297]

Thank you Joel. I’ll remove the underscores.

I thought I was still on topic because I was replying to a question about removing packages. I thought removing whole Projects would be part of that. Sorry.

Frank

[uckelman]

I’ve created a rudimentary interface where I can view flags raised on projects now. If you’ve been waiting for a reply to a flag you’ve raised, you should receive that in the next few days as I work through them.

If you have an issue with an individual project, raising a flag is now the best way to make us aware of it, as we can reply to flags directly in the forum with private messages. For all problems with page conversion, select “Other” for the flag type.

[uckelman]

@cholmcc

• Should module extensions (.vext) not also be checked against the release?
• What about saves (.vsav), logs (.vlog), and older extensions (.vmdx)? Shouldn’t they be checked too?
• It seems that checking the release number depends on the uploaded file name ending in either .vmod or .vext. Shouldn’t the check be done based on the content (mime-type) of the file? E.g., similar to what you have in image.rs for images:
• Check if file is mime-type application/zip
  • Check if archive contains moduledata
    • If so, investigate data/version in the XMLCurrently, a user could upload a module which file name ends in say .vmd and thereby by-pass the release check.

I’ve made three changes today which bear on this:

• Uploads are now all checked for containing a a moduledata file, and if they are not extensions they are assumed to be modules and thus must match the release they are in and have a .vmod extension.
• A Vassal version requirement spec is produced automatically from a module on upload, which will be >= x.y.z where x.y.z is the version of Vassal that the module was saved with. If this requirement spec is inappropriate, you’ll need to contact an admin to have it changed.
• All modules uploaded since the switchover have had their requirement spec populated.

[jrwatts]

@uckelman

If you reupload the box image (even the same one), then the next time I run the script, it will adjust the the project creation time.

Have you run the script in the past week? As suggested, I re-uploaded the same image to the 3 versions of Terraforming Mars with the incorrect date (7 days ago), but it hasn’t changed…

[uckelman]

@jrwatts

Have you run the script in the past week? As suggested, I re-uploaded the same image to the 3 versions of Terraforming Mars with the incorrect date (7 days ago), but it hasn’t changed…

I hadn’t run it since early last week. I ran it just now.

[uckelman]

There are a number of issues remaining where it’s unclear to me how to proceed:

• Adding a project creation link on library front page: We’re far enough along now that I no longer want to slow the discovery of how to create a new project. It’s not clear to me where this should go or how it should be presented.
• Adding a note about generated title sort keys in editor: I’d like to let the user know that if the generated sort key is not appropriate, then they need to contact us about that. It’s not clear to me how best to include information like this when editing.
• Tag editor: How should it behave? What should it look like? Are there any examples of this, good or otherwise?
• Image upload: How should it behave, noting that handling images in the readme, the box image, and the gallery could differ. There was some amount of thumbnailing and processing machinery that Mediawiki handled for us before. What should be done and what should do it now?

Regarding the creation link, the title sort key note, and other hints when editing: It was always my intent that the design of the library convey its intended use, so that no explanation would be required; it’s not clear to me how to incorporate such things so that they are read when needed but aren’t obscuring clutter when not. If anyone has design suggestions here, I’d appreciate having them. Examples and mock-ups are welcome.

[clanmacrae]

When you say Library front page, you mean here?

[uckelman]

@clanmacrae

When you say Library front page, you mean here?

No, I mean this.

[steenkh]

@uckelman

No, I mean this.

How do you get there normally?

[cholmcc]

@uckelman

Adding a project creation link on library front page:

How about a link “Create new project” under the “Browse all projects”? Perhaps as part of the search bar top element, so that it is present when ever the search bar is.

@uckelman

Adding a note about generated title sort keys in editor

As suggested earlier, I think adding a small paragraph of help in all input forms could help a lot.

@uckelman

It was always my intent that the design of the library convey its intended use, so that no explanation would be required; it’s not clear to me how to incorporate such things so that they are read when needed but aren’t obscuring clutter when not.

A noble intention, but very hard to get right. My suggestion above, and earlier, will only show up when editting a certainly aspect of a project, which fulfills your second consideration.

@uckelman

Tag editor

Do you see tags as key-value pairs, or as simple text strings? In the former case, I would add an element with two fields, that, when submitted, adds the key-value to the page, and then make that form visible again to allow more tags to be added. In the latter case, I would make a simple text input and separate tags be commas. Auto-completion based on known tags could be added in both cases, though tags should probably not be limited to known tags.

@uckelman

Image upload

For the Markdown part I think it should be handled like in the forum. That is, image URLs are turned into upload URLs. Perhaps insist on an ALT text so that the images can be searched more easily.

As for thumbnails, any image uploaded over a certain size, say 256x256 pixels, could have an automatic thumbnail generated. You will then need another SQL table, or similar, to tie the thumbnails to the images.

I would make galleries part of the README.

My 2¢

Yours,
Christian

[uckelman]

@steenkh

How do you get there normally?

The library front page is currently linked from the banner at the top of site pages which aren’t the forum:

[uckelman]

@jrwatts

Have you run the script in the past week? As suggested, I re-uploaded the same image to the 3 versions of Terraforming Mars with the incorrect date (7 days ago), but it hasn’t changed…

If anyone sees project creation times for newly split projects which aren’t right, please flag them for me. The only way they can be corrected is manually.

[clanmacrae]

Making images part of the Read Me makes more sense, allowing the images to be more informational, even instructional as the image descriptions would become less constrained due to the flexibility of the Read Me format.

[uckelman]

@clanmacrae

Making images part of the Read Me makes more sense, allowing the images to be more informational,

I think the suggestion was that galleries be merged into the readme, yes? Images are already allowed in the readme; there’s just no way to add more at present.

[uckelman]

If we were to rename all of the imported projects from their numerical id to the same name the module page had in the wiki, would that bother anyone?

The reason I’m asking this is that it would greatly simplify adding redirects from the URLs of the old pages to the new project pages. I am guessing that there are few links in the wild to the numerical project pages at present.

[jrwatts]

@uckelman

The library front page is currently linked from the banner at the top of site pages which aren’t the forum:

My brief testing just now seems to contradict this; every page I went to on the vassalengine.org site links to https://vassalengine.org/library/projects, not https://vassalengine.org/library/. (Yes, I cleared my browser cache when testing.)

Edit: Oops, I just realized you must have been referring to the link in the banner text, not the link from the “knight” icon. Apologies.

@uckelman

If we were to rename all of the imported projects from their numerical id to the same name the module page had in the wiki, would that bother anyone?

This sounds like a good idea to me; my only, minor, concern is that there will now be a redundant line repeating the game title when viewing the library. Perhaps suppress display of the project title if it duplicates the game title when browsing/searching the library?

[uckelman]

@cholmcc

How about a link “Create new project” under the “Browse all projects”? Perhaps as part of the search bar top element, so that it is present when ever the search bar is.

@cholmcc

As suggested earlier, I think adding a small paragraph of help in all input forms could help a lot.

For suggestions like these, it would help me to see where you think these things should go—because without that, I might be getting the wrong idea, and also if you haven’t tried it, you don’t know how it would look and behave on the page, either.

“You” here is generic. This applies to everyone, and frontend changes generally. Browsers usually have developer tools in them these days where you can live-edit the HTML and CSS on of page you’re viewing. You could literally try what you’re suggesting to see if it has the qualities for which you suggested it, and then show us.

@cholmcc

Do you see tags as key-value pairs, or as simple text strings?

Yes. Which is to say, I don’t have a definite view here. Tags are the way they are right now to preserve the data which came from the wiki.

@cholmcc

For the Markdown part I think it should be handled like in the forum. That is, image URLs are turned into upload URLs. Perhaps insist on an ALT text so that the images can be searched more easily.

Nothing in Discourse seems possible to break out as reusable components. What could we use for this?

@cholmcc

As for thumbnails, any image uploaded over a certain size, say 256x256 pixels, could have an automatic thumbnail generated.

What could we use for that?

[cholmcc]

@uckelman

@cholmcc

How about a link “Create new project” under the “Browse all projects”? Perhaps as part of the search bar top element, so that it is present when ever the search bar is.

@cholmcc

As suggested earlier, I think adding a small paragraph of help in all input forms could help a lot.

For suggestions like these, it would help me to see where you think these things should go—because without that, I might be getting the wrong idea, and also if you haven’t tried it, you don’t know how it would look and behave on the page, either.

For the text in the editor interface, see this post. For example, below line 207 of GameEditor.svelte add lines

<div class="help-text">
Game box images can be  PNG, GIF, JPEG, SVG, AVIF, or WEBP. Images should be no bigger than 200px &times; 200px, to not take up too much space on the page.
</div>

In line 44 of PackageEditor.svelte add

<div class="help-text">
Package names are free-form strings, and structures the project into sub-parts.  A typical package name would be <code>Module</code>
</div>

In line 224 of PackageSection add

<div class="help-text">
<p>
Release numbers <i>must</i> follow the conventions of  <a href="https://semver.org">Semantic Versioning</a> - i.e., be of the form <i>Major</i>.<i>Minor</i>.<i>Patch</i> (and optional pre-release or build information attached), where <i>Major</i>. <i>Minor</i>, and <i>Patch</i> are whole, non-negative, numbers.
</p>
<p>
Examples of valid release versions are 
</p>
<pre>
1.2.3
10.20.30-alpha
6.4.0-rc1
5.3.8+french
</pre>
</div>

In line 123 of NewProjectPage something a la

<div class="help-text">
<p>Project names consist solely of the letters <code>a,b,...,z</code>, 
  <code>A,B,...,Z</code>, digits <code>0,1,...,9</code>, dashes and underscores
  <code>-_</code>, and can be no longer than 64 characters.  Note that the project 
  name <i>must</i> start with a letter and <i>cannot</i> contain spaces, punctuation, 
  or other special characters.  
<p>Examples of good project names are 
<pre>
afrika_korps_username
battle_of_the_bulge_3rd_ed_username
...
</pre>
</div>

In line 246 of ReleaseSection.svelte add

{#if !uploading}
<div class="help-text">
<p>
  Remember that the version number of the module set <i>inside</i> the   
  module <i>must</i> be a valid <a href="https://semver.org">Semantic Version</a>,
  e.g., <code>1.2.3</code>.  The file name <i>should</i> contain the version number 
  too, but is not required to.  Also file name of should preferably not contain 
  spaces, punctuation, or special characters other than those in the version number,
  and as allowed by <a href="https://semver.org">Semantic Version</a> format.
</p>
<p>
  To set the version number in the module, open the module in the editor and 
  double click the top-level element.
</p>
</div>
{/if}

@uckelman

@cholmcc

Do you see tags as key-value pairs, or as simple text strings?

Yes. Which is to say, I don’t have a definite view here. Tags are the way they are right now to preserve the data which came from the wiki.

I think you need to formulate a policy before you start to think about the interface. Currently, as far as I remember, the tags are stored in the DB as key-value pairs in the form key:value, and I think all tags are of this form. Given that is the case, then I would follow that schema and insist on key-value pairs. This will also make sense when searching for projects based on tags - e.g., “find all projects with key=value” or

SELECT project_id FROM tags WHERE key = key AND tag LIKE value

@uckelman

@cholmcc

For the Markdown part I think it should be handled like in the forum. That is, image URLs are turned into upload URLs. Perhaps insist on an ALT text so that the images can be searched more easily.

Nothing in Discourse seems possible to break out as reusable components. What could we use for this?

Presumably, when a user adds an image to a README section, that image will be uploaded to the backend. The back-end will then reply with a specific URL into the object storage. This then get’s put into the README editor as

![alt-text](https://obj.vassalengine.org/foo/bar/baz.ext)

(Discourse replaces with upload://abcdef0123456789.ext, which I guess is then looked up in some DB to get the actual URL.)

This of course means that the MarkDown editor code can do this - i.e., upload images on the fly and place their URL into the text being edited.

I’m not well enough versed in Svelte or the other JS tools you use to give concrete suggestions on code to use, but I guess it can be found.

Another option would be to have a separate image list to which one can upload an image, and then look up the actual object storage URL.

@uckelman

@cholmcc

As for thumbnails, any image uploaded over a certain size, say 256×256 pixels, could have an automatic thumbnail generated.

What could we use for that?

Presumably, the thumb-nail generation would happen in the back-end, which means you could use any available Rust library for this. For example ImageMagick’s Rust binding MagickRust

My 2¢,

Christian

[cholmcc]

@cholmcc

How about a link “Create new project” under the “Browse all projects”? Perhaps as part of the search bar top element, so that it is present when ever the search bar is.

In line 130 of ProjectsPage.svelte add

<br>
<a href="/libary/new">Create new project</a>

Yours,
Christian

[cholmcc]

@cholmcc

Presumably, the thumb-nail generation would happen in the back-end, which means you could use any available Rust library for this. For example ImageMagick’s Rust binding MagickRust

This would happen somewhere around line 558 of prod_core.rs, and the ProdCore<C,U>::add_image method should probably return the storage URL so that it can be sent back to the client.

Yours,
Christian