Learning_Mapper corpus export/import problems

Posted by Symple on Sun 05 Apr 2020 03:01 AM — 11 posts, 33,278 views.

Symple Sun 05 Apr 2020 03:01 AM #0

On the latest version installed on windows 10
I am using the mapper
<plugin
name="Learning_Mapper"
author="Nick Gammon"
id="99c74b2685e425d3b6ed6a7d"
language="Lua"
purpose="AI interface to mapper to learn descriptions etc."
save_state="y"
date_written="2020-01-23 11:29:22"
date_modified="2020-02-10 09:00:00"
requires="5.05"
version="2.0"
>

and when I us the mapper corpus export command it seems to create the file just fine but when I try to load it into a new world it spits out an error.

Run-time error
Plugin: Learning_Mapper (called from world: RoD Mage)
Function/Sub: corpus_import called by alias
Reason: processing alias ""
D:\MUSHclient\worlds\plugins\Learning_Mapper.lua:2908: [string "corpus = {..."]:5533: '}' expected (to close '{' at line 1267) near '='
stack traceback:
[C]: in function 'assert'
D:\MUSHclient\worlds\plugins\Learning_Mapper.lua:2908: in function <D:\MUSHclient\worlds\plugins\Learning_Mapper.lua:2879>

Nick Gammon Australia Forum Administrator Sun 05 Apr 2020 06:07 AM #1

Can you show the last few lines of that exported file? It sounds a bit like it was truncated for some reason.

Symple Sun 05 Apr 2020 05:14 PM #2


["Avatar          Klare. <Guild of Spirit>"] = {
        score = 0.375,
        black = 0,
        red = 1,
        },
      },
    },
  ignore = {
    first_style_run_foreground = {
      },
    first_word = {
      },
    all_words = {
      },
    first_three_words = {
      },
    first_two_words = {
      },
    exact_line = {
      },
    },
  }


and here is line 1257-1277 in case the error is near 1267 where the error line is

score = 0.625,
        black = 1,
        red = 0,
        },
      nice = {
        score = 0.625,
        black = 1,
        red = 0,
        },
      },
    all_words = {
      flowery = {
        score = 0.625,
        black = 1,
        red = 0,
        },
      hole = {
        score = 0.625,
        black = 1,
        red = 0,
        },

Nick Gammon Australia Forum Administrator Mon 06 Apr 2020 03:47 AM #3

I think I need to see the whole file. Can you put it up on Pastebin or somewhere like that?

Symple Mon 06 Apr 2020 06:53 AM #4

Corpus.lua
https://pastebin.com/0ALa2ZcA

In case you would like these also,
Learning_Mapper.lua
https://pastebin.com/1Dyq7fdy
Learning_mapper.xml
https://pastebin.com/KsnE3eXv

only thing I changed was in Learning_Mapper.lua to quickly line up the text outputs.

I changed line 665

from:
local top = (((firstline - 1) * font_height) - offset) + 80

to:
local top = (((firstline - 1) * font_height) - offset) - 2

Nick Gammon Australia Forum Administrator Mon 06 Apr 2020 07:26 AM #5

Quote:

D:\MUSHclient\worlds\plugins\Learning_Mapper.lua:2908: [string "corpus = {..."]:5533: '}' expected (to close '{' at line 1267) near '='

Line 5533:


      true = {

I think you found a flaw in the system (true is a reserved word). Change that to:


      ['true'] = {

I'll look into how to fix that tomorrow.

Nick Gammon Australia Forum Administrator Tue 07 Apr 2020 01:04 AM #6

There is a bug in the serialize module, used to export the corpus.

Edit the file "serialize.lua" in the "lua" subdirectory of where you installed MUSHclient.

Find these lines around line 112 in that file:


for _, v in ipairs ({
    "and", "break", "do", "else", "elseif", "end",
    "for", "function", "if", "in", "local", "nil", "not", "or",
    "repeat", "return", "then",  "until", "while"
            }) do lua_reserved_words [v] = true end

Add "true" and "false" (with commas) so it looks like this:


for _, v in ipairs ({
    "and", "break", "do", "else", "elseif", "end",
    "for", "function", "if", "in", "local", "nil", "not", "or",
    "repeat", "return", "then",  "until", "while", "true", "false"
            }) do lua_reserved_words [v] = true end

In other words, add this to the end of the line starting with the word "repeat":


, "true", "false"

Then it should serialize correctly, and thus load correctly next time.

The raw file contents is here, if you want to use that:

https://raw.githubusercontent.com/nickgammon/mushclient/master/lua/serialize.lua

Fiendish USA Global Moderator Tue 07 Apr 2020 03:55 AM #7

Wow. I'm a little surprised that nobody hit that before. I wonder if it isn't best to avoid the tablename.keyname format altogether.

Nick Gammon Australia Forum Administrator Tue 07 Apr 2020 04:39 AM #8

I'm surprised too, it affects both ways of serializing (simple and not simple).

It's been used pretty extensively so it's probably safe now. In any case, changing it would require people to switch to the new version, which they have to do any way to implement the bug fix. So one way or another the current version (that is, until today) isn't safe with "true" and "false".

I suppose no-one ever called a variable or mob name "true" in the past. It would be a weird thing to do, generally speaking, to have a table key called "true" or "false".

Symple Tue 07 Apr 2020 07:36 PM #9

Thanks for the fix! So when I was teaching the corpus and it came across and tried to serialize true in a description is what caused the error?

I haven't gotten to learning about serializing yet so just trying to understand a little, thanks again for the fast response with the easy fix!

Nick Gammon Australia Forum Administrator Tue 07 Apr 2020 08:22 PM #10

That’s right. It was writing true that caused the error, when it was read back in.

The fix puts that in brackets. In Lua you can refer to table keys in two ways:

mytable.foo = 42

or

mytable ['foo'] = 42

The first method is a shorthand way of using a string key. The second way is needed for reserved words, and other types of keys, such as numbers. eg.

mytable [666] = 777