Trim White Spaces From Packets

Posted by Xvordan on Fri 29 Aug 2014 09:28 AM — 4 posts, 20,448 views.

Xvordan Fri 29 Aug 2014 09:28 AM #0

Greetings folks.

I'm playing an LP-based mud that for whatever reason doesn't seem to understand that clients can do their own wordwrapping and formatting. Consequently, it requires players to specify linelengths, etc, rather than just sending a burst of unformatted text. That results in long lines of text being broken up by new line characters, which are very difficult to deal with in triggers.

I, for example, can't gag these blocks of text properly. It is, therefore, a bit of a nightmare to gag undesired blocks of text, or to replace them with something more brief.

This mud does permit me to set the line length to 999, but doing so pads a boat load of white spaces to the end of the lines.

Do you guys know of a convenient way for me to trim these white spaces from the incoming packets of data? I know there's a trim function, but the OnPluginLineReceived function help text seems to imply that I can't modify the line, just toggle whether or not it's displayed. I took a look at OnPluginPacketReceived, but I don't know that Trim would work line-by-line on a received block of text. Furthermore, the help text seems to imply that modifying packets in this way could break compressed data from the mud.

I've tried modifying it with a regexp pattern, but it doesn't seem to work. ^s*(S+)s*$ doesn't seem to match these white-space heavy blocks.

I've asked the implementor if he'd consider changing how the mud outputs text so that this specific screen size setting isn't required, but he seems to think it'd be more trouble than it's worth.

Here's a packet debug post of just one line of text. By the way, I wish we had a spoilers type of forum code, so I could hide the packet debug spam below, so that those who are interested would be able to click it to display it, but those uninterested wouldn't have to see this wall.

..     Buy a com   0d 0a 20 20 20 20 20 42 75 79 20 61 20 63 6f 6d
mand using your    6d 61 6e 64 20 75 73 69 6e 67 20 79 6f 75 72 20
available comman   61 76 61 69 6c 61 62 6c 65 20 63 6f 6d 6d 61 6e
d points.          64 20 70 6f 69 6e 74 73 2e 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
                   20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20

Nick Gammon Australia Forum Administrator Fri 29 Aug 2014 09:44 PM #1

Quote:

This mud does permit me to set the line length to 999, but doing so pads a boat load of white spaces to the end of the lines.

That is very weird. I mean, adding spaces? Who does that?

Surely a line length of 999 should just mean allow 999 characters before wrapping, not add 990 spaces to the end of a 10-character line?

One problem I see from your posted packet is that you haven't actually reached the next line yet (there is no 0D 0A at the end) so this line will continue to the next packet.

Your posted line, for example, has 393 bytes not 999.

Can't the implementor at least not add the spaces? That is a huge overhead for the MUD for no reason. I mean, all he has to do is do a "trim" at the MUD end before sending the line.

Anyway, I think this plugin will do it for you, if they won't change the MUD:

To save and install the Omit_Trailing_Spaces plugin do this:

Copy the code below (in the code box) to the Clipboard
Open a text editor (such as Notepad) and paste the plugin code into it
Save to disk on your PC, preferably in your plugins directory, as Omit_Trailing_Spaces.xml
- The "plugins" directory is usually under the "worlds" directory inside where you installed MUSHclient.
Go to the MUSHclient File menu -> Plugins
Click "Add"
Choose the file Omit_Trailing_Spaces.xml (which you just saved in step 3) as a plugin
Click "Close"
Save your world file, so that the plugin loads next time you open it.


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>

<muclient>
<plugin
   name="Omit_Trailing_Spaces"
   author="Nick Gammon"
   id="a9ebc1f78f75b3bfc7c12e3b"
   language="Lua"
   purpose="Omit trailing spaces from packets"
   date_written="2014-08-30 08:11"
   date_modified="2014-08-30 16:25"
   requires="4.80"
   version="1.3"
   >
</plugin>

<!--  Script  -->

<script>
<![CDATA[

local previousPacket = { }
local lines = { }

-- trim trailing spaces, add to lines table
function trimTrailing (s)
  s = string.gsub (s, "\r", "")
  lines [#lines + 1] = (string.gsub (s, "^(.-)[ ]*$", "%1"))
  return "" -- omit from the existing line
end  -- end of function trimTrailing
                    
function OnPluginPacketReceived (packet)
  
  -- wait for compression to be turned on
  if not GetInfo (103) then
    return packet
  end -- if
  
  -- add packet to previous ones
  previousPacket [#previousPacket + 1] = packet

  -- combine into one long string
  local allPackets = table.concat (previousPacket)   

  lines = { }  -- no lines yet
  
  -- break up combined packet into lines with CR/LF in them
  local LeftOver = string.gsub (allPackets, "([^\n]*)\n", trimTrailing)
   
  -- if no carriage-return/linefeed in line return nothing
  if #lines == 0 then
    return ""
  end -- if
  
 -- this last part did not have a CR/LF in it
  previousPacket = { LeftOver }
  
  -- return existing lines with the LF added back
  return table.concat (lines, "\n") .. "\n"
end  -- OnPluginPacketReceived

function OnPluginConnect ()
  previousPacket = {}
end -- OnPluginConnect

]]>
</script>

</muclient>

[EDIT] Amended to fix a few bugs.

What this does is initially batch up incoming packets until it gets a CR/LF (0D0A). This allows for long lines to be split over multiple packets.

After each packet arrives it assembles the combined packets into one long string, and tries to break it up into lines. In the process, trailing spaces are omitted. Those lines are then returned.

Warning: It is the nature of such a process that "prompting" will not work. eg.


Enter your name:
Enter your password:

This is because the CR/LF "trigger" has not arrived.

There's nothing much that can be done about that, although possible you could amend it to have a timer, and output a partial line if (say) two seconds have elapsed.

You could disable the plugin initially (eg. during the logon process) and then enable it once logged on. You could make a couple of triggers to do that.

EnablePlugin

The documentation for the EnablePlugin script function is available online. It is also in the MUSHclient help file.

Note re MCCP.

If your MUD uses compression, I have made the plugin do nothing until compression is turned on, or it will behave strangely.

If your MUD does not use compression (MCCP) then omit these lines:


  -- wait for compression to be turned on
  if not GetInfo (103) then
    return packet
  end -- if

[EDIT] Amended again (to version 1.3) to fix a fairly major bug where previous lines would keep re-appearing.

Xvordan Sat 30 Aug 2014 08:19 AM #2

Thank you Nick. Trust me, I've tried several times to convince the implementor of this mud to do that very trim function. He, for whatever reason, seems to think it undoable without seignificant effort. I don't know anything about the LP codebase, so perhaps it is, and perhaps it isn't. In any case, I'll give that plugin a go.

The packet debug I posted did wrap the white spaces onto another packet, but I figured it was already enough spam on the post, so elected to post just that one line from that packet.

Thank you again for your time.

Nick Gammon Australia Forum Administrator Sun 31 Aug 2014 08:13 PM #3

Amended again, now version 1.3.

One important line (in bold) was missing:


-- trim trailing spaces, add to lines table
function trimTrailing (s)
  s = string.gsub (s, "\r", "")
  lines [#lines + 1] = (string.gsub (s, "^(.-)[ ]*$", "%1"))
  return "" -- omit from the existing line
end  -- end of function trimTrailing

Without that you would see previous output in front of existing output, rather confusingly.

This version also strips the \r character. MUSHclient doesn't require it, and if some packets only had a \n at the end and others had \r\n it would have behaved improperly.