the lua pcre lib seems to be not usefully complete

Posted by Fiendish on Fri 04 Mar 2016 01:52 AM — 7 posts, 29,755 views.

Fiendish USA Global Moderator Fri 04 Mar 2016 01:52 AM #0

There are only 3 exposed functions in MUSHclient's PCRE object (re:exec, re:match, re:gmatch). As far as I can tell, it's impossible to implement something like gsub using them (lrexlib has this). Adding either re:gsub or something that enables the possibility of making a re:gsub would be very useful.
Confirm/Deny?

Nick Gammon Australia Forum Administrator Fri 04 Mar 2016 02:44 AM #1

It doesn't look like that library has been updated since 2004, so you are probably correct.

Fiendish USA Global Moderator Fri 04 Mar 2016 03:25 PM #2

Well, I was wrong. It's not impossible!

Can this be added somehow?


-- like string.gsub but using PCRE
rex.gsub = function(str, re, rep)
   if type(re) == "string" then
      re = rex.new(re)
   end

   local output = {}
      
   local startfrom = 1
   local s, e, t = re:exec(str, startfrom)
   while s ~= nil do
      local captures = {}
      for i=1,#t,2 do
         table.insert(captures, str:sub(t[i], t[i+1]))
      end
      
      local filled_rep = rep:gsub("%%(%d+)", function(index) return captures[tonumber(index)] or "" end)
      table.insert(output, str:sub(startfrom, s-1))
      table.insert(output, filled_rep)

      startfrom = e+1
      s, e, t = re:exec(str, startfrom)
   end
   table.insert(output, str:sub(startfrom))

   return table.concat(output)
end

Fiendish USA Global Moderator Fri 04 Mar 2016 09:42 PM #3

Actually, my tests seem to show that this is about 25% faster (aardwolf, so luajit 2.1) using

\\\time = utils.timer(); for i=1,100000 do rex.gsub("Nick goes East Man goes West HE GOES HOME", "(\\w+) goes (\\w+)", "%1 de ho") end print(utils.timer()-time)


rex.gsub = function(str, re, rep)
   if type(re) == "string" then
      re = rex.new(re)
   end
   output = ""
      
   local startfrom = 1
   local s, e, t = re:exec(str, startfrom)
   while s ~= nil do
      local filled_rep = rep:gsub("%%(%d+)", 
         function(index) 
            local i = tonumber(index)*2
            return str:sub(t[i-1], t[i]) or ""
         end)
      output = output..str:sub(startfrom, s-1)..filled_rep
      
      startfrom = e+1
      s, e, t = re:exec(str, startfrom)
   end

   return output..str:sub(startfrom)
end

I guess that's what I get for premature optimization.

Nick Gammon Australia Forum Administrator Sat 05 Mar 2016 05:27 AM #4

"This" is faster than what? You posted two lots of code. (I think).

Fiendish USA Global Moderator Sat 05 Mar 2016 12:01 PM #5

Nick Gammon said:

"This" is faster than what? You posted two lots of code. (I think).

The code in #3 is faster for me than the code in #2 using the simple test in post #3.

Fiendish USA Global Moderator Sun 21 Jun 2020 09:23 PM #6

I think this version also allows using the replacer function form of gsub

rex.gsub = function(str, re, rep)
   local output = ""
   local as_func = (type(rep) == "function")
   local startfrom = 1
   local s, e, t = re:match(str, startfrom)
   while s ~= nil do
      local filled_rep
      if as_func then
         local substr = str:sub(s,e)
         if (#t > 0) then
            filled_rep = rep(unpack(t)) or substr
         else
            filled_rep = rep(substr) or substr
         end
      else
         filled_rep = rep:gsub("%%(%d+)", 
            function(index) 
               local i = tonumber(index)*2
               return t[i-1] or ""
            end)
         end
      output = output..str:sub(startfrom, s-1)..filled_rep
      startfrom = e+1
      s, e, t = re:match(str, startfrom)
   end
   return output..str:sub(startfrom)
end