The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Friday, January 23, 2015

Generating regression test cases

We miscounted—there are, in fact, 9,697 test cases so far.

My manager, S, has been responsible for generating the test case descriptions, stuff like:

Section 1

Originating device is XXXXXXXXXX XX XXXXX and is calling a terminator with our client on the device test cases.

It's my job to generate the data required to run all these test cases, which means I need to run through the various variables and all their values, generating the data required to run the test and there's no better way of doing this than to brute force it. I could have written the code as:

-- variable names changed to protect the innocent.

for _,A in ipairs { false, true } do
  for _,B in ipairs { 'tel' , 'sip' } do
    for _,C in ipairs { 'tel' , 'sip' } do 
      for _,D in ipairs { false , true } do
        for _,E in ipairs { false , true } do
          for _, F in pairs { 'opt1' , 'opt2' , 'opt3' } do
            ...
              test_case(A,B,C,E,E,F, ... )
            end
          end
        end
      end
    end
  end
end

and while that does the trick, it's verbose and it uses a convention (using '_' to designate a result one doesn't care about, and in this case, I don't care about the first return result from ipairs()) that the other programmers here might not immediately pick up on (as far as I know, I'm the only one using Lua at the Corporation).

No, I feel it's better to show my intent. I want the code to look like:

for A in bool() do
  for B in list { 'tel' , 'sip' } do
    for C in list { 'tel' , 'sip' } do
      for D in bool() do
        for E in bool() do
          for F in list { 'opt1' , 'opt2' , 'opt3' } do
            ...
              test_case(A,B,C,D,E,F, ... )
            end
          end
        end
      end
    end
  end
end

And that's exactly what I did. The generic for loop in Lua is defined as:

for variable in function, state-variable, initial-value

where function is repeatedly invoked as function(state-variable,initial-value) until the it returns nil; when the result isn't nil, it's assigned to the main loop variable. In the first example, ipairs() (a standard Lua function):

function ipairs(list)
  local function next_value(state,var)
    var = var + 1
    if var > #state then
      return nil
    else
      return var,state[var]
    end
  end
  
  return next_value,list,0
end

ipairs() returns a function that takes a state variable (in this case, the variable list which should be an array) and an initial value (0) and this is repeatedly called until that function returns nil (here, when the index exceeds the number of items in the array).

For what I wanted, the function list() is easy:

function list(L)
  local function next_item(state)
    state.n = state.n + 1
    return L[state.n]
  end
  
  return next_item,{ n = 0 }
end

The function next_item() only cares about the state, which holds an index into the list (it's stored in a table because this is the only way we can modify it) which we increment and return that item from the passed in list L. We only return two values, the function and the state. The missing third value will be nil, which we don't care about. The one for cycling through booleans, bool() looks a bit more complicated:

function bool()
  return function(_,value)
    if value == nil then
      return false
    elseif value == false then
      return true
    else
      return nil
    end
  end
end

Here we just return a function; the other two values for is expecting will then become nil. And since I don't care about any state (the next value is readily apparent from the previous value we've returned), that parameter in the returned function is named _ (the conventional name for ignored parameters). Since we initially gave for a nil value, that's what we get on the first call, so we return false. On the next call, value is false so we return true. On the third call, we call through, returning nil which ends the for loop.

And those two functions make the test case generation code look much better. The intent is clearer, and you can easily match the code against the English description.

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: http://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

http://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2019 by Sean Conner. All Rights Reserved.