The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Wednesday, January 29, 2020

“This is an invalid protocol because I can't open a file”

TS2 comes to my desk at the Ft. Lauderdale Office of the Corporation. “I'm running a load test of ‘Project: Cleese’ and it's not functioning. If I run a normal test, it runs fine.”

“Hmm … let me take a look.” I head back to TS2's desk. Sure enough, “Project: Cleese” is crashing under load. Well, not a hard crash—it is written in Lua and what's crashing are individual coroutines due to an uncaught error (the Lua equivalent of exceptions) where the only information being reported is “invalid protocol.” I have TS2 send me copies of the data files and script he's using to load test, and I'm able to reproduce the issue. It's an odd problem, because it appears to be crashing on this line of code:

local sock,err = net.socket(addr.family,'tcp')

I dive in, and isolate the issue to this bit of C code that's part of the net.socket() function:

if (getprotobyname_r(proto,&result,tmp,sizeof(tmp),&presult) != 0)
  return luaL_error(L,"invalid protocol");

Odd, “tcp” is a valid protocol, so I shouldn't be getting ENOENT, and the buffer used to store data is large enough (because normally it works fine) so I don't think I'm getting ERANGE. And that covers the errors that getprotobyname_r() is documented to return. I add some logging to see what error I'm actually getting.

I'm getting “Too many open files” and it suddenly all makes sense.

getprotobyname_r() is using some data file (probably /etc/protocols) to translate “tcp” to the actual protocol value but it can't open the file because the program is out of available file descriptors. “Project: Cleese” is out of file descriptors because each network connection counts as a file descriptor, and the test systems (Linux in this case) only allow 1,024 descriptors per process. It's easy enough to up that to some higher value (I did 65,536) and sure enough, the “Too many open files” error starts showing up where I expect it to.

On the plus side, it's not my code. On the minus side, you have to love those leaky abstractions (and perhaps relying upon documentation a bit too much).

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: http://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

http://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2020 by Sean Conner. All Rights Reserved.