The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Wednesday, April 04, 2018

A few days to change what amounts to about three lines of code

I spent a few days tracking down a tough bug in “Project: Sippy-Cup”. The bug wasn't in the happy path (explained here) but the “unhappy path” that happens when a timeout occurs [memo to self: check the blog before spending a few hours rewriting what you previously wrote].

We were able to reproduce the error rather quickly. What took the time was trying to isolate the root cause, and what I should have done [memo to self: reread my previous memo to self] was enable the statement logging I did a few years ago. Doing so allowed me to find the root rather quickly.

Each request is handled in a coroutine (think “very cheap thread”). The coroutine will acknowledge the incoming SIP message, extract the data we need from said message and send it to the “Business Logic Unit.” But because we have a limited amount of time to do that voodoo that we do (else we exceed our SLA and have to pay large amounts of money to The Monopolistic Phone Company), we will only wait for a limited period of time for the reply from the “Business Logic Unit.” In either case, we then send out a SIP message and wait for an acknowledgement that it was received.

The happy path—nothing times out and everybody is happy. The unhappy path is when the reply from the “Business Logic Unit” is late, and that late reply was somehow showing up as the “acknowledgement,” which my code didn't handle gracefully at all!

So normally the program does:

And the unhappy path was supposed to go like:

So really, not much difference. But in reality things were different:

The fix, as always, was easy—if a coroutine is scheduled to run, don't reschedule it. What's not so easy is finding the root cause.

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: http://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

http://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2018 by Sean Conner. All Rights Reserved.