Back in April M (and until he states otherwise, he shall remain M) brought me on as a consultant at The Corporation (not to be confused with The Company, where I work with Smirk and P) to do help profile “Project Wolowizard” (M's name for the project (which deals with telephony communications protocols) and way better than any name I came up with for use on my blog) which later lead to writing a test harness (in C, C++ and Lua) and in late August lead to full time employment with The Corporation (they made an offer I couldn't refuse).
Which is good, because we (“we” being The Company) were let go from “Project: SocialSpace 2.0” due to an internal disagreement high up on the hiring company over the direction of the website. And shortly after that, Smirk was also hired by The Corporation to help with documenation and server ops. P declined to work with The Corporation; the Ft. Lauderdale Office being too far for him to work, so P still works for The Company (along with Smirk amd me, which the Corporation has no problems with).
So between “Project: SocialSpace 2.0” (until it blew up in late August/early September) and “Project: Wolowizard” (still ongoing for the foreseeable future) I've been too busy (okay, laziness has something to do with it as well) to update the ol' blog here. But now that I've settled into a new work groove, I hope things will improve around here.
Okay, what prompted me to update the blog was an incident that happened at The Corporation.
Like I mentioned, we're working on “Project: Wolowizard,” which involves dealing with SS7, a telephony-based communications protocol. To handle SS7, The Corporation licensed The Protocol Stack From Hell™. Now, given the amount of money that exchanged hands, one would think we would get decent documentation, source code and even technical support, but no, no source code, documentation that is technically correct but completely useless and what can only be described as “obstinate technical support.”
So for the past two weeks Smirk has been tasked with setting up The Protocol Stack From Hell™ to respond to SNMP queries. The technically correct but completely useless documentation indicated that it was a simple matter to set up SNMP support; just configure the IP address, UDP port and community string (really a password) in a few files and it should “just work.”
I wouldn't be mentioning it here if it “just worked.” Not only did The Protocol Stack From Hell™ fail to reply to SNMP queries, but it wasn't even opening up the UDP port. For two weeks, Smirk, R (the office manager in the Ft. Lauderdale Office of The Corporation) and a few others from the Main Office of The Corporation (which is in Seattle, Washington—nice because they're three hours behind us) were going back and forth with the obstinate technical support. Things were going nowhere slowly.
Then R suggested that Smirk set the SNMP community string to “public”, which is a typical default setting for SNMP. Of course that worked. As an off-handed remark, I suggested that Smirk should try “private”—the other, “more secure” community string that is commonly used.
That worked too.
But not the community string we wanted to use.
Head. Meet desk.
(For the record, The Protocol Stack From Hell was selected way before R, who runs the Ft. Lauderdale Office of The Corporation, even worked for The Corporation)
I mean, yes, there certainly was some braindeath in the “Project: SocialSpace 2.0” but at least it worked even if you could measure its speed in bits per eon. This wonderous feature of The Protocol Stack From Hell™ is almost criminal in its action.
All I can say is … expense accounts rock!
A few members of upper management from The Main Office of The Corporation arrived at The Ft. Lauderdale Office of The Corporation to check things out. And because they're upper managment, they pretty much get to expense anything they want, so they took the Ft. Lauderdale Office (meaning: us) out to dinner to a Brazillian steakhouse called Chima.
There is no menu. Instead, you just flip a small disk to the side indicating you want food; then you are inundated with men carrying large hunks of roast beast offering you slices of various cuts of beef, pork or lamb. It's a never ending river of meat; vegetarians need not apply. To stem the rising tide, just flip the disk over to the other side. You can keep doing this as long as you want. As much as you want.
And insanely good.
It also appears to be one of those “if you have to ask, you can't afford it” type places.
I should have also kept the small disk on the “no” side more often.
The food was great going down. It wasn't so much fun coming back up.
Bunny and I went to see “Red” tonight and yes, it was worth it. It's an action comedy film about a retired CIA agent (played by Bruce Willis, who is “Retired but Extremely Dangerous”—thus the name of the film) trying to lead a normal life who suddenly finds himself running for his life while trying to protect an innocent love interest (played by Mary-Louise Parker).
The cast is outstanding; John Malkovich playing a paranoid conspiracy theorist who's fun to watch on the screen, the always good Morgan Freeman, still sexy Helen Mirren and Richard Dreyfuss in a small but crucial role.
The film (thankfully) keeps its humor level throughout the film and it manages to keep the action sequences thrilling to watch. It's not high art, nor does it pretent to be anything other than a comedy action film. And for what it is, it's quite good.
I finally get back into the blogging mode when the server my site (and email) is on goes offline when its power supply flames out.
It was weird today—I felt lost without access to my server, mainly because of the loss of my personal email (yes, I still run my own email server; yes, I'm masochistic like that) but yes, the website being down was also weighing on me—until yesterday (Monday) I don't think any server my site has been on has been down for more than three or four hours (oh, there have been some times recently when the site has been down for over 24 hours but that's because various processes on the box were inadvertantly killed and I didn't realize it, but that's different than the actual server being offline).
But things seem to be back to normal, and I would like to thank the crew hosting my website for their hard work in getting it back up quicker than I expected.
I received this last week:
- MXXXXX XXXXX <XXXXXXXXXXXXXXXXXXXXXXX>
- Website Inquiry
- Tue, 12 Oct 2010 15:32:04 +0800
I recently discovered your "Uhg … sushi for breakfast" page here:
I wanted to let you know that the “PublixDirect” link on your site points to a website (
http://www.publixdirect.com/) that is no longer working. Would you please consider replacing it with a link to my website called Boating.com? It is a resource that provides hundreds of used and new boats for sale, as well as reviews, tips and buying guides for anyone interested in boating.
If you think it would be of use to your visitors, would you please consider adding a link to my website on your page here:
Or in any of the pages of your site that will be most appropriate.
Here is the HTML link you could add: <a href="http://www.boating.com">Boating.com</a> - the complete boating resource.
Will look forward to hearing from you.
My original thought was to post it immediately, but I thought better of it at the time. I mean, it's nice that M pointed out a dead link on my site, but given that that particular entry is eight years old that is to be expected (also, PublixDirect was discontinued in August of 2003).
But I found it really odd to suggest replacing the PublixDirect link with a site about boat sales. It doesn't make sense in the context of the entry and it tells me that M probably has some piece of software that looks for sites with broken links that have a certain page rank and spams any address found on the page asking to replace the dead link.
Nice, except I'm not a link farm. The links in each entry provide (or at
least, I hope they provide) a context for what I'm writing about and
frankly, having the text “PublixDirect” go to a site about boat sales
doesn't serve me, or my readers, any good purpose (hmm … but what to
do about links that have since changed since I wrote an entry
… but that's another
show entry). It's also obvious that M
never bothered to read the entry in question or else M wouldn't have
bothered to suggest replacing the PublixDirect link with some spam site.
I replied to M (but without quite the snarkiness here) but have yet to receive a reply, so a week later, it's fair game. And heck, if I'm lucky, perhaps this entry will become the complete boating resource; that'll show M.
I received the sample ballot in the mail today (okay,
technically yesterday but I'm still in “today” mode) and I'm
excited to see all the non-Republican/non-Democratic candidates in the eight
races I can vote in on November 2nd. There are eight Republicans, eight Democrats (figures the
Republicans would have a
.com and the Democrats a
.org), one Libertarian (Alex Snitker), one Constitutional Party of Florida (Bernie DeCastro),
one for the Tea
Party (Ira Chester), two
Party candidates (Peter Allen and his running
mate John Zanni) and fifteen (count them, fifteen)
I find that wonderful for some reason.
Do I really expect the non-Republicans/non-Democrats to win?
Not really, but a guy can hope, right?
Hey, there! Thanks for your interest in my 8-Week Online Novel-Writing Course. It's designed to coincide with November's National Novel-Writing Month (NaNoWriMo), but includes both pre-drafting and revision instruction as well.
I've been a teacher of creative writing at a major university for more than 8 years, and all through that time I've done independent novel-writing coaching as well as written these dang things myself! When you finish my course, you will have a revised draft of a novel ready to post, sell, share, or revise further. (I offer one final read with comments after your revision!) The only way to be a novelist is to WRITE THAT NOVEL! I will be your cheerleader, drill sergeant, and best friend during this whole process.
My friend Hoade, college English instructor and zombie expert, is offering this novel-writing course.
And I think I'm going to try it.
Testing “Project: Wolowizard” isn't easy, what with dealing with The Protocol Stack From Hell™ and clarifying which optional fields of certain messages are mandatory, further complicated by the fact that some optional mandatory fields are optional if other optional mandatory fields are included, but there's also a feature that if used, means you may not include an optional mandatory field, but if you do, then you need to set another optional optional field.
It's all very confusing.
But while I'm waiting for the first extended test run to finish (24 hours of continous tests), I decided to investigate a particular phenomenon I've noticed when developing the testing program.
We need a way to test various message rates and one of the easiest is somethine like:
for count = 1 , 1000 do endpoint:send(msg) sim.sleep(thepausethatrefreshes) end
If I want to send 10 messages per second, then
thepausethatrefreshes will be
0.1 (sleep for a
tenth of a second); if I want 1000 messages per second, then
thepausethatrefreshes will be
0.001. It's an easy
way to dial up or down the number of messages per second sent.
sim.sleep() function (we're using Lua to script our test scenarios) is really a
wrapper around the C function
nanosleep(), which, in theory,
allows a very fine resolution (to the nanosecond) but in reality is limited
to the hardware on the system.
And in my testing, I've found that at best, I can get only about 100
messages per second. I wrote a program to test the limits of
nanosleep() and found that on our particular testing platform,
the lowest I can go is 0.01 seconds (my Linux system, on the other hand, can
go as low as 0.002).
As a second test, I just blasted out messages as fast as possible and was able to get a sustained (that is, actually sent and acknowledged) 1,500 per second (or one every 0.0006 seconds). I have to rethink my approach here. I think what may work is to send groups of messages before pausing a bit.
So, if I wanted to send 1,000 messages per second, I would need to write something like:
function thousand_per_second() for pauses = 1,50 do for msgs = 1 , 20 do endpoint:send(msg) end sim.sleep(0.02) end end
This sends twenty messages per batch, then pauses 0.02 seconds between batches, to give us approximately 1,000 per second (actually, a bit less due to processing overhead).
All I wanted to do was pass some data from one thread to another. I had code to manage a queue (based off the AmigaOS list code), so that wasn't an issue. And while a pthread mutex would ensure exclusive use of the queue, I still had an issue with one thread signaling the other that data was ready.
I took one look at pthread condition variables, didn't understand a word of it, and decided upon a different approach.
The primary issue really wasn't the transferring of data; it was the processing to be done with the data. I'm writing code to drive, let's say, a widget, for “Project: Wolowizard” and part of that test code is simulating, say, a sprocket. When the sprocket gets a request, the original code could return a result or just drop the request. R wanted a way to delay the results, not just drop them. To avoid a redesign of the test program, the easiest solution was to just spawn a separate thread to handle the delayed replies.
So, not only did I have to transfer the request, but signal the other thread that it had a new request to queue up for an X second delay.
My first test approach (a “proof-of-concept”) used a local Unix socket
for the transfer. This approach had the benefits of avoiding the use of
mutexes and condition variables, and the code was fairly easy to write.
poll() the local Unix socket for data with a calculated
timeout; if you get a timeout, then it's time to send the next delayed
result, otherwise it's a new request to queue (ordered by time of delivery)
and recaluclate the timeout.
But I found it annoying that the data was copied into and out of kernel space (not that performance is an issue in this particular case; it's more of an annoyance at the extra work involved).
Fortunately, I found
sem_post(). I could use
sem_timedwait() to wait
for requests, much like I used
poll(). Some mutexes around the
queue (I originally used some other semaphores, but M suggested I stick with
mutexes for this) and the code was good to go.
Only it didn't work in the actual program. My “proof-of-concept” worked, but in actual production, the second thread was never notified.
I asked M for help, and through his asking the right questions I suddenly had an idea of what was wrong—one quick change to a test script (the testing program I wrote is driven by Lua) proved my hunch right. Which is when I planted my face in the palm of my hands.
The upshot—my sprocket simulation works in one of two different ways (one method uses The Protocol Stack From Hell™ as a test) and the way I picked (which doesn't use The Protocol Stack From Hell™ and is thus, more “safe,” as both M and I are of the opinion that The Protocol Stack From Hell™ is also subtly buggy) failed to trigger the appropriate semaphore. Worse, had I used the local Unix socket, the whole thing would have worked as intended.
One of these days I'll get this whole multithreaded programming thing down pat.
Hoade's online novel writing class (disclaimer: we're “best-friends-forevah”) is really helpful. I think it's the structured nature of the course and the (so far) daily assignments (no writing yet!—that's another week or so away) that are keeping me focused.
And the constant emails yelling for completed assignments aren't hurting either (seekrit message to Hoade: Bunny's birds left extensive comments on my homework and trust me, you don't want to see it).
Maybe this time I'll have more than a pile of incoherent notes.
Nearly two weeks ago my server went down because of a faulty power supply. At the time, I made a joke about the power supply going out in flames.
Last night (technically this morning) the new server went down for a few hours (oh great!) but then I see this in email (forwarded by Smirk):
At approximately 1:49 a.m. EDT, UPS #2 experienced a catastrophic failure resulting in a fire. As a result, a portion of the XXXXXXXXX X datacenter experienced a power interruption that may have impacted some customers.
The Fire department responded to the fire in UPS #2 immediately, arriving on site at 1:57 a.m. EDT. For safety reasons, access to the XXXX XX area of the facility was temporarily restricted.
Well then … (memo to self: no more jokes about servers catching fire)