The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Wednesday, September 08, 2021

A map to a four dimensional space

In addition to expanding the output of an old program, I have also been expanding the mapping of the chaotic system (and like the last post, this post will have a very limited audience but again, I don't care). So this image:

is generated by iterating these two equations:

x_i+1 = (Ay_i + B) x_i (1 - x_i)
y_i+1 = (Cx_i + D) y_i (1 - y_i)

with the following values:

• A = +2.4376
• B = +1.5624
• C = -0.8659
• D = +4.0000

This image:

is a type of map where one can find chaotic attractors. It's a 2-dimensional slice of a 4-dimension space, where the horizontal axis is A, which runs from -4 (on the left) to 4, and the vertical axis is B, which also runs from -4 (on the bottom) to 4. If you look closely, you can make out a small red cross in the upper right hand area that marks the spot where the chaotic attractor above can be found in said space. Along the bottom, you'll see the four values listed above, with “Ax” marking the attractor location along the horizontal axis, and “By” marking the location along the vertical axis. The brighter the area, the more … um … expansive the chaotic attractor becomes.

But unlike thirty years ago, I decided to slice this a few different ways. If you imaging the above image as a slice through a 3-dimentional cube, this is the image you'd see if you were looking straight down on the cube at a horizontal slice through the cube. Shifting your view to the front, where A still runs left-to-right, C now runs up-down, and we get a vertical slice through the cube:

Looking at the cube from the left side—B values now run along the horizontal, C is still up-down and we get another vertical slice:

In each of those images, you should find a small red cross that marks the location of the attractor.

There are still three more planes we can cut through, although in this case, I can't quite make out the front, side or top. One is the A-D slice (and in each of the following images, you can make out the cross along the top edge):

The second is the B-D slice:

And the final one is the C-D slice:

Yeah … D ends up being vertical in all three … and that … kind of … makes sense … to me. Or am I going crazy? This is 4-dimensional space we're talking about.

Monday, September 06, 2021

Expanding a program from the past

I will start with a disclaimer: this post might be of interest to only two other people, neither of which read my blog (as far as I know). But it's my blog, and I don't care. I find it interesting. Anyway, on with the post.

I was playing around with an old program that plots a chaotic attractor. I won't pretent to know what the chaotic attractor is supposed to represent, but I find it fun to poke around the program from time to time. It revolves around two equations that interact with each other:

x_i+1 = (Ay_i + B) x_i (1 - x_i)
y_i+1 = (Cx_i + D) y_i (1 - y_i)

with the resulting image x and y values plotted on the graph between 0,0 and 1,1:

But then I got wondering—why the unit square? Thirty years ago I never questioned that. It should be easy enough to find out—just change the output range a bit:

Cool! I now have the range from -1,-1 to 1,1 working. So whereas before, I would have seen:

I can now see the full extent of this particular result:

And otherwise dull plots in the 0,0 to 1,1 range are now full of detail:

I am now left to wonder if this would have affected the research this program supported?

Saturday, August 28, 2021

The script kiddies have come to Gemini

Logging of just about anything on a Gemini server is, within the Gemini community, a contentious issue. Most people using Gemini dislike the web and the intensive logging of everything done with in, so of course they go overboard in the other direction. I've never agreed with that viewpoint and I do log information, even potentially personally identifiable information like IP addresses, because of crap like 5.252.227.126 is doing—repeated (and quite maliciously) trying to crawl my Gemini server for exploits.

And it would be one thing if it were well written, did a single scan, not find anything and move on. But it's not well written (hell, even commercial bots aren't well written and well behaved) and it repeately requests the same page over and over again. Until I blocked it, it had requested /index.php over 700 times. A sample:

Requests from a badly written Gemini bot that started just today

count	request
722	gemini://gemini.conman.org/index.php
721	gemini://gemini.conman.org/test/torture/index.php
86	gemini://gemini.conman.org/login.php
86	gemini://gemini.conman.org/test/torture/login.php

Among other requests.

And even now, several hours after I blocked it, it's still trying to make requests even though it's now getting the “no such port” error from the server. I just have to wonder if it's just too cheap to run these bots that it doesn't matter if they don't work all that well. Just enough to keep going and finding exploits. And hey, just becuase now we can't get that page doesn't mean it won't exist in 20 minutes, so keep making those requests.

Sigh. This is why we can't have nice things on the Internet.

More broadly though, I'm not sure what this heralds for Gemini. On the one hand, it's popular enough to attract the script kiddies to check for exploits. On the other hand, there's a large number of different servers so exploits for one server won't necessarily imply a globally workable exploit. And on the gripping hand, the fact that it's easy to write a server means the likelihood of an exploitable server is high.

But hey! Gemini hit a milestone! Script kiddies have hit the scene and now we have to contend with their crap! Woot!

---

A most persistent spam, part III

Earlier this month I nuked two email addresses being spammed by Alekandr and wanted to wait to see how things go. Two weeks later and Aleksandr (all “his” emails came from the same domain but from different subdomains) are still being sent, still being delayed by my greylist daemon only to be rejected because the destination email address no longer exists. I'm surprised that they are still attempting to deliver the emails. Is it that Aleksandr doesn't care if the emails make it or not? I noticed that the IP addresses are changing more often now—is it a test to see if the IP address being used is blocked? Is it a smear campaign against Aleksandr? What is the end game here?

Thursday, August 26, 2021

The case of the regression test regression, part IV

Like all bugs, it was a simple fix once identified—it was just identifying the bug that was difficult. The bug was due to a semantic change in an API. Lua has a file interface that includes a read() function, and the parameter it's given dictates how much to read. One of the parameters is “a” (or “*a” if you are using Lua 5.1) that indicates you want to read all possible data until the end-of-file is reached.

I ended up writing my own read() function to deal with network activity. I couldn't just open up a normal Lua file object with a network connection because that doesn't work nicely with event driven programs. An earlier version of my read() function had a different semantic meaning with “a”—it just returned any buffered data. It was in a later change where I aligned the semantics to match Lua's semantics, but forgot to change that bit of code in the one component. And those semantic changes explain the behavior I was seeing yesterday. It only took what? A week? To find the culprit.

Now the regression test with the updated code runs as fast as the previous version.

Wednesday, August 25, 2021

The case of the regression test regression, part III

When I last left off I identified the slow code in the regression test and it left me puzzled—it was a single function call that was called during startup and not when the tests were running. Faced with the fact that I had identified (for a second time) that code that did not change was causing a second of delay, I decided to change my approach. Instead of starting with the working, fast version of the code and going forward, I would start with the current, slow version of the code and work backwards. The first point of business—see if I could reproduce the fast code/slow code change with the code I identified as a sanity check. And of course that sanity check failed as both code versions were now slow.

What was that Sherlock Holmes quote? Ah yes, “When you have eliminated the impossible, whatever remains, must be the truth.” Must be the computer hates me.

Okay, so now that the slowdown was not the regression test script, but one of the mocked services. So I did to that script what I did to the regression test script—record how long it takes to handle a query. And of course it always responded in a timely fasion. No second long delay found at all!

So the delay is not in the regression test. The delay is not in the mocked service. I am slowly running out of ideas.

Okay, time to take stock: what do I know? Some queries are taking up to a second to run. They appear to be the same tests from run to run. Okay, let's run with that idea.

I modify the regression test to kick out which test cases are taking over 900,000 microseconds. I then just run those tests and yes, it's consistent. I remove those slow tests, the regression test just flows. If I just run the “slow” tests, each one takes about a second to run.

So now I know it's consistent. What's consistent about them?

Now I start digging into the logs and … yeah … something odd is going on with those tests. It's a path that “Project: Lumbergh” can take, and it's hitting a mocked service that I keep forgetting about. I mean, the regression test starts it up, but otherwise, it's pretty silent (and pretty easy to forget about). I add some logging to this particular mocked service, run the regression test and see that it responds once, then nothing ever again.

I add logging to the “fast” version and yes, it gets hit, and keeps responding.

I check the differences between the two versions:

[spc]saltmine-2:~>diff path/to/slow/version path/to/fast/version
96c93
< nfl.SOCKETS:insert(sock,'r',function()
---
> nfl.SOCKETS:insert(sock:fd(),'r',function()
106c103
< nfl.server_eventloop()
---
> nfl.eventloop()
[spc]saltmine-2:~>

Yeah … just some API updates, like yesterday. But if I run the new regression test while running the older version of this one mock, the new regression test runs as I expected. The delay is actually coming from “Project: Lumbergh” because the new version of this mocked service doesn't respond properly, causing “Project: Lumbergh” to time out, thus the delay I'm seeing.

I think I'm finally getting somewhere, but why is it so hot? And what am I doing in this basket?

Tuesday, August 24, 2021

The case of the regression test regression, part II

When you have eliminated the impossible, whatever remains, however improbable, must be the truth.

Sherlock Holmes

When last I left off I identified the slow code in the regression test and it left me puzzled—it was a single function call that did not change between versions. Now, a bit of background: eight years ago [Eight years⁈ Where did the time go? —Sean] [A world wide pandemic. —Editor] [Gee, thanks. —Sean] I wrote a custom Lua interpreter that contains all possible Lua modules we could possibly use at work in order to avoid having a bunch of code to install, which I call kslua (which stands for “Kitchen Sink Lua”). And so far, that's what I've been using to run the regression test.

Faced with the fact that the sipsock:recv() call was taking upwards of a second, I decided update just that module to the latest in the fast version of the regression test as a sanity check. Well, it failed as a sanity check, because the latest version of that module that contains that function ran fast, so my sanity wasn't saved one bit. The only conclusion I can come to is that something else has changed!

Fortunately, somethine else has changed. A bit more background: the regression test is used to test “Project: Sippy-Cup,” “Project: Lumbergh” and “Project: Cleese.” And to run those programs, I need a few more programs that those programs communicate with, and oh hey! There's a program that the regression program runs that also runs via kslua! And through a tedious process of elimination, I finally found a module that causes the slowdown—the network event driver module I wrote. I then went through a tedious process of elminiation to find the exact change that causes the slow down. The “fast” version of the function in question, which is written in C, is:

static int polllua_insert(lua_State *L)
{
  pollset__t *set = luaL_checkudata(L,1,TYPE_POLL);
  int         fh  = luaL_checkinteger(L,2);
  
  lua_settop(L,4);
  
  if (set->idx == set->max)
    /* ... */

and the slow version, which is the next literal version of the code:

static int polllua_insert(lua_State *L)
{
  pollset__t *set = luaL_checkudata(L,1,TYPE_POLL);
  int         fh;

  lua_settop(L,4);
  
  if (!luaL_callmeta(L,2,"_tofd"))
  {
    lua_pushinteger(L,EINVAL);
    return 1;
  }
   
  fh = luaL_checkinteger(L,-1);

  if (set->idx == set->max)
    /* ... */

I got tired of having to write (in Lua):

SOCKETS:insert(sock:_tofd(),'r',handler)

so I changed the code to call _tofd() directly:

SOCKETS:insert(sock,'r',handler) -- the system will know to call _tofd()

The only thing is—the program that calls this only calls this once in the program.

At startup.

Desk, meet head.

So I'm again failing to see how this causes the slowdown. I use the “fast” version and the regression runs fast. I click the version of that module one step forward and it's slow.

It's maddening!

---

All your CPUs belong to us

As if writing software without exploits is hard enough, now we have the most popular computer architecture, the Intel x86 line of CPUs, with a potential hole large enough to drive the NSA through. In a DEF CON talk, Christopher Domas shows how he found an exploit on a particular version of the x86 CPU that allowed him to gain total control over the computer without the operating system even knowing about it. All it involved is one undocumented instruction that enables access to a hidden CPU inside the x86 CPU (or rather, perhaps allow direct access to the underlying core that is simply interpreting the x86 ISA) followed by multiple copies of an x86 instruction that actually feeds instructions directly to this inner CPU that bypass all system checks because this inner CPU has access to everything (from user mode, and if you understand that statement, you know how bad it is).

As mentioned, this is only for a particular x86 implementation, but who knows what evils lurk in the heart of CPUs?

Probably the NSA.

Monday, August 23, 2021

I'm a computer illiterate dinosaur. Thanks, Microsoft!

All I needed to do was transfer a file to a fellow cow-orker. That's it. We were in a meeting, via Microsoft Teams. I was using the dreaded Corporate Overlords' mandated managed Microsoft Windows laptop. The file in question was on my work related Linux laptop (it was the first laptop I received when I started working at The Corporation 11 years earlier). So on the Windows laptop, I open up CMD.EXE:

Microsoft Windows [Version 10.0.18363.1621]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Users\sconner>cd Desktop

C:\Users\sconner\Desktop>scp spc@linux-laptop:work/file-to-trans .
spc@linux-laptop's password:
file-to-trans                        100% 6246KB  10.2MB/s   00:00

C:\Users\sconner\Desktop>dir
 Volume in drive C is Windows
 Volume Serial Number is 267D-3086

 Directory of C:\Users\sconner\Desktop

08/23/2021  06:55 PM    <DIR>          .
08/23/2021  06:55 PM    <DIR>          ..
08/23/2021  07:18 PM         6,395,862 file-to-trans
               1 File(s)      6,395,862 bytes
               2 Dir(s)  436,294,770,688 bytes free

C:\Users\sconner\Desktop>

I then open up File Explorer (sigh … that took entirely too long to find the name of that program), select “Desktop” and … the file is not there.

What?

Is the “Desktop” in the File Explorer not the same as “C:\Users\sconner\Desktop”? When did that happen?

My fellow cow-orker had to talk me through finding the file in question in the File Explorer, because “Desktop” there isn't the same one as “Desktop” in CMD.EXE.

What?

My brain broke.

Anyway, the file found, I then had to be talked through how to initiate a file upload through Microsoft Teams, and on this point, said fellow cow-orker and another fellow cow-orker had to work through how to send the file. Yes, apparently it takes three software developers to initiate a file upload through Microsoft Teams.

And then I had to describe to fellow cow-orkers that yes, this will take some time because I'm behind an honest-to-God DSL connection and NOT a multi-gigabit ethernet connection, because Chez Boca exists outside the service area for anything faster. It took several minutes of convincing them of this.

And the upload failed, probably because I'm behind an honest-to-God DSL connection and NOT a multi-gigabite ethernet connection.

Seriously, how hard is it to transfer SOME BITS?

So, not only did I have trouble with sending a fellow cow-orker a file, I somehow destroyed CMD.EXE's ability to display text as I was writing this entry. WTF? All I was trying to do was find a way to cut-n-paste the text from CMD.EXE and somehow, I manged to XXXX CMD.EXE so bad, it wouldn't show text any more. Several minutes later, I got the text back up (how? XXXX if I know) but now the cursor is missing.

XXXX it! I can deal with that.

<slow-clap>

Way to go, Microsoft!

But when did I become so computer illiterate?

• Current
• First
• Last
• Top
• Home
• About
• Archive
• Search
• Glossary
• Copyright
• Help
• Accessibility

Obligatory Picture

Obligatory Contact Info

• Comments? sean@conman.org

Obligatory Feeds

• RSS Feed
• Atom Feed
• JSON Feed

Obligatory Links

• Flutterby!
• KIRK.is

Obligatory Miscellaneous

• About the Source Code
• Source Code

Copyright © 1999-2021 by Sean Conner. All Rights Reserved.