Sunday, July 21, 2024
Fixing an Apache pthread error
After I fixed the performance issue, I started looking around for some other issues to handle, and boy, did I find some. Checking the error log from Apache I found:
[Fri Jul 12 15:04:01.762845 2024] [mpm_worker:alert] [pid 31979:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:02.731958 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 31979 exit signal Aborted (6) [Fri Jul 12 15:04:02.735360 2024] [mpm_worker:alert] [pid 32021:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:03.733536 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 32021 exit signal Aborted (6) [Fri Jul 12 15:04:03.736857 2024] [mpm_worker:alert] [pid 32063:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:04.735368 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 32063 exit signal Aborted (6) [Fri Jul 12 15:04:04.738624 2024] [mpm_worker:alert] [pid 32105:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:05.737141 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 32105 exit signal Aborted (6) [Fri Jul 12 15:04:05.740622 2024] [mpm_worker:alert] [pid 32147:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:06.739077 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 32147 exit signal Aborted (6) [Fri Jul 12 15:04:06.742500 2024] [mpm_worker:alert] [pid 32189:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:07.740898 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 32189 exit signal Aborted (6) [Fri Jul 12 15:04:07.744130 2024] [mpm_worker:alert] [pid 32231:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:08.742689 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 32231 exit signal Aborted (6) [Fri Jul 12 15:04:08.745885 2024] [mpm_worker:alert] [pid 32274:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work [Fri Jul 12 15:04:09.744445 2024] [core:notice] [pid 19646:tid 4149040832] AH00052: child pid 32274 exit signal Aborted (6) [Fri Jul 12 15:04:09.747607 2024] [mpm_worker:alert] [pid 32316:tid 3924761520] (12)Cannot allocate memory: AH03142: ap_thread_create: unable to create worker thread libgcc_s.so.1 must be installed for pthread_cancel to work
Thousands of such lines.
And the weird thing is that libgcc_s.so.1 does exist on my system.
I guess I missed that library when I installed Apache from source.
I'm not even curious as to why this is an issue,
nor why libgcc_s.so.1 is needed.
At this point I'm like “give the system what it wants,
not what I want to give it”
(which I can't say in polite company).
Running configure –help didn't show any obvious means of enabling the use of libgcc_s.so.1,
and not wanting to dive deep into a maze of twisty Makefiles all alike,
I decided on the next best thing.
I went to the Apache build directory,
deleted the existing httpd binary and ran make.
This gave me the final line used to build the executable:
/home/spc/apps/httpd-2.4.54/srclib/apr/libtool --silent --mode=link gcc -std=gnu99 -g -O2 -pthread -o httpd modules.lo buildmark.o -export-dynamic server/libmain.la modules/core/libmod_so.la modules/http/libmod_http.la server/mpm/worker/libworker.la os/unix/libos.la -L/usr/local/lib -lpcre /home/spc/apps/httpd-2.4.54/srclib/apr-util/libaprutil-1.la -lexpat /home/spc/apps/httpd-2.4.54/srclib/apr/libapr-1.la -luuid -lrt -lcrypt -lpthread -ldl
I then reran just this command but with /lib/libgcc_s.so.1 added before the -pthread option.
A quick check afterwards:
[spc]brevard:~/apps/httpd-2.4.54>ldd .libs/httpd linux-gate.so.1 => (0xf77e1000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xf77cf000) libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0xf77b1000) libaprutil-1.so.0 => /usr/local/apache2/lib/libaprutil-1.so.0 (0xf778c000) libexpat.so.0 => /usr/lib/libexpat.so.0 (0xf776e000) libapr-1.so.0 => /usr/local/apache2/lib/libapr-1.so.0 (0xf7742000) libuuid.so.1 => /lib/libuuid.so.1 (0xf773f000) librt.so.1 => /lib/tls/librt.so.1 (0xf772a000) libcrypt.so.1 => /lib/libcrypt.so.1 (0xf76fc000) libpthread.so.0 => /lib/tls/libpthread.so.0 (0xf76ea000) libdl.so.2 => /lib/libdl.so.2 (0xf76e6000) libc.so.6 => /lib/tls/libc.so.6 (0xf75bb000) /lib/ld-linux.so.2 (0xf77e2000)
and it worked—libgcc_s.so.1 is linked in!
I installed the new Apache executable, restarted Apache and … it didn't crash! I then left it to run.
That was a week ago,
and so far,
so good.
The error log is rotated weekly and for the past week no such errors have appeared.
Now only if I could nuke from orbit the crawlers sending in silly requests
like /cgi-bin/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/bin/sh—it just clutters up the error log
with “invalid URI path” errors.
Sigh.
