Unmemorable Memories

Recently I updated my Ubuntu installation. After the update many of my java applications stopped working. Any java application that attempted to display something via X11 terminates with the following error:

current locale is not supported in X11, locale is set to CX locale modifiers are not supported, using default

You can google for possible solutions, which I did. This problem is also discussed in detail in the Ubuntu Forums.

They claim that this bug is known and has been fixed.

Well it isn’t. I tried all the other suggestions for the fix, but none of them work for me.

I have two solutions:

1. Run it in a 32-bit chroot environment (see this link. But this gets pretty annoying if you have a lot of different executables. Then I might as well ditch this 64-bit OS and install the 32-bit one.
2. Hack the 32-bit libX11.so.6.2 library. Keep reading if you are interested in this hack.

The Hack

1. Create a backup of /usr/X11R6/lib32/libX11.so.6.2
2. Edit /usr/X11R6/lib32/libX11.so.6.2 in the emacs editor (any other binary editor should work)
   • search for the string /usr/X11R6/lib/X11/locale
   • replace it with /usr/X11R6/l32/X11/locale
3. Save the file and exit emacs
4. Go to the /usr/X11R6 directory
5. Create a symlink called l32 that points to lib32 in that same directory
6. Go to the /usr/X11R6/lib32/X11/locale directory
7. Create symlinks here to all the files (except lib) that exists in /usr/X11R6/lib/X11/locale
8. You’re done

After applying this hack, my 32-bit java programs now run without complaining about locales.

Why this hack works
Since none of the workarounds and fixes I’ve found in Google and the Ubuntu Forums worked for me, I decided to do some strace’ing myself.

Strace showed that the program was opening /usr/X11R6/lib/X11/locale/lib/common/xlcUTF8Load.so.2. Now I know that the Java I installed is a 32-bit executable, and this library is a 64-bit binary. So my guess is that the linker fails to link with this library. To test my hypothesis, I replaced this library (after creating a backup of course) with the one from /usr/X11R6/lib32/X11/locale/lib/common/xlcUTF8Load.so.2 (the 32-bit counterpart). Just this simple operation, fixed my problem. However it introduces another one: all other 64-bit X applications complained about my locale (although they continued to work). I had to come up with a way to have both types of applications working.

I restored the original xlcUTF8Load.so.2 to keep my 64-bit X applications happy. Now I needed a way to get my 32-bit applications to link with the 32-bit xlcUTF8Load.so.2 library.

From the strace output, I compiled a list of all the X libraries that the application opened. Then I searched those libraries for the string “/usr/X11R6/lib/X11/locale”. The culprit was libX11.so.6.2.

I needed to modify this library to look at /usr/X11R6/lib32/X11/locale in order for it to find the correct version of xlcUTF8Load.so.2. Simply replacing ‘lib’ with ‘lib32′ will not work since it will change the size of the library, and mess up whatever offsets are setup in that library. I needed to replace ‘lib’ with an equal length string that points to the correct place. So I replaced it with ‘l32′, then created a symlink called l32 which points to lib32 in /usr/X11R6.

After doing this, I tested my application, and it continued to fail. What was going wrong? Further strace output showed that the app was no longer linking xlcUTF8Load.so.2. The strace output also showed a bunch of open() failures for files that lived in /usr/X11R6/l32/X11/locale. Those files exist in /usr/X11R6/lib/X11/locale. So by creating symlinks to all those files, the behavior of the application was restored (it once again discovered that it needed xlcUTF8Load.so.2).

Now all my java programs work again without sacrificing the functionality of my 64-bit X applications. And possibly other 32-bit X11 applications will also work with proper locale support.

Update: I have made additional hacks to get acroread to work.
1. create a symlink called /usr/l32 pointing to /usr/lib32
2. modify the libraries found in /lib32, /usr/lib32, and /usr/X11R6/lib32 to use /usr/l32 instead of /usr/lib
* /usr/lib32/libglib-2.0.so.0.400.7
* /usr/lib32/libgdk_imlib.so.1.9.14
* /usr/lib32/libgtk-1.2.so.0.9.1
* /usr/lib32/libgtk-x11-2.0.so.0.400.10
* /usr/lib32/libgdk_pixbuf-2.0.so.0.400.10
** change to this is tricky. Look for gtk-2.0 followed by “/etc” then change gtk-2.0 to g32-2.0
** copy /etc/gtk-2.0 to /etc/g32-2.0 and modify the gdk-pixbuf.loaders and gtk.immodules files inside the /etc/g32-2.0 to point to /usr/lib32 versions of the files
3. modify /lib32/tls/libc-2.3.2.so
** change /usr/lib/gconv to /usr/l32/gconv
** change /usr/lib/gconv/gconv-modules.cache to /usr/l32/gconv/gconv-modules.cache
4. modify /usr/lib32/libpango-1.0.s0.0.1000.0
** change /usr/lib to /usr/l32
** change /etc/pango to /etc/pgo32
** copy /etc/pango to /etc/pgo32
** modify /etc/pgo32/pango.modules to change /usr/lib into /usr/lib32

Update 2: I have upgraded to dapper (6.06 LTS). My hacks broke in this version because the libpango-1 library uses /var/lib/pango/pango.modules. The fix is to modify /usr/lib32/libpango-1.0.so.0.1201.2 and replace all occurances of /var/lib/pango with /var/lib/pgo32, then create a directory called /var/lib/pgo32 and copy the contents of /var/lib/pango into /var/lib/pgo32. Then edit /var/lib/pgo32/pango.modules and replace all occurances of “/usr/lib” with “/usr/lib32″.

-- Posted in Geeks Paradise

Never open the door for the enemy to walk in. Once you let him in by just a little bit, an inch, a millimeter, or even less than that, he’s going to walk all over your life and you will lose control.

Recently, my office moved locations. Being a System Administrator, I would normally be in charge of moving all of the computer systems and bringing them back online with a minimum amount of downtime. Our company serves many clients who cannot afford to have their web services down for any amount of time. It was difficult to try to get them to agree to service disruptions during our move. The problem was that the move was schedule for Friday night thru Saturday. And certain systems was scheduled to go offline by 9 p.m. Friday and online by Saturday morning. There was no way I could possibly coordinate or help in any way on this move since it falls on the Sabbath.

Weeks before the move, I planned the execution of the move so that whomever will be doing the move, will have to do minimal reconfiguration and everything would work out. I tested my plan, refined it, re-tested it, refined it again until I was sure that the systems would just plug in and things would all work normally. Well, Murphy, or should I say the Devil, had different plans. 2 a.m. Saturday morning I received a call on my cell phone from work. I did not answer inspite of repeated calls. I picked up the message in the morning when I woke up. I was being called in to work as soon as possible to bring the systems online. I called back explaining that this was my Sabbath and that it would not be possible for me to work. Later that day, I received 3 more calls, from varying levels of management in the company requesting that I go in and bring the systems up.

Pressure was mounting. The request came from someone high up the command chain, who just a week before granted me my request. I felt like I was letting them down. I was feeling a mix of fear, anxiety, embarrassment, and defiance. However fear was what was growing. I started to think of ways to justify working. Could I just login remotely from home and fix the problem from there? After all I am not at work. I’m at home. I could also drive to work, and fix it. The problem is really simple and it would only take me 5 minutes to fix. That wouldn’t be working, right? Could I just phone the other sysadmins and walk them through the process of trying to get the systems running? That wouldn’t be work right? As I thought about it, I still could not convince myself that it was right. Yet I had this big fear of possible repercussions if i did not work. I started thinking, “God would understand. He is a merciful God after all. He will forgive me.” By the time Divine Service started at our Church, I was already praying continously. I was praying for anything. The prayers went from “God, please perform a miracle and get those systems running” to “God show me whether or not I should go and fix it”. If there ever was a time for me to claim the promise in 1 Corinthians 10:13, it is now.

For years, I’ve been preaching sermon upon sermon about how not to compromise our beliefs, about the importance of the Sabbath, about having faith that God will take care of you. Now is the time to put those sermons into action.

To make a long story short, I refused to succomb to the temptation to work. I kept reminding myself that my God is bigger than my bosses. My God can and will take care of me.
I made one final call on Saturday afternoon to a company VP (yes it has escalated that far up the chain). (God must have known my fragile condition at this point because this VP who had called me less than an hour ago and left me a voice message, did not pick up the phone, so I left him a message). I explained to him one more time how this is the Sabbath day, and I will not work on the Sabbath.

After Sunset, I went to work, got the systems up and running in 10 minutes. (It turned out that everything was already working as planned, it’s just that the sysadmin who did the move did not understand how to test if things were working correctly. Not his fault since I may have assumed he knew and I may not have communicated the test procedure to him).

It dawned on me that since everything was already working, this must have been a test of faith.

Two weeks later, there was another project that required my services on Friday night, thru Saturday. It made me nervous that this is the second time they are asking me to work an a project during Sabbath hours, which I will once again refuse. It was easier to refuse this time.

Later that week I was in a car with the individual who requested my involvement in that project. Enroute to a customer meeting he asked what my religious affiliation was and why I could not work from Friday evening to Saturday evening. He was simply curious.

I was able to share my beliefs openly with him.

If I had not stuck to my personal convictions, how effective would my life be as a witness? For one thing it would be very difficult for me to get out of the second request to work on the Sabbath. Further, my explanation of the Sabbath to my co-worker would not hold (if I was willing to violate it, then it why should he even consider it as an absolute truth?).

Lessons learned:
1) Stick to your guns. It gets easier the next time around.
2) Don’t open the door the slightest bit to the tempter. Once you do, it bursts out completely open, and you will be left vulnerable to him.
3) It is easy to talk, and preach about things, but living what you preach is difficult.
4) Only through constant/continuous communication with God through prayer and recitation of his promises, was I able to stand my ground. It is He, who gives me power to overcome temptation, and the power to live what I preach.
5) It is vital to safeguard ones integrity. Integrity gives your witness credibility.

-- Posted in Journal, Bible, Religion and Philosophy