Pidgin Errors with Gmail

I discovered yesterday that my Pidgin client is refusing to connect to GMail/GoogleTalk with the error:

"SSL peer presented an invalid certificate"

After half an hour of digging, I finally found this Pidgin bug with a work-around:

https://developer.pidgin.im/ticket/17118

I have no idea what's going wrong here, and I can only assume Gmail has some new cert that's not playing nice with something on my system.  But seeing as it took me so long to figure out, I thought I'd help make this easier to find for anyone else who runs into the same problem find it.

UPDATED:
After a few days, this stopped working.  I can only guess that gmail is rotate SSL certs faster than I can keep up.

But I found this Gentoo forums post:

https://forums.gentoo.org/viewtopic-t-1057862.html

Which pointed the problem at gnutls, and sure enough, emerging pidgin without the gnutls use-flag solved the problem.

Gentoo, ReadyNAS, and iSCSI

I recently bought a Netgear ReadyNAS 104 while trying to recover from a failure of my old RAID-5 enclosure.  Now that it's all put together, it looks like I can pull a maximum of 50 MB/s, with typical speeds of 30-40 MB/s.  I'm not sure what the limiting factor is at this point, since it doesn't seem to be network bandwidth, or CPU on either end.  I think something in the system is latency bound, since it seems to pull a little faster when my desktop CPU is completely idle.

At first I was going to set it up using sshfs, but the little ARM CPU in that thing can only push about 3-4 MB/s when it has to do the encryption itself.  So what I finally settled on was exporting it as iSCSI, and having my desktop do the encryption with cryptfs (LUKS).

I get the feeling that iSCSI doesn't get a lot of love on Gentoo, so I figured I'd post my troubles and what I finally got to work.  This post from the Gentoo Wiki Archives was the most helpful, though I skipped all his interface setup, which I'm assuming was designed for a dedicated storage network.

iSCSI Basics:
A target is a server which is offering up a drive for clients to use.
An initiator is the client which consumes a drive to read or write it.

Init Script:
The init script for open-iscsi was kind of primitive... I had to install the unstable version (sys-block/open-iscsi-2.0.872-r2) since it hadn't been updated since modprobe stopped supporting "-l" (which still makes me sad...).  At that point, I had to go redo all my kernel setup since I compiled the iSCSI stuff into my kernel, and the init script assumes that it can manually load and unload the modules.  I eventually commented out all the do_modules calls to work around that.

I also installed net-libs/libiscsi although I never figured out if it was required or not.

I never did figure out how to get it to automatically connect to my drive, but I got the init script to stop complaining at me by setting AUTOSTARTTARGETS="no" in /etc/conf.d/iscsid.  At this point (plus commenting out the modprobes) I could start and stop the daemon cleanly.

To manually mount a drive:

#Start a session to a target
iscsiadm -m discovery -t st -p 192.168.1.100

#Open a drive named "group1"
# on the target "iqn.1994-11.com.netgear:host:a1b2c3d4f"
# (as generated by my NAS) at IP 192.168.1.100
iscsiadm -m node --targetname iqn.1994-11.com.netgear:host:a1b2c3d4f:group1 --portal 192.168.1.100 --login

#At this point /dev/sde should appear and you can mount/format it.

#Disconnect from the target to make the drive disappear 
iscsiadm -m node --targetname iqn.1994-11.com.netgear:host:a1b2c3d4f:group1 --portal 192.168.1.100 --logout  

You can add "-d 8" (with a number 1-8) for increased debug messages if things are going wrong, but I can't say I found it very helpful myself.

The drive letter it shows up at will increment as you keep connecting/disconnecting.  Which was enough to convince me I wanted to manually mount the thing so I've given up trying to get the init script to autostart it anyway.

CHAP:
CHAP is the iSCSI authentication protocol, and was the biggest pain point of the whole experience.  I couldn't get my NAS or open-iscsi to give me any kind of useful error message other than "it didn't work" (or more precisely "iscsiadm: discovery login to 192.168.1.100 rejected: initiator error (02/01), non-retryable, giving up" ).

There are two kinds of CHAP, one used to authenticate initiators (clients), and one used to authenticate the targets (servers).  Since I was already encrypting my data on my desktop, I didn't bother setting up the server authentication, but I think it works basically the same way.  Here's what finally worked:

First I had to generate an InitiatorName and tell my NAS to restrict access to only that initiator.  My NAS only had a field for one "Initiator (IQN)" entry, while the Linux settings had 3 different values.  I don't really know what the difference is supposed to be, but I had problems until I set them all to the same thing.

## -*- mode: Conf-space;-*-
##/etc/iscsi/initiatorname.iscsi
## For examples, see /etc/iscsi/initiatorname.iscsi.example

InitiatorName=iqn.2008-10.com.example.mybox:openiscsi-a1b2c3f
InitiatorAlias=iqn.2008-10.com.example.mybox:openiscsi-a1b2c3f

Uncomment the following lines in /etc/iscsi/iscsid.conf, and set them accordingly.

##/etc/iscsi/iscsid.conf 
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.2008-10.com.example.mybox:openiscsi-a1b2c3f
node.session.auth.password = ThisIsAPassword

For whatever reason, CHAP passwords have to be between 12 and 16 characters.  I spent a while failing to get it to work with an 8-character password before I figured this out.

It probably goes without saying, but make sure you type the same password into your NAS.  (And set it for your initiator, instead of what my NAS called the "bidirectional authentication" for letting clients know they're talking to the right server.)

There are also settings in iscsid.conf for using CHAP during discovery.  You might have to set those too, but my NAS didn't seem to support it, so I left them alone.

After some trial and error, I finally figured out that my NAS was picking up password changes immediately, but my client was saving some kind of a session that was making things difficult.  Eventually I determined that to actually get my new settings to take effect I had to stop my daemon, rm -rf the session folder it had created in /etc/iscsi/nodes/, start the daemon again, and then start over from the discovery phase.

And there you go!  Good luck.

Skype and Gentoo amd64

I have a weird problem where my skype video intermittently (read mostly) doesn't work.

I found a solution here:
http://papers.ch/?p=94

Apparently it doesn't figure out it needs to load the 32-bit versions of v4l ...

Fun with gst-plugins

To anyone trying to figure out how to play .m4a files with gstreamer, try installing gst-plugins-faad.  At least, 50 plugins later, I think that was the one that finally fixed it...

Optimizations and CFLAGS in an ideal world...

I was messing around with my make.conf today, and decided to play with my CFLAGS again. Now, I realize that realistically, there are some seriously diminishing returns for turning on additional compiler flags on a system-wide basis. Even with a single package, if you don't know what you're doing you can make things worse, and if you do know what you're doing, you might just be able to spend a few hours to squeeze out that extra 1% reduction in execution time. Playing with your whole system is likely to make some things marginally better and others marginally worse. And, when all is said and done, you're probably never going to notice anyway, because amarok still takes several minutes to filter your playlist. (Amarok and I have a love-hate relationship, but that's another rant.)

But, (like many Gentoo users) I twiddle with them anyway. Partly because it's fun learning about weird compiler optimizations, but truth be told, I'm also lured by the imaginary speed gains, and the vanity of turning on weird things and knowing what they do.

So, I was reading the Linux Magazine benchmarks comparing different Gentoo Optimization levels, and there's basically no significant difference from -O2 and -O3 with a couple of weird exceptions.

One of the UT2004 demos has a 20 FPS gain for -O2 over -O3, but none of the rest of them do. So that's a weird fluke.

The Dbench filesystem tests show a huge difference between optimization levels, with -O3 losing quite dramatically, with the difference diminishing as the number of clients goes up. This one's kinda concerning, it's definitely not worth setting -O3 if it's going to halve my disk access speed. But (perhaps in my ignorance), I really have no idea how changing your optimization levels will affect disk I/O. That should all be handled in the kernel, which is optimized separately. So I'm gonna assume that's some kinda weird behavior in the test harness, because I dunno anything about Dbench.

The image processing tests show a predictable bonus for -O3, since that's the kind of CPU bound task that would benefit from vectorization and inlining.

And last but not least, is the GtkPerf results for "AddText," where -O3 takes half as long as -O2. In my ignorance, I would guess that the gain is from inlining functions, since from what little gtk programming I've done it seems like there's tons of small functions you call for everything. But I really have no idea. All the AddText test does is keep adding short text fragments into a scroll box, so maybe there's a copy loop that gets optimized somewhere, I dunno. But, I know how to run GtkPerf, so I decided to check this out.

Turns out, because of Gentoo Bug #133469, gtk filters your cflags, and doesn't even distinguish between -O3 and -O2. Since the bug was dealt with in 2006, (and they're still filtering today), and the benchmark was done sometime in 2009, I'm going to assume that they were using the standard Portage tree and were running the same gtk library in both tests. So any speed gains they saw must've been some underlying system library they were calling into. So, I'd put my money on some string copy function which runs about twice as fast under -O3.

Now, the bug report was describing some pretty catastrophic failures, (and I'm sure there were), but after hacking the ebuild to use -O3, and playing with my system, I haven't seen anything crash yet. So why is it still filtered? Well, since Gentoo actually lets users set their own CFLAGS, a bunch of crazy idiots like me decide to go through and turn on whatever they can turn on for kicks, speed and glory, and then complain and file bugs when stuff breaks. They honestly don't want to deal with that, and I can't blame them. So their general policy (as far as I understand it) is to ignore any issues where people have something other than "-march=native -O2" set. And I can't fault them there, especially when most of them are volunteers, and there are lots of idiots who want to muck with CFLAGS that break things. And granted, you can't test every gcc release to see if it stops some random, unpredictable crash. But I've seen things like this come up often enough that I wanna put in my two cents.

First off, a lot of these issues come up because there's really no way to set package specific CFLAGS. The people who are seriously about optimizing, want to set different weird flags for each program, or at very least set all their weird crazy flags, and then filter them for package they know break. (Like this guy...) Yes, I know a couple people have hacked up their own techniques to do this, but they're not integrated into Portage very well, and they're all shunned by the developers. Personally, I don't think this problem is going away. There's always going to be people (crazy or guru) who want to set custom flags for custom packages, to fix known bugs if nothing else. I was once unable to compile a working gmp with my arch setting because of some weird bug, and at that point all you can do is manually adjust the flags yourself to emerge gmp, and then put them back afterwards. Granted that works, but it kinda sucks. And the more packages you want to do that with the more it sucks. I think they should just stick a package.cflags into portage, the same way package.use works now, and let people trash their system if they're retarded. That also has the advantage of letting them put all the cflag filtering that they do into one file, the way they manage package.mask.

But what's a sensible cflags policy? Well, let me describe the imaginary, ideal world first. Ideally, there's really only 3 kinds of optimizations (for whatever you're looking for, speed, size, etc). Optimizations that always work, optimizations that sometimes work, and optimizations that make assumptions in order to work.

Things that always work, are reasonable for any user to turn on, and make their own cost-benefit determination of compile-time/running-time/memory. Setting gcc -O2 turns on more or less things that always work, and anything that breaks with these settings is a bug.

Optimizations that sometimes work, include things like loop-unrolling, and is reasonable for someone to turn on if they know what they're doing. These might actually hurt your performance, but they shouldn't actually break anything, or it's a bug. Enabling gcc -O3 turns on some things in this category like inline-functions. In my opinion, users shouldn't be protected from their own ignorance if they're slowing their system down by setting things beyond their competence level. But if these actually break things, you've probably hit a compiler bug, and to me it seems reasonable to filter them out with a warning message.

Finally optimizations that work with assumptions, are the ones you shouldn't set without knowing what you're doing, and should (probably) never set them system wide. These are things like disabling exception handling in C++, or enabling fast-math. It might help under certain circumstances, but it will break if you don't satisfy the assumptions (like you're using exceptions). Anyone using these system-wide is probably stupid, or at very least they're trying to do something weird on their own.

So ideally, you should filter out everything known to break because of a compiler bug, temporarily, until the compiler can be fixed, and then allow them again. But, we don't live in an ideal world. Compiler bugs are hard to find, hard to prove, and hard to fix. And the crazier the optimization, the harder it is to verify.

So what do you do in the real world?

First, if Portage had a package.cflags, anyone who wants to do turn on weird optimizations globally can have their own place to fix it, without filing a bunch of bugs demanding ebuild filtering for everyone. And, you now have a place to store package-specific performance information. If somebody on their own wants to publish a set of "known fastest CFLAGS for this arch" by package, they can.

Second, I think that a policy of "if you're not using -O2, we don't care" is a bit much. I think it's fair to designate -O2 as stable, and -O3 as unstable, but if there are known issues with -O3 where things actually break, it seems like you should go ahead and filter the flags. So in other words, you don't verify that anything works at -O3, but if its clear something doesn't, then go ahead and make it work for everyone else. It's not that hard, and things really shouldn't be breaking at -O3 (on a theoretical level anyway). But any weird flags making assumptions about math or runtime or whatever, it's fair to ignore, because if you were guru enough to have a reason to need that flag on, you can filter it out yourself in a package.cflags, and if you weren't that guru, you should probably turn it off. And I'm only talking genuine, verifiable crashes or errors at -O3. Slowing your system down is your problem, and if -O3 is designated "unstable," I think it's fair to let other people find and verify the problems as they come up.

Of course, I say -O3 "shouldn't" break things, but I really have no idea how risky it is, or how often bugs come up. I have random parts of my system compiled at -O3 and haven't noticed any problems, but I'm just one guy with my weird setup. If Gentoo decided to start ignoring -O3 crashes because there were too many for them to handle otherwise, I can understand. But if not, it seems like they're worth noting and fixing in the ebuilds when they come up. For the record, I think Gentoo does an excellent job on the whole, and it's really probably not worth a developer's time to chase down weird CFLAG bugs people report when they could be fixing other things. I also totally agreed with the rationale behind filtering gtk+, though if the problems are gone, it might be time to remove the filtering (but I can't tell that from my setup).


So when all is said and done, I went and ran gtkperf with just my gtk+ optimized differently. The final results?

No optimizations: Total time: 76.99
-O2: Total time: 74.25
-O3: Total time: 74.02

There's really no difference.

Hacking in Gentoo

I started using Linux as my primary OS some 4 years ago, but as much as I liked to brag that I had the theoretical ability to modify (almost) anything on my system, the reality was that getting a generic source package to work on whatever particular distribution I had was difficult (or at least intimidating). At best the paths would all be screwed up, and at worse I'd get library issues. Not to mention the hassle of specially handling that package from then on in my package manager.

Enter Gentoo.

In Gentoo's package manager, Portage, every package is installed by running a custom Python script, and you can store custom versions of it using their overlay system, while keeping all the dependency tracking information. The Bottom Line: I can patch anything as it's being installed onto my system, and I get all their fancy install scripts for free.

So, for the Gentoo aficionado, here's how I fixed xfdesktop to do exactly what I wanted when auto-sizing my wallpapers.

Howto install custom software versions in Gentoo:

1. Setup a Portage overlay
   
   See the Gentoo Handbook or this site for more details but basically you make the /usr/local/portage folder, and add PORTDIR_OVERLAY=/usr/local/portage to your /etc/make.conf
   
   
2. Copy over the package you want to customize from the regular tree
   
   ie Copy everything from /usr/portage/xfce-base/xfdesktop to /usr/local/portage/xfce-base/xfdesktop
   
   
3. Make your patch
   
   Use the actual distfiles Gentoo is already using (emerge -f xfdesktop will list them) to save yourself headaches later.
   Personally I just unpack the source, setup a git repository, muck with the code, and then use git format-patch to give me the patch file I need.
   
   
4. Modify the ebuild to use your patch
   
   If you have a particularly weird package, this can be tricky until you get better at writing ebuilds. But for most packages, this is cake. Just add your patch to the list of patches they're already applying. (Or checkout other ebuilds that have some patches to add and copy what they did). Here's the diff -u of my new ebuild.
   
   

   --- /usr/portage/xfce-base/xfdesktop/xfdesktop-4.6.1-r1.ebuild    2010-01-18 16:38:19.000000000 -0800
   +++ /usr/local/portage/xfce-base/xfdesktop/xfdesktop-4.6.1-r1.ebuild    2010-01-27 17:26:10.355164734 -0800
   @@ -56,7 +56,8 @@
   $(use_enable debug)"
   DOCS="AUTHORS ChangeLog NEWS TODO README"
   PATCHES=( "${FILESDIR}/${P}-automagic.patch"
   -        "${FILESDIR}/${P}-assert.patch" )
   +        "${FILESDIR}/${P}-assert.patch"
   +        "${FILESDIR}/0001-Rewrote-backdrop-AUTO-selection.patch" )
   }

5. Put your patch file in the files/ directory.

That's it. Emerge and enjoy!

I currently run custom versions (read 1-10 line patches) of 4 packages, and I'm getting less afraid to try to muck with things. The funniest is probably nethack, which has the username for debug mode hard-coded into the source, so you can't access it unless your username is "wizard". I guess nethack dev's make a custom user to do their debugging? I'm not really sure how that one was supposed to work.

Anyway, Happy Hacking!