SSL + svn under Apache

Ok, I really need to document this for future. This costs waaaay too much time even having clues from all kinds of places.

Anyway, I was busy making a simple script to update a site remotely by triggering an svn checkout. Easy thing except for that the connection to repository is done using HTTPS and guess what – the certificate needs to be accepted. Trying things like ‘–trust-server-cert’ (needed to upgrade to svn 1.6.x for trying this out, on Ubuntu point to the Lucid repository if anyone needs it) didn’t help. Then I played ‘smart’ and logged in under the Apache user and tried to do the same trick manually. I’ve got to the point of accepting the certificate, etc, but… next time the same story.

Well, what ought to happen was that the user home folder (/var/www in my case) was NOT owned by the Apache user, but by the root. Damn… all the certificate acceptance stuff simply went nowhere, but I didn’t get a single warning on that!

So the solution was simple – make /var/www owned by the actual Apache user (www-data in my case under Ubuntu), log in under this user (sudo so www-data) and perform the operations I needed with the svn repository manually ONCE not forgetting to accept the certificates PERMANENTLY of course. Since then everything works fine.

Life goes by…

Do I need an iPad?

I would never think I’d ever put something like this into my agenda. And yet I did mark January 27 and even started checking news that day. OK, it was a bit exciting, but not really shocking. Now, when the mystery is unveiled and numerous reviews are available I join the crowd of people asking the same question: “Do I need one?”.

Let me start answering that with the picture from the “The Economist” which I liked so much that I teared off the front page (for the first time in years I am subscribed to this nice magazine). The Economist: The Book of JobsThe front page has a headline “The Book of Jobs” and I consider it the best one I saw so far on describing the perception of the Steve and Apple in general (even my 2 years old daughter knows that my MacBook is an ‘appel’ (Dutch for ‘apple’)).

Anyway, back to the subject. Is this a holy grail everybody ‘must have’ or just another hype to hype waporized by the rivals? Well, it is A hype, and since it is one from Apple it is a sticky one. So it will stay around for a while. But looking at the specs and the added value my geeky side starts to doubt if it is indeed ‘one for all’ (avoid looking at slightly different models promised from the beginning). Let’s see.

First. I wouldn’t go for the first version. Simply because they will add camera and fix the inevitable bugs. At least I wouldn’t go for the top specs. The lowest model may be just enough to play with (and eventually give it to my daughter for watching kids stuff which she will do anyway), but it cannot even do videoconferencing. What’s the point of carrying around the dumb thing? Except perhaps for the showing off part. But this will tear off rather quickly I presume.

Second. It is a ‘closed brick’. Heck, put a (slightly downgraded) MacOSX on it and I am in a row. But having a ‘phone OS’ on a tablet. Well, it is definitely not targeted on a techies market (nothing wrong, btw), so I am afraid it is not going to work for the people like me who want to decide themselves what do they want to see running. The first app I am starting on my MacBook is the command line. Double. They it nicely covers the rest. Ok, this is perhaps not normal, but that was THE final reason for me to choose for the Mac, not because of the Aqua interface (which still sucks in keyboard support to my opinion). I am not asking for a terminal support in iPad, don’t get me wrong, but having such a thing opens the whole world of possibilities. I do want to automate things, write simple tools which do not require XCode (never got to it anyway), Perl or Python would do. But no close ecosystem for me.

Third. It does not have a single port except for the damned proprietary connector. What? No USB? Yes, it is not a computer, but there are so many things you can connect via USB that are just dumb practical that I cannot accept a ‘brick without input holes’ (a card-reader, anyone? Am I supposed to show off with my pictures only when I have connected it to a PC? And What if I am travelling???). A bluetooth is indeed a partial answer, but knowing Apple you would be limited to a list consisting of a [‘keyboard’] and guess what is the place to buy it… Of course… An Apple store!No outputs. Hey, the iPad screen is not the only thing people are watching. Heck, I can connect my iPod classic to my TV and not the latest-greatest? Oh, I see, this is for individuals, not for family/friends entertainment perhaps. Ah, stupid me, no friends, enjoy the show yourself. Hmm…

Anyway, the last drop is that I hate ‘slight curves’ in the design and the back of the iPad is exactly what I mean. I think the best thing what iPad can do for techies is dropping prices of the existing tablets or (even better perhaps) spawn a whole new idea of what a tablet can be. I am very surprised how lousy Microsoft does with their own idea of the tablet (announcement, and then what?), I think more agile guys like Asus will catch up soon and they can take into account wishes from the tech crowd much better (cheaper, anyone? :)).

So the answer is NO. I don’t need one. I need either an improved one (but no, I am not willing to pay more), or simply something else (which sadly does not exist yet). But it is OK, I am not in a hurry. Buying something which is has obvious flaws tend to be left for dust rather quickly. And this may turn to be rather expensive dust to be…

MySQL and SSL: ERROR 1045 (28000): Access denied for user…

Ok, this one was just so frustrating I cannot keep it to myself. It costs so much time and nerves to realize that you were pointed in all but the right direction after all.

So here it goes. I have added a user to the MySQL database, I require the SSL connection (actually X509) and it all works on my development system (MacOSX 10.4). So I naively didn’t expect any problems on the test server running Ubuntu. I was wrong. And my mistake apparently has a name… But not all.

So what I see is that when I try to connect to the DB using either the Python/MySQLdb or giving the command line is the same:

_mysql_exceptions.OperationalError: (1045, "Access denied for user
'xxx'@'localhost' (using password: YES)")

or

ERROR 1045 (28000): Access denied for user 'xxx'@'localhost' (using password: YES)

Great. Now what?

I tried every single thing that made sense to me:

  • Re-created the user, updated the privileges and even added
    FLUSH PRIVILEGES
    at the end (never needed it before).
  • Re-generated the certificate files
  • Different DB

Ok, after all it boiled down to the following messages in the dmesg output:

[877463.513737] audit(1263600950.291:21): type=1503 operation="inode_permission"
requested_mask="r::" denied_mask="r::" name="/xxx/xxx/certificates/server-cert.pem"
pid=11840 profile="/usr/sbin/mysqld" namespace="default"

Bingo! MySQL cannot read the f…g certificate? Why-y-y-y-y? Apparently because the
apparmor
(now I know the enemy’s name!) does not allow it. And this is all because I have installed certificates to the ‘non-standard’ folder. Well, adding the following line to the

/etc/apparmor.d/usr.sbin.mysqld
/xxx/xxx/certificates/*.pem r,

and then issuing the following commands:

#apparmor_parser -r /etc/apparmor.d/usr.sbin.mysqld 

#sudo /etc/init.d/mysql restart

finally put everything on their places.

2 hours, a lot of frustration, but it seem to work now. I hope you spend less time finding this in Google :).

Have a nice weekend!

Win Mobile – BlackBerry sync

Since mobile phone service providers are keen on bundling service with some new shining pieces of hardware (otherwise some services becomes suddenly unavailable), this was the time I could a new piece of mobile engineering for myself. It was 2 years ago when I’ve got my Windows Mobile 6 HTC Keiser II thingy and as one can imagine there is quite some information piled up in it since then.

I have downsized my field of choice to two options: hyped media-driven video-oriented iPhone (which would have been a nice addition to my range of Apple devices) and a business-oriented true keyboard but small screen BlackBerry Bold. Well, easy to guess from the title – I’ve chosen for Bold. I am not as media-oriented person and I do love typing (even if it is a query for Google); I doubt the hype of iPhone (although BB also has quote some) and… I don’t like T-Mobile which is the only option I have in the piece of land I happen to inhabit. I require good service for the money I normally pay monthly, so sorry, T-Mobile, you have some homework to do. Vodafone is still beating you left and right. So here I go, one hour later than I have expected with a new (flat and wide) piece of Canadian (well, sort of ;)) hardware in my pocket. Some more thorough review to come, but before anything else I miss some vital data on this thing. Namely contacts and calendar. I miss a part of memory in my head which was supposed to keep this data for me for years now, so I need something else (external) to help me out with it.

Well, having a Mac does not give much sync options for non-standard (read non-Apple or old/primitive) devices. Buying software is against my programming nature and since I need it once (for now) why do it anyway. And then I remembered something about Google sync… A short search around revealed that I can do just what I need, although I may get some noise from Google which saved every since e-mail address as a separate contact. Hmm. Better more than less.

Starting with the source. It took me some time to figure out, although it is described at Google sync Win Mobile page. Here are the settings that actually worked for me:

  • user: @gmail.com
  • password: your_google_account_password
  • domain: leave grayed out (whenever @ symbol is entered, done automagically)

Do NOT select synchronizing tasks! This will result in an error and a lot of time in restoring the device as you cannot change the settings (account, domain) for some unthinkable for me reason after you have tried to sync once.

Anyway, after I have figured out that I wanted too much and only put the settings as described above it finally worked. I’ve got my data in Google.

Second step, BB. Well, feel the difference. Just download the app from m.google.com/sync using the BB browser and the only info you type in is your GMail credentials. No place for mistake.

Ok, the first disappointment. No option to sync the COMPLETE calendar. Why? No idea. I could put all events from my Windows Mobile device, but not get them back. Hmmm. Not exactly what I have expected, but better than nothing and it is after all FREE :).

Working with unicode in Python (again)

This time I have stumbled (again) a unicode problem using some Python code which was supposed to be perfectly suitable for doing this since it even started with


#!/usr/bin/env python
# -*- coding: UTF-8 -*-

It was quite some time since the last post, but this does not mean I haven’t done anything interesting :). It is just that it was so much interesting that I didn’t have any time to write anything.

Anyway, this time I have stumbled (again) a unicode problem using some Python code which was supposed to be perfectly suitable for doing this since it even started with


#!/usr/bin/env python
# -*- coding: UTF-8 -*-

It went perfectly fine when running in Eclipse, but to my huge surprise I’ve got problems when running the unit tests from the command line in terminal. Whaaat? It just worked!

Well, declaring your source as UTF-8 is not enough of course. There are several things to check when getting the “UnicodeDecodeError: ‘ascii’ codec can’t decode byte … in position …: ordinal not in range(128)”-kind of errors. Googling around didn’t bring me much luck to my surprise, so there is are my findings for the next time :).

First of all make absolutely sure you haven’t forgotten the ‘u’ character before your strings containing the unicode strings. Yep, just like that you screw up the rest of the unicode support. Python (ok, I admit, I use 2.5.4) treats a ‘string’ as a regular string and not as a unicode. So write u’string’ instead!

Second, when doing things file operations don’t forget that you don’t get the unicode by default. Consider the following:


message = u'unicode message'
file_handle.write(message)

Well, guess what. You get a problem when writing the string away. It cannot be recognized. So the solution would be to do something like this


encoded_message = message.encode(u"utf-8")
file_handle.write(encoded_message)

But that’s only a half of the problem. At some point you will be reading this data back. And most probably you would like to get your beloved unicode thingy back. Just doing the following will hardly help:


file_handle = open(full_name, 'r')
line = file_handle.readline()

The following will save your day:


file_handle = open(full_name, 'r')
line = file_handle.readline().decode('utf-8')

Voila. I hope this saves some frustration to somebody, even if it will be me some month later :).

Enjoy!


Comments from Andy (thanks!):

Probably there is nothing new for you in what I am saying below, however, from my experience, it covers 99% unicode-related errors.

Unicode string is s sequence of code points in range 0 to 0x10ffff. Encoding is a way of serializing this sequence, so thay can be represented in memory, written to a file, sent over a socket etc.
Encoding unicode string is needed _at least_ because its ‘as is’ byte representation is not portable due to byte order issues.

It is _recommended_ that you work with unicode string internally provided the language/API supports unicode
It is _must_ that you encode the string to be consumed by another program.

XPlanner setup

I like XPlanner, but I am also very suspicious on beta’s being frozen for years (ok, GMail being a nice exception :)). Anyway, today I was happily changing XPlanner configuration to get around a known bug in XPlanner, which fortunately didn’t cost too much time to find solution for. XPlanner on Tomcat 5.5 + Java-6 on Ubuntu 8.04 apparently result in an exception during startup. The following explanation (thanks Alex!) gives a quick fix.

  1. in /xplanner/WEB-INF/classes/sbring-beans.xml
  2. for Find the bean id="metaRepository"
  3. replace
    <map>......</map>
    with

    <property name="repositories">
    <bean class="java.util.HashMap">
    <constructor-arg>
    <map>.....</map>
    </constructor-arg>
    </bean>
    </property>

Arrgggh, I feel I am hacking too much just to get get things working while I already expect them to work out of a box. Is this the way all admins feel?..

Guest WinXP VM host share access

I use VmWare Server 1.0.6 for testing purposes (great thing because of snapshots and it is free!), but for a while I could only connect to the guest VM from my host. Somehow my guest Windows XP refused to connect to the host machine shares. After all the following worked:

  • Create a regular user on the host computer.
    e.g. go to Control Panel | User Accounts | Advanced (tab), click on the Advanced button
    And select Users | Right mouse click | New User
    enter new user name and password.
    Make sure the “User must change password at next logon” checkbox is not checked; then check then “Password never expires” checkbox.
  • add a new share to the disk/folder you would like to share
  • give the new user rights to use share (remove Anybody from the list for security reasons)
  • connect from your guest machine using
    net use * \\ /user:<host_name>\<new_user>

    you will be prompted for a password.

Voila!

MySQL TIMESTAMP

Original date: 2008-12-27

Apparently using the TIMESTAMP fields in the MySQL the MySQLdb library in python results in the datetime.datetime() objects. A valid value can be set by providing a valid datetime.datetime instance.

Giving up on PHP

Quick and dirty prototyping in python instead of PHP (+MySQL).

Original date: 2008-12-22

I have spent a lot of time trying to figure out how to get my time registration app (php + mysql) up and running on my old FreeBSD laptop. After several unsuccessful attempts, abscence of properly installed mod_php and perhaps a lot of other things gave up. I can quickly do the minimal thingy using python!

I have added .py to the cgi script handlers:

AddHandler cgi-script .py

Made a simple script to connect to the MySQL database, but it failed miserably. No output was generated. After several attempts it seems that the import statement fails. Ok, put a try/except clause around and output the result. Hmmm, the result is rather interesting:

(Can’t extract file(s) to egg cache The following error occurred while trying to extract file(s) to the Python egg cache: [Errno 13] Permission denied: ‘/nonexistent’ The Python egg cache directory is currently set to: /nonexistent/.python-eggs Perhaps your account does not have write access to this directory? You can change the cache directory by setting the PYTHON_EGG_CACHE environment variable to point to an accessible directory. )

Ok, it seems that my www user (which does not have any home folder of course!) attempts to create cache folder under his (non-existing) home folder. Adding (perhaps not very secure)

SetEnv PYTHON_EGG_CACHE /tmp/apache/

to the virtual host configuration helped for now (you need mod_env for this). The cache can be created.
Ideally you would need a proper home directory with a restricted access for the apache user account.

If the server comes up with a very descriptive message like

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.
[...]

This may simply means that your beauty (script) contains just a small typo. Very handy indeed. The following two lines help a lot:


import cgitb
cgitb.enable()