Joomla! upgrade

Unbelievable that such a simple thing as an upgrade to a newer version of CMS may cause any headache these times. Recently I have received a warning message from my host (lunarpages.com) that I still use an old version (1.7.2) of Joomla! for my company site, while a newer version is available. I have tried to follow official upgrade procedures for about an hour without success, resulting in wrong layouts and half-working admin functionality, and finally ended up just downloading the complete package for Joomla_3.1.5-Stable-Full_Package.zip and simply unzipping it to the current installation folder. That did the trick (well, almost) leaving me with the similarly popular warning message

Joomla upgrade warning Error
Your host needs to disable magic_quotes_gpc to run this version of Joomla!

After some googling around the following solution worked. I have created two files as shown below in the folder with Joomla! installation (so not affecting other websites) and that did the trick.


[php.ini]
magic_quotes_gpc = Off
session.save_path = "/home/<user>/tmp"


[.htaccess]
/home/<user>/www/<site>/php.ini

Enjoy!

Configuring Django under wsgi

A small reminder for settings up Django under wsgi. Apparently the following snippet helps getting the path correct avoiding the dreadful


self.load_middleware()
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/core/handlers/base.py", line 39, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/utils/functional.py", line 184, in inner
self._setup()
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/conf/__init__.py", line 95, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'xxx.settings' (Is it on sys.path?): No module named xxx.settings

messages from Apache (well, instead of ‘xxx’ your project name will be mentioned of course).

The fix
import sys

# Correct path.
app_path = os.path.abspath(os.path.join(os.path.dirname( __file__ ), '..'))
if app_path not in sys.path:
sys.path.append(app_path)

For completeness, the sites-enables/xxx looks like

WSGIPythonPath /var/www/path/to/project

<VirtualHost *:80>
DocumentRoot /var/www/path/to/project
ServerAdmin admin@example.com
ServerName my.example.com

Alias /static/ /var/www/path/to/project/

Order deny,allow
Allow from all

WSGIDaemonProcess project_name
WSGIScriptAlias / /var/www/path/to/project/wsgi.py

</VirtualHost>

Happy Djangoing!

UPDATE: when multiple Django applications are configured under Apache apparently a request could be routed to the Apache instance hosting wrong application. The server will not load application and Apache log files would show:


Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/core/handlers/wsgi.py", line 219, in __call__
self.load_middleware()
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/core/handlers/base.py", line 39, in load_middleware
for middleware_path in settings.MIDDLEWARE_CLASSES:
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/utils/functional.py", line 184, in inner
self._setup()
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/conf/__init__.py", line 95, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'xxx.settings' (Is it on sys.path?): No module named xxx.settings

basically failing to load settings from the wrong application. More on this topic in this blog post, but the fix seem to be specifying WSGIDaemonProcess per application.

Setting up Dango i18n, i10n

Anything related to i18n/i10n subject seem to be somehow quirkier than it looks at first glance. Python (2.x) itself and handling of unicode is a story apart.

This time I was looking into building a small website that has to provide UI in different languages. As this is one of the things you want to have right away I’ve started experimenting with adding i18n and i10n support.

First step is easy, the settings.py already had proper settings. Then for .py files it is rather straightforward to add e.g. (for forms):


from django.utils.translation import ugettext_lazy as _
...
city_name = forms.CharField( required = False, label = _('City:'))

For the .html files something like


{% load i18n %}

{% trans "Hello there!" %}

Then create under project folder folder ‘conf/locale’ (if you don’t do this it will complain), and then run

django-admin.py makemessages -l ru

Edit the resulting django.po file, add translations to your messages.

Warning: don’t forget to edit the following field, which comes EMPTY first, even while you have given it a parameter! Otherwise this file will be not used properly.

"Language: ru\n"

Then compile your nice and shiny translations:

django-admin.py compilemessages

Now we get all messages available. At least they should. But there is another trick missed in the Django documentation/tutorials: you HAVE TO specify the location of the message files explicitly in your settings.py otherwise your texts will continue coming up in default (en) language no matter how hard you try. E.g.:

LOCALE_PATHS = (
os.path.join(os.path.dirname(__file__), 'conf', 'locale').replace('\\','/'),
)

Well, after all this it seems to work. But it costs quite some searching and poking around to come to this. I can imagine after several rounds this becomes obvious, but you don’t get any errors, warnings, whatsoever, it just does not what you want it to do. Well, I hope it will do it for you now :).

Happy Djangoing!

(Re-)configuring Apache SSL on Ubuntu server

Since my old server died there were many things waiting to happen. First of all the main ‘carrier’ for my project information was located on a webserver obviously hosted on the dead server. For security reasons it only serves HTTPS, so, guess what, I needed to setup it up again.
The very first thing I found that my SSL certificates were no longer valid as the key used to generate them was happily buried on (both) dead hard disks. Arrrgghhh. OK, luckily it takes only few mouse clicks on GoDaddy site to re-generate the certificates. You perform

openssl req -new -newkey rsa:2048 -nodes -keyout yourdomain.key -out yourdomain.csr

on the server where the domain is to be hosted (in my case my smallish-greenish server) and copy/paste the content of the .csr file to the GoDaddy’s interface. Viola! You can download your certificates now and put them under e.g. the corresponding VirtualHost:


SSLEngine on
SSLCertificateFile /etc/apache2/certificates/watchitforme/watchitfor.me.crt
SSLCertificateKeyFile /etc/apache2/certificates/watchitforme/watchitfor.me.key
SSLCertificateChainFile /etc/apache2/certificates/watchitforme/gd_bundle.crt
ServerSignature Off

Good. Restart Apache using e.g.

sudo /etc/init.d/apache2 restart

…browse to your https://domain.tld and WTF-WTF-WTF…

Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.

Watta… well, this is something new apparently, but you are also required to edit the Apache ports.conf file, and add

Listen 443
NameVirtualHost *:443

and then in your virtual host setup you need to say instead of

<VirtualHost domain.tld:443>

put

<VirtualHost *:443>

Restart Apache again and you get your pages nicely served through HTTPS. C’mon, why so much trouble each time?..

p.s. There are a lot of posts about Chrome, IE, etc settings, but check the root cause first!

FreeBSD vs Ubuntu server

Some frustration from installing FreeBSD on a Foxconn A3550 with a Realtek 8168B/8111 card and Ubuntu saving my day (again).

After a pretty sad death of both harddisks in RAID1 setup on my old server that was sucking a lot of electricity and making pretty annoying level of noise for several years (don’t ask when the last backup was performed) I was obviously up to another setup. This time I decided to go for a ‘greenish’ setup without unnecessary redundancy and splitting the concerns of large storage and running all kinds of things like Apache, MySQL, etc for private projects data.

After some small research I settled down on Foxconn nT-A3550 E-350. Dropped some 4Gb of memory and 64 Gb SDD gave me a tiny silent box that can be used to exercise all kinds of things. There was only one small thing left… Installing an OS.
Somehow I had a FreeBSD as the first candidate when thinking of a server. Previous server had a rather old Ubuntu server version, but this time I was up to something new. Except… well, one by one.

My first idea was getting recent version of FreeBSD, which is at the time of writing was 9.0. Few minutes of download, dd’ing a spare USB stick and we’re done. Installation went smooth, although I’ve only realized later that it was due to the fact that 9.0 recognized the Intel wireless card in the little Foxconn and downloaded everything using wireless connection. The problem started when I’ve tried to setup wired connection (preferred for stability, as wireless may drop once in a while). Well, there was no luck for two sleepless nights and a lot of digesting of forums. I’ve tried many things and narrowed the problem down to the driver of the Realtek 8168B/8111 under FreeBSD. While it was recognized there was no luck in getting either DHCP acknowledgement or getting connection using static IP configuration. Symptoms: card fires up, link is detected, DCHP client tries to get IP, but does not “see” the DHCP offer from the router. Checking the logs of the router show that the offer is sent, but apparently never received/understood by the client. Apparently Realtek did release a driver version for FreeBSD 8.0 and some people claim that there was better support after 9.0 release this didn’t really help. I’ve tried the Realtek version, but it didn’t compile under 9.0 due to cleanup of the internal software interface. The version in 9.0 looked like a more advanced version apparently it didn’t work with my card. Bad luck…

Next trying FreeBSD 8.3 The idea was that the drivers are further developed since the 9.0 release. Well, I haven’t even got to the drivers. The Intel wireless card was not recognized and the wired Realtek still didn’t work with the same symptoms. That was it for FreeBSD.

And then Ubuntu (12.01 LTS server to be specific). Creating a bootable USB stick involved just another conversion step (from .iso to .img) as nicely described on the Ubuntu site and off we go. Guess what, the installation went smooth and all hardware detected at once. About 30 minutes later I had a fully functioning system with the software installed. So after all I am back to Ubuntu again.

Second though on FreeBSD. I may give it a chance later on when 9.1 comes out. Having manually built driver that relies on a (apparently deprecated) interface just asks for a trouble with the next update (which may be needed due to e.g. security reasons).

Look up a value in Excel

Unbelievable, but once in a while Excel can spoil your day when you “just want to have a quick analysis” of your data. Having all similarly looking but totally different FIND, SEARCH, LOOKUP, etc methods just does not help. So how can you find whether a value exists in a selection of cells?

Unbelievable, but once in a while Excel can spoil your day when you "just want to have a quick analysis" of your data. Having all similarly looking but totally different FIND, SEARCH, LOOKUP, etc methods just does not help. So how can you find whether a value exists in a selection of cells?

Lyrics apart, once again I was building up a cross-reference trying to find mentioning of a value in a selection of cells. After all it appeared to be pretty simple. If you know what you’re looking for:

Excel search text in cells

=MATCH(A1,$B$10:$B$100,0)

This will give you "#N/A" when no value of A1 is found in the range of B10:B100 or a number when a value is found. If you want it to be a bit more ‘boolean’ you can use the following formula

=ISNUMBER(MATCH(A1,$B$10:$B$100,0))

so that the result is TRUE or FALSE. Add some conditional formatting with red/green and it will give you a good start for something comprehensible.

Starting a company in The Netherlands

Since I have stopped with my “regular job” and started freelancing (a term which does not exist from official point of view of e.g. Tax office) I am regularly approached with questions regarding starting a business in The Netherlands. While there is a lot of information available, it takes time to compile and most of it is (of course) in Dutch. As many people that are interested in this information are not necessarily speak Dutch fluently hereby a small compilation of what I have discovered so far and insert corresponding terms in Dutch here and there for further reference.

Disclaimer

This post is in no way representing the official information. Please consult the corresponding websites of Dutch Chamber of Commerce (KvK) and Tax Office (Belastingdienst) for the complete and actual information. The information below represents solely my personal experience and opinion. I am not responsible for any consequences of following my advices or tips. I am only sharing my own thoughts for the ones interested in the subject. Feel free to contact me for more details or if you are interested in working together with my company.

Introduction

Since I have stopped with my “regular job” and started freelancing (a term which does not exist from official point of view of e.g. Tax Office) I am regularly approached with questions regarding starting a business in The Netherlands. While there is a lot of information available, it takes time to compile and most of it is (of course) in Dutch. As many people that are interested in this information are not necessarily speak Dutch fluently hereby a small compilation of what I have discovered so far and insert corresponding terms in Dutch here and there for further reference. Note that I am describing the situation from the point of view of an IT freelancer, e.g. someone who is doing programming work for (somewhat larger) clients, although many things do not depend on particular area of application. This post actually "grew out" from few notes I was sending to several people answering their (general) questions, I hope this will be also useful for other people as well.

So what does it take to register a company (in NL)?

Well, it depends on a type of business you want to register. For most the Self-employed without staff (Dtuch "ZZP = zelfstandige zonder personeel") is the easiest way to start with. It is easy to setup and to deal with the (unfortunately) corresponding overhead, but it also has disadvantages like liability with your own possessions in case something bad happens. As long as you are not taking (too much) responsibility it is OK. Next step is a BV, which is made somewhat simpler to register from October 2012 by the so-called "Flex-BV" which among other things takes away the previsly needed 18.000 eur initial capital, but that one is mainly makes sense when you are taking somewhat more risk and therefore want to offset it. I’ve decided to start easy with ZZP and then have a good look on whether a BV is a better option on a longer term.

Actual registration of a self-employed is very easy. You make an appointment in one of the local offices of KvK after about 15 minutes (electronic) paperwork and small fee you get your registration number and a small bottle of sparkling wine for a good start :). Is that it? Well, almost. Before registering you have to think of a name and what are you exactly going to do, etc. This all is available in the corresponding registration forms.

Tip: search Google for "kvk inschrijvingsformulier" and fill in the form before you go to KvK. Registering a domain and getting proper e-mail address instead of your private one may be a good idea as well, but it is absolutely not necessary and completely up to you.

Beware that as soon as you do register you are subject to pay annual fees to KvK and providing information on your turnover and VAT payments to Tax office (even if you have a zero balance!). This is small overhead, but does require some time.

Tip: you can get a lot of useful information by signing in to a regular KvK starters day. Google

Finding clients and assignments (opdrachten)

Here are the companies I was contacting for freelance projects. Note that your situation may depend totally on what you’re going to do with your business, the type of assignments you’re looking for, type of clients, etc. I have decided to start with on an easy route and to provide service to the same type of clients I used to work for (multinationals working in large high-tech machines control). So the following companies may help you with finding clients.

On most websites you can upload your CV and get inside their database (for some you need one a Word format… yuk!). The ones marked with a start I have worked with. More information is available on request.

I have also registered with the following ones, but didn’t have direct contact with the following ones:

Note You may also ask why do you need an intermediair for this? Well, from experiences of people that have tried with and without them seems that going through an agency simplifies life especially (but not only) when dealing with big companies which tend to pay somewhere around 80-90 days after you invoice them and sometimes forget, mess up administration, etc. I am not in shape of fighting financial departments of large companies, I prefer having people doing it for me and getting my payments in time (normally around 30 days from the date of issuing the invoice). This does cost about 10-25% on top of your hourly rate (depending on agency), but I find around 10% a fair amount for the things they take care of. Again, totally up to you.

Insurances

You need to have several insurances, the ones you definitely need to take care of are the following:

  • "Rechtbijstand" (legal assistance). Note that there are 4 modules, all 4 are needed, as the last one includes e.g. going after companies that refuse to pay you, etc.
  • "Wettelijke aansprakelijkheidverzekeringbedrijven" (AVB). Basic liability required by law. Some intermediairs (like Yacht I believe) requires this one to work with you.
  • "Arbeidsongeschikheid" Income insurance in case of inability to work. Very handy in case you have a family depending on your income… Note that above certain amount (as of 2012 around 53 kEur) you need an explicit medical check and a business plan confirming you have a solid basis for your insured amount, but it is no problem if you stay below the threshold amount.
  • Optionally (but still required by some companies) you may get so-called "beroepsaansprakelijkheidverzekering" (professional liability), but beware that this is a costly one (next ot the "arbeidsongeschikheid"), you only want this one if it is required or you need a coverage in case you are responsible for project results as opposite of only being paid for hours, which is true most of the time with larger clients. All of the companies mentioned above ensure you are only "selling" your hours but not the result of your work, so you are only responsible for spending your claimed time at the proper things (and not watching movies instead :)).

Financial administration

It is advisable (although not a must) to go for a financial advisors. The price will be around 100 eur per month (tax deductible), but those guys can potentially save you more than that! They also will keep up with the changing tax and law landscape. So unless you’re willing to spend your precious time on non-technical stuff look up an accountant nearby.

Banking

It is advisable (but not necessary) to get a business bank account. It makes life of accountants but also yourself if the business and private life are split up properly. Note that a business account does not mean just another account, but there are also additional rules. E.g. you have to pay for each transaction a small fee and an additional yearly fee. Normally the first year comes with a discount and all transactions are for free, but after a year everything becomes "serious". I have conducted some research and while some banks seemed cheaper at the first glance the other come as cheapest in total package over several years. In any case it is worth looking at the current offers and get a feeling of the service, etc. The very first reply I’ve gotten from one (cheapest on my list) bank put a fat cross on it’s name as it stated something like "we are very busy, please get back in a month"! What a … service!

Background

After several years of prolonging and "investigatin the issue" I took a step "to the other side" and registered my own company Profigent. While taking some risk I was sure this was the only way forward and if I do not do it now I will most likely have to forget the idea and accept life as-is. No way…

Other interesting tips

Recently I have attendend a seminar organized by IT-Staffing with an interesting guy
Robby Coelho. Although not everything was new for me it was worth the time and I did note down few points of attention for myself. Keep an eye on the upcoming seminars, perhaps something for you as well.

References

  • Pitfalls for enterpreneurs (original title in the brochure mentioned 10 of them, but the KvK website now has only 5, I will list them before they reduce it to 1 :)).
    • Agreements not on paper
    • Administration is not OK
    • No insurances
    • Let non-payers go on
    • Not putting money for taxes aside

A catch with the clean_xxx methods in Django forms

As I am getting more into Django stuff I also seem to step into rookie mistakes. Hereby some (beginner) notes.

As I am getting more into Django stuff I also seem to step into rookie mistakes. Hereby some (beginner) notes.

For validating of a particular field in Django a form class method can be defined, e.g.

from django import forms

class MyForm(forms.Form):
    first_field = forms.CharField() 
    second_field = forms.CharField()

def clean_first_field(self):
    first_field = self.cleaned_data['first_field']

Well, so far so good. But there is a catch which cost me another sleepless hour last night. When the method for field1 is called the ‘self.cleaned_data’ dictionary does not contain the ‘field2’ yet!!

Try the following:

def clean_field1(self):
    first_field = self.cleaned_data['first_field']
    second_field = self.cleaned_data['second_field'] # This will result in a KeyError exception!

Apparently Django will call the clean_xxx() methods in the same sequence as they are defined and will only supply the values UP TO AND INCLUDING the field being ‘cleaned’, but NOT the fields defined AFTER it. Watta…

This means if you want to check e.g. field1 and field2 between each other (yes, there are other ways of doing that as well, but when you’re only so far through the Django book you don’t know about them :)), then you have to check for the LAST field you want to check against to get values of this and PREVIOUSLY defined fields. So the code becomes:

def clean_first_field(self):
    first_field = self.cleaned_data['first_field'] # This is OK, field1 is defined before field2
    second_field = self.cleaned_data['second_field'] # As this is called for field2 you can get its value as well.

Of course you should have more error-checking logic, the examples above are simplified for illustrative reasons.

Note that if you use the higher-level clean() method it will be called when all values are filled in.

Missing article in Joomla! Article Manager

“Suddenly” missing an article in Article Manager in Joomla!? Try selecting “All” in the “Status selection”. Apparently by default it will filter out e.g. archived articles. I have accidentally clicked on the “Archive” button having one article selected and then spend good 15 minutes looking for that article.