Installing HTTPS/SSL certificate on Nginx

OK, this writing is specific to Godaddy certificate since that is what I’ve been doing recently – but the steps below can be applied to any Certificate Provider

You need to have request CSR signature of the Domain. This is basically to generate a private key that is unique to you (your domain). If you already have one then reuse that! if you don’t have any request CRS – create one CSR By Using this link https://www.digicert.com/easy-csr/openssl.htm

The link basically help you in generating terminal command similar to the one below (WARNING: do not use the command below 😛 in case you didn’t notice on the CN= part, it’s for my domain. Use the link I told you above)

openssl req -new -newkey rsa:2048 -sha256 -nodes -out star_mydomain_co_id.csr -keyout star_mydomain_co_id.key -subj '/C=ID/ST=Jakarta Barat/L=Jakarta/O=My Organization/OU=Web Administration/CN=*.mydomain.co.id'

After executing the command line, you’ll get *.csr and *.key file. The .csr file is Certificate Signing Request – you will need to provide this file to your SSL provider later. The .key file is your private key that is unique to you – of course you MAY NOT spread these files.

Keep these files – we will need it for later. Now use the generated Certificate Signing Request / *.csr to register for SSL in Godaddy (or your provider), they will ask you to upload your certificate and then they will provide you a signed Certificate File .crt. Depend on the provider – they will also provide you with intermediate/bundle certificate.

For example GoDaddy give me 2 files:

the-signed-domain-certificate.crt
gd_bundle.csr

Deploy these files to your machine – In Nginx the provider certificate need to be combined. I combine it using

cat the-signed-domain-certificate.crt gd_bundle.crt > star.mydomain.intermediate.2014.01.01.crt

Is a good practive to append expiration date to the certificate name. Save these files to local path in your server. We usually use /opt/key/. Remember to change the permission of the files to 600 (root readable)

Now go to your nginx sites-available config and add

server {
    listen          443;
    server_name     mydomain.co.id;

    ssl on;
    ssl_certificate         /opt/key/star.mydomain.intermediate.2014.01.01.crt
    ssl_certificate_key  /opt/key/star_mydomain_co_id.key

Restart your nginx. And test out your HTTPS.

Update:
Since google is trying to kill legacy SSL signature that is using SHA1, here’s a good article that will help you to avoid “Connection not Trusted” on future Google Chrome versions

notprivateSo, if you use Godaddy – at first they will give you SHA1 intermediate certificate – but then you can Re-Key your certificate to have SHA2 signed one. Here’s Godaddy explanation about re-key

Bash Script (!#) Shebang Issue (Vagrant, Linux, Windows)

Now, just found this issue yesterday. This happen on Vagrant that have Linux guest machine and using Windows as host machine.

Our shebang script suddenly give us

: No such file or directory

In one of the developer machine. But it works normally on the others. So I try to take a look at it.

The error is happening on one of the machine that is using Windows as the host machine and it does not happening on Linux / Mac’s. So it must be Windows thing right?

Judging from the editor – I cannot see any differences from the shebang lines below.


#!/usr/bin/env php
#!/usr/bin/env node

But suddenly I remember that Windows use CRLF and Linux use LF as line endings.

Never heard of such case in shebang script, I try to prove it by creating a simple hello world that is using proper LF line ending and it works properly.

So yeah if you got : No such file or directory error on your shebang script and you use Vagrant and have multi-OS developers – check your line-ending 🙂

The cause:

  1. Using Vagrant
  2. Using Linux as Guest Machine
  3. Using Windows
  4. Have some shebang scripts
  5. Does not properly set .gitattributes to properly resolve line endings

The fix:

  1. Properly set .gitattributes to properly force line ending – we use text eol=lf (see Github Article about dealing with Line Endings)
  2. Temporary fix to use dos2unix tool to convert the file line-ending

Install Vagrant and free your Machine (more memory for Games!!)

Why is it necessary?

I am using Mac OS 10.7 for my development machine, and using was using Homebrew to install all Open Source Development needs. But there’s some drawback about installing those Open Source tools on Mac Os, those are:

  1. I install many backgrond-services to serve my development needs (nginx, mysql, pgsql, mongodb, redis). And in mac – to turn those off is not as easy as `sudo service nginx stop`
  2. Most Open Source Stack are developed for NIX*, so on some rare-case compiling source from scratch will need `some` manual hacks to be compiled on Mac environment
  3. On some rare case I broke Mac’s default PHP and Ruby (specially the basic gems that Mac use for their internal Server Tool on Mac Mini)
  4. Mac does not emulate my EC2 environment, and I want to match local and server environment as close as possible so I can do CI properly
  5. My team need whole day to setup a development machine to fit some project needs – and when they jump to another project – there’s many left-over. I need faster machine setup and clean environment
  6. Sometimes developers still need assistance in `setup` a perfect environment, not all developer familiar with #devops aspect 😀

And there’s Vagrant to the rescue, combined with VirtualBox and become an awesome Development tools. The Pros are:

  1. My development tools are contained inside a box, I can turn on, suspend and stop a box easily
  2. Each box contain one OS, so I can install Ubuntu, Centos, AppScale, Heroku Celadon Cedar or even Windows Server 2008 R2 – Datacentre full (go here for Vagrant compatible base boxes)
  3. Vagrant support Chef and Puppet (and even simple bash script) to init my Box. So it’s all automated.
  4. I can tell developers to just clone a project and then `vagrant up`within minutes they will have the complete development tools on their machine (no more googling for tutorials or resolving issues)
  5. I can suspend a vagrant box anytime and resume it later, it will free my memories for a quick LAN game session (That is why the guys my office have Steam IDs / LOL IDs)
  6. I can jump from Mac Machine, to Windows and Linux easily since vagrant is available for those major OS platforms

Getting Started

Of course install lastest Vagrant and VirtualBox (or if VmWare is your thing.. there’s a section for Vagrant to run using VmWare.. it’s not free though).

Go to your project folder and run

> vagrant init ~/public_html/experimentation

A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

It will create a Vagrantfile, basically Vagrantfile consists of configuration on how your machine-project will be handled by Vagrant.

Since most the server that I maintain are Ubuntu server, I’ll show you how to setup an Ubuntu box.

This is my standard VagrantFile

# environment
env = "local"

# Instance name prefix, so all site will be vhosted on "instanceprefix" + "appname"
instance = "vagrant.epm"

# default user:group used for www-data
user = "vagrant"
group = "vagrant"

# vm tweak
cpuexecutioncap = "100"
memorysize = "512"

Vagrant.configure("2") do |config|
 # the VM box and location (will download if not having any) - just add a box to your local
 config.vm.box = "precise64"
 config.vm.box_url = "http://files.vagrantup.com/precise64.box"

 config.vm.provider :virtualbox do |vb|
   vb.gui = false
   # cpu-execution
   vb.customize ["modifyvm", :id, "--cpuexecutioncap", cpuexecutioncap,"--memory",memorysize]
   # this is a fix for symlink in VirtualBoxFS
   vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"]
 end

# This is the direct IP so we can host-alias this machine
config.vm.network :private_network, ip: "10.10.11.1"

# port redirects
config.vm.network :forwarded_port, guest: 80 , host: 8080 # http - since hosting on port 80 will need ROOT
config.vm.network :forwarded_port, guest: 1000, host: 8010 # redis
config.vm.network :forwarded_port, guest: 3306, host: 8036 # mysql

# folder to share to vagrant machine
config.vm.synced_folder "./", "/var/www/appsindo-baseline"

# Chef Scripts
config.vm.provision :chef_solo do |chef|
  chef.cookbooks_path = "./provisioning/cookbooks"
  # nginx in vagrant use vagrant:vagrant user
  chef.json = {
    # core setup for file owners
    "instance" => instance,
    # base web user and group (change this to www-data on deployment/production)
    "www" => {
       "user" => user,
       "group" => group
    },

    # Webapp needed
    "webapp" => [
     {
      "server_name" => "#{instance}.business",
      "public_path" => "appsindo-baseline/business/public/",
      "log_path" => "appsindo-baseline/log/",
      "env" => env
     }
    ]
 }

 # the basic custom recipes
 chef.add_recipe("appsindo-ephemeral")
 chef.add_recipe("appsindo-ephemeral::nginx")
 chef.add_recipe("appsindo-ephemeral::node")
 chef.add_recipe("appsindo-ephemeral::redis")
 chef.add_recipe("appsindo-ephemeral::mysql")
 chef.add_recipe("appsindo-ephemeral::phpmyadmin")
 chef.add_recipe("appsindo-ephemeral::integration")
 chef.add_recipe("appsindo-ephemeral::finalize")
 end
end

The most important parts are

  # the VM box and location (will download if not having any) - just add a box to your local
  config.vm.box     = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"

I want to install Ubuntu precise64. So I change `config.vm.box` to `precise64`. and `config.vm.box_url` to `http://files.vagrantup.com/precise64.box`. Box is a machine definition that will be used as a base-template. By using the `box` concept I can create few box and serve it on my company server (e.g. “precise64_with_nginx_and_nodejs”, “windows_server_plain”, “precise64_plain”) so the team can use appropriate boxes depend on the projects.

I also add

  config.vm.provider :virtualbox do |vb|
      vb.gui          = false
      # cpu-execution
      vb.customize ["modifyvm", :id, "--cpuexecutioncap", cpuexecutioncap,"--memory",memorysize]
      # this is a fix for symlink in VirtualBoxFS
      vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"]
  end

The `vb.customize.modifyvm` is to tweak the Virtual Machine CPU and Memory Cap. And the `vb.customize.setextradata` is some tweak for Symbolic Link issue within VirtualBox (due to Virtual File System limitation)

Make sure the network to have internal-ip (you can set it up as NAT or Bridged network.. see this)

  # This is the direct IP so we can host-alias this machine
  config.vm.network :private_network, ip: "10.10.11.1"

I set some port redirects from the guest-machine to host-machine. Not using port 80 since it will need root access on my host machine. Besides having
`http://vagrant.host:8080/` is useful as a constant reminded that I am on the local-development machine.

  # port redirects
  config.vm.network :forwarded_port, guest: 80  , host: 8080  # http
  config.vm.network :forwarded_port, guest: 1000, host: 8010  # redis
  config.vm.network :forwarded_port, guest: 3000, host: 8030  # kueue
  config.vm.network :forwarded_port, guest: 3306, host: 8036  # mysql

Then the folder to be synchronized/shared from host to guest machine.

  # folder to share to vagrant machine
  config.vm.synced_folder "./", "/var/www/appsindo-baseline"

Basically means – all changes on the current folder will be applied to `/var/www/appsindo-baseline` folder on the guest machine.

And finally – a Chef instruction to prepare my development machine, it’s a Custom Chef recipes written in Ruby to prepare my machine by installing `nginx`, `nodejs`, `redis`, `mysql`, `phpmyadmin` and also preparing an nginx site to be accessed from outside.

  # Chef Scripts
  config.vm.provision :chef_solo do |chef|
      chef.cookbooks_path = "./provisioning/cookbooks"
      # nginx in vagrant use vagrant:vagrant user
      chef.json = {
         # core setup for file owners
         "instance" => instance,
         # base web user and group (change this to www-data on deployment/production)
         "www" => {
            "user"  => user,
            "group" => group
         },
         # webapp needed
         "webapp" => [
          {
             "server_name" => "#{instance}.business",
             "public_path" => "appsindo-baseline/business/public/",
             "log_path"    => "appsindo-baseline/log/",
             "env"         => env
          }
         ]
      }

      # the basic recipes for PHP development
      chef.add_recipe("appsindo-ephemeral")
      chef.add_recipe("appsindo-ephemeral::nginx")
      chef.add_recipe("appsindo-ephemeral::node")
      chef.add_recipe("appsindo-ephemeral::redis")
      chef.add_recipe("appsindo-ephemeral::mysql")
      chef.add_recipe("appsindo-ephemeral::phpmyadmin")
      chef.add_recipe("appsindo-ephemeral::integration")
      chef.add_recipe("appsindo-ephemeral::finalize")
  end

That is all. All I need to do is typing `vagrant up` and let Chef do the installation works. It will spend some times – but it’s only for the initial works.
The next time I do `vagrant up` all the installation won’t be repeated.

If it’s a simple project, You can also use simple `bash script` to provision your machine, see the section about Using Shell to Provision Your Machine

PHP FPM refuse to start after upgrading Ubuntu

This weird thing happen after I upgrade my Ubuntu Production server. Suddenly php-fpm refuse to start.

sudo service php5-fpm start

sudo service php5-fpm status

And I always get

* php5-fpm is not running

Checking on the error-log located at sudo tail /var/log/php5-fpm.log show nothing, the last log entry is from previous machine-reboot, so it indicates that php-fpm is not running at all.

So I check wether the standard port 9000 is being used by other service by using

netstat -tap | grep 9000

It give me nothing, so then I try to run php-fpm executable directly from console

php5-fpm

And it show some errors related to configuration file, apparently the upgrade process messing around with my custom config.

Fixing the config and voila! it goes back to work.

Node.js + Nginx

So this will explain about what I’ve been doing recently. Basically I have iOS mobile app that consume node.js based API (usual JSON REST stuff). And now the challenge is to deploy it
to production (and hope for it to be approved by Apple :))

So the node.js server itself is an Express based web-server application and configured to be persistent by using ‘forever’ forever will ensure our backend-server to restart itself after crash (not quite a lot, but somehow it happen)

I use this command to run forever using custom log location (in which will be logrotated)

forever start -a -o /opt/myserver/myserver.log -l /opt/myserver/myserver.log -e /opt/myserver/myserver.log app.js

Nginx will be the one who listen on port 80 and reverse-proxy-ing all outside request to our node.js web-server. This is the setup that I use

server {
  listen 80;

  server_name myserver.com;
  root /var/www/myserver;

  access_log "/var/www/myserver/access.log";
  error_log "/var/www/myserver/error.log" error;

  charset utf-8;

  default_type  application/octet-stream;
  sendfile        on;

  # would be awesome if your mobile-app can utilize keep-alives!
  keepalive_timeout  65;

  # enable gzip
  gzip on;
  gzip_comp_level 6;
  gzip_vary on;
  gzip_min_length  1000;
  gzip_proxied any;
  gzip_buffers 16 8k; 

  # we only gzip these mime-types (since there's no use to gzip jpegs)
  gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

  # tell-client to cache all 'assets'
  location ~* \.(?:jpg|jpeg|gif|png|ico|gz|svg|svgz|mp4|ogg|ogv|webm)$ {
    expires 1M;
    access_log off;
    add_header Cache-Control "public";
  }

  # disable logging for some `common` files
  # Disable logging for favicon
  location = /favicon.ico {
        log_not_found off;
        access_log off;
  }

  # Disable logging for robots.txt
  location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
  }

  # Prevent clients from accessing hidden files (starting with a dot)
  location ~* (^|/)\. {
    return 403;
  }

  # Prevent clients from accessing to backup/config/source files
  location ~* (\.(bak|config|sql|fla|psd|ini|log|sh|inc|swp|dist)|~)$ {
    return 403;
  }

  # reverse-proxy here, if your have multiple machine/cores would be better to use UPSTREAM so nginx can load-balance requests
  try_files $uri $uri/ @mynodeserver; 
  location @mynodeserver {
        proxy_redirect off;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_set_header   Host $http_host;
        proxy_set_header   X-NginX-Proxy true;
        proxy_set_header   Connection "";
        proxy_pass         http://127.0.0.1:8088;
  } 
}

That is all, easy peasy huh? 🙂

The nginx setup is a combination from awesome h5bp boilerplate

Yii Building Search Query Using CDBCommand

I am new in Yii framework, and I found it interesting, so today at work I found a problem to create conditional search query in Yii, of course I can create ordinary SQL text and execute it, but a framework as mature as Yii should have it’s own SQL query helper / generator helper, so I google and mingle with the Yii documentation and found CDBCommand. I avoid Yii ActiveRecord on this case due to performance limitation 🙂

This is a simple example to create query using CDBCommand

$sql = Yii::app()->db->createCommand();
$sql->select('id,name,phone');
$sql->from('{{mytable}}');
$sql->where('id=:ID AND name=:name', array(':ID' => $id, ':name' => $name);

But the thing is, Yii CDBCommand does not work like CodeIgniter’s SQL query helper, you cannot put multiple where to create complex query. Yii behave similar to SQL syntax, you can call multiple join but you can only call select and where once.

So if you an ex-CodeIgniter programmer, you cannot do this in Yii

$sql = Yii::app()->db->createCommand();
$sql->select('id,name,phone');
$sql->from('{{mytable}}');
$sql->where('id=:ID AND name=:name', array(':ID' => $id, ':name' => $name);

if ($phone) {
    $sql->where('phone=:phone', array(':phone' => $phone);
}

The workaround is to use array as where parameter, fill the array as you go and pass it to where once.

So this is an example where I want to filter my data based on optional criterias.

        $sql = Yii::app()->db->createCommand();

        // create the where filter
        $filter_where = array('AND');
        $filter_where_data = array();

        // construct a search/filter query
        if (isset($id)) {
            $filter_where[] = 'people.id=:ID';
            $filter_where_data[':ID'] = $id;
        }

        if (isset($name)) {
            $filter_where[] = 'people.name=:name';
            $filter_where_data[':name'] = $name;
        }

        // a join here.. you can create multiple joins by calling join method multiple times
        $sql->join('{{table_address}} addr', 'addr.addr_id = people.addr_id');

        // construct your query 
        $sql->select('people.id, people.name, addr.description', 'SQL_CALC_FOUND_ROWS');
        $sql->from('{{table_people}} people');
        $sql->limit($pagesize, $offset);

        // where will be generated based on passed filter_where and filter_where_data arrays
        $sql->where($filter_where, $filter_where_data);        

I prefer this method rather than constructing SQL text query by myself, it looks cleaner and in my opinion can construct a cleaner SQL since we use Yii parameter binding and cleanup.

Well, I hope you got the idea 🙂

Javascript check Opera Mini

So today I got caught on the weird-side of mobile web-development, that is to create consistent javascript on major mobile browsers. The good news is Android and IPhone are using webkit so it’s kind of easy. The Bad News is Opera Mini.

Don’t get me wrong, I love opera mini. But the different java-script engine behavior could cause headache sometimes. After googling for a while I stumbled on this http://dev.opera.com/articles/view/javascript-support-in-opera-mini-4/ that mention some things to consider:

  1. Opera Mini JS has a limited DOM event
  2. No background scripting
  3. Very limited AJAX support

And the most important thing is this snippet to check whether we are inside Opera Mini (of course you can use PHP or Server Side script as a better method)

is_operamini = Object.prototype.toString.call(window.operamini) === "[object OperaMini]";

So to avoid more headache, I just treat opera-mini as a non-javascript-capable browser *smirk*