Let’s see if it’s working

Terminologies

REST API centers around resources that are grouped into collections

• Services expose a set of collections and resources in order to provide a correlated functionalities.

  Services are customarily named with single nouns.

  Example: /auth service provides functionalities related to authentication, e.g. login, logout, refresh token.

• Resources are pieces of any kind of information, e.g. documents, audio/video files, xml/csv/json data, web pages…

  A resource is identified by a {resource_id}.

  Example: /users/1 is a user resource belonging to the collection users, identified by resource id 1.

• Collections typically are plural nouns. Collections give an identity to all the resources they contain.

  A collection can be a part of a resource.

  Example: /auth/tokens is a collection of tokens, belonging to the authentication service.

REST API URL structure

# Standalone collection
/<collection>

# Collection managed by a service
/<service>/<collection>

# Nested collection and resource
/<parent_collection>/<parent_id>/<child_collection>

# Accessing a resource
/<collection>/<resource_id>
/<service>/<collection>/<resource_id>
/<parent_collection>/<parent_id>/<child_collection>/<child_id>
<service>/<parent_collection>/<parent_id>/<child_collection>/<child_id>

Since every resource is identified with a {resource_id}, /<collection>/<resource_id> (#1) and /<parent_collection>/<parent_id>/<child_collection>/<child_id> (#2) are basically referring to the same resource. The notable difference is that #1 refers purely to the resource, while (#2) stresses more on the relationship between the resources.

Resource and collection relationship: composition vs. aggregation

Composition is a type of containment relationship, in which a child collection belongs to a parent resource, but not any others. To rephrase, the child resource is managed by the parent resource; it cannot exist independently of the parent resource, and thus, the destruction of the parent resource will lead to its own destruction.

For example:

• A house is composed of several rooms. The rooms cannot exist on their own.The demolishment of the house will cause the rooms cease to exist.

• A time table has many time entries. The time entries don’t have any meaning without belonging in a time table. If the time table is removed, the time entries must as well be deleted.

Aggregation is a type of containment relationship in which the the child resource can exist independently of a parent resource.

For example:

• A course has many students. However, the students are free to join multiple courses. If a course is removed from the school’s curriculum, the students are simply disenrolled from that course.

• A chatroom channel may have many users. A user can join multiple chatroom channels. If the channel admin suddenly decides to delete the channel, other channel users would simply be forced to leave the original channel.

• A user can has many friends, who are just other users. When a user delete his accounts, only his relationships with the other users are removed, not the users themselves.

Interacting with collections and resources

REST APIs use HTTP methods to interact with collections and resources. There are more HTTP methods, but the following five are the most used, and are often adequate for any REST API implementations.

GET

The HTTP GET method is used for retrieving a resource or collection from the REST API service.

When applying GET method onto a collection, it is expected get a list of the collection members.

GET is safe and idempotent. It means that GET requests do not have any side effects to the API server. The consequence is that calling the same request multiple times would produce the same result (unless the server’s state has been changed by something else)

When implementing GET APIs, it is recommended to provide filtering, sorting, fields selection and pagination for collections.

Examples:

• Filtering:

    # Get user’s device whose type is electronic
    GET /users/1/devices?type=electronic
  
    # Get user’s female friends
    GET /users/1/friends?gender=female
  
    # Show friend requests dated after Dec 31st, 2015
    GET /users/1/friend_requests?after=12-31-2015
  
    # List user posts between Jan 1st, 2016 and Dec 31st, 2017
    GET /users/1/posts?after=1-1-2016&before=12-31-2017
  

• Sorting:

    # Sort posts by ascending title and descending created_date
    GET /users/1/posts?sort=+title,-created_date
  

• Fields selection:

    # Get user friends, but only retrieve their id and username
    GET /users/1/friends?fields=id,username
  

• Pagination:

    # Get news articles from the first page, 10 articles per page
    GET /news?page_number=0&page_size=10
  

POST

POST method only performs on a collection. It creates a new resource in the collection.

POST is non-idempotent. Calling the same request twice, if succeeds, will create two resources in that same collection, each with a different resource id (unless the server returns some errors due to logical constraints, e.g. disallowing duplications in certain fields).

POST can as well be used to perform actions in the API server, though this is considered hacky and should be avoided if possible. However, in case you really need it, create a new action resource in the collection actions.

Example:

# Resend activation code to user
POST /auth/actions { action: "resend_activation_code", "user_id": 1 }

# Tell a car to stop
POST /car_manager/cars/1/actions { action: "stop" }

# Tell all cars to stop
POST /car_manager/actions { action: "stop all" }

PUT

PUT methods only performs on a resource. PUT request replaces the resource with new data.

It is recommended to include all fields when using PUT, else non-included fields would replace the resource corresponding fields with empty data. To update a resource, use PATCH.

PUT is not safe, since it changes the state of the API server by updating the resource with new data. It is idempotent, though; calling the same request multiple times would create the same result.

PATCH

PATCH methods only performs on a resource. PATCH is similar to put, except that it updates instead of replacing the resource. Non-included fields will not update the resource’s corresponding fields.

DELETE

DELETE performs on both resources and collections:

• If performs on a resource, it will delete that resource.
• If performs on a collection, it will empty that collection.

DELETE is non-idempotent. Calling the same request the second time should returns a 404 error if the first call was successful.

Notes: When designing DELETE APIs, we must pay attention to what we really want to delete.

• If we want to delete a resource, use direct access URL structure

    # A user is a standalone entity, thus we can delete it directly
    DELETE /users/1
  

• If we want to dissociate the resource with its parent (especially in aggregation relationship), use the nested collection URL structure.

    # A friend is not a user. 
    # It is a relationship between a user and another user. 
    # Delete a friend doesn’t really make any sense, 
    # but rather dissociate the two users. 
    # Therefore, we apply the nested collection and resource URL structure.
    DELETE /users/1/friends/1
  

Receiving responses

HTTP elements

The JSON (JavaScript Object Notation) format is a popular way to represent resources in order to communicate between the API server and the client. There are two types of JSON object:

• JSON Object
• JSON Array

However, though it’s possible, not all information should be included in the json object. REST API makes use of HTTP message elements to attach additional metadata:

• REST API uses HTTP status codes to indicate if a request is successful or errorneous; and give additional information about the result if applicable.
• Sometimes, HTTP message header fields are utilized to communicate metadata not directly related to the information being exchanged, e.g: authentication information, response format, expected content language, content language…

The main HTTP status codes used are generally of class 2xx and 4xx. The following are the most frequently used ones.

2xx: success

• 200 OK: The standard response code for a successful request. A response body is expected.
• 201 Created: The request was successful, and a resource was created

• 202 Accepted: The request was accepted and being processed. However the result of the process would be unknown. This status code is used when we do not immediately care about the result of the process

  For example:

  # Asking the server to shutdown
  POST /system/actions { action: shutdown }
  

• 204 No Content: The server successfully processed the request, but is not returning any content. This status code is used when we only care if the request is successful or not.

Examples of 2xx responses:

POST /auth/tokens
Host: https://api.theitfox.com
Accept-Language: en-US
Accept: application/json
Content-Type: application/x-www-form-urlencoded

grant_type=credentials&username=<username>&password=<password>
--------------------------------------------------------------
HTTP/1.1 200 OK
Date: Fri, 2 Feb 2018 17:26:00 PST
Server: Nginx
Content-Language: en-US
Content-Type: application/json; charset=utf-8

{
    "expiration": <unix timestamp>,
    "access_token": <access token>,
    "refresh_token": <refresh token>
}

POST /auth/tokens
Host: https://api.theitfox.com
Accept-Language: en-US
Accept: application/json
Content-Type: application/x-www-form-urlencoded

grant_type=refresh&token=<token>
--------------------------------------------------------------
HTTP/1.1 200 OK
Date: Fri, 2 Feb 2018 17:26:00 PST
Server: Nginx
Content-Language: en-US
Content-Type: application/json; charset=utf-8

{
    "expiration": <unix timestamp>,
    "access_token": <access token>,
    "refresh_token": <refresh token>
}

POST /users
Host: https://api.theitfox.com
Accept-Language: en-US
Accept: application/json
Authorization: Bearer <token>
Content-Type: application/x-www-form-urlencoded

username=<username>&password=<password>&email=<email>
--------------------------------------------------------------
HTTP/1.1 201 Created
Date: Fri, 2 Feb 2018 17:26:00 PST
Server: Nginx
Content-Language: en-US
Content-Type: application/json; charset=utf-8

{
    "id": <user id>,
    "username": <username>,
    "email": <email>,
}

4xx: client errors

• 400 Bad request: The server cannot process the request due to a client error (i.e malformed request syntax, size too large, general logical error…).
• 401 Unauthorized: Authentication failed, expired, or not yet provided. This is used when the server cannot identify the user.
• 403 Forbidden: The server is refusing to serve the user. This is generally used when the server knows who the user is, but the user does not have the required permission to access the resource, or due to some other logical constraints.
• 404 Not found: The resource does not exist
• 405 Method not allowed: The HTTP method used in the request was not supported for this resource. For example, you can only create (POST), access (GET) or delete (DELETE) a friend relationship, not replacing (PUT) or updating (PATCH) it.

Examples of 4xx responses:

GET /users/@me
Host: https://api.theitfox.com
Accept-Language: en-US
Accept: application/json
Authorization: Bearer <token>
Content-Type: application/x-www-form-urlencoded
--------------------------------------------------------------
HTTP/1.1 401 Unauthorized
Date: Fri, 2 Feb 2018 17:26:00 PST
Server: Nginx
Content-Language: en-US
Content-Type: application/json; charset=utf-8

{
    "code": "UNAUTHENTICATED",
    "message": "Session expired"
}

GET /cars/2018
Host: https://api.theitfox.com
Accept-Language: en-US
Accept: application/json
Authorization: Bearer <token>
Content-Type: application/x-www-form-urlencoded
--------------------------------------------------------------
HTTP/1.1 404 Not found
Date: Fri, 2 Feb 2018 17:26:00 PST
Server: Nginx
Content-Language: en-US
Content-Type: application/json; charset=utf-8

{
    "code": "NOT FOUND",
    "message": "Resource not found"
}

JSON response format convention

There are several different JSON formats to represent the response data

The naive style

In the naive style, JSON object and JSON array are utilized directly.

• Requesting a single resource

    GET /tracks/1
    Host: https://api.theitfox.com
    Accept-Language: en-US
    Accept: application/json
    Content-Type: application/x-www-form-urlencoded
    --------------------------------------------------------------
    HTTP/1.1 200 OK
    Date: Fri, 2 Feb 2018 17:26:00 PST
    Server: Nginx
    Content-Language: en-US
    Content-Type: application/json; charset=utf-8
  
    {
        "id": 1,
        "title": "Happy new year",
        "artist": "ABBA"
    }
  

• Requesting a collection

    GET /tracks
    Host: https://api.theitfox.com
    Accept-Language: en-US
    Accept: application/json
    Content-Type: application/x-www-form-urlencoded
    --------------------------------------------------------------
    HTTP/1.1 200 OK
    Date: Fri, 2 Feb 2018 17:26:00 PST
    Server: Nginx
    Content-Language: en-US
    Content-Type: application/json; charset=utf-8
  
    [
        {
            "id": 1,
            "title": "Dancing Queen",
            "artist": "ABBA"
        },
        {
            "id": 2,
            "title": "SOS",
            "artist": "ABBA"
        },
        {
            "id": 3,
            "title": "Mamma Mia",
            "artist": "ABBA"
        }
    ]
  

The advantages of this approach are:

• The JSON is small, reducing the bandwidth
• The approach is straight forward, and easy to understand

The disadvantage of this approach is that this doesn’t contain metadata, including pagination for collection resources. To overcome this problems, some developers include these metadata as response header fields, for example:

GET /tracks
Host: https://api.theitfox.com
Accept-Language: en-US
Accept: application/json
Content-Type: application/x-www-form-urlencoded

page_number=1&page_size=20
--------------------------------------------------------------
HTTP/1.1 200 OK
Date: Fri, 2 Feb 2018 17:26:00 PST
Server: Nginx
Content-Language: en-US
Content-Type: application/json; charset=utf-8
X-Page-Number: 1
X-Page-Size: 10
X-Total-Size: 3

[
    {
        "id": 1,
        "title": "Dancing Queen",
        "artist": "ABBA"
    },
    {
        "id": 2,
        "title": "SOS",
        "artist": "ABBA"
    },
    {
        "id": 3,
        "title": "Mamma Mia",
        "artist": "ABBA"
    }
]

However, since RFC 6648, custom header fields are officially deprecated.

The enveloped style

The enveloped style solves the pagination problem with collection resources by wrapping the collection inside a named element.

GET /tracks
Host: https://api.theitfox.com
Accept-Language: en-US
Accept: application/json
Content-Type: application/x-www-form-urlencoded

page_number=1&page_size=20
--------------------------------------------------------------
HTTP/1.1 200 OK
Date: Fri, 2 Feb 2018 17:26:00 PST
Server: Nginx
Content-Language: en-US
Content-Type: application/json; charset=utf-8

{
    "records": [
        {
            "id": 1,
            "title": "Dancing Queen",
            "artist": "ABBA"
        },
        {
            "id": 2,
            "title": "SOS",
            "artist": "ABBA"
        },
        {
            "id": 3,
            "title": "Mamma Mia",
            "artist": "ABBA"
        }
    ],
    "_metadata": {
        "page_number": 1,
        "page_size": 20,
        "total_size": 3
    }
}

This approach is generally good for most REST applications.

The JSON API style

The JSON API uses the enveloped style for both single resources and collection resources format, since metadata can in some cases be useful to single resources as well. It is a specification created to standardize data communication with JSON. The full specification can be read at jsonapi.org.

Here’s an example of a JSON API response.

{  
    "links":{  
        "self":"http://api.theitfox.com/articles",
        "next":"http://api.theitfox.com/articles?page[offset]=2",
        "last":"http://api.theitfox.com/articles?page[offset]=10"
    },
    "data":[  
        {  
            "type":"articles",
            "id":"1",
            "attributes":{  
                "title":"JSON API paints my bikeshed!"
            },
            "relationships":{  
                "author":{  
                    "links":{  
                        "self":"http://api.theitfox.com/articles/1/relationships/author",
                        "related":"http://api.theitfox.com/articles/1/author"
                    },
                    "data":{  
                        "type":"people",
                        "id":"9"
                    }
                },
                "comments":{  
                    "links":{  
                        "self":"http://api.theitfox.com/articles/1/relationships/comments",
                        "related":"http://api.theitfox.com/articles/1/comments"
                    },
                    "data":[  
                        {  
                            "type":"comments",
                            "id":"5"
                        },
                        {  
                            "type":"comments",
                            "id":"12"
                        }
                    ]
                }
            },
            "links":{  
                "self":"http://api.theitfox.com/articles/1"
            }
        }
    ],
    "included":[  
        {  
            "type":"people",
            "id":"9",
            "attributes":{  
                "first-name":"Dan",
                "last-name":"Gebhardt",
                "twitter":"dgeb"
            },
            "links":{  
                "self":"http://api.theitfox.com/people/9"
            }
        },
        {  
            "type":"comments",
            "id":"5",
            "attributes":{  
                "body":"First!"
            },
            "relationships":{  
                "author":{  
                    "data":{  
                        "type":"people",
                        "id":"2"
                    }
                }
            },
            "links":{  
                "self":"http://api.theitfox.com/comments/5"
            }
        },
        {  
            "type":"comments",
            "id":"12",
            "attributes":{  
                "body":"I like XML better"
            },
            "relationships":{  
                "author":{  
                    "data":{  
                        "type":"people",
                        "id":"9"
                    }
                }
            },
            "links":{  
                "self":"http://api.theitfox.com/comments/12"
            }
        }
    ]
}

The problem

I happen to come accross this programming problem, which I find a bit challenging.

Tom is playing his favorite adventure game and got stuck in a puzzle. This puzzle requires him to press a​ sequence of buttons to unlock a door. However, the game does not tell Tom the button sequence directly but via a encoded string.

Tom knows that one unique character​ of the encoded string must correspond to ​one button​. However, he has no further clues on the mapping, i.e. which character in the string is which button. As a result, Tom has to try all the possible mappings to find the correct button sequence.

For example:

The sequence is AABBABAB, and there are 3 buttons, labelled 1, 2, 3

There are 2 uniques characters: A, and B. There are 3 buttons 1, 2, and 3. So, there are only 6 possible mappings:

• 11221212 # A = 1, B = 2
• 11331313 # A = 1, B = 3
• 22112121 # A = 2, B = 1
• 22332323 # A = 2, B = 3
• 33113131 # A = 3, B = 1
• 33223232 # A = 3, B = 2

At first, I thought “Ok, this is easy. I have to try all the cases. This is just brute-forcing, so I just need a few for loops”, but then I realized it wasn’t that simple.

The number of possible mappings is the k-permutation of the buttons, in which k is the number of unique characters in the sequence.

While it’s totally possible to implement the permutation algorithm without recursion, I find the recursion algorithm much easier to understand.

Algorithm

The problem was an example of a interview question for senior software engineer, in which the candidate is required to write the test in C/C++. However, I long have forsaken C/C++, thus I am going to write the algorithm this in Python.

Personally, I find Python much more friendly to algorithm learning than C/C++.

Partial permutation (k-permutation)

Applying a function to all arrrangements of k distinct elements selected from a list

def _partial_permute(prefix, suffix, k, func):
    if len(prefix) == k:
        func(prefix)
    else:
        for i, c in enumerate(suffix):
            _prefix = prefix + [c]
            _suffix = suffix[:i] + suffix[i + 1:]
            _partial_permute(_prefix, _suffix, k, func)

def _partial_permute(sequence, k, func):
    _partial_permute([], sequence, k, func)

Example usage

def func(permutation):
    print "".join(permutation)

partial_permute(list("ABCDE"), 3, func)

Full permutation

Applying a function to all permutions of elements in a list

def _permute(prefix, suffix, func):
    if not suffix:
        func(prefix)
    else:
        for i in xrange(len(suffix)):
            _prefix = prefix + [suffix[i]]
            _suffix = suffix[:i] + suffix[i + 1:]
            _permute(_prefix, _suffix, func)

def permute(sequence, func):
    _permute([], sequence, func)

Notes: The full permutation can as well be written as the k-permutation of the list, in which k is the length of the list.

def permute(sequence, func):
    partial_permute(sequence, len(sequence), func)

Example usage

def func(permutation):
    print "".join(permutation)

permute(list("ABCDE"), func)

The solution

def _partial_permute(prefix, suffix, k, func):
    if len(prefix) == k:
        func(prefix)
    else:
        for i, c in enumerate(suffix):
            _prefix = prefix + [c]
            _suffix = suffix[:i] + suffix[i + 1:]
            _partial_permute(_prefix, _suffix, k, func)

def partial_permute(sequence, k, func):
    _partial_permute([], sequence, k, func)

def print_possible_mapping(sequence, n):
    sequence = list(sequence)
    uniques = list(set(sequence))

    if n < len(uniques):
        print "There are more buttons than the number of unique characters in the sequence"
        return

    buttons = range(1, n + 1)

    def func(buttons_permutation):
        charmap = {}
        for i, c in enumerate(uniques):
            charmap[c] = buttons_permutation[i]
        _sequence = "".join([str(charmap[c]) for c in sequence])
        print _sequence

    partial_permute(buttons, len(uniques), func)

# Test
print "------ Test 1 ------"
print_possible_mapping("AABBABAB", 2)
print "------ Test 2 ------"
print_possible_mapping("AABBABAB", 3)
print "------ Test 3 ------"
print_possible_mapping("EFEBBBFF", 3)

I didn’t start following reactive programming on purpose. I started with using ReactiveX on Android as part of the clean architecture, the reactive approach, and eventually I had some ideas about reactive programming as a side effect.

A few days ago, an ex-colleague asked me to explain to him what Reactive Programming is, and thus I decided to write this blog post.

Reactive Programming

I find the definition on Wikipedia’s page about Reactive Programming a very nice one.

In computing, reactive programming is a programming paradigm oriented around data flows and the propagation of change.

I reckon this sentence is enough to convey the definition of reactive programming.

A data flow comprises of a source that generates data. Data in data flows can be anything, from user input, a click event, a keyboard input event, to chunks of bytes from a file stream, system notifications, or the response of a web service request.

When a change happens, the information regarding that change is generated, then flows from the source to a handler that handles that particular change; that is the propagation of change.

Some languages has reactivce programming built-in, while techniques must be used with the mainstream multi-paradigm languages.

Less words, let the codes explain:

The following piece of code is an example of reactive programming that calculates result = x + y written in HTML, javaScript, and jQuery. You can try the running example here

<p>x = <input name="x" value="0"/></p>
<p>y = <input name="y" value="0"/></p>
<p>x + y = <span id="result">0</span></p>
<script>
    jQuery(function($) {
        var application = {
            $x : $("input[name=x]"),
            $y : $("input[name=y]"),
            $result : $("span#result"),
            update : function() {
                // Get the int values of x and y.
                var x = parseInt(this.$x.val());
                var y = parseInt(this.$y.val());
                this.$result.html(x+y);
            }
        };
        // Propagate the input events to application.update() function
        application.$x.on("input", () => application.update());
        application.$y.on("input", () => application.update());
    });
</script>

Everytime the value of application.$x or application.$y changes, the value of application.$result is updated. We can say that $result is reactive to the changes of the inputs, or the input events propagate to the application.update() function.

JavaScript is event-driven, thus matches extremely well with reactive programming. However, the flow of data is not always obvious, and sometimes complex codes must be produced to achieved desired effects. This is the reason why libraries like ReactiveX are produced.

ReactiveX

What the fuss is ReactiveX?

ReactiveX stands for Reactive Extension. It is a library that provides tools to compose and manipulate streams of data or events using observable sequences. You can read more on ReactiveX’s website

How does ReactiveX aid reactive programming?

Let’s first rewrite our previous example in RxJs. You can try the running example here

<p>x = <input name="x" value="0"/></p>
<p>y = <input name="y" value="0"/></p>
<p>x + y = <span id="result">0</span></p>
<script>
    jQuery(function($) {
        var application = {
            $x : $("input[name=x]"),
            $y : $("input[name=y]"),
            $result : $("span#result"),
            update : function() {
                // Get the int values of x and y.
                var x = parseInt(this.$x.val());
                var y = parseInt(this.$y.val());
                this.$result.html(x+y);
            }
        };
        // We have two streams of input events
        var xInputObservable = application.$x.onAsObservable("input");
        var yInputObservable = application.$y.onAsObservable("input");

        // Merging the two streams of data into one single stream
        var inputObservable = xInputObservable.merge(yInputObservable);

        // Subscribe the stream to a handler (technically called a subscriber)
        var subscription = inputObservable.subscribe(() => application.update());
    });
</script>

The changes, though seem minor, actually make the code much cleaner, and more extensible. Let me demonstrate how it is.

In the original code, we propagated the input event to the handler using these lines.

application.$x.on("input", () => application.update());
application.$y.on("input", () => application.update());

It looks pretty darn fine. However, this is just a simple example. What if we want to do something else with a more complicated logic, e.g. print out the current time when an input event is fired for the first time for each input? Of course we can still do it like this.

var is_x_first_input = true;
var is_y_first_input = true;

application.$x.on("input", () => {
    if(is_x_first_input) {
        var now = new Date();
        console.log(now);
    }
    is_x_first_input = false;
}));
application.$y.on("input", () =>{
    if(is_y_first_input) {
        var now = new Date();
        console.log(now);
    }
    is_y_first_input = false;
});

Let’s solve the above problem with RxJs.

var xInputObservable = application.$x.onAsObservable("input")
    .map(e => new Date()) // Transform the input event into a Date object
    .take(1); // Only take the first object
var yInputObservable = application.$y.onAsObservable("input")
    .map(e => new Date()) // Transform the input event into a Date object
    .take(1); // Only take the first object

// Merge the two streams
var inputObservable = xInputObservable.merge(yInputObservable);

// Subscribe the stream to the handler
var subscription = inputObservable.subscribe(date => console.log(date));

Conclusion

ReactiveX is great at visualize, compose and manipulate streams of data. It can merge multiple streams into a single stream, transform a stream into another stream, convert, filter, limit the data of the streams… and many more functions.

When doing reactive programming, ReactiveX is not a requirement. However, it makes your life easier, so much easier that it gradually becomes a style of programming.

ReactiveX is not only good at handling user input. It can treat anything as a data stream. A stream of plain text from the server can be manipulated and converted into a stream of json object, which then can as well be filtered, limited, or combine with other streams of data.

It is just as much useful when implementing a clean architecture, but I will leave it for another blog post.

I have just moved from wordpress to Jekyll as my new blogging platform, and it is pretty fun. I now can write my blog posts in Markdown in my favourite text editor, have my blog version controlled with git, and deploy my site with a simple git push.

Deploying Jekyll is easy, but it can as well be confusing to newbs. Thus, I am writing this tutorial.

Choosing a Jekyll theme

There are lots of free Jekyll themes out there for you to choose with a little bit of googling. The theme I am currently using is Jekyll Mon Cahier. If you’re incline to develop your own theme, well, you can do it later after getting used to Jekyll first.

For the sake of this tutorial, I will be using Jekyll Mon Cahier.

git clone https://github.com/btquanto/jekyll_mon_cahier.git ~/blog

Installing and running Jekyll

You can either choose to install Jekyll, or use a pre-built Docker image

Installing Jekyll to your machine

1. Jekyll is a ruby gem, thus installing ruby is the pre-requisite

    curl -L https://get.rvm.io | bash -s stable --ruby=2.0.0
   

2. Now is the actuall installing Jekyll part

    gem install jekyll
   

3. Running your site is easy

    cd ~/blog
    jekyll serve
   

   Learn more about the jekyll serve command with jekyll serve -h

4. Checkout your blog at localhost:4000

Using docker

1. Using docker is even neater, but the pre-requisite is that you have to install Docker first. Checkout the instructions on Docker’s website

2. Then, just pull a pre-built docker image for jekyll. I am using btquanto/docker-jekyll here.

    docker pull btquanto/docker-jekyll
   

3. Just run your site (follow the instructions of whichever image that you use)

    cd ~/blog
    docker run -u $UID -d \
        --name jekyll \
        -v `pwd`:/src \
        -p 4000:4000 \
        btquanto/docker-jekyll jekyll serve -H 0.0.0.0
   

Deploying your blog

1. Generate a new ssh key without specifying any password. Save the key to ~/.ssh/id_rsa_np

    ssh-keygen
   

2. Go to github.com and add the public key of the newly generated key pair to your account.

3. Open ~/.ssh/config, and add the following content

    Host github.com-np
        HostName github.com
        User git
        IdentityFile ~/.ssh/id_rsa_np
        IdentitiesOnly yes
   

4. Create some folders for deploying your blog

    sudo mkdir /var/www/
    sudo mkdir /var/www/blog
    sudo chown $USER:$USER /var/www/blog
    mkdir /var/www/site
    mkdir /var/www/scripts
   

5. Clone your blog

    git clone [email protected]:<username>/<your-blog>.git /var/www/blog/src
   

6. Add a upstart script for serving your website, save it in /etc/init/jekyll-watch.conf

    description "Start jekyll watch for generating blog"
    author "Quan To <[email protected]>"
   
    start on runlevel [2345]
    stop on runlevel [!2345]
   
    exec su <username> -lc "jekyll build -ws /var/www/blog/src -d /var/www/blog/site"
    respawn
   

   Remember to replace <username> with your username

   Then start the upstart job

    sudo start jekyll-watch
   

7. Go to your DNS, create an A record, and point to your server.

8. Create an nginx config in /etc/nginx/sites-available/jekyll-blog

    server {
        listen 80;
   
        server_name your.domain.name;
   
        root /var/www/blog/site/;
        index index.html index.htm index.php;
   
        location / {
        expires off;
            try_files $uri $uri/ =404;
        }
   
        error_page 404 /404.html;
    }
   

   Remember to adjust the file appropriately

   Enable the configuration file using a symlink

    sudo ln -s /etc/nginx/sites-available/jekyll-blog /etc/nginx/sites-enabled/jekyll-blog
   

   Reload nginx

    sudo service nginx reload
   

   Now you can access your website at your.domain.name

9. Write a script for updating your src folder, put it in /var/www/blog/scripts/update.sh

    #!/bin/bash
    SRC_DIR="/var/www/blog/src"
    cd $SRC_DIR
    git fetch
    git reset origin/master --hard
   

   Give it execution permission

    chmod +x /var/www/blog/scripts/update.sh
   

10. On your local machine, add an alias for convenient use

    echo "alias update_blog=\"ssh user@host /var/www/blog/scripts/update.sh\"" >> ~/.bashrc
    

That’s pretty much done. Now, everytime you want your server to fetch new blog posts from github and regenerate your blog, just call udpate_blog.