SQL Select with Write Lock

SQL transaction is usually used when we want to make database changes in a consistent state. When inserting/updating row in a transaction, a write lock will be held by the transaction so other transaction cannot do update at the same time (*). This makes sure the updated row is not changed from the execution of update to the end of transaction.
*) this is an oversimplification, please read the transaction manual about isolation level for more detailed explanation

However in some cases transaction is not enough to prevent race condition. A race condition is something that occurs when 2 concurrent execution tries to access and change a piece of data at the same time.

Let’s use an example with following database schema in MySQL

CREATE TABLE account {
  id int NOT NULL PRIMARY KEY;
  balance decimal(10,0) NOT NULL default 0;
} 

Consider the following piece of code in NodeJS with bookshelf.js ORM

function addBalance(id, amount){
  return bookshelf.transaction(function(t){
    return Account.where('id', id)
    .fetch({ transacting: t })
    .then(function(account){
      account.set('balance', account.get('balance') + amount);
      return account.save(null, { transacting: t })
    })
    .then(function(account){
      console.log('Balance: ', account.get('balance'));
    });
  });
}

Promise.map([1000, -1000], function(amount){
  return addBalance(1, amount);
});

If we run the code, we will get the following result

Balance: 1000
Balance: -1000

It happens because when both “addBalance” function executes the SELECT statement, the balance value is still 0 for both. This is definitely not the result we want. Even if we execute 2 functions in parallel, we want a safety measure to keep the balance in consistent state, so the read and update process must happen atomically.
This is where a Select Write Lock comes into play. For MySQL, we can read the documentation here (Locking Reads).

I won’t talk about “Lock in Shared Mode” in detail because I haven’t tested it thoroughly, so I will only show what happens if we use “Select … For Update” statement.

function addBalance(id, amount){
  return bookshelf.transaction(function(t){
    return Account.where('id', id)
    .query(function(qb){
      qb.forUpdate();  // this is the important part
    })
    .fetch({ transacting: t })
    .then(function(account){
      account.set('balance', account.get('balance') + amount);
      return account.save(null, { transacting: t })
    })
    .then(function(account){
      console.log('Balance: ', account.get('balance'));
    });
  });
}

When we run the code again, it should give a consistent result

Balance: 1000
Balance: 0

As the documentation stated, “Select … For Update” tries to acquire write lock for the selected row(s) just like an actual “Update” statement. So the second “addBalance” function cannot execute its select statement until the first transaction finishes. This feature is very useful for atomic read & update functionality.

However as an important note, this is only valid if both “addBalance” function are done in separate transaction. It will break if we try to call them in the same transaction in parallel. I will talk about it in later post.

Posted in Programming | Tagged , , , | Leave a comment

A Little Misconception about Promise (nodeJS)

Consider the following code

var Promise = require('bluebird');

function f(x){
  return Promise.try(function(){
    console.log("start " + x);
    return Promise.delay(500);
  })
  .then(function(){
    console.log("finish " + x);
  });
}

var promises = [];
promises.push(f(1));
promises.push(f(2));

Promise.each(promises, function(){
  return f;
});

Expected result:

start 1
finish 1
start 2
finish 2

But when we run the code, the actual result is

start 1
start 2
finish 1
finish 2

So what part is wrong?
Until now I have a very wrong assumption that the function inside Promise.try() will be executed when it is called inside Promise.each. But it is actually called synchronously the moment f(x) function is invoked.

So the correct way to write the code is wrap the f(x) function call inside another function like the following.

var promises = [];
promises.push(function(){
  return f(1);
});
promises.push(function(){
  return f(2);
});

Promise.each(promises, function(p){
  return p();
});
Posted in Programming | Tagged , | 2 Comments

Parallel asynchronous functions with NodeJS promise

Long time since I last wrote a post. So recently I’ve been doing NodeJS and meets a cool feature: Promise. Promise is a way to make asynchronous callback look like serial, so instead of writing

doSomething(function(err,obj){
  if (err) ...
  ...
  doSomethingElse(function(err,obj){...})
});

we can instead write

doSomething()
.then(function(obj){...})
.then(function(obj){...})
.catch(function(err){...});

and we can even catch all the errors in one function, that’s neat!
NodeJS library for promise is bluebirdJS. You can learn more about it on the site.

What’s even cooler about promise is the ability to make several asynchronous functions run in parallel. Well not true parallel because NodeJS is single-threaded, but the point is we can run several promises simultaneously and wait until they all finish then continue our code. This is done with bluebird’s Promise.all() function. Promise.all() receives an array of object and the resulting promise will return an array of result for each object.
I’ll give 2 examples.

Example 1: we want to make a database query for several ids in parallel and process the results at once. This can be done with

var ids = [1,2,3,4,5];
Promise.all(ids.map(function(id){
  return Model.query({id:id});
}))
.then(function(results){...});

Example 2: we want to run 2 different database queries in parallel and process the result at once.

Promise.all([
  (function(){
    return Model.queryOne();
  })(),
  (function(){
    return Model.queryTwo();
  })()
])
.then(function(results){
  // results[0] contains result of first query
  // results[1] for second query
});

That’s very easy and short. Now imagine how many lines and functions are needed to write the above queries with callback. Promise paradigm is indeed very promising and helpful for asynchronous programming.

Posted in Programming | Tagged , , , , | Leave a comment

[ios] Drawing unicode string in CGContext

There’s one big, big defect in CGContextShowText function. It only works for ASCII characters. There’s actually another way to draw in CGContext using NSString drawInRect: method, but it doesn’t work in my case because I need to calculate all the transform coordinates manually.

So after some googling I finally found an alternative using CTLine in CoreText framework. After tweaking the code here and there, finally it worked! To make things easy I’ll just paste the code here. Note that I use ARC feature on this code.

CTFontRef font = CTFontCreateWithName((__bridge CFStringRef)_fontName, _fontSize, &CGAffineTransformIdentity);
CFStringRef keys[] = { kCTFontAttributeName, kCTForegroundColorAttributeName };
CFTypeRef values[] = { font, _fillColor.CGColor };
CFDictionaryRef attributes = CFDictionaryCreate(kCFAllocatorDefault, (const void**)&keys, (const void**)&values, sizeof(keys[0]), &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
CFDictionaryRef attributes = CFDictionaryCreate(kCFAllocatorDefault, (const void**)&keys, (const void**)&values, 2, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
CFAttributedStringRef attrString = CFAttributedStringCreate(kCFAllocatorDefault, (__bridge CFStringRef)text, attributes);
CFRelease(attributes);
CFRelease(font);

CTLineRef line = CTLineCreateWithAttributedString(attrString);
CFRelease(attrString);
CTLineDraw(line, context);

Edit: The deleted line causes crash in Release Build, so I had to change it

There’s good and bad things about CTLineDraw. The good thing is it respects most of CGContext states that work for CGContextShowText. Some CGContext functions that still work on CTLineDraw are CGContextSetTextPosition, CGContextSetTextMatrix, CGContextSetTextDrawingMode, CGContextSetStrokeColorWithColor, and CGContextSetLineWidth. So we can safely set the text position and text stroke.

Now the one function that doesn’t work is CGContextSetFillColorWithColor. The text color must be passed via the values array in above code. (see the _fillColor.CGColor part) The second thing that doesn’t work is more tricky. After calling CTLineDraw and you call CGContextGetTextPosition, the value doesn’t change from the value before you call CTLineDraw, while for CGContextShowText the cursor will move to the end of the string. In order to simulate the same situation with CGContextShowText, we can set the text position manually by adding the text width to the previous position.

float textWidth = CTLineGetTypographicBounds(line, NULL, NULL, NULL);
CGContextSetTextPosition(context, prevPosition.x + textWidth, prevPosition.y);

Of course it works for single line text. I didn’t try on multiline text because I handled multiline text by drawing it line by line.

So that’s how I handle the problem. If you know an easier solution to do the same thing, please do tell me.

Posted in Programming | Tagged , , , , , | Leave a comment

[iOS] Automatic Reference Counting and cross reference

Even with Automatic Reference Counting (ARC) feature in iOS 5, cross reference between objects should still be treated with care. A cross reference is where 2 or more objects are referencing each other, making circular reference. In which case, the reference count on the objects won’t go down to zero and they won’t be automatically deallocated. For example, it can happen on a case like this.

@interface A: NSObject {
  A * parent;
  A * child;
}
@end

If an object is keeping a reference of both its child and parent, the parent and child will have cross-reference between each other. The way to resolve it is by having “__weak” reference on the object with higher hierarchical status, i.e. the parent. In ARC, weak reference means keeping pointer to object without increasing its reference count.

@interface A: NSObject {
  __weak A * parent;
  A * child;
}
@end

Or if you use properties, you can set the property attribute to weak.

@property (weak,nonatomic) A * parent;

Another common case is the delegate pattern. It’s better to set delegate as weak reference too. But one thing should be kept in mind: if you try to access a weak-referenced object which has been deallocated, it will result in bad memory access. So when the delegate object goes out of scope, remember to set the delegate reference to nil.

@protocol B<NSObject>
@end

@interface C: NSObject
@property (weak,nonatomic) id<B> delegate; 
@end
Posted in Programming | Tagged , , , , | Leave a comment

After Effects: Exporting alpha channel on separate file

Lately I’ve been playing with videos at my workplace (programming related). The widely supported video codecs, H.264, cannot have alpha channel, so in some cases I have to export the RGB channel and alpha channel into 2 files and then combine it in the program. In After Effects this can be done via Output Module Settings.

To export RGB channel, choose “RGB” in “Channels” and “Straight (Unmatted)” in “Color”. The “Color” choice shouldn’t be “Premultiplied (Matted)” because the 2 channels will be combined later. To export alpha channel, duplicate the settings and change the “Channels” to “Alpha”.

Posting this on blog just as a reminder for myself 😉

Posted in Komputer | Tagged , , , , | Leave a comment

Easy comment-uncomment with 1 character

This is commented.

/*
do_something();
//*/

This is not.

//*
do_something();
//*/

Got this from a friend who got this from a friend who got this from a friend who…

Posted in Programming, Tips | Tagged , , | Leave a comment

[iOS] Pointer to Block Pointer

iOS block programming feature is pretty cool, it makes asynchronous programming easier. This time I want to talk about block pointer. A block pointer is a type of variable that we can use to refer to a block, for example

void (^blockPointer)(void) = void ^(void) { NSLog(@"inside a block"); };

We can then execute the block like normal function call.

blockPointer();

What’s interesting is that a block pointer is actually an Objective-C object. We can do copy, retain, release, autorelease (more on this can be read here) on it, and also assign a pointer to it :D.

I’ll show you a sample use case where I want to make a pointer to a block pointer. Suppose I have a function that does something asynchronously and executes a block at the end of it.

- (void) asyncFunction:(void (^)(void)) block {
  // do something asynchronously, then execute the block at the end
  block();
}

Then I have another function, a nasty one with many nested ifs, that passes a block to asyncFunction depending on the conditions. When it doesn’t call asyncFunction, the block still has to be executed. So the block must be called exactly once, no matter which conditional route it takes.

Solution A: do some brute force with myriads of if-else

- (void)nastyFunction:(void (^)(void)) block {
  if (...) {
    if (...) {
      [self asyncFunction:block];
    } else {
      /* myriads of other if-else */
    }
  } else (...) {
    block();
  }
}

Solution B: Because a block pointer is an Objective-C object, we can assign nil to it. So after calling asyncFunction, we assign the block to nil. At the end of nastyFunction we execute the block only if it’s not nil.

- (void)nastyFunction:(void (^)(void)) block {
  if (...) {
    if (...) {
      [self asyncFunction:block];
      block = nil;
    }
    /* myriads of other ifs */
  }
  if (block != nil){
    block();
  }
}

The number of else decreases, the number of ‘block = nil‘ increases.

Solution C: this is where pointer to block pointer becomes useful. Instead of writing ‘block=nil‘ numerous times, we write it only once inside asyncFunction. But to change the value of block inside nastyFunction, we have to pass it to asyncFunction using a pointer. We declare a pointer to block pointer using this statement.

void (^pointerToBlockPointer *)(void)

So asyncFunction becomes

- (void) asyncFunction:(void (^*)(void)) block {
  // do something asynchronously, then execute the block at the end and assign it to nil
  *block();
  *block = nil;
}

And we pass the block from nastyFunction using its address.

- (void)nastyFunction:(void (^)(void)) block {
  if (...) {
    if (...) {
      [self asyncFunction:&block];  
    }
    /* myriads of other ifs */
  }
  if (block != nil){
    block();
  }
}

The declaration for pointer to block pointer is pretty funny, I got it from XCode compilation error message :D.

Another useful tips: if you want to simplify the declaration, you can typedef the block pointer.

typedef (void)(^BlockPointerType)(void);

So the pointer to it can be written as

BlockPointerType * pointerToBlockPointer;
Posted in Programming | Tagged , , , , , | Leave a comment

[AS3] Nested function and asynchronous model

In early days I’m used to synchronous programming model, everything can be called sequentially so I don’t have to deal with callback, I can write the code very neatly. When I meet AS3, I learn much about asynchronous functions, callbacks and event handlers.  AS3 uses asynchronous model so extensively, almost everything that needs “loading time” is asynchronous, like URLLoader, Loader, SQLStatement, and more. On one side it makes my job easier because I don’t need to create a background process (as in Java / C#) to do the loading in background and prevent the application from hanging. On the other side it makes the code I write pretty messy.

Imagine a case where I need to do multiple INSERT statement to an SQLite database and do something when all insertions finish. I can either 1) do all insertions sequentially and handle all success callbacks. Then when the number of callbacks equals the number of insertions, I continue. Or 2) do first insertion, wait callback, insert the next, wait callback, and so on until the last callback. In either case, I need to add a new object variable to count the number of callbacks. That makes the code more unclean because the variable should only be used temporarily.

This is where I find nested function to be really useful. I have known that AS3 has nested function since way back then, but I prefer not to use it in my code for the sake of clean and more readable code. However in this case of multiple asynchronous functions, I find no better way to do it than nested function. This is due to one magical property of nested function: it preserves the local variable of its parent function, even across asynchronous calls.

So for example, if I want to do multiple insert to SQLite using method 2) above, the code will look something like this.

function insertAll(){
  var i:int = 0, max:int = 10;
  var statement:SQLStatement = new SQLStatement();
  statement.sqlConnection = this.connection;
  function insert(){
    if (i < max){
      statement.text = "INSERT INTO table VALUES ("+i+")";
      statement.execute(-1, new Responder(insert));
      i++;
    } else {
      //do something after
    }
  }
  insert();
}

In function insertAll(), the value of local variable “i” will be preserved across calls to nested function insert(). So I don’t need to add any new object variable. In this case I find asynchronous model and nested function are meant to be used together, maybe that’s actually the idea behind them anyway.

Posted in Programming | Tagged , , , , , | Leave a comment

Cara Sederhana Membedakan Script PHP untuk Development/Deployment

Saat sedang kodang-koding di kantor, tiba-tiba saya menemukan penggunaan require_once pada php yang agak menarik. Seperti yang kita tahu, require_once memasukkan sebuah script hanya pertama kali dipanggil. Jika script yang sama dimasukkan 2 kali, pemasukan kedua tidak dilakukan.

Masalah yang sedang saya kerjakan seperti ini: saya membuat script php yang menerima input parameter dan melakukan validasi sederhana dengan hashing parameter. Yang merepotkan, untuk melakukan testing pada script tersebut saya harus menghitung hash dari parameter yang saya input -___-, padahal saya ingin melakukan testing cepat dengan memasukkan parameter lewat browser. Salah satu solusinya, saya dapat menonaktifkan dahulu fitur hashing untuk melakukan pengujian lewat browser, tapi untuk menguji aplikasi sebenarnya saya harus mengaktifkan hashing lagi, jadinya bolak-balik deh.

Ada cara lain yang ternyata lebih mudah, yaitu membuat script terpisah untuk testing pada browser dan meng-require_once script asli. Pada script asli saya menambahkan konfigurasi aktif/tidaknya hashing dengan variable $use_hash. Variable $use_hash ini saya definisikan pada script terpisah “config.php” yang di-require_once oleh script asli. Untuk lebih gampangnya, isi script kurang lebih sebagai berikut.


//script config.php
$use_hash = true;

//script asli.php
require_once("config.php");
if ($use_hash) {
//hitung hash
} else {

}

//script testing.php
require_once("config.php");
$use_hash = false;
require_once("asli.php");

Mekanisme pada script testing.php: karena require_once hanya meng-include sekali, script config.php hanya dibaca pertama kali. require_once(“config.php”) yang terdapat di script asli.php tidak akan terpanggil, sehingga nilai akhir variable $use_hash adalah false. Dengan cara ini saya dapat melakukan testing pada browser cukup dengan membuka testing.php (muhaha). Mungkin trik ini juga bisa dipakai untuk keperluan lain, seperti membedakan script yang akan digunakan untuk tahapan development/deployment.

Posted in Programming | Tagged , , | Leave a comment