https://github.com/gideondk/scaliak/commits/master

The latest commits add preliminary support for MapReduce to the Scaliak client.

MapReduce jobs are modelled into a simple case class (MapReduceJob) which together with the MapReduceBuilder abstracts the Riak MapReduce job into a much simpler flow.

The MapReduceJob class takes a MapReducePhasePipe as a argument and has optional arguments for a Search Query, Bucket, Object Map (Bucket => KeySet), Binary Index or Int Index with a Int or a Range as argument.

The MapReducePhasePipe makes adding phases easier: By using the |* or |- method, you can add a new Map or Reduce phase to your job. For instance:

MapReducePhasePipe(mapValuesToJson(false) |- sort(field, sortDESC))

Both mapValuesToJson as the sort function, are named or anonymous javascript functions stored in MapReduceFunctions.scala. I will add more common MapReduce functions into this object for easy job building.

Once a job has been created, the job can be send to the mapReduce function in the ScaliakBucket class, which uses the MapReduceBuilder to create a JSON representation of the job and sends in to the client.

Next to the added MapReduce functionality, I've added a DomainObjectWrapper, which makes DomainObjects easier to use.

To use it, simply create a case class extending the DomainObject class and create a companion object:

object Photo extends DomainObjectWrapper[Photo](classOf[Photo])

Through this companion class, you can fetch, store and delete objects in several ways, classes are automatically (de)serialized to JSON and if you use annotations (RiakKey and RiakIndex for instance). Keys and indexes are automatically stored and retrieved.

The code is untested and still in progress, but hope to finish most parts and the end of next week!

JavaScript + structured concurrency. Bring structure to your JavaScript based asynchronous applications. A small but powerful extension to the language of the web.

http://onilabs.com/stratifiedjs

Progressive Stream Processing and manipulation is an important task in modern Web Programming, starting from chunked upload/download to Live Data Streams consumption, creation, composition and publishing through different technologies including Comet and WebSockets.

Read more about it: Wiki link

ActorKit provides an Objective-C implementation of asynchronous inter-thread message passing.

The purpose of ActorKit is to facilitate the implementation of concurrent software on both the desktop (Mac OS X) and embedded devices (iPhone OS). On the iPhone, thread-based concurrency is a critical tool in achieving high interface responsiveness while implementing long-running and potentially computationally expensive background processing. On Mac OS X, thread-based concurrency opens the door to leveraging the power of increasingly prevalent multi-core desktop computers.

Try it now!

Needed something leaner myself and if you need one too, here you go:

import javax.sound.sampled.AudioInputStream

class Bucket {
    var min = 0;
    var max = 0;

    def add(x: Int) = {
      if (x > max)
        max = x
      if (x < min)
        min = x
    }

    def peak: Int = {
      List(Math.abs(min), Math.abs(max)).max
    }
}

object AudioStatter {

    val NUM_BITS_PER_BYTE = 8

    def numberOfChannels(audioInputStream: AudioInputStream): Int = 
      (audioInputStream.getFormat().getFrameSize() /  (audioInputStream.getFormat().getSampleSizeInBits() / NUM_BITS_PER_BYTE))

    def maximalPeakFromBucketArray(bucketArray: Array[Bucket]): Double = 
        bucketArray.par.foldLeft(0) {(peak, bucket) => if (bucket.peak > peak) bucket.peak else peak} / 65535.0

    def normalizedSampleList(bucketArray: Array[Bucket], maxPeak: Double): List[List[Double]] =
        bucketArray.par.map(bucket => List(bucket.min, bucket.max).map(i => i / 65535.0 / maxPeak)).toList

    def sampleListToWaveFormCoordinateString(sampleList: List[List[Double]]): String =
        sampleList.flatten.map(f => "%.2f" format f).grouped(20).map(a => a.mkString(",")).mkString(",\n")

    def byteToSample(firstByte: Int, secondByte: Int): Int = (secondByte << 8) + (firstByte & 0x00ff)

    def waveformFromAudioInputStream(audioInputStream: AudioInputStream): String = {
      val resolution = 1600
      val bucketArray = Array.fill(resolution){new Bucket}
      val inputSize = audioInputStream.available
      val bucketSize = ((inputSize - 1).toFloat / resolution.toFloat + 0.5).toInt + 1
      var index = 0

      val bufferSize = audioInputStream.getFormat().getFrameSize() * 1024 * 1024 * 2 // With a frame size of 2 bytes, a buffer of around 4MB
      val buffer = new Array[Byte](bufferSize)

      var bytesRead = 1

      while (bytesRead >= 0) {
        bytesRead = audioInputStream.read(buffer, 0, buffer.length)
        if ( bytesRead >= 0)
        {
          for (doubleByte <- buffer.grouped(2) if index / bucketSize < resolution)
          {
              bucketArray(index / bucketSize).add(byteToSample(doubleByte(0),doubleByte(1)))
              index = index + 2
          }
        }
      }

      val maxPeak = maximalPeakFromBucketArray(bucketArray)
      val normalizedSamples = normalizedSampleList(bucketArray, maxPeak)
      sampleListToWaveFormCoordinateString(normalizedSamples)
    }
}

You try to create a optimal object-oriented structure, use attributes on objects to set and get data. And when you’re using less object-structured data like dictionaries, you use the (almost same) key-value coding as you do with dictionaries.

And then you have to store things… Using traditional relational databases, the next step is to create a ORM (object-relational mapper) to map those beautiful dynamic lists of objects into a table. A completely different approach of data representation: a two dimensional structure with tables, columns and rows.

NoSQL

Thank god for document and key-value stores like MongoDB and Redis! With the right tools, they enable you to create, store, change and extend your objects without thinking about database migrations.

Using such storage on you server side, makes you frustrated you have to revert to SQL for storage on the iPhone. Especially when you simply want to embed objects into other objects.

Countries with cities

For instance: you want to create multiple “Country” objects, which stores a ordered list of “City” objects (by alphabet). Perfectly fine in the object-oriented mind of a Objective-C programmer. But if you want to store these objects, and you want to retrieve a country with its cities. You’re out of luck…

First you have to retrieve the city you want to fetch, by name for instance, and you have to fetch all cities with a specific “city_id” matching the identifier of the city. Finishing the task by putting the list of city objects in the “cities” attribute of the country object.

Could be simpler right? SQL brings a lot of power when it comes to filtering and searching through your data. In this case, and many cases: SQL brings more pain then joy.

BNRPersistence

So let’s bring some noSQL goodness onto the iPhone by using BNRPersistence. BNRPersistence is a Objective-C wrapper for the key-value store “Tokyo Cabinet”.

With large data sets, BNRPersistence can be 10 to 17 times faster then the traditional CoreData. While sacrificing complex searching and filtering, you get a new storage layer which is not only faster but also much simpler!

Once you get past the (bit more difficult) installation, adding the storage functionality is more then easy. Start off by extending the BNRStoredObject class in your object.

#import "BNRStoredObject.h"
@class City;

@interface Country : BNRStoredObject {
    NSString *name;
    NSMutableArray *cities;
}

- (void)setName:(NSString *)n;
- (NSString *)name;

- (NSArray *)cities;

@end

Next step is to add the data serialization and deserialization functions to your class, like this:

- (void)readContentFromBuffer:(BNRDataBuffer *)d
{
    [name release];
    name = [[d readString] retain];

    [cities release];
    cities = [d readArrayOfClass:[City class] usingStore:[self store]];
    [cities retain];
}

- (void)writeContentToBuffer:(BNRDataBuffer *)d
{
    [d writeString:name];
    [d writeArray:cities ofClass:[city class]];
} 

And your up and running! For other functions you can use, just check the header file of BNRStoredObject.

At the moment, BNRPersistence shows some immaturity… So you can walk into some drawbacks:

• The framework isn’t thread safe currently, so lock your stuff or use one thread for storing data.
• Stored data can be quite a bit larger then when using CoreData, so think things through what you want to store and what not.
• Next to full-text searching, you don’t have a lot of other added functionalities. If you need things from CoreData, just use them side by side.

All in all, if you are frustrated by the cumbersome managed object contexts and managed objects or looking a fast and simple storage method (with built in versioning). Definitely give BNRPersistence a try!