Archive for the ‘software’ Category

I completed Andrew Ng’s machine learning class on Coursera. While I’ve done other machine learning (ML) classes, I found this class’ Octave exercises enabled me to understand what’s happening in the algorithms. Many of my previous ML classes involved proofs of the mathematics, whereas this class focused on applying the vector and matrix operations to work with datasets, whether it’s a small m or large m (number of data points). After all, we need to work with the data to derive any meaningful insights.

Much of the supervised learning was devoted to figuring out the cost function and applying gradient descent to minimize the cost function whether it’s linear regression, logistic regression, neural network, or SVM. Unsupervised learning included k-means, PCA, and collaborative filtering. Most important is the approaches to develop better algorithms specific to the data: diagnosing bias and variance, add features to improve underfitting, regularization to reduce overfitting, use of ceiling analysis to determine where to spend effort to get improvements. These concepts all come to life for me in the programming exercises.





Read Full Post »


This is the claim by Nvidia CEO Jen-Hsun Huang, that 3 Nvidia Titan Z CUDA based GPU cards could offer the same performance in running deep learning neural nets as done in the Google Brian project using Intel processors.

  • 1/150 the acquisition cost reduction
  • 1/150 heat consumption

If this could be done in general for deep learning type problems, we could have many more machines to do machine learning on the explosion of data. At the same time, to use the CUDA cores, software programmers would need to learn program this hardware and / or use OpenCL. The cost savings could warrant pushing over the learning curve.

This paper is referenced: “Deep Learning with COTS HPC Systems” by A. Coates, B. Huval, T. Wang, D. Wu, A. Ng, B. Catanzaro, published on ICML 2013

Read Full Post »

Tried Glass

I finally got to tried Glass. I attended Yosun Chang‘s Code Camp session on hacking Glass and she let the audience test some of the hacks she did with Glass. It was rather light and felt comfortable on the head.

I saw up close the structure that holds the OMAP4430 processor, 500+MB memory, 16G storage, GPS, camera, accelerometer, speaker, mic, light sensor and touch sensor.  There is a battery that’s designed to hang behind the ear when Glass is worn. The most visible piece is the clear block of plastic with a builtin prism to reflect the lights from the LCD screen to the eye’s upper right field of view.

I think it has huge potential for sensor fusion where the user intent could be surmised from the sense information of the accelerometer, mic, light, touch, and other sensors.

But NOT the use case of deploying the head as a mouse. One of hacks had the wearer browse items projected on a virtual cylinder by moving the head.  A person could easily get a neck cram and maybe develop a new form of carpal tunnel. Using gaze of the eye to track cursor position could be interesting, although currently there is no camera point at the eye. Opportunities for future versions of Glass.

In the mean time, I took a picture with me wearing Glass.


Read Full Post »

Photo with Matz

I had the opportunity to attend an event where the creator of Ruby was speaking. Matz was extremely friendly and he was gracious enough to take a picture with me.


Read Full Post »

I just received my certificate for the “Introduction to Artificial Intelligence” online course offered by Sebastian Thrun and Peter Norvig. I’m one of 23,000 who received the certificate, for doing well in the homework, midterm, and final exams. The class was one of three open online courses offered as an experiment by Stanford.   Enrollment reached 160,000 students from 190 countries.  The course started in September and ended in December.


I finally got ot understand how probability and statistics could be applied to make sense of data.  Dr. Thrun explained very well the applications of bayesian statistics and especially particle filter. I had years and years of statistical theories in math, physics, finance, computer science classes.

These 3 classes have launched the Massively Open Online Course revolution in education.

Read Full Post »

carlh-robeypOn June 24, we had a thought provoking set of presentations at the SDForum SAMsig, arranged by our departing Co-Chair Paul O’Rourkesamsig-attendees-june24The presentations ended with how Twitter is scaling today with the rewriting of the backend infrastructure from Ruby on Rails to a new language called Scala.  Notably, a queuing system, kestrel, that mediates between Twitter’s web user interface and the processing of “Tweet following” and sending tweets was written in Scala and implements Actor.  This implementation is much simplified from other implementations, is more reliable, and scales to handle billions of tweets.

The switch from Ruby to Scala for Twitter’s backend is detailed at Artima developer, run by Bill Venners, who also mediated the presentations by the 3 speakers (Hewitt, Sommers, Pointer) at SAMsig.

3 themes came out in the meeting, that pointed to Twitter’s switch to Scala:

  • Actor model enables simple programming of applications involving concurrency
  • Scala language features make programming fun and interesting
  • JVM has solid reliability and thread scaling

Actor model

Created by Carl Hewitt some 35 years ago at MIT, the Actor model is surfacing as a good way to think about and to implement systems that involve multiple simultaneous communications, such as chat and Twitter.  Actors are objects that do not operate on the same content that changes (mutable states) and communicate with each other via messages (message passing).  As the result, the actor model eliminates many of the headaches that programmers face when solving the concurrency issues involving millions of senders and receivers of messages. 

At SAMsig, Hewitt reviewed some of these issues and the history of the creation of the Actor model.


Scala is a recently created language that runs on top of the Java Virtual Machine (JVM) and thus uses all the facilities of the Java environment.  It takes advantage of the proven reliability, performance, and other capabilities offered by the JVM.  However, many programmers find coding in Java could be tedious with its formal requirements.  Scala makes it fun for programmers with its simplicity of passing functions (functional programming like LISP) and pattern matching (more general and powerful than C case statement).  Scala also implements Actor using the multi-threading capabilities of JVM, while removes the complexity of thread communication for the programmers. 

At SAMsig, Frank Sommers presented many of the features of Scala in a short 40 minutes.  There was much to take in in a short time.

JVM reliability and multi-threading

JVM has proven to be reliable and can scale easily to take advantage of the latest multi-core processors and large cluster of servers running together in the data center.  Robey Pointer indicated that  by using Scala to write the Twitter queuing system, kestrel, he was able to take advantage of all the goodness in the JVM, without having to write in Java.  And yet, Java could be a backup language if Scala fails.   Furthermore, by using Actor within Scala, the coding was much simplified from similar code written in Java, as the Actor enforces a share-nothing form of communication, designed for concurrent environment.  The code size for kestrel is estimated at half the size of similar code written in Java.  Without the complexity of managing threads and locks explicitly in kestrel, along with smaller code size, the support and maintenance of the code is much easier.  In fact, kestrel runs on multiple servers and processes billions of tweets without failing.  System responsiveness is fast.  Pointer’s slides are here.

The combination of Actor, Scala, and JVM makes the kestrel queueing system and Twitter  reliable, scalable, and fast.  By writing the code in Scala and using Actors, the code is easier to develop and simpler to maintain.  Such combination of these elements points to the continuing innovation happening with software.  This serves as a good example for us, as we endeavor to develop new products, that we consider new advances in software (such as Scala), rather than just being stuck with doing things the same old way.

Copyright (c) 2009 by Waiming Mok

Read Full Post »

I was  at an SDForum party in its offices and I noticed this chart, which shows the various Special Interest Groups (SIG) that were active through the years.


The chart ended on 2004.  As of the end of 2008, there are 16 active SIGs, some of which did not appear at the end of 2004.  The software landscape is ever changing.

Copyright (c) 2009 by Waiming Mok

Read Full Post »

Older Posts »