Oracle OpenWorld 2009 started today and it appears to be an exciting event.  Larry Ellison in his keynote today spoke about Sun and Oracle and how that combination would beat out IBM.  At the same time, Oracle has other systems vendors as partner — especially the very important HP, which also owns EDS, crucial for Oracle deployment.  At the same time, HP and EDS, along with Oracle, have IBM as major competitor.

Looking at the sponsor list for Oracle OpenWorld 2009, there are also many other consulting sponsors.  These major consultancies’ businesses do have a big dependency on Oracle.  So to some extent, Oracle has systems and storage vendors, along with consulting companies, all aligned against IBM.

IBM has moved its business more and more into services and consultancies, with IBM software being a major component.  Could IBM, being nearly 4X bigger than Oracle, put Oracle at bay, even when Oracle has many folks with common interests supporting it?

Copyright (c) 2009 by Waiming Mok

Inspiring video on IEEE’s role on technologies and its impacts.

As a tool is used by more and more people, we see more and more flavors of the tool, and eventually evolution of the tool.  How many types of screw drivers are there? 

As Michael Stonebraker points out in his blog on ACM:  there is no one database that solves all problems.

Same thing can be said of data stores:  it’s no longer a choice between direct attached disks, SAN block storage, NAS file systems.   Many of the developments came about because of the use of multiple servers to build web services, cloud computing, and grid computing.  The developers found that they can significantly improve system performance by locating data closer to the application or to the edge (where the data is accessed), that they could ease the complexity of development by relaxing consistency constraints (which are not needed) — see CAP Theorem, and that they could significant lower costs by avoiding or minimizing the use of expensive SAN storage or CDN data delivery.

As the value of storage management shifts beyond SAN and NAS, the question now is:  “Should storage vendors be offering products in this area?”

When Yahoo! announced that it’s outsourcing search to Microsoft, some folks became concern that Yahoo! could end its sponsorship of Hadoop.  This is cleared up in the Yahoo! Developer Network Blog:  Yahoo! will continue to sponsor this effort.

Open source software requires sponsors to staff engineers in order to develop and maintain the software; and Yahoo! is the key sponsor for Hadoop.  Yahoo! has other uses for Hadoop in addition to search.  Like Linkedin and Facebook, Yahoo! has lots of users and groups, and like these other web 2.0 companies, Yahoo! can run data mining and analytics for its purposes, such as targeted advertising.  Many data analytics algorithms, including machine learning, can scale linearly with a Hadoop architecture.  So it’s very believable that Yahoo! has a big stake in having a robust Hadoop implementation,.

For discussion sake, if there were no Hadoop, the existing alternatives are somewhat limited:

• Google is not open sourcing it’s GFS and MapReduce.
• Microsoft has yet to open up its DryadLINQ.

At the same time, if Yahoo! were to pull Hadoop sponsorship, it’d likely that others could come out to sponsor Hadoop.  My guess would be Linkedin, Facebook, and even Google.

Copyright (c) 2009 by Waiming Mok

IEEE Computer Society in Silicon Valley is holding its annual conference on Saturday July 18.   Moving not far from last’s year’s theme on Cloud Computing, this year it is on mobile devices and their impact to cloud computing.

Last year’s event was a great conference.  I’d expect this year to be the same.

Details here. 

Copyright (c) 2009 by Waiming Mok

On June 24, we had a thought provoking set of presentations at the SDForum SAMsig, arranged by our departing Co-Chair Paul O’Rourke.  The presentations ended with how Twitter is scaling today with the rewriting of the backend infrastructure from Ruby on Rails to a new language called Scala.  Notably, a queuing system, kestrel, that mediates between Twitter’s web user interface and the processing of “Tweet following” and sending tweets was written in Scala and implements Actor.  This implementation is much simplified from other implementations, is more reliable, and scales to handle billions of tweets.

The switch from Ruby to Scala for Twitter’s backend is detailed at Artima developer, run by Bill Venners, who also mediated the presentations by the 3 speakers (Hewitt, Sommers, Pointer) at SAMsig.

3 themes came out in the meeting, that pointed to Twitter’s switch to Scala:

• Actor model enables simple programming of applications involving concurrency
• Scala language features make programming fun and interesting
• JVM has solid reliability and thread scaling

Actor model

Created by Carl Hewitt some 35 years ago at MIT, the Actor model is surfacing as a good way to think about and to implement systems that involve multiple simultaneous communications, such as chat and Twitter.  Actors are objects that do not operate on the same content that changes (mutable states) and communicate with each other via messages (message passing).  As the result, the actor model eliminates many of the headaches that programmers face when solving the concurrency issues involving millions of senders and receivers of messages. 

At SAMsig, Hewitt reviewed some of these issues and the history of the creation of the Actor model.

Scala

Scala is a recently created language that runs on top of the Java Virtual Machine (JVM) and thus uses all the facilities of the Java environment.  It takes advantage of the proven reliability, performance, and other capabilities offered by the JVM.  However, many programmers find coding in Java could be tedious with its formal requirements.  Scala makes it fun for programmers with its simplicity of passing functions (functional programming like LISP) and pattern matching (more general and powerful than C case statement).  Scala also implements Actor using the multi-threading capabilities of JVM, while removes the complexity of thread communication for the programmers. 

At SAMsig, Frank Sommers presented many of the features of Scala in a short 40 minutes.  There was much to take in in a short time.

JVM reliability and multi-threading

JVM has proven to be reliable and can scale easily to take advantage of the latest multi-core processors and large cluster of servers running together in the data center.  Robey Pointer indicated that  by using Scala to write the Twitter queuing system, kestrel, he was able to take advantage of all the goodness in the JVM, without having to write in Java.  And yet, Java could be a backup language if Scala fails.   Furthermore, by using Actor within Scala, the coding was much simplified from similar code written in Java, as the Actor enforces a share-nothing form of communication, designed for concurrent environment.  The code size for kestrel is estimated at half the size of similar code written in Java.  Without the complexity of managing threads and locks explicitly in kestrel, along with smaller code size, the support and maintenance of the code is much easier.  In fact, kestrel runs on multiple servers and processes billions of tweets without failing.  System responsiveness is fast.  Pointer’s slides are here.

The combination of Actor, Scala, and JVM makes the kestrel queueing system and Twitter  reliable, scalable, and fast.  By writing the code in Scala and using Actors, the code is easier to develop and simpler to maintain.  Such combination of these elements points to the continuing innovation happening with software.  This serves as a good example for us, as we endeavor to develop new products, that we consider new advances in software (such as Scala), rather than just being stuck with doing things the same old way.

Copyright (c) 2009 by Waiming Mok

This book by Richard Denny provides good insights for people wanting to understand sales and the selling process.  Denny provides good summaries and illuminating “wise words” at the end of each chapter.  Some chapters are very applicable to those looking for employment.  Best “wise words”:

If you want to increase your successes, double your failure rate.  (Go for the ‘no’)

Yesterday is history.  Tomorrow is a mystery.  Today is a gift.  That is why it is called the present.

Some good info from the book:

4 ways to grow a business:

• increase number of customers
• increase average transaction value
• increase frequency of repurchase
• acquisition

6 stages of successful presentation:

1. show the results
2. go to prices and fees, get a reaction
3. go into detail
4. give one or two case histories appropriate for prospect
5. provide something to give you credibility
6. outline your guarantee (risk reversal)

Definitely well worth reading, and keeping a copy for reference.

Copyright (c) 2009 by Waiming Mok

OPENforum’09 was well attended, albeit on a Saturday, with a full audience at the large auditorium at the Computer History Museum.  I saw 2 of the 3 keynotes and I found both inspiring.

Warren Packard of DFJ described 3 curves that drive technology, entrepreneurship, and the economy:

• Bell curve:  the statistical nature of the world
• S-curve:  dynamics of competition and relationship between effort and product improvement and eventually replacement by new technology breakthroughs
• turkey curve (ala Taleb’s Black Swan):  there are the unpredictable, high impact rare events
  

  Turkey Curve

Packard projected optimism in emerging technologies and in entrepreneurship being key to solve the difficult problems in the world.  He spoke on the positive contributions of the real time web, social networks, MEMS, new materials, dramatic cost reduction of DNA sequencing that would lead to new ways of medical diagnosis,  data center transformations, emerging user interfaces that combine the virtual and the real world.  These disruptive changes would bring about good. 

When asked about today’s economic and environment calamities, Packard’s response was that these are huge issues, absolutely surmountable, by the drive of entrepreneurs to develop solutions to take advantage of the economic opportunities offered by these challenges.  Packard’s answer to political oppression in the world is to give people hope, to lead them to see the future, and people will work toward that future. 

What an inspiration!  Just consider:  our actions are driven by our beliefs.  So believe in hope and the positive future, then we can work toward that future.

Masood Jabbar, formerly of Sun Microsystems, gave the second keynote and he spoke from his life experiences and lessons learned:

• 10 years of experience is a myth, it is more like 1 year of experience, 10 times.  We learn quickly when we hit the wall or when things are new and uncomfortable.  But we are not good at learning when things change very slowly or the environment is familiar and comfortable (i.e., over 10 years of doing similar things).  Lesson is:  when it ain’t broken, change it; lest the world changes on us and it becomes very costly for us to change.
• There is over-emphasis on team work.  If the team does not have active debates, the result one gets is the lowest common denominator of the team members.  Lesson:  teams without debate produce mediocre results.
• Todays’ economy is like a Formula One race car going into the pitstop, where the actions of the pit crew would determine the outcome of the race.  Lesson:  act now to take advantage of the opportunities as the economy recovers.
• Lesson from Fred Smith of Fedex:  “No company can ever get customer satisfaction without employee satisfaction.”

I consider the following the best insight from Jabbar:  To win for the future, you have got to move the ball forward today, and every day.

This was a great event.  Very inspiring.

Copyright (c) 2009 by Waiming Mok

At the Hadoop Summit ‘09, the large auditorium at the Santa Clara Marriot was overflowing with people.  There was huge enthusiasm for this open source software and its applications.  Some key notes I got from this event:

 The Hadoop ecosystem provides the framework to run big data applications using computer science techniques (distributed systems, algorithms, machine learning, …)  faster, and at much lower costs. 

The underlying compute infrastructure is cloud computing, pioneered by Amazon, that offers low cost to run multiple computers, on an as-needed basis.  The ease to scale quickly to thousands of computers makes cloud computing elastic.  This elasticity shortens the time for innovations/problem solving by deployment scalable parallel computing resources.  Hadoop simplies the programming of the large distributed system with the map-reduce programming model and a distributed file system.

In the 3 years since its inception, there is a large ecosystem around Hadoop, including Pig, Hive, HBase, Zookeeper, Cascading, Mahout, Katta, …  Cloudera has stepped in to provide a distribution to ease the deployment of Hadoop.

The range of problems that could be solved by using Hadoop is expanding:  from gene sequence matching, to Large Hadron Collider data analysis, to matching potential mates, to SPAM filtering, to classifying and organizing photos, to machine understanding of human language, to analyzing large mobile call data.  The practitioners in these diverse fields say by using Hadoop in a cloud infrastructure, they could solve their problems at much lower cost and faster– in hours/days rather than weeks.

Linkedin (at ScaleCamp the night before) claimed that they could produce revenue generating products in less than 2 weeks using Hadoop and machine learning on their data.  For example, Linkedin is analyzing the profile data to map out career paths.  Other companies like Facebook are banking on growth strategy that leverage the extraction of information from the large subscriber data available from their services. 

The was excitement at the Hadoop Summit.  Hadoop and its ecosystem offer a promising platform to address big data problems and applications.

Copyright (c) 2009 by Waiming Mok

I caught this at the end of day on the show floor at TiECON 2009.  Done by a Japanese company, Speecys Corp.  Have a laugh.

Copyright (c) 2009 by Waiming Mok