Oracle OpenWorld 2009 started today and it appears to be an exciting event. Larry Ellison in his keynote today spoke about Sun and Oracle and how that combination would beat out IBM. At the same time, Oracle has other systems vendors as partner — especially the very important HP, which also owns EDS, crucial for Oracle deployment. At the same time, HP and EDS, along with Oracle, have IBM as major competitor.
Looking at the sponsor list for Oracle OpenWorld 2009, there are also many other consulting sponsors. These major consultancies’ businesses do have a big dependency on Oracle. So to some extent, Oracle has systems and storage vendors, along with consulting companies, all aligned against IBM.
IBM has moved its business more and more into services and consultancies, with IBM software being a major component. Could IBM, being nearly 4X bigger than Oracle, put Oracle at bay, even when Oracle has many folks with common interests supporting it?
As a tool is used by more and more people, we see more and more flavors of the tool, and eventually evolution of the tool. How many types of screw drivers are there?
As Michael Stonebraker points out in his blog on ACM: there is no one database that solves all problems.
Same thing can be said of data stores: it’s no longer a choice between direct attached disks, SAN block storage, NAS file systems. Many of the developments came about because of the use of multiple servers to build web services, cloud computing, and grid computing. The developers found that they can significantly improve system performance by locating data closer to the application or to the edge (where the data is accessed), that they could ease the complexity of development by relaxing consistency constraints (which are not needed) — see CAP Theorem, and that they could significant lower costs by avoiding or minimizing the use of expensive SAN storage or CDN data delivery.
Problem being solved
Implementation
Characteristics
Hadoop HDFS
Ensure availability of data for distributed batch processing.
Distributed servers, with one master metadata server and multiple data servers. Data is replicated at 3X normally.
Changes to data require creation of new copy of data.
Facebook Haystack
Minimize use and cost of CDN to provide photos to users.
Distributed servers: memory for metadata and local disks for photos.
Mostly read system; CAP less an issue.
Yahoo! MObStor
Provide fast and highly available access for unstructured content, for operational storage
Multi-layer: Global Object Management, Local Object Management, Object Store that accepts different types of storage
Offers eventual consistency
As the value of storage management shifts beyond SAN and NAS, the question now is: “Should storage vendors be offering products in this area?”
When Yahoo! announced that it’s outsourcing search to Microsoft, some folks became concern that Yahoo! could end its sponsorship of Hadoop. This is cleared up in the Yahoo! Developer Network Blog: Yahoo! will continue to sponsor this effort.
Open source software requires sponsors to staff engineers in order to develop and maintain the software; and Yahoo! is the key sponsor for Hadoop. Yahoo! has other uses for Hadoop in addition to search. Like Linkedin and Facebook, Yahoo! has lots of users and groups, and like these other web 2.0 companies, Yahoo! can run data mining and analytics for its purposes, such as targeted advertising. Many data analytics algorithms, including machine learning, can scale linearly with a Hadoop architecture. So it’s very believable that Yahoo! has a big stake in having a robust Hadoop implementation,.
For discussion sake, if there were no Hadoop, the existing alternatives are somewhat limited:
Google is not open sourcing it’s GFS and MapReduce.
At the same time, if Yahoo! were to pull Hadoop sponsorship, it’d likely that others could come out to sponsor Hadoop. My guess would be Linkedin, Facebook, and even Google.
IEEE Computer Society in Silicon Valley is holding its annual conference on Saturday July 18. Moving not far from last’s year’s theme on Cloud Computing, this year it is on mobile devices and their impact to cloud computing.
Last year’s event was a great conference. I’d expect this year to be the same.
On June 24, we had a thought provoking set of presentations at the SDForum SAMsig, arranged by our departing Co-Chair Paul O’Rourke. The presentations ended with how Twitter is scaling today with the rewriting of the backend infrastructure from Ruby on Rails to a new language called Scala. Notably, a queuing system, kestrel, that mediates between Twitter’s web user interface and the processing of “Tweet following” and sending tweets was written in Scala and implements Actor. This implementation is much simplified from other implementations, is more reliable, and scales to handle billions of tweets.
Created by Carl Hewitt some 35 years ago at MIT, the Actor model is surfacing as a good way to think about and to implement systems that involve multiple simultaneous communications, such as chat and Twitter. Actors are objects that do not operate on the same content that changes (mutable states) and communicate with each other via messages (message passing). As the result, the actor model eliminates many of the headaches that programmers face when solving the concurrency issues involving millions of senders and receivers of messages.
At SAMsig, Hewitt reviewed some of these issues and the history of the creation of the Actor model.
Scala
Scala is a recently created language that runs on top of the Java Virtual Machine (JVM) and thus uses all the facilities of the Java environment. It takes advantage of the proven reliability, performance, and other capabilities offered by the JVM. However, many programmers find coding in Java could be tedious with its formal requirements. Scala makes it fun for programmers with its simplicity of passing functions (functional programming like LISP) and pattern matching (more general and powerful than C case statement). Scala also implements Actor using the multi-threading capabilities of JVM, while removes the complexity of thread communication for the programmers.
At SAMsig, Frank Sommers presented many of the features of Scala in a short 40 minutes. There was much to take in in a short time.
JVM reliability and multi-threading
JVM has proven to be reliable and can scale easily to take advantage of the latest multi-core processors and large cluster of servers running together in the data center. Robey Pointer indicated that by using Scala to write the Twitter queuing system, kestrel, he was able to take advantage of all the goodness in the JVM, without having to write in Java. And yet, Java could be a backup language if Scala fails. Furthermore, by using Actor within Scala, the coding was much simplified from similar code written in Java, as the Actor enforces a share-nothing form of communication, designed for concurrent environment. The code size for kestrel is estimated at half the size of similar code written in Java. Without the complexity of managing threads and locks explicitly in kestrel, along with smaller code size, the support and maintenance of the code is much easier. In fact, kestrel runs on multiple servers and processes billions of tweets without failing. System responsiveness is fast. Pointer’s slides are here.
The combination of Actor, Scala, and JVM makes the kestrel queueing system and Twitter reliable, scalable, and fast. By writing the code in Scala and using Actors, the code is easier to develop and simpler to maintain. Such combination of these elements points to the continuing innovation happening with software. This serves as a good example for us, as we endeavor to develop new products, that we consider new advances in software (such as Scala), rather than just being stuck with doing things the same old way.
This book by Richard Denny provides good insights for people wanting to understand sales and the selling process. Denny provides good summaries and illuminating “wise words” at the end of each chapter. Some chapters are very applicable to those looking for employment. Best “wise words”:
If you want to increase your successes, double your failure rate. (Go for the ‘no’)
Yesterday is history. Tomorrow is a mystery. Today is a gift. That is why it is called the present.
Some good info from the book:
4 ways to grow a business:
increase number of customers
increase average transaction value
increase frequency of repurchase
acquisition
6 stages of successful presentation:
show the results
go to prices and fees, get a reaction
go into detail
give one or two case histories appropriate for prospect
provide something to give you credibility
outline your guarantee (risk reversal)
Definitely well worth reading, and keeping a copy for reference.
OPENforum’09 was well attended, albeit on a Saturday, with a full audience at the large auditorium at the Computer History Museum. I saw 2 of the 3 keynotes and I found both inspiring.
Warren Packard of DFJ described 3 curves that drive technology, entrepreneurship, and the economy:
S-curve: dynamics of competition and relationship between effort and product improvement and eventually replacement by new technology breakthroughs
turkey curve (ala Taleb’s Black Swan): there are the unpredictable, high impact rare events
Turkey Curve
Packard projected optimism in emerging technologies and in entrepreneurship being key to solve the difficult problems in the world. He spoke on the positive contributions of the real time web, social networks, MEMS, new materials, dramatic cost reduction of DNA sequencing that would lead to new ways of medical diagnosis, data center transformations, emerging user interfaces that combine the virtual and the real world. These disruptive changes would bring about good.
When asked about today’s economic and environment calamities, Packard’s response was that these are huge issues, absolutely surmountable, by the drive of entrepreneurs to develop solutions to take advantage of the economic opportunities offered by these challenges. Packard’s answer to political oppression in the world is to give people hope, to lead them to see the future, and people will work toward that future.
What an inspiration! Just consider: our actions are driven by our beliefs. So believe in hope and the positive future, then we can work toward that future.
Masood Jabbar, formerly of Sun Microsystems, gave the second keynote and he spoke from his life experiences and lessons learned:
10 years of experience is a myth, it is more like 1 year of experience, 10 times. We learn quickly when we hit the wall or when things are new and uncomfortable. But we are not good at learning when things change very slowly or the environment is familiar and comfortable (i.e., over 10 years of doing similar things). Lesson is: when it ain’t broken, change it; lest the world changes on us and it becomes very costly for us to change.
There is over-emphasis on team work. If the team does not have active debates, the result one gets is the lowest common denominator of the team members. Lesson: teams without debate produce mediocre results.
Todays’ economy is like a Formula One race car going into the pitstop, where the actions of the pit crew would determine the outcome of the race. Lesson: act now to take advantage of the opportunities as the economy recovers.
Lesson from Fred Smith of Fedex: “No company can ever get customer satisfaction without employee satisfaction.”
I consider the following the best insight from Jabbar: To win for the future, you have got to move the ball forward today, and every day.
At the Hadoop Summit ‘09, the large auditorium at the Santa Clara Marriot was overflowing with people. There was huge enthusiasm for this open source software and its applications. Some key notes I got from this event:
The Hadoop ecosystem provides the framework to run big data applications using computer science techniques (distributed systems, algorithms, machine learning, …) faster, and at much lower costs.
The underlying compute infrastructure is cloud computing, pioneered by Amazon, that offers low cost to run multiple computers, on an as-needed basis. The ease to scale quickly to thousands of computers makes cloud computing elastic. This elasticity shortens the time for innovations/problem solving by deployment scalable parallel computing resources. Hadoop simplies the programming of the large distributed system with the map-reduce programming model and a distributed file system.
In the 3 years since its inception, there is a large ecosystem around Hadoop, including Pig, Hive, HBase, Zookeeper, Cascading, Mahout, Katta, … Cloudera has stepped in to provide a distribution to ease the deployment of Hadoop.
The range of problems that could be solved by using Hadoop is expanding: from gene sequence matching, to Large Hadron Collider data analysis, to matching potential mates, to SPAM filtering, to classifying and organizing photos, to machine understanding of human language, to analyzing large mobile call data. The practitioners in these diverse fields say by using Hadoop in a cloud infrastructure, they could solve their problems at much lower cost and faster– in hours/days rather than weeks.
Linkedin (at ScaleCamp the night before) claimed that they could produce revenue generating products in less than 2 weeks using Hadoop and machine learning on their data. For example, Linkedin is analyzing the profile data to map out career paths. Other companies like Facebook are banking on growth strategy that leverage the extraction of information from the large subscriber data available from their services.
The was excitement at the Hadoop Summit. Hadoop and its ecosystem offer a promising platform to address big data problems and applications.
I spent May 16 at TiECON 2009. There appears strong interests in entrepreneurship:
The announcer said attendance was 20% above last year’s, while attendance at othere conferences were reduced;
Michael S. Malone mentioned that there is growth in ”encore entrepreneurs”, those over the age of 40, vs the younger entrepreneurs.
Three of the sessions were especially noticeable:
Keynote by Tony Hsieh of Zappos.com. He spoke about Zappos, its focus on customer service, reaping customer life time value, delivered by company culture, achieved by employee selection, training, and incentives. He also spoke about achieving happiness, which ahs 4 factors: perceived control, perceived progress, sense of connectedness, and finding meaning and value in life.
Be proactive, don’t just post, submit, participate, comment on others.
Reward good behaviors.
Do not spin.
Interview of Reid Hoffman by Michael S. Malone. Linkedin is not about how much time you spend online, but more on how efficiently you leverage your network. Entrepreneurship is like throwing yourself off the cliff and assembling the airplane on the way down, you either survive or die.
Just finished reading Lawrence Lessig’s Remix. In the book, Lessig calls for changes to how we deal with conflict between commercial interests of content owners (music, video, …) and the emerging creativity using these content in mashups and remixes. He makes the argument for the advent of the RW culture, the new hybird economy, and he calls for changes to the copyright laws.
Lessig describes the resurgience of the RW (read/write) culture, where with the advent of internet technologies, we now can mashup and remix software, pictures, text, video, music, and other content created by others into new creations. RW culture contrasts with the RO (read only) culture that predominates in the 20th century prior to the internet, with creations like movies on tape, music on CD, software on DVD, etc. This, of course, runs counter to the economic interests of owners of (commercial) content and their lawyers.
Lessig highlights the conflict in the way we exchange content via 2 economies::
Commercial economy, where money or “price” is a central term of the normal exhcange. Netflix, Amazon, and Google are 3 commercially successful enterprises based on the Internet, due to the Long Tail principle, which says that as the cost of inventory falls, the efficient range of inventory rises. As transaction costs fall to zero, the efficient inventory rises to infinity.
Sharing economy, where content are exchanged for the purpose of relationships, and when money comes in, poisons the relationships. Examples include open source projects, where people are sharing their work without the exchange of money, for the benefits of the greater good or creation.
Lessig advocates the internet now allows for the hybrid of the 2 economies. He pointed to Warner Brother, who has began to adopt this, has benefited from this approach. By being less restrictive with its intellectual property, allowing Harry Potter fans to uses images in their fan sites, it strengthened it’s fan’s loyalty to the brand.
Lessig says that the copyright laws need changing, as they do not address the new world as the result of the internet. He advocates 5 key changes:
Deregulate amateur creativity, allow for remixes.
Clear title and reinstitute opt-in copyright, where the copyright owner has to actively re-register after a period of time, in order to retain title to the work.
Simplify the law.
Decriminalize the copy.
Decriminalize file sharing.
Very valid, very logical. This would surely take a long time for the world to accept and implement.
SDForum is holding its annual developer conference on April 17-18, with emphasis on cloud computing platforms and applications, as well as the emerging social network business models. There is also an unconference on April 18.
Speakers include: Bill Grosso, Eric Klein, James Staten, Clara Shih, and others.
The focus of Scrum is to deliver results with small teams (less than 10 people). Issues are identified and resolved by the team, product requirements are put in a backlog and prioritized by the product owner. In the latest iteration of the Scrum definition, the product owner has responsibility (his/her neck is on the line) for the results delivered by the team and interfaces with the stakeholders to ensure proper communication with the stakeholders. As a full-fledge member of the team, the product owner adheres to the Scrum rules in working with the team. The Scrum Master ensures that the product owner follows the rules of Scrum. Influence is key to the success of the Scrum Master.
While scrum was developed for software development, it could easily be applied to marketing launches and other product deveopment, as the focus is on team work and marching toward sprint cycles. There are ways to coordinate multiple Scrum teams to form larger development work.
Many companies are already practicing scrum, including Microsoft, Oracle, Yahoo!, and others. This is a good practice to apply to teams.
With less than 1K servers, redundant data centers included, Salesforce.com supports over55K customers, including Google and Dell. This feat is achieved by an ingenious group of Oracle database experts who have taken an enterprise class relational database and turned it into a multi-tenant system that runs customer relationship management for these many customers. BTW, this system supports close to 200M transactions each weekday, at less than 1/4 of a second response time.
On March 25, Craig Weissman, CTO of Salesforce.com, gave an illuminating presentation on the internal architecture at his company to a room full of attendees at the SDForum SAMsig. Some highlights:
There are 15 or so “pods”, each consisting of 50 or so servers, running a 4-way Oracle RAC and application (Java) and support servers. Each pod supports thousands of customers.
Each Oracle RAC database consists of static tables that store the data from thousands of customers all mixed together. One row of data belongs to one customer and the next row belongs to another, and the columns may have completely different types of data between the rows. Control and access to the data are managed by metadata. In essence, the Oracle database is transformed to a multi-tenant database.
Customer data in the columns are stored as human-readable strings. Some customers have requested certain data to be encrypted. Appropriate transformation functions are used to convert to the correct data types when the data are accessed.
Apex is a new language to enable customers to write data processing applications, like a new generation of 4GL. It resembles Java. Governors are deployed to prevent abuse of the system resources.
The Salesforce.com architecture is an engineering feat in leveraging the strength of an existing product (reliability and scalability of Oracle RAC) to build a new system that supports thousands of customers on the web with million of transactions per day and fast response time. Interestingly, Salesforce was able to move the underlying hardware from Sun Sparc systems to x86 and updated the Oracle version, over the course of 10 years, while retaining the higher level software architecture and getting substantiate customer growth.
I just finished reading this book Managing Humans by Michael “Rands” Lopp. Great book for software managers, managers, and individual contributors. I find Rands provide realistic and practical advice on dealing with people who run your company, the person you work for, and people you manage. Rands also provide interesting insights on people and classify them in his terms: NADD, incrementalists, completionists, organics, mechanics, inwards, outwards, holistics, free electrons, joe.
Like usual, I borrowed this book from the library so I’d read it, as I tend not to read books I buy. With this book, I purchased a copy via AllBookstores.com after I’ve read it. It’s worth the purchase, if only for future reference.
With full audience, the CCICECloud Computing event on March 19 was a very interesting event. The event host, Sara Rauchwerger, drove the program with a combination of video presentation, live teleconference, and live presentations by 2 speakers. Furthermore, the entire event was broadcast live via the Internet using PAPItv.
In the use of multimedia, a pre-recorded video with Dr. Maik A. Lindner of SAP was shown to the audience. A Windows PC ran the video. Dr. Lindner spoke about the hype cycle of cloud computing.
With the live teleconference via Skype, Dr. Patrick Wong of IBM spoke about IBM’s cloud computing effort, while Sara drove the slides.
The live presenters had some interesting ideas about cloud computing:
Praveen Kumar of Packet Island spoke about the buzz word cycles and that “cloud computing” is replacing “SaaS”.
Praveen Kumar spoke about the importance of ensuring Quality of Experience (QoE) in order for SMB to adopt cloud computing. His way is to minimize CAPEX by managing the entire service life cycle.
CCICE is pushing the envelope in conferences by incorporating what companies already do (video, teleconference, live video streaming, live presentations) to conduct an event. For the people who attended the conference, they still get to do networking.
At the Monte Jade annual conference on March 7, Dr. Arun Majumdar of Berkeley gave a very interesting talk on energy use and the opportunities for improvement. He presented lots of data. I found 3 of his points especially interesting:
1. When looking at overall energy use, there are 9 sources of supply and 6 areas of demand. Even high growth rate in solar would take years to be of significant impact — that should not stop solar.
2. Abatement of energy use is the most efficient and effective way in the short term to solve the energy crisis. There are many ways, including changing the way we build buildings and how a building is put together. These savings are extremely significant.
3. Battery, energy storage, has improved little. Storage capacity has doubled in 60 years. New development in nanotechnolgy could help.
By focusing in these areas with new technologies and software, significant headway could be made in the next 3-5 years to improve the energy problem.
Orion Letizi gave an interesting talk on Terracotta at the SDForum SAMsig tonight. His example on the deployment of Terracotta to solve a database scaling problem was especially compelling:
Examinator example
Before
After
Concurrent web-based tests
5K
10K
Main database server utilization
75%
30%
Additional hardware
-
Additional commodity servers
Software changes
-
Management of session transient data from main database server to Terracotta networked memory
By moving the processing of various transient data used in the web service from the main Oracle database to Terracotta’s network attached memory model, with minimal changes to the Java code. There were some additional hardware added to support the Terracotta servers, but these were low cost commodity hardware.
This examples shows that often web/app/database architectures tend to rely on the backend database servers to manage the transactions, but also the transient data for sessions. Transient session information includes data like tracking the page the user is on, the data that he has entered, etc. Terracotta can manage the transient data in place of the main database. The resulting system is much more efficient than having the entire web service architecture relying on the main database to maintain all the states in the system.
Terracotta is open source and works with Java based systems. Orion mentioned that other languages can be supported, such as Ruby via jruby.
voltage potential brings about an electric field (battery and wire)
electric field generates magnetic field
2 magnetic fields in same direction cause repulsion (magnet and wire)
and the marvel of man’s ingenuity:
converting this knowledge into a device (electric motor) that’s used in so many applications from disk drives to garage door opener to cars.
I certainly have learned from this simple experience, reminding me why I was a physics major a long time ago.
To make the motor work, be sure of the following:
the wire must be insulated wire, so the coil does not short
one end of the wire has the insulated wire removed by sanding; the other end has only one side of the insulation removed (this ensures that the wire generated magnetic field is turned on and off as the wire spins, enabling the repulsion of the 2 magnetic fields to continue the spinning of the wire)
regularly sand the ends of the wires again, as there are occasionaly electric arcs that bring about deposit (carbon?) on the wire that prevents electrical conduction
In the week of Feb 15, 2009, I had the opportunity to take a 3 full day class on WPF from Charles Petzold at Microsoft. I had not written software for the .NET or Windows, but it was a good opportunity to understand this fundamental technology in the Microsoft world.
I learned some interesting things from the class:
Silverlight is not a complete subset of WPF; Microsoft is pushing Silverlight, so the hope is that the 2 would sync somehow
WPF can be done with XAML in many cases, even 3D presentation, C# would be needed for more complicated actions and controls
WPF is tablet ready — an important support as we move forward with new user interface devices
I was able to follow along, because Charles showed lots of examples. Great way to learn programming is by looking at examples and seeing the code work. To become an example is writing XAML and C# with WPF, I’d have to study the APIs and use it for at least 6-12 months.
Separately, I looked into other alternatives for RIA:
MS WPF and MS Silverlight
Adobe AIR / Flash / Shockwave
OpenLaszlo
Appcelerator
and a few other ones (e.g. Sun JavaFX)
Charles Petzold is very well known and admired in the Microsoft community. I’m thankful to have the opportunity to learn from and interact with him.
I had a situation where the boot block for a Vista machine is on one disk drive and the Windows directory is on a different disk drive and the first disk apparently failed and I could no longer boot. The solution is to rebuild the MBR on the second disk. The following resources helped.
I attended my first SHDH. This time it’s held at Sun’s Menlo Park campus and it has over 350+ people on a Saturday afternoon that extended to 11pm. People were doing programming and some even had computer controlled devices. An interesting afternoon.
At the SDForum SAMsig January meeting, Klaus Roder of IBM showed the promise of mashup for the enterprise. Klaus indicated that IBM is providing a set of tools to help enterprise customers to create quick mashups to access data. Of course, in order to have good mashups, good data is a key requirement. The tools that run on customer premises are available to enterprise customers for a fee. A free test version of the capability is also available on the web, the web version (IBM Mashup Center) is similar to Yahoo!Pipes (not available for the enterprise). Other than IBM, there are some vendors who offer similar capabilities, such as JackBe.
Microsoft’s James Senior showed a demo of Live Mesh to the SDForum Cloud-Services SIG on Jan 27. Live Mesh provides desktop in the cloud. By itself, might not be very interesting, but it promises to synchronize the content in various devices. That could be sellable?
Although it looks like a simplistic desktop with few features (save files, view files), there are many elements that could surface quickly. In the developer area of Live Mesh, there are already demo applications. Would Word, Excel, etc. be far behind?
At the Jan 27 session of the SDForumCloud Services and SOA SIG (just renamed from Web Services SIG), a bigger than expected crowd showed up, …, over 100 people. Many were new to the SIG — there was an entrance fee as well. There appeared to be big interests in cloud computing.
Demos on Microsoft Live Mesh and Google AppEngine likely increased the interests.
In CACM, Feb 2009, Ashish Arora, Matej Drev, and Chris Forman surveyed the current status of software innovations around the world. They use the number of software patents as a measure of software innovation. As can be seen from the chart below, the US still dominates in the number of software patents, thus software innovation. Do note that these are software patents filed in the US. Other countries could have underdeveloped patenting process, so there could be undercounting of software innovation.
There has been much outsourcing of software projects to India, China, and other countries. These software developers over time take on more of the software project, including design. And as demand for software grows in the local regions, the local software talents come to innovate on software developed for the local regions. The authors made the point that software development tend to be located close to the lead users. Over time, as local demand for software products and services increase, the local talents would create more innovations.
In CACM, 01/09, Werner Vogels, in his article “Eventually Consistent“, describes how the construction of large distributed systems, such as Amazon EC2 and S3, would require the tradeoff between consistency and availability. This follows Eric Brewer’s CAP theorem (conjecture?). The idea is that in any shared-data system, only 2 of following 3 properties could be satisfied: consistency, availability, and tolerance to network partitions. In large distributed systems, e.g. cloud-computing, tolerance to network partitions is a given. So there is a trade-off between consistency and availability.
Vogels’ proposition is that availability of the system (and the resulting services) has higher priority to consistency in most cases. He identifies different types of relaxed form of consistency, other than strong consistency — that all data stored in the entire system could be retrieved, and when compared, are the same. The relaxed consistencies allow the Amazon implemention of the cloud to be highly available, and eventually, transaction updates are made consistent.
This article points to the fact that in order to build cloud-computing architectures, previous assumptions such as ACID may need to be relaxed.
Sandia National Lab published a study that shows the current design of multi-core processors with high core count actually slow down the system performance. Key issue is memory bandwidth. The implication is that processor vendors like AMD and Intel need to improve memory and data access in addition to CPU computation.
Writing a regular blog could take a lot of time. If I could take less than 1% of a day’s time, then the benefits from the following purposes could make the time well spent:
1. describe results on my actions each day, in a public manner
2. reinforce discipline to make optimal choices on how I use my time
3. develop clarity of thought by writing quickly and succinctly
I had thought about writing a blog for more than a year. I could not settle on what to write about. Then Carol Emerson at Right Management suggested: write on “Where is Waiming?”, like “where is Waldo?”. That helped me come to the purposes for this blog.
As an estimate, it’s less than 15 minutes each day. This can be computed without any calculator as follows:
There are 24 hours each day. ‘24′ is near ‘25′ and 25 times 4 equal 100. So as an approximation, 1 hour or 60 minutes consist of 4% of each day. Then, 1/4 of an hour or 15-minute is 1%.
Now, 1/96 (15 minutes) is bigger than 1/100, so 15 minutes is longer than the actual 1% of a day. To check the number, we use a calculator, which says 1% of day consists of 14.4 minutes.
This method of thinking is inspired by Herb Simon, whom I had the opportunity to listen to when I was at Carnegie-Mellon. Dr. Simon’s point was that using our mind and the available information, we can figure things out quickly, without turning on a calculator or a computer. One example he told was on how to quickly figure out the circumference of the earth:
From Los Angeles to New York is approximately 3,000 miles and crossing the continental U.S. spans 3 times zones, so each time zone is approximately 1,000 miles. There are 24 times zones in going around the earth, so an estimate for the circumference of the earth is 1,000 times 24 or 24,000 miles. Googling “earth circumference” comes up with 24,902 miles. So we obtained a good estimate just using our mind and some common knowledge.
When choosing a theme for this blog, I picked MistyLook by Sadish for its clean design, putting forth the blog writings first on the left hand side and having options for indices and other info on the right. I did want to change the top graphics that’s my very own. So I spent more than half an afternoon on 9/13 to pick out some of my photos and to do some photoshop adjustments. Resulting in more than 17 options and I chose the one that has overlapping pines and bamboos (#8, counting from left to right, then top to bottom), somewhat reflecting who I’m: someone who has lived straddling between American and Chinese cultures.
I especially like these other ones (counting from left to right, then top to bottom):
#6: double dragons: a dragons fly sitting on top of a stone dragon head
#10: orange train in a depot, with tracks extending far away
#12: reflections of taxes and a truck from glass windows in a high rise in Hong Kong; there is
”Keep Clear” sign in English and Chinese
#13: leopard from San Diego Wild Animal Park
#15: masks on display at the Shanghai Museum
Since now I’m producing writing and graphics outside of work, I’m re-instituting a habit I had developed as a software developer some time ago: putting “Copyright (c) year by Waiming Mok” on writings and graphics I put on WordPress.
In addition to Photoshop, I found the latest Google Picasa 3.0 beta to be extremely useful. It helps me find all the photos I’ve on my disk drives and it’s user interface allows me to quickly pick out the photos I needed. In addition, it has great features like collage production that makes the following collage easy to do.
I heard about this event while it was happening, so did not get a chance to go. I spoke to 2 people who went. One had a good impression: startups are given 8 minutes to present and demo their products and 8 minutes for Q&A; 50 companies went thru that process; very good format for the audience to see all these startups and their products and services. The other person indicated that most of these companies were offering feature-adds to the web (vs a platform plays by Google or Amazon) and his impression is that the resulting business models would not be sustainable.
I was curious, so went to the web site and found videos of many of the companies. Two companies I saw could be interesting:
1. Mint.com: quick and easy way to manage one’s money online: free, anonymously, and the website even suggests ways for one to save money. Only question is how much would someone trust a website with his or her personal financial info.
2. fitbit plans to make Americans healthier with a small wearable device with a wireless interface and free software that tells the wearer how much he has walked/exercised or how well he has slept. The device is simple: an accelerometer with a wireless interface; the rest is done in software. This offers similar capability as the Nike+, but smaller, and you can sleep with it. The concept of using sensors to track a person’s walking for weight loss purposes is also described in a book: chapter 9 of The Design of Things to Come by Vogel, Cagan, and Boatwright. The book talks about how a product could be created by using information from different areas: using 10,000-step program to weight loss and an automatic means to measure number of steps a person takes, and putting all that in a shoe, it could make weight loss easier. Fitbit has now made this concept potentially real and useable by offering a $99 device and free software.
The current examples are accelerometers in the iPhone and the Wii, which allows you to play games by sensing your hand movements. Another is the million-mirror DLP that’s the key to today’s video projectors. Around for 25 years now, the boutique MEMS industry is fast growing, producing all types new devices that are lower costs, lighter, better functioning: such as pressure sensors, RF switches, fuel cells, microengines, rockets, quartz crystal replacement, needles, drug delivery, AFM. MEMS uses all types of materials, many are bio-compatible, so cell scaffoldings can be built to grow organs, devices can be implanted, new experimental retina are being made.
Dr. Alissa Fitzgerald gave an exciting presentation on MEMS at the IEEE CNSV today. The MEMS industry is the wild west and great possibilities are out there. Simple devices could be fabricated with less than $50K – even startups could do this. More complex ones for consumer use could cost $20M-$50M. It could take 5 to 10 years to perfect a device as there are still unknowns and the tools are not yet mature. Nevertheless, this is an improvement from TI’s 25-year investment into DLP. As Dave Wyland said: the MEMS industry is like the semiconductors in the 1970’s. New ways of doing things are possible. Costs could come down quickly once someone finds a volume use for a device, such as the accelerometer (now at $1 per axis) or DLP.
Paul Maritz’s keynote spoke about the company’s attempt to broadly address the changes in the computing infrastructure: existing means of computing (e.g. Windows infrastructure), emerging web frameworks (e.g. AJAX, …)-people are no longer writing traditional windows applications (experimentation), diversity of clients (e.g. iPhone, …). 3 initiatives (i.e., ideas that would be products ==> opening for competitors):
* “data center OS” to address internal “clouds” — these are delivered as products today (ESX, …)
* federation of different external clouds outside the firewall
* dealing with the client dilemma: multiple clients access to same personalized set of applications
VMware appears to be still driving the virtualization agenda in 2008. Intel announced 6-core Dunnington, and others are announcing new products at VMworld.
With changes in management and the commodization of the hypervisor, VMware is trying to adjust itself. And competitors like MS, Citrix (Xen), RH, Oracle, Sun, … are releasing products in the space, while web 2.0 vendors (Google, Yahoo, …) are changing the landscape.
Bob McDonald’s talk on managing/leading in a crisis at the SDForum Engineering Leadership SIG illustrated what good managers do. Bob is brought into companies in trouble to get them back on course. Bob described a number of things he does–I only summarize the following aspect.
During times of turmoil in a company, something is amissed: maybe the org is messed up, maybe something is wrong with the product, etc. To get things going, management has to establish structure quickly: structure in the business plan, product roadmap, resource allocation, structure in the organization (whom to eliminate, who’s responsible for which roles, who does what project), structure to move the company forward (weekly operations meeting), structure to solve problems (calling special meetings–not solving problems in the weekly meetings), structure to build trust and empower the employees (regular 1:1, ask what employees want, help them to get them). If there is trust, then these structures enable the people in the company to work again, to start solving the issues.
To build a sound structure composed of people (i.e. a company), trust is the best fastener.
If a manager is to do these actions during crisis, why wouldn’t the manager do these in OK times? Thus, management = structure + trust.
Yubikey is a new way to provide security and authentication for web 2.0 deployment. It consists of a USB device with a single button, which when pushed, generates a unique identity and a different pass code that is effective for one second. The authentication is done on a server over the internet. The API is open so any web services developer can quickly incorporate the authentication capability into the service.
What a neat way to combine a physical key, server based computing, and web services.
The company, Yubico, that invented it is based in Stockholm.
Tony suggested to me to enjoy being hit, so I could focus on thinking thru the blade actions. Every time he asked me what happen in an action, I could not say, as my mind was stressed on protecting myself from being hit.
What I needed to learn is to have control of the blade and to feel thru the blade what the opponent is doing in order to develop my moves. That awareness, plus a ready mind to apply the right speed and action would make me a better fencer. So let the opponent hit (fail), and focus on control and awareness, and learn. As I watch the more experienced folks, their actions are more crisp, not out to score a hit or avoid being hit. They spend their time on thinking thru the actions, faints, deceives, parries, ripostes… I see this mode of learning more with foil than sabre.
I find this 33+ miutes speech by Tim O’Reilly inspiring (keynote at the Web 2.0 Expo NY). He posed the challenge to the developers and startups to solve the problems in the world, not just to dip into the great money of venture capital. Watch it, if it’s still on youtube.
Quote of Irving Yalom:
“First will what is necessary, and then love what you will“
and reading of “The Man Watching” by Rainer Maria Rilke:
“… What we choose to fight is so tiny!
…
Winning does not tempt that man.
This is how he grows: by being defeated, decisively,
by constantly greater beings.”
O’Reilly:
“Create more value than you capture“
“Web meets world”
instrumentation of the web
strategy to deal with an upredicable future: work on stuff that matters, rather than get caught up in the reality bubble
InSTEDD, real diseases not stopped by vaccination, but isolation (web 2.0 application)
Ushahidi, open source platform to gather reports by mobile phone, web; premise: gathering crisis information from … public … provides insights … real-time
23andme, “get your gene sequence, … build big pool of genetic data … for further research”
“Visionary Company: Big Hairy Audacious Goals”, Jim Collins, Jerry Porras, Built to Last
Microsoft 1978, bunch of kids with a great idea
“Organize all the world’s information”, Google
“Web meets world is a challenge for all of us, … to build company that matters, … to create value for all of us”
The Man Watching,Rainer Maria Rilke, “… what we choose to fight is so tiny! …”, “… we’re all going to be defeated in this life, let’s fight with things that matter, that makes us stronger, to makes us better, that’s the challenge …”
Oracle Beehive is the new collaboration suite of software that aspires to replace MS Exchange + SharePoint + Office Communication Servers, and similar offering. VMware B-Hive Control is a VM performance monitoring software delivered as a virtual appliance. Oracle Beehive was developed over the last 3 years with lots of internal Oracle engineers. VMware B-Hive was a May’08 acquisition of B-Hive Network by VMware. I saw both at Oracle OpenWorld 2008.
Charles Phillips said this during his keynote at Oracle OpenWorld 2008. He continued to say that (my paraphrasing) a software business needs to achieve largest market share; the bigger you’re in software, the more likely you’d win.
Goal is to turn variable costs into noise, and focus on investment in R&D (and marketing). As long as there are markets for the products, keep investing in engineering. This idea reflects Oracle’s strategy to be “Complete”, “Open”, and “Integrated”, as well as its many acquisitions.
Wei Zheng from Informatica gave an interesting talk on the the company’s plan to do federated database or virtual DB using an abstraction layer for data access. She indicated that previous attempts at virtual DB failed because people were trying to solve the BI problem, which requires access of multiple TBs of data all at once. Now Informatica is looking to solve the web service data access problem (e.g. customer service query on customer) on multiple sets of data (e.g. from merger of companies), where results of the query tend to be small. Naresh Govindaraj then followed with how that the abstration layer could be used for enterprise mashup and showed examples involving Google Maps. Event held by SDForum Web Services SIG.
As I tried these, most of these do not work with the WordPress service — they would work with a WP server you control… TwitterFeed should work using OpenID. I don’t know enough about this to use it yet. Also looking into some form of RSS feed from WordPress to Twitter. I’m now trying TwitterFeed, setup to poll every few hours for changes in WordPress and send any updates to Twitter.
Hagel, Brown, Davison described the concept of shaping strategies, which are initiatives that mobilize ecosystems to reshape industries — usually occur in a world of constant disruptions, like now… Such a strategy includes 3 elements:
view, more than a vision, that provides focus and directions for participants to help grow the ecosystem; e.g. salesforce.com’s view in 1999: (1) managing customer relationship would become competitive advantage as customers gain more power, (2) these applications would be best delivered as network-based services
platform, a set of clearly defined standards and practices that organizes and supports many particpants; e.g. salesforce’s Force.com makes it easy to create 3rd party applications
acts and assets, what the shaper does and what it owns to assures and encourages participants to stay on the platform and grow the ecosystem
Changes and disruptions spell opportunities for shapers to develop these 3 elements to reshape an industry.
Another article in HBR Oct’08 (free online) by Scott Cook speaks to his insight on the use of volunteers or user contributions to change the economics of industries. Some examples are:
Skype reduced capital costs to zero by using users’ PCs to run the internet based phone system
Wikipedia leverage the expertise from its constituents, with no costs to acquiring the expertise
open source minimizes costs for software development
Setup Google Apps today. Costs $10/year for a domain name, get a web site, email to/from the web site domain, Google Docs tools. The Google app web site edit feature is rudimentary. Still trying to figure out how to publish a web page, even after choosing a template …
11.1 You retain copyright and any other rights you already hold in Content which you submit, post or display on or through, the Services. By submitting, posting or displaying the content you give Google a perpetual, irrevocable, worldwide, royalty-free, and non-exclusive licence to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute any Content which you submit, post or display on or through, the Services. This licence is for the sole purpose of enabling Google to display, distribute and promote the Services and may be revoked for certain Services as defined in the Additional Terms of those Services.
2 interesting videos on Investment vs Spending for the federal government. The message is the country needs investment in infrastructure, education, science, …
(on $700B and/or baby boomer social security funds, …) looks like accretion of power in federal government, and this is likely to hamstring the federal government for the foreseeable future
(on individual spending funded by debts) 105B credit cards, or 9 per card holder
short term problem with Wall Street / financial system collapsing, requiring quick action
long term issues with people loosing homes/jobs, protected industries not creating new products, education system not preparing for the future, retirement system not going to be there
Congress has been unwilling to address fundamentals of this country: energy problems that make sense, infrastructure, healthcare
Regulations are designed for world 50-years ago, inconsistent with regulations around the world, when we need to operate world-wide
Cause: instant gratification, … wanted something for nothing
I saw a presentation by Dawn DeBruyn on her service WeMeUs. This has the potential to be my contact manager. The system is a web service with your data in the cloud, so you can access the data anywhere. It is meant to do what Goldmine and other similar desktop contact manager does. Features include tags, logs, and action reminders on individual contacts, also grouping of contacts, import/export, esp. with LinkedIn. These features are free. WeMeUs has more capabilities than Outlook, but is not as complex as CRM system, like Saleforce or SugarCRM. Nor is it a social network like LinkedIn or Facebook (which apparently could do not do import/export of contacts), so only you have access to your data, which Dawn said is “your relationship IP”.
I’m in the limited beta. Hopefully this is better than MS Business Contact Manager 2007, which I had encountered problems even trying to start on both 32-bit and 64-bit versions of Vista, after installation.
I attended my first unconference, CloudCamp Silicon Valley 08. It was well organized and well attended, in fact sold out. Dave Nielsen was able to arrange sponsors so the venue (Sun EBC) and the food were free. The attendees suggested topics and we held 5 discussion sessions at 7pm, 7 sessions at 8pm, and 4 sessions at 9pm. I attended the sessions on storage, resource sharing, and hadoop. Lots of interesting technical discussions, and somewhat unstructured. Good event.
MagCloud enables anyone to publish a magazine. It costs nothing to publish. Rather, when someone wants a hard copy, it costs 20 cents per page, plus shipping. You set the issue price, so you can even make $ from the printing. So a blogger in New Zealand with a small following is publishing her blog as a magazine for those who want hard copies.
A neat way to go after the long tail using the web.
Russ Daniels of HP described MagCloud as an example of creating value using cloud computing, in today’s all day SDForumCloud Computing event. BTW, MagCloud is an HP company.
At the RAS meeting, Zico Kolter presented his work on learning algorithms for LittleDog (a quadruped) for path planning and control, as well as some initial work on perception. In path planning, the problem is framed as solving a convex optimization problem on footsteps and foot placement. The human provides advice on where Littledog should step when it makes a mistake. Littledog translates the advice to adjusting parameters in the optimization problem. With 10 minutes of training, Littledog can plan and walk across new terrains the it has not seen before.
On control, the problem is framed as making LittleDog learn to walk fast on a specific complicated path with specific orientations. Walking fast involves having 2 diagonally opposed legs up in the air at each step. After 100 simulations by varying some parameters, the learning algorithm based on SVM reduces the problem from 98-dimension to 4-dimensions. This takes 2 minutes. LittleDog is then able to manueuver thru the path.
Of course, if we talk about LittleDog, we should look at the BigDog video. All automated quadruped. Some look at it as scary, others might think this is another drain of the world’s energy, … At the same time, what marvelous things man creates. BigDog:
My article on Pandora is now published on pp. 16-17 in the Oct/Nov 2008 issue of the SDForum Newsletter. It’s currently printed and available from SDForum. It should be archived online here by December 2008.
HYSTA held its annual conference today. Some quotes:
Milton Chang: “Give, to take.”; “Business is about relations.”
HP Jin: “In time of turbulence, stay calm, see the opportunities.”
Bill Huang: “Time … to incubate: cost to start business is low”; “Information industry is going thru huge shift in technologies (web, …, cloud)”
Ping Wu: ”If yon have money, you can buy a business.”
Hong Chen: “(Now is) best time to get people to join you.”
“(If you’re going to China,) sharpen your Chinese communication skills-Chinese have confidence now, …”; “When the storm comes, the stronger survives.”; “Business owner: be stingy; cash is king today.”
Last 50 years, CO2 in earth’s atmosphere from 280ppm to 384ppm
Jeff Wacker (EDS): “innovators are those people who know the 99 percent that everybody knows and … able to create the 1 percent that nobody knows. If you don’t now the 99 percent, … will not have foundation to create the new 1 percent.”
‘… true costs … becoming visible, … days of a “subprime planet” are over…’
“green is the new, white, and blue: … strategy … ease warming, …, energy supply shortages–and make America stronger …”
“green revolution is … innovation challenge — not a regulation challenge.”
“… bubbles … can drive innovation at a fever pitch” — Daniel Gross’ book
“Everything is connected …”
“Japan’s obsession with conservation stems from an acute sense of insecurity in a resource-poor nation …”
“We need to develop a system for clean energy generation … also need to develop a global strategy for the preservation of our forests, oceans, rivers, and endangered biodiversity hot spots, …”
“Though covering only 1.3 percent of the earth’s land surface, Indonesia’s forests represent 10 percent of all the world’s tropical forest cover and are home ot 20 percent of all the world’s species of flora and fauna, …”
“Think Small” is the advice Milton Chang gave to aspiring entrepreneurs. Focusing on a niche and delivering value there, tend to result in minimizing resources, competition, burn rate, and risks and maximizing potential. Once there is success, then expand and grow. This philosophy applies to new products entering new markets as well. This is the same message in Susan Friedmann’s “Riches in Niches”.
In 2 minutes, you can make phone calls to 3, 10, or 100 people, to ask them to come to a meeting, or to tell them about your new born. And do this for free. If you want to call 15 local plumbers for a quote, you can make that call in 2 minutes, for free.
http://techix.blogspot.com/2008/10/orkut-usage-stats-more-from-usa-log-in.html, also http://www.orkut.co.in/Main#MembersAll.aspx
The social network Orkut is hosted by Google, created by a Turk, and yet has more than 1/2 its users being Brazilians, followed by Indians, Americans; also very populated in Iran (now censored) and other Arab countries (some are blocked). BTW, Orkut is the 2nd biggest social network (MySpace is biggest), having over 120M active users. When I saw this in Ellen Spertus‘ talk at SFACM on applying similarity measures for a recommendation system for Orkut, I did slightly more research.
In my search for better ways to look at information on a computer screen, I came across Spacetime 3d. It’s a free program that integrates with IE on Windows which shows multiple browser windows over time in 3d. It can even tilt the browser windows at an angle to the screen. All type of neat views. Runs somewhat sluggish for me, on initial tries.
Unconscious, automatic mimicry (listener emulating the speaker’s gestures and facial expressions) in salary negotiations and sales have shown to improve financial results by 20%-30%. Mimicry is a form of Honest Signals, which are unconscious human behaviors that show the rhythms of interaction in a group of people. According to Alex Pentland and Tracy Heibeck in SMR, Fall 2008 (pp. 70-75), these signals can be measured with a “sociometer” device. The measurements can predict accurately the results of job interviews or salary negotiation or sales success. Honest signals include influence, mimicry, activity, consistency and these have associated brain structures and biology. The authors postulate that with more sophisticated statistical models and sensor capabilities, we would see the creation of a quantitative, predictive science of human organization, leading to the possibility of social engineering. Farfetched? Controversial? Maybe that’s why I do not see the article mentioned in the SMR web site. In any case, more info are here:
I guessed wrong in this article. I looked up on the web in late November’09 and saw that you just have to run the QuickGPS program in the Tilt and that downloads some data for the GPS and the GPS now works.
I just discovered today that AT&T has turned off the GPS in the Windows Mobile 6.1 upgrade for the Tilt. I upgraded my phone several weeks ago to the latest 6.1 and liked the new features: enabling user assignment of the PTT button to user defined function, and large time display. But, when I use Google Maps, the GPS does not work, whereas with Windows Mobile 6.0, it did. I did not see any clear statement from AT&T. I checked on the web and found many others have the same issue. I can only guess that AT&T has asked HTC to turn off the GPS in the 6.1 upgrade. Reason is that AT&T charges $10/month for its navigation service; and there was no reason for me to buy that while I could use Google Map and the GPS feature in Windows Mobile 6.0. (Do note there is an organization, HTCclassaction.org, that’s collecting evidence to sue HTC on putting out drivers for the mobile phone it makes.)
[If my guess is correct...]
As a product manager, I can see why AT&T would want to do this: turning a previously free feature (either enabled free by mistake or by intention) into a for-fee feature. But AT&T’s service would increase my monthly bill by 18%+, while the free Google Maps is sufficient. Yet, I do not see AT&T offering sufficiently compelling value to ask for such an increase in price. I believe that in order to starting charge $ for something that was free requires offering much more value, and must be followed by clear articulation/messaging (AT&T did not clearly articulate this feature/$ change). Otherwise, …
I’m revert back to Windows Mobile 6.0, giving up some of the new features in the 6.1 upgrade, to get back the free GPS feature.
Kai-Fu Lee, 李開復, spoke to over 280 people at the Google Crittenden campus today. He spent two hours talking and answering questions about what Google China is doing.
Some highlights:
Google strategy in China: 5,000 years–patience; humility– understand the users, Chinese internet user behaviors are different those in the US; build local expertise; advertising based business model
China might be more advance in development and application of cloud computing because of software/media piracy
Google creating Chinese specific tools, e.g. Google Chinese input IME that uses search results and latest search data to improve selection of Chinese words with same sound
KFL’s advice to those who want to do a startup: be hungry, i.e. you’ve no safe alternatives and it’s live or die.
I just read Kai-Fu Lee’s insights into parenting* and also saw his writing printed on a bulletin board in a school. So others also think these points are useful:
Develop self sensibility, self reasoning
not blind obedience
on discipline: set clear rules, but allow complete freedom within the rules’ bounds, set expectation on punishment for rule violation, minimize rules
to shape character, parents must set good examples
Cultivate skills of learning (4 realms of learning)
熟能生巧: thorough understanding creates new ideas; must know content, requires practice
举一反三: know, and know why
无师自通: master self study, no guidance needed
融会贯通: achieve mastery by applying to life
Nurture independence, with responsiblity (5 do’s and 5 don’ts; ‘he’='he/she’)
do teach your child “to solve his own problems”
do let the options be left to your child so he becomes master of of his own
do foster sense of responsibility
do foster curiosity
do trust your child
do not have too many rules
do not punish failure
do not preach
do not do his work for him
do not interfere too much
Foster positive self-confidence
Let your child be happy, let him follow his interests
Be your child’s friend (4 suggestions)
play with your child, act as he does
say what’s in your mind, listen
tell him he’s important to you, share your time with him
take the time to learn about the latest fads that your child would be exposed to
(Waiming’s opinion:) Let every child grow up positive, with sensibility and self-confidence, knowing how to learn, and ready for the world. How do we make that priority #1?
* The text is in Chinese, you can use Google translate to convert to English.
Enterprise search is emerging as an enterprise application in the last few years. The goal is to help employees of a company quickly find the right information buried in all the databases, wikis, files in the company’s network, while retaining the access permissions appropriate for each person, and securing the data from unwanted disclosure. In terms of the search process, it is a harder problem than web search, because there is less information to aid the search, so some of the web search techniques like PageRank would not necessarily help. There are many players, some big ones, some less known. Here are the ones I know about:
Of course, IBM, Oracle, SAP, etc. are also involved. Timothy Chou of Openwater suggested that there would be an industry of new applications developed on top of search data, similar to what happened with SQL data in the last 20 years.
Warren Buffet wrote an opinion column in the New York Times. The essential message is: convert cash to US equities, as equities would appreciate from today’s low prices, whereas cash would depreciate due to inflation. He wrote in the paper:
“A simple rule dictates my buying: Be fearful when others are greedy, and be greedy when others are fearful,”
“if you wait for the robins, spring will be over.”
“Today people who hold cash equivalents feel comfortable. … They have opted for a terrible long-term asset, one that pays … nothing and … depreciate in value. … the policies that government will … prove inflationary … accelerate declines in the real value of cash accounts. …Equities will almost certainly outperform cash over the next decade…”
“Before one could manage a family, rule a country, or make peace in the world, one has to manage oneself” — this is the essence of a Chinese proverb – 正心、修身、齐家、治国、平天下.
Peter Drucker provided specific guidance on achieving excellence to manage oneself in this article that appeared several times on HBR, last in the January 2005 issue. The article came from Drucker’s book Management Challenges for the 21st Century, 1999. Key points:
what are my strengths?
it takes time to really find out, 9-months or more using feedback analysis
how do I perform?
am I a reader or a listener?
do I learn by writing or by talking?
do I work well with people or am I a loner?
do I work best as subordinate or as team member or alone or as a coach/mentor?
do I produce results as a decision maker or as an adviser?
Eric Jackson is a good writer; the book is a good read, as it reads like a spy novel, with intrigues that range from fierce competitive battles with eBay, cat-and-mouse with the Russian mafia, and the company’s continual struggle for survival. Read it as a historical novel, with some truth. Jackson made no mentions of the engineers and other key players at PayPal — either he did not know them or did not think they were important to PayPal’s story:
Sanjay Bhargava, a former Citibank exec, who figured out the world’s first low cost method of authenticating terrestrial bank accounts
Also, key decisions at PayPal were referenced by Jackson, but he did not appear to be present when the decisions were made. So Jackson’s explanations of the decisions should be read as reconstruction and conjecture.
The book gives a good account on how Confinity was formed (later merged with Elon Musk’s X.com to become PayPal Inc.) by Peter Thiel, Max Levchin, David O. Sacks, and Luke Nosek; and that Thiel recruited many of his former associates from the Stanford Review, a conservative newspaper he started at Stanford. This latter point speaks to how fortunes could be made from a person’s earlier associations.
The book is a good read and I recommend it, but read it along with Elon Musk’s response; and don’t believe everything you read, from either person.
Faraz Syed, CEO/co-founder of DeviceAnywhere, presented at Vlab. The company addresses the needs of mobile service and application providers to validate mobile application across the myriad of phones and phone networks at low cost. The mobile world is fragmented:
Thousands of handsets, while each handset is unique platform with its own hardware, OS, executive engine (J2ME, BREW, …)
Hundreds of phone operators, each with its own unique network
Each handset must connect to the phone network locally
However, on-device, in-network testing is a must for end-user experience in order to retain customers
DeviceAnywhere addresses the problem by offering thousands of actual phones located in the local networks (currently across 10 countries). The phones are remotely accessed via replicated on-screen interfaces. All physically accessible control on the phones can be remotely accessed. Subscription starts at $160 / month for 10 hours of access to this capability.
By offering this as a service using a shared infrastructure, customers can do validation at much lower cost than purchasing their own handsets and phone plans. Furthermore, customers can begin to offer their applications to remote locations.
50% of the business is SaaS-based, and interestingly, the other 50% consists of replicating portions of the SaaS infrastructure for enterprise customers. Syed says the company is profitable.
Just saw a lecture by Dr. Carl Hewitt. He posits an interesting perspective on cloud computing. He thinks clients are key to cloud computing for a number of reasons: (1) client devices will be very powerful in next 5 - 10 years due to multi-core technology and Moore’s Law, (2) data would be in the cloud, but also in the client for redundancy and security reasons. From that, he suggests that 4 virtual machines in clients would be significant and new features would need be added to the VMs to take advantage of ever more powerful compute capabilities. I noted that these VMs are associated with application development environments, so I researched the following table:
Google just recently introduced its own JavaScript virtual machine in the Chrome browser. Apple does not have its own virtual machine development environment, instead relying on MacOS and associated tools.
From this perspective, we’d see the competitive market places for software applications and web services revolve around these VMs/tool chains, and maybe other clients (such as iPhone and sensor devices) with significant market share.
Until reading this article in HBR, Nov 2008, I understood the sources of revenue and profit for free software or free web accounts were advertising, conversion to paying customers, or services. Then Sunil Gupta and Carl Mela explain the means to model the network effect of free (non-paying) customers, and the lifetime value of these free customers. This paper and associated articles present a more general and rigorous way to understand the impact of free customers.
Network Effect
Positive impact
Negative impact
Direct
(how a buyer attracts more buyers)
e.g. Xbox Live
e.g. Employment site or malls
Indirect
(how a buyer attracts more seller/other parties, which then attracts more buyers)
e.g. online game player attracts more game developers, then garner more players
e.g. advertisers want more viewers, but too much advertising could turn off viewers
Methodology:
Collect historical data on paying customers and free customers (authors’ model require 4-5 years of data)
Examine growth in number of both customer types by (a) marketing strategies; (b) direct network effects; (c) indirect network effects
Assess the monetary value of acquiring a new free customer at different points in time; computation model using dynamic system optimization (see 2006 article):
Adjust marketing strategy (e.g. pricing) and analyze different scenarios
Results of applying this technique in the example given in the article:
improved marketing operations,
catering more to free customers,
making a stronger case to investors
The authors refer to the two-sided market of free and paying customers. The concept of two-sided market provides a refined way to understand the customers of a business and the network effect of 2 distinct user groups on each other. Examples: credit cards have cardholders and merchants; operating systems have end-users and developers. Also, the authors said that social networks like Facebook have 3 or more players, so these would be multi-sided markets.
The methodology might not be directly applicable to startups. However, keeping in mind the customers in these terms and estimating the impact of network effect could guide the success of a startup.
From Mercury News, interview with Peter Chou 周永明 of HTC by John Boudreau:
“Q How do you feel about competing with Steve Jobs and his iPhone, and Research In Motion’s BlackBerry?
A In terms of business, especially for Chinese, we don’t always think about competition. We think about opportunity. So getting opportunity is good. What is more important is you need to be in an attractive marketplace and you need to have a competitive strength. If it’s an attractive marketplace, there is room for more people to be successful.”
Where should we focus a business? Opportunity first.
After reviewing the 51 blog entries published in the last 45 days, I’ve a better idea on the direction of this blog. Rather than just talking about anything that comes across my eyes, my blog will focused on the following categories:
doing: things I do or have done
technology marketing: my current profession, includes product management, product marketing
management: my future work, includes business strategies
cloud-computing: technology area for my work, includes virtualization
software: technology area for my work, includes web services, software development, “tin-wrapped software”
new-stuff: on the lookout for the next big thing
I’ve adjusted the categories and the tags for the published blog entries and the stats are in the chart. Looks like I have more blogs to do for some of the categories.
Just setup a free account on Evernote. The service allows me to put notes in the cloud. Notes include:
typed notes
handwriting
pictures
voice
I can enter the notes from my cell phone (Windows) or PC. It has an app for the iPhone as well. Once the notes are on line, I can search by dates or by word. Could be very useful. Edmond Hui pointed me to this service.
3 issues which if resolved, could make me a continual user:
Speed. Even on the 3G network, sometimes the updates are slow. Also, the app switches with IE on the phone.
No easy integration with Microsoft OneNote. I use OneNote on my laptop. OneNote for Windows is a good tool, whereas OneNote on the cell phone is not very useful. Evernote would be a better alternative on the cell phone. Now if Evernote and OneNote could have the same respository.
Trust. It’s not clear if EverNote do anything with my personal notes. For some reason, I feel easier to put data on Google Docs. Maybe there is so much data on Google that mines are but unimportant traces.
As we consider the premise that the cloud becomes the keeper of data and applications for consumers, PC and smartphones are the likely dominant clients to access the cloud. The question is which OS for smartphones?
Let’s simplify the market analysis by only looking at the clients and their OS and avoid the complications from mobile carriers, dumb phones (3.3B mobile subscriptions WW in late 2007) and other issues.
There are 4 types of clients: PC, thin-client, Smartphones, new or specialized device (e.g. Amazon Kindle). Thin-client shipments are still small: 0.78M units in Q4-2007, growing at 15.2% YTY (source IDC).
POINT #1: PC and Smartphones are the likely clients. PCs have more than 2X the volume of Smartphones and both grew about the same rate in Q2-2008. There is always a chance some new device or specialized client could come into large volume.
World-wide Q2 2008 unit shipment (not installed base):
I reviewed the PDC 2008 presentations and some webpages on Windows Azure. The messages are:
internet scale cloud platform hosted at Microsoft data centers
reduce up-front server infrastructure purchase, reduce expertise needed to setup and maintain server data center
quick application deployment in the cloud for developers
if you know how to program in .NET and Microsoft tools, you can use this now when you get CTP access
more is possible when the enterprise data center integrates with the outside world thru the cloud
just the beginning, more features to come (scalability, HA, …)
Good demo by Chris Anderson and Don Box putting applications on Windows Azure. The Bluehoo demo reminds me of some toy and does not look like a killer app (as claimed) for the cloud.
While Microsoft does not offer the capabilities of Google nor Amazon in terms of scalability and running Linux based Hadoop, this is a good defensive move by the compnay. Now Microsoft developers can be kept on Microsoft platforms and have cloud too, no need to move to Google or Amazon.
With flattening of the world, rapid technology changes, and social relationship changes between companies and society, Dr. Yoram Wind called for the marketing discipline to adjust to today’s realities and he suggested strategies to increase rigor and relevance to marketing [ SMR, Summer 2008 ].
Marketing leveraging the groundswell of customers in blogs, social networking sites has emerged as an important tools for marketers. Josh Bernoff and Charlene Li in SMR, Spring 2008 described some examples of the good use fo groundswell:
Salesforce.com developed IdeaExchange enables customers to vote on feature ideas. The company doubled the number of product release to four in 2007.
Fiskars was able to combine customer advocates (“ambassadors”) for their paper and crafting supplies and tools and online discussions to increase sales at stores. It turns out that their customers (crafters) think their craft/hobby is more important than life and death.
Dell built a team to actively track blog posts to find distruntled customers and reach out to these customers to proactively address their service and technical support issues. This team, along with straight forward communications on product issues, has helped Dell weather thru issues like the flaming laptops.
According to the authors, marketing to the groundswell has its risks. But appropriately applied, it significantly helps companies advance objectives in R&D, marketing, sales, customer support and operations.
Started to learn about web analytics; first book is Actionable Web Analytics, by Jason Burby and Shane Atchison. The key elements to web analystics from the book are summarized below:
Web analytics is collecting data about activities on a web site and analyze the statistics and conduct hypothesis testing
Just did some hands-on installation of VMware ESXi and a virtual machine. Works fine, easy and straight forward. Below is an initial comparison of ESXi vs an installation of Microsoft Hyper-V. I do not have another system with AMD-V or Intel VT, so I have not tried the latest Xen – maybe it would be quicker if I just get on EC2.
ESXi
Hyper-V
Hardware/OS requirements
Runs fine on an old AMD 180 system with 2GB of memory. Installed easily on bare metal and a blank SATA disk.
(for me) I had to buy a new AMD 5200 system with AMD-V, and had to upgrade from 2GB to 4GB to in order run sufficiently fast. I installed Windows Server 2008 along with Hyper-V
I just signed up for Fitness 19 for $9/month, with a $149 enrollment fee. That’s a great deal to access 11 gyms in the Bay Area. It does have lots of elliptical and various training equipment. I needed to either buy a replacement elliptical or find a gym. A replacment elliptical cost between $500-$1K, that would be 3-8 years worth of Fitness 19. Thus Fitness 19. This would augment my 2 times a week at the pool, each time 1,800 yards (> 1 mile), and occasional fencing.
Some interesting marketing techniques used by Fitness 19 (from the guy who signed me up):
The gyms are located half-way between two 24-Hour Fitness gyms as the founder of Fitness 19 came from 24.
The prices are lower than 24-Hour Fitness, in order to cater to the more price conscious segment. The product offering is less in that there are no basketball court, handball court, swimming pool, or shower (some of the affiliates to Fitness 19 have showers). While not 24-hours, it’s open 5a-11p on weekdays and 8-8 on weekends.
Fitness 19 offers a permanent 10% reduction in the monthly bill if the customer refers another customer to the gym. So if were to refer 10 people, then my bill would go from $9/month to 9 * (90%)^10 = $3.14/month.
And of course, the $9/month promotion for the first 100 customers to sign up November is a great way to bring in prospects.
I had the opportunity to sit in a breakfast session with Ashok Narasimhan, who founded Wipro IT and VeriFone and other companies. In his response to a question regarding VC funding in today’s financial crisis, he said that from his perspective as investor, he’s seeing the following:
VCs would tend to fund more series A than later stages. The rationale is that there could be higher chance of success with new startups than with startups that have been around a few years.
With the advent of cloud computing, the funding dollars for many startups (e.g. web 2.0) are less and as the result, VC would need to find more companies to fund in order to use their available cash. VCs would be managing many more deals than before.
Adam Coates of Stanford presented at the IEEE RAS meeting. He has his colleagues, including Pieter Abbeel and Andrew Y. Ng, used machine learning to enable a 2-processor computer to quickly learn complicated flight patterns of large model helicopters controlled by a skilled pilot. The model helicopter pilot took 20 years of learning to perfect some of the complicated and aggressive flights and the computer was able to learn it in a few hours. Thereafter, the computer can repeatedly control the model helicopters thru the same flight pattern. Adam indicated that they looking into applying these techniques for other types of robot learning tasks.
Larry Page and Kevin Martin talked about wireless boardband and the need to use white space spectrum for faster wireless internet access at the WCA Symposium. The increased use of the internet with faster wireless access (initially in the US) would push more data and applications to the cloud, and help Google generate more revenue. Page dismissed broadcasters’ claim that the use of white space would interfere with TV signals by saying that technologies would solve that problem, also that wireless microphones, which are unlicensed while overlapping with the boardcasters’ spectrum, today do not pose that issue.
More wireless broadband => more data / applications in the cloud.
At the Web 2.0 Summit, John Doerr suggested the following advice for entrepreneurs in face of the economic downturn. I think many of these points are very applicable for individuals in planning to face the unknown tsunami of economic downturn that’s silently coming after the credit and financial earthquakes [conjecture].
Act now, act with steed (get a loan, secure financing, make cuts, …)
Protect the vital core of the business
Cut once, cut deeper than you need to (don’t do a second time) – make sure you have 18 months of cash, on a conservative revenue forecast; ask for a convertible note from investors
Defer facility expansion (don’t spend money on facilities, no building, no software, use Google tools, …)
Re-evaluate R&D
Renegotiate any existing contract (lease could be re-negotiable)
Everybody ought to be selling (selling company’s value prop)
Offer people equity rather than cash bonus
Secure cash in treasury securities, e.g. government-backed treasuries; people do not know what’s going on — another big bank could fail
For revenue plan, figure out what the leading indicators are, such as customer engagements, and plan conservatively and realistically based on these analytics
Over communicate with everybody–do not sugar coat; let people know where things stand; while cutting things, do not cut hope
Copyright (c) 2008 by Waiming Mok (above points are all John Doerr’s)
In an emerging market, there are many entrants, offering their own solution. Same with the cloud. Each of the offering today for cloud computing are designed specific to the vendor’s way of doing business and as the result, forcing lock-in for the users.
At SiliconVally codecamp_08, Vertigo demonstrated a Microsoft Surface table and ease of programming using WPF. Microsoft Surface uses 5 internal cameras looking up at the table surface to detect reflected infra-red. This technique enables multi-touch from several people. The harder you press your finger, the bigger the image on the surface and the cameras and software interpret that as increased pressure on the surface; and the system can respond accordingly. Microsoft Surface can interact with anything that reflects infra-red, so it can see a pencil or writings on paper, as graphite is a good reflector of infra-red.
People can interact with Microsoft Surface from all sides, images on the surface can be rotated in any orientation, so it’s great for several people to interact on the device.
If you know WPF 3.5, then Microsoft Surface is easy to program. Vertigo demonstrated a quick application by just adding a few lines of code to a generic WPF framework. In 5 minutes, a directory full of photos are displayed and users can manipulate the photos on the surface table.
Price is over $5K, so not for the home yet. AT&T is installing Microsoft Surface at its retail locations.
The SecondLight demo at PDC 2008 showed additional devices combined with Microsoft Surface to add more dimensions to the user interface.
So, Microsoft Surface and new devices like the Apple iPhone are beginning to change the computer user interface from the almost 25-year old version consisting of a vertically oriented screen, keyboard and mouse.
Tim O’Reilly and John Batelle have been saying ”Web Meets World” this year to described the convergence of web 2.0, wireless, low-cost internet devices, and sensor technologies, and the resulting emerging applications. Tim O’Reilly describes an aspect of this as the instrumentation of the web. This is a very broad area, and there are many emerging devices for the instrumentation and resulting applications, including military deployment. Some examples are:
At the Web 2.0 Summit, John Battelle introduced Michael Pollan, who suggested a way to look at the global system in terms of the food system. He says that the current government priorities on energy dependence, climate change, health care crisis are all dependent on the food system. At the same time, there is an inherent conflict between the food business and the healthiness of food based on today’s system: profit is increased when food is more processed, while more processed food is not healthy. The system requires rethinking and rework. Michael Pollan suggests we need to change the rule of the game by changing government policies.
At the DM SIG of SFBay ACM, Rion Snow spoke about using Amazon Mechanical Turk as a cheap and fast way to do various tasks, such as collecting annotations, to train machine learning and natural language recognition algorithms. He was charging as low as $1 for 1,000 labelling tasks.
MTurk shows Amazon’s foresight, along with EC2 and other services. MTurk was established in 2005 to leverage the idea of crowdsourcing, where people online can request specific tasks to be done (such as doing paper review and editing) with a payment. Other people in MTurk would just the task and get paid. Most people get paid on average of $1/month, while some really diligent folks could get up to $100/month.
Rion indicated that the result could be made close to experts could produce, at low cost and fast, provided some of the biased/noise data could be eliminated. The biased/noise data include those that were generated by a few people who were just doing the job for the money (conjecture on my part).
Premise: There is need for higher network bandwidth in server clusters. This is driven by Moore’s Law on the processor side; this would be evident in the next 2 generations of processors from companies like Intel. 10 GigE is the next jump from Gigabit Ethernet. Costs would come down as the chipsets are integrated into server motherboards over time. The switching infrastructure would correspondingly decrease in costs with higher volume. The desired target cost for the entire switching infrastructure is in the very low 3-digit $ per server — not there yet.
Value-add: (a) hardware integration with low-power, high density 10 GigE ports as network fabric. (b) software architecture that’s standards based, modular, and extensible. (c) a bunch of smart people, with track record.
Software architecture: based Fedora linux without any change (to avoid issues with GPL license). Switch software is entirely in user space, running in processes that do not own states. States are stored in a transactional based data structure. The architecture is modular and extensible.
Andy indicated that transition to 10 GigE would take time, as in the history of 100BaseT to GigE transition, it took 4 years for the complete transition to GigE on server motherboards.
The trend line does point to this transition to the next level of network infrastructure. Whoever get established as key players could reap big rewards.
As part of the investigation for an event topic for VLAB, I started looking into government money (such as the $700B financial bailout dollars) instead of private capital for fundings of new products and services. Not sure where that would lead. Using Freemind, I’ve captured an evening’s investigation into a mind map:
As part of the VLAB investigation, some folks suggested into blogs related to Chris Larsen. Looking thru some of the blogs on Proper.com, I have a better understanding of peer-to-peer lending. This could be another source for fundings for new products and services, provided people have money to lend out.
I could not find specific blogs by Chris Larsen. Also, there is some quiet period for Proper.com, so much of the commentary by him have been removed. I did find audio of interviews with him as well as blogs on prosper.com on the topic of peer to peer lending. So I started a mind map on the topic related to him.
First day of Mashupcamp was held at the Computer History Museum today. Companies are coming from all over the US: Florida, Maryland, Massachusetts, New York, Utah, Washington, …, and the local Bay Area. This is an exciting facet of the cloud. Data and applications can be put together and new applications and different views of data can be constructed very quickly.
One discussion centered on whether standards are needed, when many APIs could be used as-is. There are many interesting mashup tools:
While manning the IEEE RAS SCV booth at Robo Development Conference, I also had the opportunity to listen to Sebastian Thrun’s keynote. He spoke for some time on the 2 DARPA Grand Challenges, Stanley and Junior. Toward the end, he made a call to action on changing the way we build cars to address the issues of traffic fatality and inefficiency of the current designs of the automobile. The problems and inefficiencies include:
Traffic fatality in the US costs 2% of GDP, at 42K deaths and 2.7M injuries each year. Traffic is the leading cause of death for ages 3-33. But theonly response to vehicle safety is bigger vehicles (SUV), which is inefficient.
We average 1.25hr / day in traffic.
Of the 660 million cars in the world, the average utilization is 3%. For much of the time, cars are parked.
22% of the nation’s energy is used by cars.
30% of the energy used by each car is wasted on extra weight added for safety reasons.
At peak capacity, only 8% of the highway’s capacity is used, spacing is required to ensure human drivers have enough distance between cars. (What if fast moving cars could be paced closer together without accidents?)
Dr. Thrun suggested that the above problems could be addressed with new innovations involving more sensors and intelligence to make cars run autonomously. He called on engineers to rethink how cars are designed and built, as well as how they are used and shared by people.
According to Tim O’Reilly at the Mashup Camp, the trend is moving toward an “operating system” for the internet. Rather than relying on the desktop or laptop to provide applications and data, the internet would become the platform to provide these services. Thus the rise of the likes of Google. O’Reilly described the fundamentals of this internet OS are driven by data. Web 2.0 is a manifestation of this evolving internet OS, where (according to O’Reilly), “data is the Intel inside”. Data is the lock-in point for anyone developing the Internet OS platform, like Intel is the lock-in and source of value for the PC.
O’Reilly sees 3 data sources that would offer great value for this Internet OS. Whoever builds businesses around these would come out the winner. These are:
namespaces, which are unique, reliable keys used to look up actual data
primary data
derived meaning
O’Reilly gave several examples for these data sources. It was another thought provoking speech.
Take 2 or more photographs from different perspectives of an object or a scene, Microsoft Photosynth can build a realistic 3-dimension model of the object or scene. This is what Michael Bruch of the DoD demonstrated at the Robo Development Conference. The 3D reconstruction could then be used in visualization or simulation, such as training a robot to manuever thru real life obstacles.
Photosynth is usually presented as a way to stitch together photographs so you can get a more emersive 3D experience from 2D photos. The photographs could even come from different sources on the internet, so there are potentially leveraging and social aspects.
Photosynth is computational intensive, so could be a good application for the compute cloud.
To me, Photosynth demonstrates a converging application of the storage cloud, compute cloud, and the processing of the data to derive meaning. The derivation of meaning from data is called out by Tim O’Reilly as a key to success of an internet endeavor.
Just saw my article on “Cloud Computing and Beyond” published on the Dec/Jan 2009 SDForum Newsletter. We should see the article on-line by mid January here. In the mean time, we could get a hardcopy from SDForum.
I heard someone said the semiconductor industry is forecasting up to 20% contraction in 2009. The SIA is projecting 5.5% contraction. Semiconductor is a likely leading indicator for computer sales, consumer electronics, which would be leading indicators for software and internet advertising. I spoke to another person involved with networking and his (large) company is projecting up to 10% contraction in 2009. It seems companies are forecasting reduction in revenue and thus are planning headcounts accordingly.
As an individual with no impactful control on this situation, I’m approaching this situation as follows:
plan and optimize use of budget and resources
network to build relationships, best way is to help others whenever I can
Nassim Nicholas Taleb (in Fooled by Randomness) presents a more interesting observation of how the world works. Build a sand tower, keep piling sand as high as possible; reach the heaven if you can. There is a point when adding one more grain of sand would crumble the tower. He uses the sandpile to illustrate the concept of non-linearity in nature and the onset of the tipping point. A small change could impact an enormous structure built over time. No easy way to deal with these situations; one means is to be conscious that these things happen, have an open mind, and be ready to adjust accordingly.
By the way, the ancient towers could only be built so high before the bottom layers of the structure would be crushed by the weight on rest of the structure above the layers, causing the crumbling of the entire structure. Gravity at work.
In the mean time, the pictures of Tower of Babel and sandpile remind me of the various software architecture diagrams people put up…
By understanding that hidden non-linearity and tipping points exist and seeking them out, as man is building out architectures, software, building, or other structures, man could continue to build and grow, leveraging these whims of nature.
Just finished reading Nassim Nicholas Taleb’s Fooled by Randomness, 2nd Ed. Taleb illuminates simply, with stories, that reality is embedded with randomness (not predictable), nonlinearity (not easily forecasted using past data assuming a straight line) and skewness (not symmetrical), yet we humans often get confused by our biases and mis-apply statistics to describe reality. Experts could be multiply fooled by the same biases and mis-applications.
We confuse what’s noise and what’s signal; and our news and social sciences (in attempts to apply scientific methods incorrectly) inject more noise into the system. On top of these are rare events, some black swans, that are rare and happen unexpectedly. The bad ones cause havocs in the “normal” progression of life, while the good ones could reap great benefits to those who could catch them. But these rare events could not be foreseen.
We don’t know “what we don’t know”, whilst we think we know. According to Taleb, success is often the result of luck, or randomness, but we attribute success to one’s skills. This is especially more prevalent with professions that involve randomness, like securities trading, or upper level management (big decisions are make infrequently and the consequences of the decisions are not seen until years later). Even a streak of successes could be the result of randomness if the sample size of people in the profession is large. Only repeated actions over a long time would one really distinguish between luck and skills. Such is the distinction between a securities trader and a dentist.
Biases:
hindsight bias – overestimating what one knew at the time of a rare event due to subsequent information: “I knew it all along”, when we did not see the rare event coming
selection bias - misinterpretationof the statistics from sample selection
attribution bias – cognitive bias that affects the way we determine who or what was responsible for an event or action
survivorship bias – the tendency to excluded failed companies from (finance) performance studies because they no longer exist
As Taleb is a voracious reader and reads in many fields, he injects lots of ideas into the book. Most notable is that Taleb reveals himself to be a disciple of Sir Karl Popper, who challenged the singular reliance on observations and inductions to form scientific theories — any theory could be overturned by just observing a counterexample that was not previously observed. This last point hints at Taleb’s Black Swan, described in another book of his. There is much to grasp as one reads Fooled by Randomness.
I highly recommend this book, as Taleb challenges us to be vigilant of what we don’t know, and maybe on occasions, be less a fool to randomness. Very insightful, yet readable.
I heard a speech by Frank Esposito about getting real, focusing on the now, and committing fully. He said mental toughness, believing in yourself, being inspired, and inspiring others are key to fulfilling one’s life’s callings. What’s more, Frank suggested one action that I think really works: “Do something physical and do that mindlessly.” E.g., if you had only walked 3 miles, walk 5, …
I did that once. For 2 years, I could only swim 20 laps (2×25 yards each lap) non-stop and I felt very tired and I had to stop. Then one day about 2 years ago, I decided to push beyond 20 laps. As in the past, I began to feel tired after 8 laps, then a headache came on soon after. But by lap 16, I noticed I could start to ignore the pain. Once I passed lap 20, I kept on going… 24, …, 28, …, 30, 31, 32, …, 36. I was able to sustain myself till lap 36. I swam 1,800 yards, slightly farther than a mile, non-stop. I’m now swimming 36 laps (2×25 yards/lap), non-stop, when I could get an hour in the pool.
I was able to ignore the pain because something seemed to be emitting from inside my head. This was endorphin at work, easing pain and instead bringing a feeling of exhilaration. This allowed me to persist to 36 laps.
Other quotes from Frank:
“Belief without action is self-delusion.”
“There is only 100%”, there is no such thing as putting in 110% effort.
“Get real”.
Say the word “Cancel” to remove negative self talk.
If you ever get a chance to hear Frank speak, I’d recommend it.
The beginning and ending stories in Fooled by Randomness remind me of a story I read in grade school:
In 16 Chinese characters and dating 2 millennia, this story tells of an old rancher in Northern China.
One day, one of the rancher’s horses ran away. The rancher’s friends said: “We are sorry for your loss”. The old rancher replied: “How do we know that’s not good luck?”
A few days later, the horse came back and brought a stallion with it. The rancher’s friends then said: “You are lucky.”. The older rancher replied: “How do we know that’s not bad luck?”
A few more days later, the rancher’s son rode on the stallion, fell, and broke his legs. The rancher’s friends again said: “We are sorry for your son and your bad luck.” The old rancher again replied: “How do we know that’s not good luck?”
A few more days later, a northern tribe of warriors came to take young men as soldiers for their war. They did not take the rancher’s son, as he could not walk…
What impressed me is the old rancher’s attittude on gains and losses. You don’t know what these gains and losses would bring. This Taoist view of the world resonates with Taleb’s book.
I was trying to get all the PDF files a conference put their web page, without scrolling down the page and clicking through each link. I heard about Yahoo!Pipes, looked into it, setup a pipe and that solved the problem of getting a list of PDF files without the rest of the content on the web page. Then I generalized the pipe to get any document type.
This is my first try at Yahoo! Pipes. The pipe extracts a list of files of a type you specify that appear in a web page for easy download. For example, if the web page has a list of PDF files, the pipe would only list all the PDF files on the web page for easy downloading, eliminating the rest of the content from the page. You can specify any file type (PDF, MP3, JPG, DOC, PPT, …) and this Pipe would list only those files with that type. You can also feed this pipe to a RSS reader and if the page changes with added files, it would send you an RSS feed notification.
It took me a while to get this working. After numerous tries, I finally used an example pipe that turns a web page with MP3 into a Podcast feed.
I applied it to the Yahoo!pipes page I created yesterday, using a filter for the file type, then the firefox extension prompts once on where to download, and it automatically does the rest. It should work without the Yahoo!pipes.
Go to a web page, the computer running the web browser scans through the web page and bring out potentially key information for you, automatically. That’s what Thomson Reuters’ Opencalais is doing. In its easiest deployment, just load the free Gnosis plug-in for IE or Firefox, add additional web pages for semantic processing.
The Gnosis plug-in scans through the web page you browse and creates lists like “cities”, “companies”, “industries”, …, that it thinks are relevant to you, based on semantic analysis of the content of the page. I find the list of companies very useful for the search for “cloud-computing” in wikipedia.
There is another plugin, Tagaroo, designed for stand-alone wordpress blog server. Tagaroo automatically uses Opencalais’ semantic processing to suggest tags that would be appropriate for a blog entry, saving you time from having to figure out what tags would make sense for the entry. However, this plug-in is not currently available for wordpress.com that runs this blog; so I still need to spend a minute to do my own tags.
There are many tools, including mashups, interface to Yahoo!Pipes, …, that are based on the Calais Web Server. Calais illustrates a powerful application of cloud-computing, where the infrastructure is running computing intensive data analysis (semantic processing) on web data content, thereby helping us to sort through information faster.
Clusty is another search engine. What makes it unique is that it groups similar results together (clustering), so it helps you get to the what you need to find quicker.
Using the remix feature, clusty generates more groupings, in case the initial set of groupings are not what you are looking for.
Clusty leverages results of the other search engines, such as Google, Amazon’s A9, Microsoft Live Search, …, to do the web crawling and “raw” searching (e.g. tf-idf, link analysis, etc.). Instead, Clusty, aggregates the results of these search engines, and generates an ordered list based on comparative ranking to present as search results.
Clusty is another illustration of applying compute algorithms (clustering) in a cloud-computing infrastructure, leveraging other compute (search) and data (web sites) capabilities on the internet, to help people find information faster.
I wanted to setup a single news feed that would aggregate the business, technology, and science news from Google, Yahoo!, Microsoft Live, Business Week, Scientific American, Science News, and then list the top 20 of the latest news items on the right hand side of this blog.
So I setup a Yahoo!pipe to take these feeds and aggregate them into one feed. Then I use the “unique” function to remove redundancies and then sort by the latest available news and only display up to 20 of the latest news in these areas.
The output of the Yahoo!pipe is fed to the WordPress’ RSS widget, which is on the right hand side on this blog. However, 20 news entries would stretch too far down the page, so I limited to only display the top 5 news item on the right hand side of this blog. Even this is too long.
A delegation headed by the City of Wuxi visited Silicon Valley. On Dec 1, the delegation held a ceremonial signing of a MOU between IBM and the City of Wuxi for a long-term strategic cooperation for IBM on software development. This follows IBM’s previous establishment of its first cloud-computing data center in Wuxi in early 2008.
In addition, the delegation from Wuxi also signed a number of agreements with other companies. A day earlier, the City of Wuxi also signed another agreement with Microsoft. Even in today’s economy, Wuxi is pushing for growth of its outsourcing development.
In addition to being one of China’s economic development zones, Wuxi is a major tourist stop, with attractions including Lake Tai… With the bullet train, it takes 45 minutes to travel from Shanghai.
Other than IBM, Microsoft, Wuxi claims many other companies who are outsourcing there.
Just finished reading Part 1, first 133 pp., of NNT’s Black Swan. Teleb presents his observations and conjectures on how we humans are not equipped to see and learn from the (1) rare events that have (2) significant impact and we tend to (3) ignore these events. These 3 points define the Black Swans. For me, Teleb jogs me with very interesting set of ideas, while prompting me to question what I do, how I think and interpret the world. As he presents his argument with stories, some real-life, some with names changed, some thought experiments, it’s hard for me to say that he’s scientific in his approach. At the same time, Teleb suggests that traditional scientific approaches in social sciences (e.g. economics, or looking at things like Black Swan) have too simplified models (e.g. Bell Curve) that do not reflect reality (more Mandelbrot like). From reading Part 1, I gain new perspectives and a recurring pinging in my mind to question how I think and look at the world. Worth reading.
There following are some interesting concepts from Part 1:
Teleb says that Black Swans are invisible to most humans because we are not equipped to think about randomness, and ignore what could have happen, as it is hard to see and think about such abstractions. Experts often use simplified models like the Bell Curve erroneously to describe what we see and then derive decisions from these models. Sometimes, by luck, one among millions, could end up successful over a long period of time, but could still hit the Black Swan because it is not in model. And the Black Swan wipes out all the gains that were gotten from the previous successes.
Teleb tells a story of casinos spending millions to manage risks with gambling and monitor cheating with sophisticated security equipment, and yet for the Mirage Casino which lost hundreds of millions of dollars because they lost Siegfried and Roy due to an unexpected tiger mawing of Roy.
World environments are either Mediocristan (non-scalable, winner takes a bit … ) or Extremistan (scalable, winners takes almost all … )
Humans think in 2 modes: System 1 (“intuition”, effortless, automatic, fast, opaque, …) vs System 2 (“thinking”, effortful, reasoned, slow, logical, …, self-aware)
“Most of our mistakes in reasoning come from using System 1 when we are … thinking that we are using System 2.”
The world is more nonlinear than we think. E.g. you play tennis every day with no signs of improvement, then after years of playing, one day, you start beating the pros.
Don’t be like turkey. A turkey is fed every day for 1,000 day — the turkey would be thinking the 1001th day bring food again, but on that day, the turkey goes on the Thanksgiving table.
The dead and failures are silent. We tend not to count them, when we look at statistics of those who have succeeded, and then start attribute what led to these people’s success are due to what they did rather than randomness…
To deal with these biases and innate handicaps, Teleb suggests that we look at reality, avoid interpretation based on our simple models of the world, be always skeptical, and keep an open-mind. For, to think out of the box, especially to catch a Black Swan, you cannot be stuck inside the box, where Black Swans could not be seen.
The modern web architecture is built using the Danga stack that was first used to construct Livejournal.
The web architecture consists of a long chain of components, starting with the browser, to the web site’s front-end proxy, to the web server logics, to the backend behind the web servers. The backend may not necessarily have databases, but could consist of long chains of services. Many of the services are offered by different vendors. One service could call on another, down the chain. Example:
I saw today a group of people really focused on extracting $ out of offering their services, many of which are moving to the internet and mobile. These folks at the SDForum Gaming event, speak in earnest about the business using these terms:
talent and strong teams
indy communities vs established vendors
deriving fun and bring about great experience of playing
social, story telling, mission, leveling, goals, …
content
…
It’s a serious business in the world of gaming and there are a lot of smart people dealing with rapid changes in this intriguing world, including
Emergence of the iPhone as a platform. The iPhone is turning out to be the entry point for new games, as developing on the iPhone is quicker than other platforms. Even at $4-$10 per download, a game could be run as a profitable small business. In addition, the iPhone is causing the untethering of users from the consoles (Playstation, …)
Social network and gaming are colliding, whether they become partners or competitors, time will tell
Converging interests in games from TV, movies, music; also from the military and NASA (use games to encourage STEM)
Business models are changing, with internet distribution impacting retail sales of game packages, use of adversement, and the rise of micropayments and virtual economies
Micropayments are points used in the transaction of virtual goods and contents and services created in the game worlds, and now in social network, like Facebook. Many companies (e.g. Twofish) are already there or trying to get there to provide the framework in the backend operation of the virtual economies. Other companies (e.g. Offerpal Media) are there to deliver the infrastructure to manage consumer offers, targeting in social networks and game worlds.
One of the most successful company pioneering micropayment is Tencent, the progenitor of QQ. Tencent is a public company with $300M in annual revenue, offering gaming and chat and various online services, mixed with advertising and micropayment virtual economies. It made its success in China in the past 10+ years. Now, the micropayment model is coming to the US.
According to some, micropayment could be a way to solve the free-loader problem in the use of the internet. We’ll see in time.
On Dec 5, I had the opportunity to talk to Dr. Carl Hewitt at the SDForum New Media III Event, and we spoke about cloud storage. He said that’s going to be a big deal. That would fit into his idea of client cloud computing. He said one issue is today’s cloud storage is “blob storage”, no cloud-based (generally available) federated database and no JOIN’s on the data set. That prompted me to look into what’s available in terms of distributed database options:
The Long tail is the concept that there are large number of customers who’s unique needs (in small quantities) are not addressed by the high-volume products. It is often represented as a the power law distribution curve. The idea goes further with internet technologies and other happenings, which lower costs for product development and/or distribution. Over time, the demand curve could shift away from the high-volume product, making the tail longer and fatter.
The Fat tail is a different concept where the probability of events away from the mean might be significant. Traumatic events (e.g. oil shock, political turmoil) could disrupt the well-behaved mathematical models of financial investments, incurring considerable more risks than estimated. The variance might not be finite. In some situations, achieving six-sigma might not provide the comfort of high quality or the feeling of being safe.
Both concepts are part of the evolving maturation of our outstanding of uncertainty and risk. The world may not be simply be modeled by the bell curve and/or the Central Limit Theorem.
As part of a VLAB project, I had to make calls to folks who do not know me. The uncertainty of the type of responses I’d get and the possibility of rejections made these calls very hard. But I had to make the calls, lest my part of the work would not be done.
So here’s what I did to make the cold calling easier:
Wrote out a list of people I’d be calling, with phone numbers.
Wrote a one-paragraph script for I’d be saying. The script is a concise elevator pitch explaining what I am trying to achieve. The following elements are included: my name, whom I’m representing, why am I calling, how much time I’m asking, benefits to the person, my contact info, next steps. All should be 2 minutes or less.
Practiced the script briefly.
Made the calls, with smiles. The first call was like jumping into a cold swimming pool. The subsequent calls were easier.
I did make the calls and mostly left voice mails. At least the calls were made.
In parts 2, 3, 4, Taleb went into further details on actions and decision making in face of the unknown unknown. He contrasts the lopsidedness of how things work and how Black Swans changes things. Winners tend to win more often. The winners might have gotten there by luck, not skills. So life is not fair. At the same time, due to Black Swans, those on top could be toppled quickly, unexpectedly. In that sense, the world offers opportunities for the little guys.
Central to his theme is that we tend to look at the world by creating or believing in theories to fit the world to the theories, rather than looking at what’s happening. This could apply to math, science, religion, history, and especially more so with economics and social sciences. And, when economists apply flawed theories to real world finance and governance, dire consequences could happen.
As we do not know the level of risk and extent of uncertainty of the future, Taleb suggests the following approach to deal with that uncertainty, to avoid the negative Black Swans, and to increase the likelihood of catching positive Black Swans:
Be prepared for all relevant eventualities
Barbell strategy:
Be extremely conservative (very, very safe) with 85-90% of your assets, in your approach
Be extremely, wildly aggressive with 10-15% of your assets
Modest tricks:
Make a distinction between positive contingencies and negative ones
Don’t look for the precise and the local. Don’t be narrow minded, be open to possibilities.
Seize any opportunity, or anything that looks like opportunity.
Beware of precise plans by governments.
Don’t waste time trying to fight forecasters, …
Taleb also writes an entire chapter on his mentor, Benoit Mandelbrot, who introduced fractals to the world. Fractals have scale-invariance, which is an interesting property that Taleb says is prevalent in how we compare things. Further, Taleb says looking at the world in terms fractals could make Black Swans gray.
Taleb speaks like an outlier, not part of the mainstream. That’s what’s appealing about the book. He forwards arguments that make the mainstream and those in power, as mostly the result of randomness. After reading the book, I feel less afraid of those in power. In the end, they are not that different from you and I.
Taleb practices an open minded approach to view, think, and act about the world. You have to read the book to get that approach. He drills these notions into your head. The last advice (don’t run after trains) in the book is especially uplifting. Read the book.
Just going thru this book “The McKinsey Way” by Ethan M. Rasiel. It’s a quick read, with some good points about approaches to solving business problems:
Beware that books, games don’t necessarily apply to the real world
Avoid noise, usually generated by news and media
Don’t be the turkey: future projection using history could be detrimental
We do not know much about the past, especially causes and effects — “look at a puddle of water, … and try to reconstruct in your mind’s eye the shape of the ice cube it may once have been.”
We just can’t predict, esp. Black Swans
Work to be a skeptical empiricist, for the brain tends to operate using intuition, making mistakes relative to reality, and the brain is not built to deal with Black Swans
Think fractal, Black Swans could turn gray
Be open minded, don’t be stuck with a particular theory
Beware and leverage the scalable: winner takes all, Matthew Effect
80/20 rule is also the 50/01 rule
Apply the 90/10 rule: be ultra-conservative 90%, hyper-aggressive 10%
Winners might have gotten there via luck, Black Swans could change things
Prepare and be ready for serendipity
Seize any opportunity — they are very rare
Don’t run after the train – “Missing a train is only painful if you run after it!”
Be the one to reign. … It’s difficult to be a loser in a game you set up yourself.
You always control what you do; make this your end.
Just read thru the book Endless Referrals by Bob Burg. Lots of good advice on networking and building contacts to help one’s business and sales. Toward the end, Burg quotes Simeon Ben Zoma and his father on the following questions:
Who is the wise person?
One who learns from others.
Who is a mighty person?
One who can control his emotions and make of an enemy a friend.
Who is a rich person?
A truly rich person is one who rejoices in his lot.
Who is an honored person?
One who honors others.
Who is a brave person (from Burg’s dad)?
A brave person is one who is smart enough to be afraid and still do their job.
These are keys to networking and people relationships.
The tougher the negotiation, the more critical it is to understand that if someone in the room has to feel okay, it is not you. If someone has to feel not-okay, it is you.
Separate payside and nonpayside activities, minimize non-payside. Only payside activities are directly related to a specific negotiation. Examples of nonpayside activities include presentations to non-decision makers.
Some theories say we need 800 hours of practice to learn something new and complex.
A negotiation is the effort to bring about an agreement between two or more parties, with all parties having the RIGHT to VETO (i.e. to say no).
Never take responsibility for the other side’s decisions. Never “save the relationship.”
In any negotiation, your mission and purpose must be rooted in the world of the other side.
As of yesterday, I’ve logged 100 entries since mid September, 2008. So far, I’ve been able to keep to the target of 1 blog a day. To maintain this pace, more planning will be required, as I spent on average 4% a day to do this blog and the goal is less than 1% a day.
In the mean time, …
Woot! is an interesting web retailer: 1 item a day – until the item is sold out and a new item is put on sale. The site sells geeky products, similar to items you could find at Fry’s. Woot! seems to be able to maintain high volume of these select items. E.g., on Dec 10, 2008, between 9:44pm-10:03pm CDT, 19 minutes, it sold 1,499 pieces of USB AM/FM radio for $5.99 + $5 shipping. Sales receipt is ~$9K in 19 minutes. In total, the web site sold 59 different SKUs on Dec 10.
Woot! has built up a community with following features:
community
blog
statistics on who and where the items are sold
RSS feed
ads
This is interesting, as the web retailer can minimize inventory costs, but still can leverage the web to move inventory at high volume. The operation is also more streamlined — with one item being sold at any one time, the web retailer can optimize on shipment and minimize handling errors (shipping wrong item to customers). By offering good deals, the web site is able to sustain a sufficiently large pool of buyers to move the inventory as well.
With revenue per hour that ranges from $10K to $131K (random sampling), Woot! is likely to log over $200M of sales receipts in 2008. That has grown from $117M in 2007.
Whenever I have the opportunity, I’d ask companies if they use Amazon EC2. For several startups I’ve spoke to, all have used or intended to use EC2:
One company is deploying linux
Another one is planning to deploy on EC2
3rd one has converted their .NET application to run on Windows under EC2
There are a few who have no intention to run on EC2. Major reasons are:
Not control on the data — 2 of these companies manage their customer’s data and one have to deal with government compliance.
No SLA — one company is concerned about the downtimes at EC2, another has moved servers from a competitor of EC2, because there is no SLA
Cloud computing on EC2 has address many issues for startup, notably the setup costs for infrastructure deployment. At the same time, there are issues that not yet addressed by EC2.
Just finished reading “The Post-American World” by Fareed Zakaria. The book reflects the cogency of thought that I’ve seen in Zakaria when he appears on television. He writes about the trends in today’s world with the rise of developing countries, China and India and others.
Today, America is the most powerful nation in the world. The metrics include:
economic power
influence – other nations seek America to resolve disputes
higher education – America has the best universities in the world and lead the world in the number of Nobel prize winners
technology innovations (e.g. there are more nanotech laboratories in the US than any other country)
continued population growth and renewal with immigration
At the same time, because of their population sizes and recent economic growth, China and India are casting long shadow across the world, both economically and politically.
With its centralized government and pragmatic approach to the economy, China is growing quickly. Through autocracy, the new Bejing Airport could be built in a few years, the same amount of time needed for the British government to do an environment study to add a new addition to the existing Heathrow Airport. At the same, economic growth is spurring spiraling decentralization and the Communist Party officials are not selected for retail political skills, which would be needed to deal with the decentralization. Furthermore, China’s population is aging with birth rate below the rate needed to sustain the current population size.
Zakaria makes an interesting point that China’s (and Japan’s) pragmatic approach to everyday life and social issues derives from that the country has no general belief in an all-knowing, all-powerful authority, who has laid down a set of abstract moral laws that must be followed. He pointed out that Voltaire, Rousseau, d’Alembert, Kant, Leibniz, Thomas Fuchs, and Federick the Great had once idealized Confucianism, China’s alternative to the moral laws, to promote secularism.
In contrast, India’s Hinduism is all tolerant. It encompasses everything, it could be either spiritual or materalistic, rational or irrational, practical or severely moral… For example, Buddhism was founded in India, but does not exist there today, because it has been absorbed into Hinduism.
India is very diverse, free, and democratic. However, “democracy in India too often means not the will of the majority but the will of organized minorities – landowners, powerful castes, rich farmers, government unions, local thugs.” As the result, economic growth in India is organic and happens despite the government. Growth is slow but robust. Because its population is sustaining its youthfulness thru increased birth rate, India has time to grow.
Zakaria spends much time comparing America to Britain in the 1800’s and how America is different. Notably, America has been the world’s largest for 130 years, whereas Britain has only sustained this status for a few decades.
Zakaria does point to issues about America today and suggests some ways to address them. Among the biggest issues is the do-nothing politics, where there is “ceaseless, virulent debate about trivia”, as the country’s politics “has been captured by money, special interests, a sensationalist media, and ideological attack groups.” This impacts government policies that bring about economic decline. For this, Zakaria could only asks questions whether Washington could adjust to the changing world, the rise of the rest, and respond.
Garage Technology Ventures provided the following top 10 lies of entrepreneurs to provide some perspective to entrepreneurs about what they say vs reality. You can get the top 10 here. I’ll paraphrase them.
The startup’s projections are conservative.
Some big name analyst says the market will be $xxB by year 20yy.
This big company will sign the contract next week.
Key employees will join when there is funding.
There is no competition – the startup is the only one making this product.
There are already investors interested in putting in $.
This big guerilla in the industry moves too slow to be a threat.
People will pay for the pre-release version.
Patents will protect the business.
Getting 1% of the market will make the startup into a big business.
I was talking to a system software engineer and he asked me whether cloud-computing is just the generalization of virtualization, i.e., a much bigger, multi-tenant version of server virtualization deployment. I thought about it and there is some merit to this argument. At the same time, I indicated there are at least 2 other elements to cloud-computing:
DATA (per Tim O’Reilly), where information is distributed in various parts of the clouds, belonging to different entities, and there are ways to allow operations on the merged data. E.g. mashups, APIs.
CONNECTING (per Russ Daniels of HP), where the cloud allows people, machines, computer programs, and things to connect to each others to form interesting relationships and applications. So what are the protocols and applications that would bring the connecting to reality?
Add these to
COMPUTE VIRTUALIZATION, where VMWare/XEN/HyperV are likely to be just the basics. How could computing be pooled, ala mashups and grid? How could computing be localized, ala at a client? Also, in addition to carving up a system into multiple VMs, there are also computing needs where multiple systems are combined to perform a task, e.g. search or grid computing.
there are likely other key elements to the cloud:
PAY BY USE and ECONOMY OF SCALE to make the service model viable (ala Amazon)
Marcus Buckingham’s “Go Put Your Strengths to Work” emphasizes spending time on a person’s strengths and avoiding weaknesses. This is a very reasonable strategy that’s echoed in other self-help books. Buckingham has a good framework to identify a person’s strengths and weaknesses:
Relative to a type of activity
Strength
Test
Weakness
Test
SUCCESS
·Have been successful
·Other people tell me I have a gift for this
·Received prizes or recognitions
·Little success from repeated tries
·People tell me I need to improve on this
·Received no prizes or recognition
INSTINCT
·Do everyday
·Find myself volunteering for this
·“gut reaction” to me
·I look for ways to avoid doing
·I keep thinking of other people who can do this instead of me
·I have to brace myself
GROWTH
·I pick up this activity quickly
·I find myself thinking about this every day
·I can’t wait to learn new techniques for doing this
·I took a long time to learn
·It bores me
·Little improvement in performance, no matter how much I try
NEEDS
·I always look forward to doing this
·It’s fun for me to think back to when I was doing this
·Greatest personal satisfactions
·Leaves me feeling tired and drained
·I cringe when I think back on myself doing this
·Makes me happy to think of a world where I would never have to do this
If you’re at a time of self reflecting, this is a good book.
Using Amazon Web Services, Animoto has created a sensational service: create professional quality video from photographs and sound tracks — in minutes. The service is free for 30-second videos and $30/year for longer videos–non-commercial use. Here’s a sample creation done after several initial tests:
Jeff Younker aptly described agile development in his book “Foundations of Agile Python Development.” Younker highlighted the following key practices for agile development:
pair programming
user stories (not use cases)
system metaphor
on-site customers
unit tests
test-driven development
refactoring
simple design
short iterations
collective code reflection
continuous integration
documentation
With this methodology, the code should always be working, although features might not have implemented. More info on the Manifesto for Agile Software Development.
Copyright (c) 2008 by Waiming Mok
As I create more and more videos with animoto, which piece of music I could use become critical. Animoto uses an audio track, mostly music, to drive the presentation of the video. However, I cannot arbitrarily use any MP3 file I find on the internet, as each piece of music was created by someone and there is a copyright. The licensing involved would dictate what rights I’d need to get in order to use the music as background.
So far, I’ve found 2 legal routes:
Get formal licensing of the piece of music, which could be done mechanically via HFA, more info at NMPA. BTW, these rules likely apply only in the US, other countries have potentially different rules.
Use music with the Creative Commons licensing. Some sites for music with this licensing are:
People do make a living off their creations, so copyrights, rights to use and money are intertwined. It’s more complicated than maybe necessary, with non-creators, some necessary, in the music supply chain involved.
People like me who just want to create personal videos (and other things), with great tools from folks like animoto, are spending unnecessary time trying to understand licensing, and not to be tangled up in the cob web of rights and money. It is important to attribute creation rights and to pay for their services. There is no hassle-free, cost effective way to use music or other digital content that I come across. Instead, I’ve to search for content with free to use licenses.
which means “Live till you’re old, learn till you’re old.” So I looked up the origin of the phrase. One source says it dates back to 600 BC, to the Greek philosopher Solon. He was quoted to say, “so that I may learn it then die.“, when someone asked him, why he should waste his time learning a poem that his young nephew was singing.
To look at why people use cloud computing, I start to look at what solutions are available on Amazon AWS. The Solution Catalog shows the number of solutions available to the 5 types of audience. (There could be some double counting.)
Continuing to review the published solutions using S3, comparing the star ratings. Over 50% of solutions have no ratings. 40 of these solutions were modified in 2008, so there are fairly new. As a guess, there might be little use of them, thus no star ratings.
I was trying to not run after trains and I found myself running after trains 3 times today.
Taleb presented a compelling story at the end of “Black Swan” about “not running after trains”. The message is that one has control over one’s actions and can make choices accordingly.
The problem is that trains run on relatively stringent schedules. If I had missed a train, I’d have to wait for 30 minutes or more waiting for the next one — a big loss of time.
I knew the schedules of the trains. I ran in case the trains came earlier by a minute or two. In this sense, I was still in control of my choice to get on the trains on the planned schedules.
Movie creation is an iterative process, where the starting point is some rough concept that gets refined with more details in subsequent iterations. The work between iteration is problem solving. This process can be seen in creation of other products, although the time frames could be shorter than movie making.
Creativity is not a solo act or a single idea, but is embedded in each facets of a product, that’s developed over the course of time, by various team members.
Talent is rare. Finding good people is hard, much harder than finding ideas.
Management’s job is not to prevent risk but to build the capability to recover when failures occur. The key to recovering from failures is talented people.
Getting talented people to work together cannot be mandated, but is built with trust and respect over time — thus building a community is key. A community that’s a magnet for talent is even better.
Pixar has specific methodology to form the creative team – read article for details.
During the product incubation stage, one can’t judge a team by the materials it produces, but has to assess whether the team’s social dynamics is healthy, whether the team is solving problems and making progress.
While talent is not equally distributed, Pixar adheres to the following principles:
Everyone must have the freedom to communicate with anyone.
It must be safe for everyone to offer ideas.
Pixar employees must stay close to innovations happening in the academic community.
Very insightful article that has other ideas as well. Although Catmull talks about movie making at Pixar, similar principles could be applied to product creation.
A key factor for the advent of cloud computing is the so-far inexorable march of cost reduction of key components that make up a server. Let’s look at 3 key components: processors, memory, disks.
Computing processors have been following Moore’s Law. Although pricing has been relative stable as we move from one generation of processors to the next, the number of transistors in each new generation of processors have doubled every eighteen months or so. Performance essentially has been doubling from one generation to another, as well. This has enabled server virtualization: more independent operating environments could be run, without loss of performance, in the same physical system.
Disk costs have been exponentially decreasing as well.
Memory costs have been exponentially decreasing as well (see above black dot trend line). Interestingly, IDEAS studied memory pricing for one vendor, and memory has been coming down $200 / GB each year for the last few years.
These 3 components constitute (est.) 60-80+% of the cost of a server, so as they come down in cost, servers get cheaper. More servers could be deployed in the data center and thus the cloud is formed. There are many other elements of the cloud, such as the network, internet communication, power, cooling, etc. Many of the technologies have come down in costs as well.
Mark Elgan wrote a good article in Computerworld on useability and design requirements that differentiates a product that’s loved by users vs one that’s frustrating to use.
His key point is that products (hardware or software) that support the user’s feeling of being in control tend to be loved by the users:
what comes next is what’s expected
in control of what you need to do
no time wasted
not being forced by the product to do or to wait, …
This point describes why the Microsoft Ribbons (that’s in MS Office 2007) is so frustrating to use — note Elgan did not mention Ribbon at all in the article. Microsoft apparently plans to put Ribbons in future products, including Windows 7, creating opportunities for MS’ competitors to offer better alternatives.
For a user, there is a trade-off between UI and product capabilities. When the product offers more capabilities (such as the ability to do 3D design ala Solidworks (you need to take courses to use this product) or ability to format a table quickly in Powerpoint), the user might choose the product’s capabilities over bad UI design. This is why I use Office 2007 — choosing features over bad design. In a sense, product capabilities (that work) enhance the user’s feeling of control.
There are too many other trade-offs to mention here, such as the trade-off between the feeling of being in control vs the feeling of using the latest version…
When Karen Flynn mentioned that she created a vision map at the beginning of the year, I was intrigued. So l looked into what it is.
A vision map is a picture or description of your dreams, what you want to accomplish, and key steps to get there. In creating this map, the emphasis is to turn your deep desire to accomplish something positive into goals and actions. Part of the process is to flush out the vision with close friends and confidants. The resultant map could be pictures, collages, or writeup of what you’d like to achieve, based on your dreams.
When Yahoo! announced that it’s outsourcing search to Microsoft, some folks became concern that Yahoo! could end its sponsorship of 
On June 24, we had a thought provoking set of presentations at the 
The presentations ended with how Twitter is scaling today with the rewriting of the backend infrastructure from Ruby on Rails to a new language called 
This book by 
keynote and he spoke from his life experiences and lessons learned:
At the 
Hadoop, including 
I spent May 16 at 
Just finished reading 
With less than 1K servers, redundant data centers included, 
customers. BTW, this system supports close to 
each weekday, at less than 1/4 of a second response time.
access to the data are managed by metadata. In essence, the Oracle database is transformed to a multi-tenant database.
I just finished reading this book 
With full audience, the 
presentation, live teleconference, and live presentations by 2 speakers. Furthermore, the entire event was broadcast live via the Internet using 
infrastructure 2.0: 
At the 
there are 9 sources of supply and 6 areas of demand. Even high growth rate in solar would take years to be of significant impact — that should not stop solar.
are extremely significant.
Orion Letizi gave 
an interesting talk on 
I attended my first 
At the 
meeting, Klaus Roder of IBM showed the promise of mashup for the enterprise. Klaus indicated that IBM is providing a set of tools to help enterprise customers to create quick mashups to access data. Of course, in order to have good mashups, good data is a key requirement. The tools that run on customer premises are available to enterprise customers for a fee. A free test version of the capability is also available on the web, the web version (
Microsoft’s 
At the Jan 27 session of the 
expected crowd showed up, …, over 100 people. Many were new to the SIG — there was an entrance fee as well. There appeared to be big interests in cloud computing. 
Finished reading 
I upgraded my phone several weeks ago to the latest 6.1 and liked the new features: enabling user assignment of the PTT button to user defined function, and large time display. But, when I use 
DeviceAnywhere addresses the problem by offering thousands of actual phones located in the local networks (currently across 10 countries). The phones are remotely accessed via replicated on-screen interfaces. All physically accessible control on the phones can be remotely accessed. Subscription starts at $160 / month for 10 hours of access to this capability. 
, with a $149 enrollment fee. That’s a great deal to access 11 gyms in the Bay Area. It does have lots of elliptical and various training equipment. I needed to either buy a replacement elliptical or find a gym. A replacment elliptical cost between $500-$1K, that would be 3-8 years worth of Fitness 19. Thus Fitness 19. This would augment my 2 times a week at the pool, each time 1,800 yards (> 1 mile), and occasional fencing.
He has his colleagues, including 
about 
At 
and ease of programming using 
First day of 
While manning the 
changing the way we build cars to address the issues of traffic fatality and inefficiency of the current designs of the automobile. The problems and inefficiencies include:
toward an “operating system” for the internet. Rather than relying on the desktop or laptop to provide applications and data, the internet would become the platform to provide these services. Thus the rise of the likes of Google. O’Reilly described the fundamentals of this internet OS are driven by data. Web 2.0 is a manifestation of this evolving internet OS, where (according to O’Reilly), “data is the Intel inside”. Data is the lock-in point for anyone developing the Internet OS platform, like Intel is the lock-in and source of value for the PC.
Take 2 or more photographs from different perspectives of an object or a scene, 
The 
They did not take the rancher’s son, as he could not walk…
Go to a web page, the computer running the web browser scans through the web page and bring out potentially key information for you, 
automatically. That’s what Thomson Reuters’ 
I wanted to setup a single news feed that would aggregate the business, technology, and science news from Google, Yahoo!, Microsoft Live, Business Week, Scientific American, Science News, and then list the top 20 of the latest news items on the right hand side of this blog. 
his observations and conjectures on how we humans are not equipped to see and learn from the (1) rare events that have (2) significant impact and we tend to (3) ignore these events. These 3 points define the Black Swans. For me, Teleb jogs me with very interesting set of ideas, while prompting me to question what I do, how I think and interpret the world. As he presents his argument with stories, some real-life, some with names changed, 
some thought experiments, it’s hard for me to say that he’s scientific in his approach. At the same time, Teleb suggests that traditional scientific approaches in social sciences 
(e.g. economics, or looking at things like Black Swan) have too simplified models (e.g. 
by the high-volume products. It is often represented as a the 
The 
the item is sold out and a new item is put on sale. The site sells geeky products, similar to items you could find at Fry’s. Woot! seems to be able to maintain high volume of these select items. E.g., on Dec 10, 2008, between 9:44pm-10:03pm CDT, 19 minutes, it sold 1,499 pieces of USB AM/FM radio for $5.99 + $5 shipping. Sales receipt is ~$9K in 19 minutes. In total, the web site sold 59 different SKUs on Dec 10.
