Our Software Industry needs to step up to the crease and tell the world that we can write code

8 years ago I had a very embarrassing meeting. I laughed at a major global company that had invested over $ 600 million in a shared system. It took 5 years for the companies representative to talk to me again.

I did not actually laugh initially. I did something much worse.

They explained what they had done, and I kept trying to understand how they could have taken so long and spent so much money, and they explained again and again, as I looked at them in complete disbelief.

I think they may have initially thought that I was impressed or astounded. They certainly thought that their system was amazing, a first of its kind, the best of the best etc. etc., and they had been jumping up and down shouting from the roof tops about its brilliance.

Eventually, I said “is that it, is that really all it does, its trivial” I was shocked. It was about 3 months effort for a competent developer on a decent platform. We had done something much much better 5 years before, and regarded it as nothing to boast about.

My shock showed. They were affronted and angry. I laughed from embarrassment, and was promptly shown the door.

It has taken a very long time to rebuild the relationship.

So it would be very nice, if other people could point out that the UK software industry does create great things, and that when someone from another culture says theirs is “Amazing” and we say that we have created something that is barely adequate. Our barely adequate may only be 100 or 1,000x better than their Amazing world beater.

Advertisement

Caching is king or how to make a fast web app

Your developers have created a brilliant new application, and you are new starting to get some traffic, but it is collapsing under the load, and you are having to throw hardware at the problem.  A solution is at hand in the form of caching.

Varnish is the first thing to give your app some polish, it is relatively slow as a reverse proxy compared to lighttp or nginx, but a massive improvement over dotNet, Java or php.  However these all use naïve caching where content is cached for a time out period and then checked again after the time out, so the load will soon over take you and you will need to move to the next level.

Intelligent caching has a number of levels so lets move to the top end the @UK  Platform Generator Cache.  The key differences are as follows:-

• Compressed content
• No time to live, but updating as content changes
• Html Block level compression and de-duplication

The result is that the @UK  Platform Generator Cache can handle millions of pages per gig of memory and eliminates the issue of having to balance between out of date content and system load  since it never needs to go back to the database/middle ware to check if the cache is stale, because the cache is kept up to date in real time.

The result is that @UK can provide customers with free infrastructure since even high volume customers do not produce a measurable load on the system.   A typical customer using Php, java or dotNet for their web layer will see both web and database servers collapse before the @UK system starts to register a measurable load.

This means that @UK offers a migration service where it will provide an expanded infrastructure to the customer and some tuning to their application as part of an upgrade onto the @UK Platform, so that they can continue their expansion and then take the brakes off once they are on Platform Generator.

Understanding the cloud

First it has been around a lot longer than you think and in a lot of different formats. The modern cloud came about from the requirements of online companies to solve the problems of providing 100% uptime and security on cheap hardware. Mainframes solved these problems a long time ago, but were not cheap.

The modern cloud was independently created by a number of ecommerce and software as a service companies whose revenues and reputations depended on providing 100% Security, and Availability to their customers. In the case of the ecommerce companies both @UK and Amazon derived 100% of their income from online sales, and had carried out pioneering work into low cost clustering, which is part of the basis for the modern cloud technology.

Amazon had virtually unlimited cash for their operations and consistently lost significant sums of money as they built the business. This meant that they threw hardware at the problem and needed tools and systems to manage a vast infrastructure.

@UK was created in 1999 by a leading software house @Software PLC, and missed out on the dot com boom for funding. Thus, @UK developed its cloud between 1999 and 2006 with minimal financial resources. This changed with the @UK IPO in 2006, however the philosophy of high quality software as the solution rather than hardware remained.

Cloud Application Stacks

The result is that @UK has developed a cloud application stack that is at least 100x more efficient than well know web software stacks such as

• Linux Apache PHP and MySQL
• Oracle Java and
• Microsoft IIS, DotNet and SQL server

For most real world applications the level of change is not measurable since they collapse under load before a measurable change in load is recorded for the @UK system.

Clustered Storage and Virtualisation

The other parts of the modern cloud are clustered storage, and virtualisation. The first large scale deployments of clustered storage were by Google who created their own file system to cope with their requirement for large amounts of inexpensive storage. Cern (the creators of the web) also contributed to this effort with low cost hardware designs for their storage requirements.

Virtualisation has been around since the mainframe, however the x86 chip is difficult to virtualise and there are a limited number of virtualisation solutions for the older chips and they were often very inefficient. The new VT enabled chips mean that commodity x86 hardware now provides reasonably efficient virtual hosts.

It is the combination of low cost virtual hosts, clustered storage and the experience of managing clustered application stacks that created the modern cloud. However the problems that they were trying to solve of providing secure available applications have been around for a much longer time.

Cloud Delivers Security and Availability

This is why for most major cloud players the current concerns about cloud security and availability are things that they normally started to address over 10 years ago and solved a long long time ago.