Monday, 3 May 2010

Impact Of Scalable Computing On Software Engineering Processes

Of late, scalability has become a key quality characteristic which is demanding increased attention from the developer community. There are many factors for this – including evolution of cloud computing and also due to advances in hardware such as multi-core processes.
Software Engineering processes, if followed true to its spirit, easily accommodate scalability requirement of the software being developed. But the width of the gap between process definition and its practice is left to anyone’s imagination.
So, we need to mind the gap, eh?
In this post I have listed few key elements that deserve an increased attention from software designers and developers for few of the software engineering processes. My intention is to link industry best practices and key concerns to software engineering processes.
If your organisation is using review checklists, perhaps you could consider including appropriate points to check these elements.

Requirements Gathering and Management
  • Difficulty in
    • visualising requirements from a wide base of subscribers for a SaaS application.
    • capturing service management related requirements such as metering, billing etc. due to either inexperience or lack of a strong business process backing it.
  • Increased focus is needed in capturing and managing
    • Interoperability requirements as in future SaaS applications will increasingly inter-operate with each other. As a result, APIs that need to be used and exposed to other products must be known at this stage.
    • Security, Privacy, Trust requirements
    • User interface requirements as cloud application are likely to be accessed from a variety of mobile devices.
  • Need to separate requirements into two groups – common features available for all SaaS tenants and potential customisations

Software Design
  • Good assumptions to keep in mind include:
    • Design must be akin to that of SOA elements – that is – loosely coupled service components
    • The applications must work in a clustered environment
    • Database server on the cloud is virtualised and is more susceptible to failure than the physical counterpart.
    • You have to design the application for performance which is likely to be bound by SLAs with commercial and/or legal consequences.
    • Underlying infrastructure in which your application will run will be of lower quality and lower bandwidth.
    • In almost all cases, you have to design your application to be remotely managed in real time.
    • Stateless architecture is preferred. In this architecture you are not getting rid of the state data, but you will be either keeping it on a peer buddy server or on the network but never on the server serving your application.
  • You have to consciously design
    • Security and privacy provisions. You need to be aware of not only standard SSL but also OpenID (Open source solution for unique username and password), PCI DSS (Payment Card Industry Data Security Standards) etc.
    • Scalability – including providing inputs for capacity planning, dynamic scalability, load balancing –these factors could be manually controlled or automatic.
      • To design for scalability, you must use new modelling methods such as Cube based models.
  • Be aware of new protocols, standards and when to use them. For example:
    • XMPP – Extensible Messaging and Presence Protocol – which is a way communication protocol between the client and the web server designed to eliminate polling/pinging the host.
    • JSON – JavaScript Object Notation – which is a low resource consuming alternative to XML when sending/receiving data using JavaScript
    • REST (Representational State Transfer) versus SOAP (Simple Object Access Protocol)
  • You should be aware of open source tools available which you can make use of.

  • Be ready to learn or re-learn
    • new programming languages such as APEX or Python.
      • They could be new to you not necessarily to the IT industry.
    • distributed transaction management
    • new models for state management
    • multi-threading
  • Your aim should be to develop lean code that uses as fewer resources (mainly memory and CPU) as possible.
  • It is safe to assume that bad performance of the code will ultimately lead to a commercial impact on your company’s services.
  • Forget stored procedures! They aren’t easily portable across databases.
  • You are likely to use a development infrastructure provided by a “Platform as a Service” provider or an “Infrastructure as a Service” provider. Hence your development practices and review methods are likely to be influenced by these provider’s facilities and practices.
  • In order to achieve scalability, you must split your application threads in real time based on functionality and demand.

  • You need to test your applications in an environment similar to a commercial cloud in which you will not have visibility or control. This is likely to be a challenge. You are likely to outsource this to a “Testing as a Service” provider.
  • Troubleshooting on the cloud is not easy.
  • You should be in a position to test and certify application performance.

  • You need re-design your deployment processes in light of virtualised servers.
  • You may have to optimise your releases minimising impact on a large number of user community.

Change Management
  • If you are developing a SaaS application, expect a part of your software subscribers to come back to you with change requests to your software. Compared to traditional software development, implementing change requests in a cloud environment is not easy. You might want to keep your basic software offering stable yet you will be compelled to honour variety of change requests submitted by your tenants.
Of course it is only a partial list of impact on processes. I will update this post when I collect few more relevant points.
I appreciate your feedback on this post.

No comments:

Post a Comment