Case Study: Incorporating Disruptive Technologies into Existing Products (part 3)

In the previous article we looked at how SAP handled building a disruptive technology, HANA, into the core of some of the largest, most complex, most critical, and slowest moving systems in existence. While certainly not perfect, this is a good example of how a technology transition can be handled.

SAP is also working with another disruptive technology – AI. Unlike the core ERP database, AI is new. AI can be applied to new applications, making it easy to add to existing systems without replacing or modifying existing systems and functions.

At its core, AI adds value to data. It takes existing data and provides guidance. ERP systems are built on data – they store data, process data, and add value to data. You can use AI as a new tool to add value to data – where it makes sense.

Consider one subsystem for ERP: credit card fraud detection. Every time a customer buys something you want to make sure that this is a legitimate transaction. Fraud detection has been a part of order processing for decades. Existing tools and applications check each order for a variety of indicators of whether or not the purchase is valid. For example, has the credit card account expired or been canceled? It is a cat and mouse game between companies and criminals – you don’t want false positives (denying a valid order) or false negatives (allowing a fraudulent order to go through).

AI can provide an additional check. After training on both good and fraudulent transactions, an AI system can provide a probability rating of whether a new order is good or bad.

In this case, AI would extend an existing function. It would be an additional test, building on existing tests – there is no need to use AI to see if an account has expired! This also allows AI to be deployed incrementally. For example, the initial deployment might run the AI based check but just report the results, allowing comparison of AI with existing techniques and further training and adjusting it without customer impact. The AI check could then be used as an additional data point for human evaluation of suspect transactions, and then ultimately used as “just another check” in the fraud detection system in full production deployment.

As revolutionary and disruptive as it promises to be, AI can be introduced incrementally. A good starting point would be to provide an AI Toolkit for customers to use to build AI applications on top of the ERP systems. This toolkit would provide access to data (stored in HANA!), APIs for integrating with the rest of the ERP system, and AI functions for both training and inferencing.

Since ERP systems work with critical and often private data, an AI ERP toolkit would also integrate with the ERP access controls.

This is exactly what SAP has done – provide an AI toolkit that customers can use to build applications.

The AI Toolkit can also be used by SAP – and partners and other members of the SAP ecosystem – to add AI to existing functions and applications as well as building new functions and applications.

This is an example of how existing systems can be extended by disruptive technologies without impacting existing customers or applications and how the disruptive technologies can build on and take advantage of the core capabilities, infrastructure and strengths of an existing system and ecosystem.


Posted in Product Development | Leave a comment

Case Study: Incorporating Disruptive Technologies into Existing Products (part 2)

The previous article introduced ERP and touched on some of the challenges around using the huge amounts of data that ERP requires.

SAP developed a new technology called HANA. Unlike a disk based SQL database, HANA is entirely in-memory (with a disk based backing store for backup, recovery, and data archiving). Unlike the row oriented SQL model, HANA is a hybrid row/column datastore which supports high performance queries and analytics. HANA is also a compressed data store – in addition to allowing a complete ERP database to fit into memory, the compressed data can also be searched and transferred more quickly.

Enabling this software was the development of large servers from companies like Dell, HP, and IBM with dozens of processor cores and 10+ TB of physical memory. Since the introduction of HANA servers have continued to grow – today you can get servers with over 100TB of physical memory! There are other developments now emerging which will radically grow the amount of memory that can be attached to a server and which will lower the cost of this memory. In short, an in-memory approach has proven to be better than on-disk.

HANA provides dramatic performance improvements as well as new capabilities – it is a disruptive change at the very core of ERP. SAP wants to use HANA as the foundation of all ERP deployments, replacing all current databases.

Consider how SAP has implemented and deployed HANA:

Much of the architecture of HANA is designed to support analytics. So, analytics applications were the first target for HANA. Like other ERP Analytics implementations, data would be extracted from the OLTP database, stored in an analytics database (HANA in this case), and used by analytics applications.

Analytics applications would be developed and modified to take advantage of the new capabilities of HANA. Over time more and more analytics applications would begin to use a single HANA datastore.

At the appropriate time, HANA would also be used for OLTP. This would initially be done for customers with special needs and mainly targeted at new installations.

Finally, HANA would be offered as a single datastore for mainstream customers.

Note the incremental approach which is key to successful deployment of disruptive technologies:

  • Start with an application that benefits from the unique capabilities of HANA, has minimal requirements for broad features and integration, delivers business value to a relatively small set of users, and is outside the mainstream use cases. This provides immediate business value from a limited set of capabilities and allows you to improve and mature the technology in a constrained environment. At this point it is critical to minimize the impact of failure and to provide an opportunity to grow and extend the technology.
  • Over time add additional analytics applications, either by developing new applications that are now feasible or by modifying existing applications that benefit from the unique capabilities and performance of HANA. Test and extend HANA with each new application, and don’t release any applications until they and HANA work well together.
  • Once HANA proved itself with a core set of analytics applications add more applications. Over time the focus moves from applications that need HANA capabilities to the business value and cost savings of having a single datastore for analytics.
  • HANA fully supports the SQL model used by the current databases. It can be treated as  “just another SQL database” and requires minimal modification of the rest of the system. This allows SAP to begin to use HANA with minimal changes to the ERP ecosystem built on top of the database.
  • Next, begin to deploy fully integrated ERP systems that use HANA as the single master database for both transactions and analytics. The initial targets will be specialized companies that are “relatively simple” as far as ERP goes.
  • At this point you can begin to use HANA for new customer deployments. These customers will start with a fully integrated system. This is the entry of HANA into the majority market.
  • Next focus on moving existing customers to HANA as part of major version upgrades of the ERP system. Major version upgrades are expensive and time consuming. They require significant migration and integration work, and provide an opportunity to introduce new technology. At this phase you are addressing the Early Majority market.
  • The next step is to make HANA the standard choice so that it is simply part of all new systems and upgrades – anything that is not HANA based becomes an exception. You are now addressing the Late Majority market.
  • Finally, retire the older versions of software that do not support HANA. Over time phase out support for these older versions. This will encourage the Laggards to move to HANA based systems.

SAP has a long timeline for customer adoption. SAP understands the impact of major changes on their customers, partners, and ecosystem. HANA was initially introduced in 2008 and the current target for full customer migration is 2025 – a 17 year migration!

You can see how this approach starts with Innovators and then moves through Early Adopters, Early Majority, Late Majority, and ultimately laggards.

SAP HANA is a good case study on how to successfully integrate and deploy a disruptive technology in an existing product in a mature market.

Next: Case Study: Incorporating Disruptive Technologies into Existing Products (part 3)

Posted in Product Development | Leave a comment

Case Study: Incorporating Disruptive Technologies into Existing Products (part 1)

In a previous article we looked at delivering disruptive technologies by developing a new product to replace an existing product. We touched on the impacts of replacing an existing product with a new product. The alternative is to incorporate disruptive technologies into existing products – a daunting task, fraught with risk and danger!

SAP is one of the leaders in ERP (Enterprise Resource Planning) systems. They provide several interesting examples of how to develop and deliver disruptive technologies. Let’s look at how they are building disruptive technologies into their products:

Perhaps the most complex and critical systems in use today are the ERP systems that provide the backbone of all large companies, most medium size companies, and many small companies. ERP systems are highly integrated across the company and touch all parts of the company. If the ERP system goes down, the company can’t build or ship product – or collect money, pay suppliers, pay employees, comply with regulations, plan business, or track the market. ERP is literally the definition of mission critical. ERP systems have lifespans of decades. They are continually evolving and adding new capabilities, all while maintaining compatibility and stability. They are also heavily customized for each customer, and sit at the center of an ecosystem of applications built around company data.

An ERP system is fundamentally three components.

  1. A datastore of all information created or used by a company.
  2. A set of functions and applications that create, update, use, or add value to this information. The two primary uses are updating the data through transactions and using the data for analytics. These two use cases place very different demands on the underlying datastore.
  3. A set of controls on who can access and manipulate the information and applications.

At the heart of an ERP system is the datastore. Traditionally this is a SQL database. The size of ERP data, performance requirements of the applications accessing this data, and the critical requirements of data integrity have driven the high end of the database market. Only a few databases can scale to the levels required – most notably Oracle, although other databases are also used.

As the use of ERP and ERP data has grown, the style of access has grown from batch processing to OLTP (OnLine Transaction Processing), to interactive queries and reporting and stream processing. SQL databases are great for batch and OLTP, but are not as well suited to many popular queries or stream processing.

As a result ERP data has been divided between an OLTP database, typically an SQL database, and copies of this data in forms better suited to analytics. This Analytics data is typically stored in  data lakes, Business Intelligence data cubes, and similar query focused formats including non-SQL databases.

Using multiple technologies to contain what is essentially the same data is bad for data integrity, for avoiding stale data, for integration, and bad for the increased cost of maintaining multiple technologies and applications. Using the same ERP datastore for both transactions and analytics has been a long term goal – one that could not be met with existing technologies.

From a business perspective, SAP and Oracle are competitors. The Enterprise level SQL databases that can support large ERP systems are extremely expensive and represent a large part of the cost of ERP systems. The ERP database is the central integration point for all corporate data – many applications using ERP data will integrate with the database itself rather than directly with the ERP system.

All of these factors made it attractive for SAP to be able to offer a new technology for storing ERP data – a disruptive technology that would replace the traditional SQL databases and other forms of data storage in ERP systems. The next article will cover what SAP did from both technology and customer perspectives.

Next: Case Study: Incorporating Disruptive Technologies into Existing Products (part 2)

Posted in Product Development | Leave a comment

Incorporating Disruptive Technologies into Existing Products

The previous article explored investing in disruptive technologies.

How can you incorporate disruptive technologies into existing products? With much difficulty…

Developing the new technology is likely to be the “easiest” part (not that it is actually easy). Actually using a disruptive technology in an existing, successful, proven product has an incredible range of forces aligned against it – technical, product, process, people, corporate, and especially customer! Note that we are talking about disruptive technologies; if a technology can easily be incorporated into an existing product, it is probably not really disruptive.

Before we go into implementing a disruptive technology, let’s focus for a moment on the disruptive aspect. In addition to technology, the disruption applies to changes to the product needed to support the new technology. Disruption also applies to how the product is used, processes and systems using the product, the ecosystem built around the product, support, and especially customers.

Since we are looking at incorporating disruptive technologies into existing products, the most important thing is to ensure that customers continue to give you money – if you interfere with a customers ability to give you money you have failed. I plan to explore this topic in future articles; for now remember that the ultimate goal is revenue.

There are three main ways a disruptive technology can be used:

  1. Create a completely new product addressing the same market. Initially this will address customers not well served by the existing product. Over time it may replace the existing product.
  2. Replace core parts of the existing product with the new technology.
  3. Extend the existing product.

There are many advantages to creating a new product. The adopters of a new technology have specific needs or goals that are not currently being adequately addressed. You can focus on these unmet needs, work directly with the customers, establish initial success, and build on it. In other words, address the Innovators and Early Adopters. Hmm, looks like we are actually applying the Product Life Cycle model we’ve been talking about for the last several articles!

Creating a new product provides a clean slate and a clean foundation to build on. This new product is smaller and does not have as many dependencies as a mature product. The new product is easier and cheaper to support and doesn’t have a backlog of thousands of bugs that need to be fixed The result is that it is much easier to modify, extend, and grow the new product.

A new product is appealing to customers who have reached the limits of existing products or who do not need the broad scope, integration, stability or cost of a mature product.

There are three things a new product does not have: a customer base, breadth of capabilities, and revenue. Looking at the product life cycle model we see that the path for profitability in a new product is for it to assume the characteristics of the product it is replacing!

If you assume that a new product built on disruptive technology is going to immediately replace an existing mature product, you will be disappointed. If you try to force a new product – or an existing product with major changes – onto existing customers you will receive massive resistance. Note that if you force a customer to move to a new product, you also open the door to them considering other products, not just your replacement…

In general, forcing customers to make changes in how they use a product is a bad idea. Changes in interfaces, capabilities, integration, or operation can have great impact on a customer’s business. Changes to a product can easily break existing processes and workflows, causing problems with little or no perceived benefit.

One common reason for forcing new products on customers is “we can’t afford to support two products that do the same thing”.

In this case you don’t have two products that do the same thing: you have an existing product that is widely used and is generating business value for customers – it is in the majority market segments of the product lifecycle. And you have a new product that is (hopefully!) interesting to innovators and early adopters. Even though the old and new products appear to be addressing the same market, they are addressing different segments of the market. They have very different requirements and expectations; attempting to force them together will not end well.

Next: Case Study: Incorporating Disruptive Technologies into Existing Products (part 1)


Posted in Product Development | Leave a comment

Investing in Disruptive Technologies

Previous articles have highlighted some of the challenges in attempting to develop disruptive technologies and products in a traditional corporate environment – basically it doesn’t work.

So, how can disruptive technologies and products be developed? This apparently simple question is actually three separate questions:

  1. How can disruptive technologies be developed?
  2. How can disruptive technologies be incorporated into current product offerings?
  3. How can new markets be addressed?

All of these questions have technical, economic, product, and corporate components.

Let’s start off with the first question.

In many ways, the easiest approach to developing new technologies is “somewhere else”. That is, let someone else develop a technology and acquire rights to the technology once it is proven. This is the fast follower or second mover approach. Like all potential approaches it has strengths and weaknesses.

The most obvious danger in fast follower is in missing a new technology. However, the greatest risk is actually a corporation’s ability to implement and execute disruptive changes to products, internal processes, existing resources, and corporate structure. There is massive resistance to major changes – the common outcome is to resist change until it is too late.

A good place to start is by asking the question “do we have a corporate strategy and mechanism for investing in small, poorly defined, low margin markets with high risk of failure?”

The two most common answers are “no” and “are you crazy?!?”. Companies like large, well defined markets with good margins. A good return on investment needs margin and predictability.

Where does this leave us?

What we are talking about is actually research, not product development. It needs to occur outside of traditional corporate environments. The logical thing is to do it in a corporate R&D group.

Unfortunately, corporate R&D groups are typically not tightly connected to discovering and nurturing new markets. Our focus here is on profitable products, so classic R&D only addresses part of what is needed.

So, what is needed?

  • We need a product and market focus – technology is a tool, profit is the goal!
  • We need minimal yet sufficient resources for exploration.
  • We need a place for unreasonable people to follow their passions.
  • We need the ability to experiment, try alternatives, learn as we go, and to change directions.
  • We need the ability to seek out and work with non-traditional customers.
  • We need the ability to target small market opportunities.
  • We need the ability to fail – and to learn from the failures, adapt, and move on.
  • We need time – time to explore and mature concepts.
  • We need freedom from many of the constraints, processes, rules, and overhead of mainstream corporate life.

Experience shows that this needs to be done outside of traditional corporate environments. “Skunkworks” projects and business incubators provide a model for how this can be done. You need a small, highly motivated team with not quite enough resources, probably in an isolated facility.

The funding model needs to focus on learning, not revenue. This is a way to address the risk inherent in research and to provide an appropriate level of control and structure. When the goal is learning you have a mechanism dealing with failures without failing. As long as you learn something actionable and can use it to guide the next steps you are making progress.

Next: Incorporating Disruptive Technologies into Existing Products

Posted in Product Development | Leave a comment

False Dichotomy

A previous article suggested that the two ways to deal with potentially disruptive technologies are to either invest in all potentially disruptive technologies or to ignore all potentially disruptive technologies.

One of the most powerful ways to go wrong is the False Dichotomy, which is also known as false choice, black and white thinking, or either/or thinking. Typically this involves a situation which is presented as a binary choice of either A or B, with the implicit or explicit assumption that these are the only possible choices – there are no alternatives C, D, or E.

In real world situations this is almost never the case. There are always options, or at least variations on the two choices presented. In many cases the alternatives presented are the extreme positions, ignoring the many alternatives between them.

Further, in the case of disruptive innovation the best alternative may be neither A nor B but “kumquat” – something completely unexpected and entirely outside the range of alternatives being considered! A popular saying is “On a scale of 1 to 10, what is your favorite color in the alphabet?”. While these examples appear nonsensical, they illustrate the need to consider alternatives that may not be obvious – an approach often called thinking outside the box.

Two points are worth making: first, there is almost never The Right Answer – that is, a single correct answer that solves all problems and where any other answer is Wrong. Instead, there are a range of alternatives that can be made to work with various levels of effort and trade-offs. Part of product planning is to explore these alternatives and to determine the benefits, cost, and risk associated with each.

Second, interesting problems are invariably multi-variate. Instead of a single parameter that can be optimized, several interacting parameters must be considered. Any real world situation is going to involve a series of trade-offs, typically between capabilities, cost, investment, resources, integration, and time. Other factors include side effects and consequences: for example, one material being considered for a product may have all desired physical properties but be toxic.

Also important is understanding whether a constraint is an absolute constraint like the speed of light¹ or is flexible. In the example above, a toxic material might be used if it is carefully packaged.

Looking for The Right Answer can lead to ignoring acceptable solutions and approaches that can be made to work. A better approach is to consider multiple potential solutions, determine the strengths and weaknesses of each – including what can be done to address these weaknesses – and choose the best overall solution. Note that the best overall solution will often include elements from multiple approaches – even elements from both halves of the false dichotomy!

For product development the challenge is to understand customer needs well enough to provide a product that meets their needs at a price they are willing to pay. Note that the customer has the final word on what their needs are – if a feature is not wanted or used by a customer, that feature does not meet their needs. The ideal is a product that meets current customer needs and can be extended to meet future needs.

While product definition is often done informally, there are structured approaches that can be used. A powerful technique is Design Thinking.

Design Thinking uses a five step process of: defining (or redefining) the problem, needfinding and benchmarking, ideating, building, and testing. Design Thinking is a team based approach, best done with a multidisciplinary team bringing different knowledge, expertise, and viewpoints to the project.

Much of the power of Design Thinking comes from applying a structured methodology to complex problems – much more than a blog post is needed to really understand it, much less apply it. Fortunately there are many resources available, including books and courses. Both edX and Coursera offer courses on the subject, with edX even offering a five course “micromasters” program.

Next: Investing in Disruptive Technologies

¹ This is something of a trick example. The speed of light in a vacuum can’t be exceeded. However light travels through other materials at different (slower) speeds. For example, light in a fibre optic cable is roughly 30% slower than in a vacuum. This means that it can be faster to use a microwave link than a fibre optic cable, leading to interesting designs for systems with minimal latency.

There is a lot of interesting work going on around quantum entanglement that may allow information exchange to exceed the speed of light – this would definitely be a disruptive technology! Thus this is actually an example of the importance of understanding your constraints and exploring novel options to possibly work around fixed constraints!

Posted in Product Development | Leave a comment

Unknowable Markets

The previous article began our exploration of innovation.

Clayten Christensen is very clear about understanding the markets for disruptive technologies: “Markets that do not exist cannot be analyzed: Suppliers and customers must discover them together. Not only are the market applications for disruptive technologies unknown at the time of their development, they are unknowable.”


In the early stages of a disruptive technology you don’t know what it is, how it works, what is required to develop it, who will use it, what they will use it for, or how they will use it. Based on this you have to build a business plan, establish a return on investment that meets corporate thresholds, prepare a development plan and budget, get approval, obtain and assign resources, deliver on schedule, and meet sales forecasts. And, of course, the new technology is inferior to existing more mature technologies for most use cases.

Right. Easy. No problem!

The only things you know at the early stages are that the initial markets will be small and that your early beliefs and assumptions are almost certainly wrong. Just to make things even better, there is an excellent chance that any truly new technology won’t work out. When it does work it is likely to take significant time to mature – more time than most companies are willing to accept for their investments. Until the new technology matures it will be inferior to existing technologies for most applications. Welcome to the wonderful world of pioneering new product development!

Based on this, no reasonable person would want to be involved in developing a disruptive technology.

Fortunately we have unreasonable people! People with vision, passion, and the skills needed to go after things that haven’t been done before. People with the determination to continue even after setbacks and failure. People that believe “impossible” is just a word in the dictionary between “imposition” and “impost”.

The next challenge is that well run companies have systems designed to prevent investment in disruptive technologies. Well, they aren’t specifically designed to do that, but it is one of the side effects of planning systems.

There are always more potential projects than available resources. Well run companies have well developed systems to allocate and assign resources, most notably money and people, to the projects that have the highest payback. The best companies have systems that give priority to projects that take advantage of the core resources and competencies of the company and align with the largest and most profitable market segments.

These companies know their markets, their customers, their resources – and themselves. They have a laser focus on excellence in execution and predictability. A learning company will continuously improve their processes and products. They are well aware of the adage “it is easier to keep an existing customer than to capture a new customer”. They dedicate themselves to keeping their customers, making customer service a top priority and seeking more ways to deliver value to their customers.

“But” proclaim the believers in an unproven and immature technology, “this new technology will revolutionize our industry and put us out of business if our competitors have it and we don’t!” As shown in the Gartner Hype Cycle they are joined by analysts and the press in proclaiming how everything is changing and you have to fully commit now!

There are a couple of sayings to keep in mind: the noted economist Paul Samuelson observed “the stock market has predicted nine of the last five recessions.” Also “even a blind squirrel finds a nut once in a while.”

We know that disruptive innovation happens and that it can greatly impact even – or perhaps especially? – the best run companies. What can we do about it?

  1. Invest in every new potentially disruptive technology. This will divert critical resources from sustaining innovation in the short term and cause the company to become uncompetitive.
  2. Ignore disruptive technologies. This approach works – until it doesn’t.

Some industries experience disruptive changes every few years. Other industries go decades without disruptive changes. The risk is that disruptive changes build up slowly and then hit with such speed and impact that it is too late to respond when they do happen.

Fortunately this is a false dichotomy – there are choices between “everything” and “nothing”. The next article will begin to explore these alternatives.

Next: False Dichotomy

Posted in Product Development | 1 Comment