The SUGEN Numbers

January 20, 2010

I argued in the previous post that SAP’s new, “two-tier” pricing system for maintenance offers customers less choice than meets the eye, and commentators like Dennis Howlett agree.

So why did they bother? If one offering is “good support for a fair and reliable price” and the other offering is “less good support for roughly the same price (only no one will really know for six years)” why would anyone pick the latter? And why would SAP risk a public relations nightmare when the people who pick the apparently lower-cost alternative find that they’ve been snookered?

Is it just that SAP needs to offer the enterprise application version of “small coffee,” the coffee size that nobody ever orders, but you need on the menu, so people will order medium?

The question is particularly salient because SAP has data that, one could at least argue, shows that Enterprise Support really is better.

This data comes out of a program embarked on last summer, sponsored by SUGEN (SAP User Group Executive Network) and SAP. In this program, companies were put on Enteprise Support, and the benefits thereof were measured in 11 benchmarks and the sum of those benefits added into something called the SUGEN KPI Benchmark Index. SAP had vowed not to raise the cost of Enterprise Support until this program showed that a gain in the Index that gained justified the increased cost of Enterprise Suppory.

SAP reported the results at the Influencer Summit last December, which I attended. According to the numbers they showed me, the SUGEN KPI benchmarks had indeed been achieved.

These numbers were disclosed to me on the condition that I not repeat them until the full results were published, and while I’ve been given informal permission to speak about them, I will try to respect this request.

I think, though, that I can convey a fairly accurate idea of what is going on without actually citing the numbers.

Before I can get to this, though, I need to explain something about the program and the expectations that people had for it. When SAP first announced a new, improved class of support at an increased price, which all customers were required to use, many customers thought that this was just price-gouging. They didn’t believe that SAP’s across-the-board price increase for maintenance would be accompanied by any benefits. When SAP started hearing from these customers, they were clearly taken aback, since the executives in charge of this new support program clearly did (and do) believe in its benefits.

So SAP (and SUGEN, the customers’ self-appointed representatives) agreed to put the question to an empirical test.

Now anybody reading the press release about this program or anybody attending the journalist session at last May’s Sapphire (as I did) would believe that this test would be done along traditional social science lines. A representative sample of the SAP customer base would be given the opportunity to take advantage of Enterprise Support, and the benefits would be measured.

After attending that session, I told my client base (people who are professionally interested in tracking what SAP does) that it was impossible for this program to show so much benefit that it would justify the across-the-board increase. The reason was simple. To get the available benefit from Enterprise Support, a customer must get and install a software product called the Solution Manager and must then do a lot of process documentation and process modification. A representative sample of SAP customers simply wouldn’t include very many customers who had done all this installation, documentation, and change, because the total amount of work was considerable, and most customers weren’t going to do it, at least not any time soon.

Isn’t it sort of squirrelly, expecting Enterprise Support customers to get software, install it, and then do a fair amount of work before they get the benefit that SAP promised them? Well, yes, but it isn’t quite as squirrelly as it sounds.

At the Influencer Summit, Uwe Hommel, the person behind this idea, expressed it roughly this way. A lot of customers don’t really run support as well as they could. The Solution Manager provides them with a framework for the practices that they should be using, plus it enables SAP support personnel to give better, more accurate, and faster help, because the Solution Manager gives them better information about what was going on at a customer site. It would be nice if SAP could wave a magic wand and improve support without any effort from the customers. But that just can’t happen. All we can do is provide a framework.

As far as Hommel is concerned, what SAP is saying is roughly what the trainer at the gym offers. “We’ll make you better, bigger, stronger, and leaner, but of course you have to do your share.”

Fair enough, of course. But that’s not actually what SAP said. SAP actually said something more like, “You need to do support better, and to do it better, you’ll need a trainer and you’ll have to put in some effort, but oh, by the way, you have to pay for the trainer whether or not you actually get around to going to the gym.”

Perhaps the oddest thing about the test that SUGEN and SAP ran is that both parties pretended that SAP had said the first thing and not the second.

You see this, for instance, in the way they [SUGEN according to Myers, below] chose the subjects for the test. Rather than choosing the representative sample of the customer base that I was told they would choose, they asked for volunteers to apply for the program. 140 customers did apply; of those, only 56 were chosen for testing. This, of course, simply guaranteed that the test would not prove what SAP wanted it to prove, that the price increase was justified. At best, it would prove that those customers who decided to go to this metaphorical trainer would get some benefit from it.

So, did they get some benefit?

Well, um, uh, sort of.

As I said above, SAP and SUGEN agreed before the test that there were 11 areas where benefit might be provided. The areas ranged from the obvious things–fewer outages, faster problem resolution, and fewer problems–to the less obvious, but still important things, like more efficient CPU utilization and better use of disk storage.

In the actual test, the benefits of Enterprise Support was measured in only 6 of these 11 areas. The areas chosen had to do with total cost of operations (use of CPU and storage), the cost/effectiveness of managing patches, and the extent to which customers used SAP’s current software effectively. Clearly, this made things harder for SAP, since they were trying to prove benefit, but the benefit was actually measured in less than half the areas where benefit might be available.

Nevertheless, SAP thinks that it succeeded, and technically they did. They measured benefit by giving the SUGEN Index an arbitrary value of 100. The way I understood it at the session, the aim was to show that the increased benefits at least offset the roughly 7.6% price increase in 2009. [According to Myers, below, the actual aim was 4%].

Both aims (what I thought was the aim and what Myers said was the aim) were actually achieved. The benchmark index dropped by 6.89 percentage points. Even though only 6 benchmark areas were measured, the benefit achieved did offset the 7.6% increase (at least within 1 percentage point).

There is, however, a little, tiny, “but.”

All the benefit was achieved by massive improvements in only two out of the six areas: storage utilization and number of failed changes. (A “failed change,” is an attempt to install a patch which fails.) In all the other areas that were measured, the average improvement was very small.

Both of these measure appear to me to be one-time-only improvements. Take, for instance, storage utilization. If you have one of those awful Windows machines, and your disk is sluggish, you can run a utility that compacts your disk and frees up disk space. You’ll show massive improvement in storage utilization. But running this utility once is not the sort of thing that justifies a permanent yearly increase in maintenance costs. Yes, you can do it next year, but it won’t show the same level of improvement, because you gained most of the benefit the first time you did it. The same thing goes for reducing failed changes. Changes in process (and use of the Solution Manager, or Sol Man) can reduce this number a lot. But once you’ve made the changes, further reductions aren’t really available.

I certainly hope that somebody from SAP is reading this; if you are, you’re probably upset, because you’re saying to yourself, “Well, the benefit is permanent; for the rest of time, people will have fewer failed changes and use less disk space.” [Myers does in fact argue this. See below.] You are, of course, right. But you’ll have overlooked the larger question: does helping people to a one-time improvement justify a permanent, yearly price increase. That is hard for me to see. If Enterprise Support promised to bring these kinds of improvements in regularly, then it would be OK to pay more for it regularly. But this test doesn’t show that these regular improvements will be forthcoming.

In any case, it’s all moot now. SAP has scrapped the SUGEN benchmark process. In a way, it’s a shame. This is one of the few times that any enterprise application company has ever tried to run a systematic test of whether its software and services work as advertised. And the results of this test are very interesting. In some areas, the software and services don’t seem to work; the benefits are minimal. In other areas, though, they work very well indeed; the benefits are startling. Who woulda thunk it?

At the very least, shouldn’t SAP keep going with this, so it can go back and fine-tune its software, figure out why some benefits aren’t forthcoming and do something about it?

Apparently not.

SAP announced yesterday that it was creating a two-tier support system, effectively reinstating its old Standard Support offering at a slightly increased price. (The new price is 18% of net versus the old 17% of net.)

This has been hailed as a U-Turn by press and analysts, all of which proves something to me: most writers can’t do math.

SAP begins its press release as follows (emphasis mine):

In a demonstration of its commitment to customer satisfaction, SAP AG (NYSE: SAP) today announced a new, comprehensive tiered support model that is being offered to customers worldwide. This support offering includes SAP Enterprise Support services and the SAP® Standard Support option and will enable all customers to choose the option that best meets their requirements.

So let’s look at the choice that’s being offered to customers; after you look at it, you can judge how much satisfaction it’s going to generate.

The cost of Enterprise Support this year is 18.36% of a base number, a number that usually stems from (but may not be identical to) the net amount paid for the SAP licenses that are being supported. So, this year, assuming that the base for a company was $100,000, the total cost of Enterprise Support is $18,360 and the total cost of Standard Support is $18,000.

Next year, the cost of Enterprise Support goes up to 18.9%, increasing to 22% by 2016. That means that in 2011, it is $18,900, and in 2016, it is $22,000. Those of you who are writers, I apologize for all these numbers, I know they do get confusing.

Now to Standard Support. With Standard Support, the percentage is fixed. But the base is not. It is subject to cost of living increases. We don’t know what COLA (cost of living adjustment) SAP will impose. But let’s just say for the sake of argument that it is 3.00%/year. In 2011, the cost will be $18,540.00, and in 2016, it will be $21,493.00.

All this is in a spreadsheet which you are welcome to look at and play with. (In the spreadsheet, I rounded 18.36% down, so the Enterprise Support costs are slightly low.) Assuming you’re not a writer and you want to play with the numbers, here’s what you’ll see. If the COLA is 3%/year, the costs of either kind of support will be very close for a long time to come. If the COLA is 1%, Standard Support will be quite a bit cheaper. And if it is 5%, Standard Support will be quite a bit more expensive.

It’s confusing, I know, but it’s true. If the COLA is 5%, then “18%” support will cost more than “22%” support. If the COLA is 3%, then “18%” support will cost about 2% less than “22%” support. And if the COLA is 1%, then the lower tier of support will cost about 10% less than the upper tier.

So what will it be. 5%? 3%? 1%? 0%? At the press conference, SAP didn’t say. There is no commitment to impose these increases and there is no commitment not to impose them. SAP, according to Léo Apotheker, “[has] the liberty of linking Standard Support [] to the cost of living index.” (Thanks Information Week.) Whatever their decision, the imposition of COLA will not be uniform. The cost of living index is the index for the country whose currency is the master currency for the contract, and the actual linkage to this actual index depends on the contract language, which varies.

So what are we to think? Whatever SAP is doing, it is not a U-Turn, and it is not a rescission of the price increase. It is offering customers a new choice, which I’ll characterize as follows:

1. Return to Standard Support and get less support than you got with Enterprise Support (though how much less is unclear) and price increases in the form of COLA (though how much increase is unclear).

2. Stay with/sign up for Enterprise Support and get more support (how much more is unclear, but I’ve been posting on this and will post more) and definite, clear price increases in the form of increases in the maintenance percentage.

It’s a choice. But is it really the sort of choice customers want?

And is offering this choice really an example of a commitment to customer satisfaction?

Two Cheers for Léo

November 2, 2009

The German Financial Times today took a bead on Léo Apotheker, SAP’s CEO, saying that on his watch, SAP had lost touch with its roots [verlorene Wurzeln]. No longer, as in the days of Hasso Plattner and Dietmar Hopp, is SAP customer-focused, the article says, and as a consequence, customers no longer think the software is worth the money. (The article cites the current customer unhappiness about increased maintenance prices as evidence for this.)

Helmuth Gümbel, no stranger to this blog, is cited frequently in the article; clearly, he was persuasive about the current state of affairs between SAP and its customers. Clearly, too, one disagrees with Helmuth at one’s peril.

Still, I wonder whether it’s really fair to hang all these problems on Léo. Take the infinitely hashed-over introduction and semi-withdrawal of Business By Design. Léo tends to get the blame for this because he was there on the podium claiming that SAP would get $1 billion in revenue from this product by, what is it, next year? But he had nothing to do with the original product. He was working in sales when Peter Zencke was put in charge of Project Vienna, and he was still in sales when Nimish Mehta’s team was developing T-Rex (a great product, no question), and he was still in sales when Hasso was insisting that T-Rex be incorporated into Business by Design, and so on.

Certainly, it was injudicious for him to promise that his organization could sell the heck out of a product that wasn’t ready for prime time. But why does this make him responsible for SAP’s lost roots? If anybody lost their roots, it was the development organization, which somehow or other couldn’t build the product it thought it would be able to build.

Don’t blame Léo for problems that were not of his making.

Now, I have my own issues with Léo, as my readers know. But frankly, when he came in, I think he was right about SAP. Too much time and money was being spent on stuff that wasn’t what the customer needed (or wouldn’t sell), and this had to stop. Hence the acquisition of Business Objects, the downsizing, the restructuring in the development organization. All of these things deserve at least one cheer, and I hereby give it. Hip, hip, hoorah.

Where I criticize Léo–and I’ve told him this to his face–is in his view of SAP’s role vis a vis the customer. He thinks what SAP has always thunk, that it’s up to SAP to make the best possible tools and it’s up to the customer (aided by the SI community) to figure out what to do with them. I think this view is wrong; SAP has to take more responsibility for making sure that the stuff works.

Ten years ago, when the heroes of the FT Deutschland article were fully in charge of the SAP business, I agree, SAP didn’t need to do that. SAP knew about businesses and about software, and they could figure out what they should do next without fretting about the problems customers were then having. (Believe me, there were a lot of them.) Today, though, with so much development time and development effort squandered, they can no longer believe that they can just build the right tools and count on the customers to get the benefit. Instead, they need to find out what went wrong with the tools they’ve been building and what needs to be done in the future. And the only way they can do that is to figure out EXACTLY what is preventing customers from getting the value they think they ought to get.

You can see why this problem is so important if you look at the SAP Solution Manager. This product ought to be what justifies the maintenance price increase. But as Dennis Howlett and Helmuth himself have both said, the product itself does not yet do what SAP needs it to do. If SAP really wants to justify this price increase, it needs to figure out why customers aren’t getting the value that SAP needs them to get. And they need to do it fast.

This is not easy; indeed, when I said this to Hasso late one night at an analyst party, he said, roughly, “We don’t know how to do that.” But let me just say, of all the executives I know at SAP, the one who is most likely to figure it out is Léo.

If he can figure it out, he will be able to get his customer base back again; indeed, I think they’ll be cheering, and when we’re convinced, so will I and so will Dennis and maybe even Helmuth. So, just in case this actually happens, let me give my cheer now, before it is actually deserved, as a way of saying, “I think you can do the right thing.”

Hip, hip, hoorah.

With Siemens asking for and apparently getting a huge break on SAP maintenance costs, it is time to take a look once again to take a look at the whole issue. Understandably, there is a lot of emotion and name-calling and confusion around this; it’s not quite the health-care debate here in the United States, but the real issues have been buried under rhetoric in a strikingly similar way.

SAP Charges for Improved Support

Let me first state SAP’s position as sympathetically as possible. SAP believes (quite correctly) that the customer’s TCO (total cost of ownership) ought to go down, as software and hardware gets better and cheaper. It also believes that if it does something that would help TCO go down, it should get a share of the benefits.

Who would disagree with either point?

Roughly two years ago, therefore, it introduced a series of software and support improvements that it believed would indeed reduce TCO. These improvements largely revolved around a newly improved version of the Solution Manager, a piece of software that is supposed to do what its name implies.

The Solution Manager (or Sol Man, as it is called familiarly) was actually introduced roughly ten years ago. In its original version, it was a separate piece of software (one that ran on its own Windows box) that one used to communicate with SAP support (filing bug reports, etc.) and to monitor the performance of your SAP installation.

In the version introduced two years ago, the Sol Man EE (or enterprise edition), it did considerably more: it allowed you to do more extensive monitoring, test and manage upgrades, and even document your business processes. This new edition also beefed up the connectivity with SAP Support, so that support could use it to troubleshoot your installation more rapidly and effectively.

When SAP introduced its new, higher-priced Enterprise Edition support package, it placed the Solution Manager EE front and center. In every speech and every press release, it wasn’t, “We’re raising prices because Oracle got away with it.” It was, “We’ve developed new tools and support services based on those tools, and we’re increasing the cost of maintenance, because the maintenance has improved.”

The tools it was referring to were the various components of the Sol Man EE, and the improved support services were made possible by and delivered through the Sol Man.

To sum up, SAP’s position is that it is improving enterprise support by providing customers with new and better support tools. It is in the software business. So it is only reasonable for it to charge for those tools. That it charges via a maintenance price increase rather than by charging for the product itself is reasonable, presumably, because many of the benefits involve improved services, which are provided through the maintenance contract.

The Customer Reaction

It’s just a plain fact, of course, that nobody paid much attention to this. It took me, for instance, almost a year (until John Krakowski’s excellent presentation at ASUG last May) to figure out what SAP was getting at when it talked about new tools. (Before that, I wrongly, but honestly believed that the talk about tools was pure hand-waving.)

Other commentators on this, like Vinnie Mirchandani or Ray Wang or Dennis Howlett , may have gotten to a proper understanding of the argument faster than I did, but for the most part, they didn’t try to address its merits.

Today, for instance, the Enterprise Advocates, gave a webinar on Reducing SAP Maintenance Costs. (The Enterprise Advocates include the aforementioned three, plus Frank Scavo and Oliver Marks.) Not once during the main body of the talk did they even mention SAP’s recommendation for reducing maintenance support costs, which is to implement the Solution Manager and use it.

I don’t blame the advocates for this; it’s not their job. But I do blame SAP. If the Sol Man is what justifies the price increase, then SAP need to explain this in clear language.

Once SAP fails to do this, the Advocates and the SAP customer base are entitled to believe what you and I would believe when somebody offers an unclear explanation for something that seems to require some explanation: they dismiss the explanation that’s offered.

At some point, though, it does seem that someone should give SAP the benefit of the doubt and ask the question that SAP wants you to ask, namely, “Can the Sol Man EE deliver so much benefit that it justifies the maintenance price increase?”

If the answer is, “Yes,” that would of course be the best thing all around. Customers would have a clear path to reducing maintenance costs. SAP would have a product that keeps its customers paying maintenance. Total cost of ownership would go down.

And the Answer Is…?

Over the past four months, I’ve spent a fair amount of time finding out what I could about the Sol Man. I don’t have access to the documentation (all 1000 pages of it), but I do have the more public documents that SAP has issued, and I have talked to a number of Sol Man users and consultants.

What I found out is so complicated, and this blog is already too long. So I’ll delay a full report to another post. But here’s the answer in a nutshell:

1. To get the benefits of the Sol Man absolutely requires significant investment on the part of the customer.

2. It does not appear to be the case that the Sol Man was designed with the goal that SAP now has for it top of mind. It appears to be a product that was designed to be one thing that is now being turned to a different purpose.

3. The areas of benefit that the Sol Man promises are indeed important, and it is at least possible that customers can get significant benefit from it, if they put in the work.

4. At the end of the day, though, it appears to this humble observer that SAP needs to put more skin in the game.

Hope all this whets your appetite for the next post on the subject.