Tuesday, January 27, 2009

Redefining the Mid-Market and Its Business Intelligence Requirements

Small and mid-sized organizations seem to get the short end of the stick when it comes to their software needs. Until recently, business intelligence solutions were largely targeted at the enterprise market. These solutions required solid IT infrastructures to support the deployment and maintenance of data warehouses, data marts, OLAP and the like. Consequently, even many upper-mid market corporations (i.e., those at $850 million and above) were unable to take advantage of what was available. With limited budgets and lack of internal resources to develop and manage solutions, business intelligence (BI) became the elite software that was relegated to a subset of super users within enterprise organizations.

With the expansion of solution options and the shift toward the mid-market, the BI industry became inundated with newer solutions and niche players that removed the former barriers to entry that existed for many organizations. Although many of these solutions existed beforehand, renewed focus on marketing activities from the larger BI vendors and general confusion caused by a number of vendor consolidations enabled newer players and diverse offerings to shift positions and to become valid options for mid-market companies.

Even with this recent shift, however, the mid-market itself is still ill-defined. Instead of breaking out the market in a way that organizations can relate to, in too many cases, the mid-market is considered to be organizations that make less than $1 billion in revenue per year and small organizations considered anywhere below $250 million or $100 million depending on the source. Consequently, these organizations are lumped together but may have very different needs based on size, revenue, resources, budget, vertical industry, etc.

This article explores whether mid-market needs are being met by the solutions available, some solutions that benefit the mid-market specifically and the need to redefine the mid-market in order to better meet the business requirements of these organizations.

Are Mid-Market Needs Being Met?

Within business intelligence and especially with the more recent expansion of solutions that target mid-market companies, the question still remains: Are these new solutions actually meeting the needs of small and mid-sized organizations? Do stripped down enterprise solutions make the cut, or do some mid-sized organizations actually require the same solutions as their enterprise counterparts but are being pushed aside because of their inability to pay for the infrastructure and upkeep? These are questions that have long been debated; and although they are valid in certain respects, as diverse solution offerings become available and organizations have more choice, they become secondary.

Whether or not pared-down solutions targeted to mid-market companies actually meet their requirements or fall short, little can be said. There is a lack of cohesion existing within the market in terms of what small and mid-sized companies really need to make business intelligence successful. Aside from pointing out what mid-market companies may lack – for instance, budget and IT infrastructure – there is little agreement on the similarities within mid-sized companies and the key requirements needed to meet their BI needs. In addition, lumping a group of companies together based on how they fall outside of traditional BI offerings does little to offer those companies the additional value or unique requirements that will help them get the competitive advantage necessary to grow their business or to maintain their positioning.

Mid-Market Specific Solutions

The mid-market solution landscape is broad. With traditional BI vendors offering targeted mid-market products that resemble a subset of an overall suite, with on-demand and open source vendors offering solutions at lower price points and disparate pricing models, and alternate solutions targeting the smaller enterprise, companies are actually well poised to take charge and demand the types of solutions that most suit their unique requirements. Organizations have the upper hand as they can pick and choose their way through various solution choices, pricing options, and deployment methods. This means that although the business requirements of mid-market organizations have not been analyzed to the depths of their enterprise counterparts, the sheer volume of solutions to choose from balance out that deficit.

On-Demand on the Rise

On-demand solutions may be the best poised to meet the needs of small and medium businesses (SMBs) simply because many of these organizations are used to and comfortable with the on-demand model. Because many smaller organizations preferred to have solutions hosted and kept off premise, basically managed as a set of services, the rise of BI-based software as a service (SaaS) provides many SMBs with the type of solutions with which they are most comfortable. In general, on-demand gives organizations the advantage of full data management, reporting and analysis without the hassle of development and maintenance.

Aside from traditional business intelligence turning to the on-demand model, the rise of embedded analytics as hosted solutions enables organizations to take advantage of operational or pervasive business intelligence without having to identify and integrate any solutions in house. In addition to solutions such as LucidEra, that provide embedded analytics and reporting for salesforce.com, other solutions are available that provide analytics, reporting and visualization on top of ERP and CRM solutions to give organizations a broader view of what is happening within their organizations.

Leveraging Open Source

With the expansion of the data warehouse appliance market, organizations are not stuck paying hundreds of thousands of dollars for a data warehouse or series of data marts. Microsoft’s SQL Server is just one example of how organizations can implement a solution at a lower price point. Vendors such as Greenplum and Infobright take advantage of open source solutions as the core of their products, enabling a lower overall price point. In addition to data warehousing, solution providers such as Information Builders and their integration of R – an open source statistical analysis tool – enables them to use high grade predictive analytics without affecting overall pricing to customers.

Open source enables mid-sized organizations to take advantage of solutions, either directly or through OEM solutions that they may otherwise not have access to. With open source offerings such as those provided by JasperSoft, Pentaho and Actuate, organizations can take advantage of business intelligence for free. The catch is that generally open source solutions are not plug and play, so without a developer in house the solutions may not provide the added benefits of business intelligence with a clean user interface. However, the option to deploy a solution free of charge has pushed many vendors in the market to offer free trials of their products to help end users get a feel for what is available before committing to a single solution.

Final Thoughts

Overall, business intelligence can offer great advantages to small and mid-sized organizations. The problem becomes sifting through the plethora of options to select offerings that meet the organization’s needs and that offer the flexibility required in an agile marketplace. In addition, generalized categories of where small and mid-market companies fit are too broad and need to be reorganized to take into account the requirements of these companies. Because using an umbrella, such as companies with yearly revenues of under $1 billion, it would be wise to generalize the needs of companies small and large alike without taking into account their differences.

Source:b-eye-network.com

Mobile BI Emerges on the Apple iPhone

I was disappointed to find Apple missing from the IE Editors' Choice list. There's a "perfect storm" brewing that just might elevate the iPhone as the tool of choice for corporate types, not just for e-mail but for BI on the go.

Apple has, of course, long been the consumer's delight and the corporation's despair. Apple leads digital convergence in our personal lives – the iPod, iPhone and Mac work together like no other set of these technologies and are beautiful to boot (pun unintended) – but is still a rarity on the corporate landscape. This is partly because for years, Apple has maneuvered itself out of corporate reach (price uncompetitiveness being no small a factor) and partly because Microsoft Windows and Windows applications are so firmly entrenched on the corporate desktop.

But it looks like things might be about to change, with a groundswell led by the acclaimed iPhone.

"At least 15 to 20 percent of people who buy the iPhone are going to use it for business reasons," says one research analyst. He believes people "have gone out and bought these things and have taken it to a company and said 'make it work,' or have made it work somehow." This isn't just talk: Companies like Genentech, Nike and Walt Disney are reportedly planning to support it.

Looming over this groundswell are business and BI solution providers striving to reach down and increasingly empower users on the go, with features such as location awareness, single authoring capabilities (so that developers don't have to develop multiple versions for multiple deployment devices) and incremental reporting capabilities (e.g. Cognos 8 v4 now supports prompted reports for a more interactive experience). Applications like Oracle Business Indicators and SalesForce.com CRM Mobile for iPhone are opening the corporate doors ever wider for iPhone. (While on the topic, check out this list of 10 Great iPhone Apps For Your Business.)

Solution or content componentization, leading to collaborations and mashups (e.g. using Google Maps for location-aware BI solutions), compounded by the unparalleled usability of the iPhone, will only help increase iPhone penetration in the corporation. On the desktop, features such as native Windows execution on the Mac and desktop virtualization are equally well poised to accelerate Apple's corporate convergence.

It would be interesting to see if this groundswell for the iPhone (and Mac) leads us to a new trend: employees purchase their own devices, subsidized (to a varying extent) and supported by the employer.

One way or the other, Apple is likely to a force in the hunt for business-on-the-go solutions, and business intelligence stands to be a major beneficiary. And it would be nice if IntelligentEnterprise.com finds a category to place visionary and potentially influential entrants like Apple in the Editors' Choice listing.

Source:intelligententerprise.com

Wednesday, January 14, 2009

Netezza Announces New Interoperability with Microsoft's BI Platform

Netezza's interoperability play with Microsoft is either a big deal for Netezza or an even bigger deal for Microsoft
By Stephen Swoyer


Netezza Inc. last week appeared to close the loop on one of its perceived competitive shortcomings when it unveiled a new interoperability strategy for Microsoft Corp.'s business intelligence (BI) platform

Until now, Netezza's data warehouse (DW) appliances have often been seen as rip-and-replace propositions, designed chiefly to displace existing DW systems. Competitors Dataupia Corp. and ParAccel Corp. claim to work with (to complement and accelerate) existing DW platforms; Netezza, they contend, requires that customers move away from these systems, at least as platforms for enterprise data warehouses (EDW).

Netezza's new strategy focuses on improved interoperability with Microsoft's Office, Performance Point Server, and SQL Server-based BI assets: namely, Analysis Services, Integration Services, and Reporting Services. The announcement may be seen as a big deal for Netezza, a bigger deal (with broad DW ramifications) for Microsoft, or a subtle refinement of the status quo.

There are respects in which Netezza, a pioneer of the modern DW appliance model, is perceived as more of a greenfield DW option -- or, in some cases, as a rip-and-replace alternative to existing IBM, Microsoft, or Oracle data warehouse systems. Consider the perspective of a DW architect based in the UK, who -- when interviewed late last year about Oracle Corp.'s then-new Database Machine -- indicated that he'd opt for a Netezza-only solution, had he his druthers. "[I]f it were purely down to me, I'd rather just scrap the current system [mixed Oracle and SQL Server data warehouses] and start again with Netezza. We'd have a brand new, dazzlingly fast solution within a few weeks," he said. "The best project I ever worked on was a greenfield telco [in] which I was lucky enough to select and implement a Netezza system back in 2003. It was a dream come true to someone who has worked with Oracle for many years."

Netezza officials acknowledge as much. "We're looking at a broadening market for Netezza as we go forward. We're starting to move into more and more mainstream types of accounts in the industry," says Phil Francisco, vice-president of product management and marketing with Netezza. "In doing so, what this gives customers of Netezza in those sorts of environments is the ability to stay with a Windows or Microsoft-based application platform, which they've probably had for quite some time, and still enjoy the benefits of a very high-performance data warehouse behind it."

Francisco rejects the idea that Netezza has been more of a greenfield- or rip-and-replace proposition, however. "Previously, we were able to use ODBC to connect to the Microsoft applications, so you'd have OLE DB running over ODBC," he points out.

If that's the case, what's particularly new about Netezza's claimed "interoperability" with SQL Server and Microsoft's BI stack? In this case, too, Netezza is touting OLE DB connectivity. It's via OLE DB, in fact, that Netezza claims to deliver on its promised interoperability with Office, PerformancePoint, and SQL Server-based BI services.

The difference, Francisco insists, is in the implementation. "In the past, [interoperability] was possible, [but] it wasn't particularly high performance. This OLE DB [connector] delivers high-performance connectivity. One of the things we've done is we've implemented the connector on our platform without creating a lot of overhead, so we're able to get extremely high performance out of the data warehouse system itself."

Francisco demurs when asked to talk more specifically about Netezza's interoperability strategy vis-à-vis Microsoft. Right now, he said, there's the OLE DB connector; tomorrow, Netezza is mulling several options -- including, possibly, .NET connectivity. "We were at a crossroads of adding either a .NET provider or an OLE DB connector. In consultation with Microsoft, we decided to go with the OLE DB connector," he indicates.

"The fact that now you can use the full [SQL Server] toolset, including Integration Services, means that there can be high-throughput data sets getting moved back and forth." Will Netezza commit to delivering the .NET provider? What other collaborative efforts is it mulling with Microsoft? Francisco punts: "There's nothing that I can talk about today. It's all part of a larger strategy that we're not talking about in great detail right now. Certainly, our desire is to make sure that that level of interoperability increases."

Trouble with Madison?

One thing that Francisco and other appliance industry watchers do want to talk about is Microsoft's Project Madison strategy, its ongoing project to incorporate (assimilate) the assets of the former DATAllegro Corp. -- including that company's shared-nothing, massively parallel processing (MPP) technology.

With this in mind, Francisco says, the partnership between Microsoft and Netezza suggests that Redmond envisions a big DW tent, with room for several different players. "In the near term, DATAllegro doesn't complicate it [our relationship with Microsoft]. In the long term, I don't know. It may or may not. It's really hard to say. The Project Madison work that they have going on at Microsoft [is designed to] incorporate the DATAllegro elements to get higher performance and a higher level of scalability to multiple terabytes, but that is a long-term project. Even by Microsoft's current reckoning, they're talking about availability sometime in 2010," he comments.

"What they see now is that Netezza has growing market momentum. We've now reached the point where we have hundreds of customers online, and we've continued to grow, so Microsoft is clearly a good partner for Netezza and vice-versa."

Netezza veteran Foster Hinshaw, now a principal with rival appliance player Dataupia, has a much different take on the announcement, as you might expect. To Hinshaw, Netezza's accord with Microsoft suggests that Redmond is having more trouble than it first anticipated with Project Madison. "I think it's really Microsoft's acknowledgement that the DATAllegro play didn't work for them, and they have to look at other solutions in order to solve some of the issues that they have," he argues.

News Attracts Little Attention

Perhaps not. After all, the Netezza interoperability story generated nary a blip -- much less a press release -- on Microsoft's in-house PR page. Microsoft officials, for the record, did not respond to requests for comments from BI This Week. In fact, the partnership seems to be a Netezza-driven affair, with Netezza building -- or enhancing -- linkages to Microsoft's BI platform.

Francisco says that Netezza is an "active" partner with Microsoft. Moreover, he maintains, the partnership -- including any as-yet-undetermined areas of collaboration -- will benefit Netezza and Microsoft customers alike.

"It would suffice to say that we are part of the Microsoft Partner Program and that we do have active conversations with them," he comments. "I think that prior to this, we had customers that had Microsoft applications, but this was really a good stepping-off point for making that partnership stronger."
Click Here!

Dataupia's Hinshaw maintains that Netezza has always had connectivity solutions for Microsoft BI. More to the point, he contends, Netezza has always had OLE DB connectivity to Microsoft BI.

"That connector has been around forever, and they've been using it with different customers," he says. "I'm sure they've optimized it from where it had been," Hinshaw adds, returning -- again -- to the issue of Microsoft's Project Madison play. "I think it is a more significant [announcement] than the little bit the press gave it, because it does show Microsoft's interest in the data warehousing space. It shows that DA didn't do what they needed it to do for SQL Server."

Francisco argues that it behooves Dataupia to raise doubts about Microsoft's execution with Project Madison, as high-end SQL Server data warehousing is more of a threat to Dataupia -- with its SQL Server accelerator strategy -- than to Netezza. "[Dataupia's] strategy tends to be more focused around the acceleration of existing data warehouse platforms, including SQL Server," he concludes. "What we're really talking about is the ability to use the application layer services with SQL Server with the base data warehouse system being a [Netezza] NPS platform, so you get the performance of the NPS appliance and the familiarity of the Microsoft applications. We're not talking about using [NPS] as an accelerator [for SQL Server] like they do."

Source:esj.com

Thursday, January 8, 2009

How Microsoft plans to make its mark in CRM

Microsoft Dynamics CRM has become a key product for the company, according to CRM division general manager Brad Wilson--and it's an area the software maker plans to invest in further.

According to Wilson, when Microsoft earlier this year committed itself to investing US$1 billion annually in the construction of new data centers to support the Microsoft Live portfolio of on-demand software, some of that sum--he declined to specify how much--was earmarked for CRM (customer relationship management).

ZDNet Asia's sister site ZDNet UK spoke to Wilson in November 2008 to find out how he intends to make that CRM investment and whether he thinks it will be sufficient to beat on-demand CRM leader Salesforce.com on its own turf. Wilson made it plain that, despite the small and midsize business bias that Microsoft CRM may have acquired, it is equally aimed at the enterprise.

Q: Microsoft has been in CRM for some time, but just how long?
Wilson: We've been in the CRM market for five-and-a-half years, and have more than 16,000 CRM customers and more than 750,000 users.

Microsoft is investing more annually on data centers than the complete revenue of all the on-demand players--US$1 billion annually. As a company, we are investing heavily.

How is that customer base made up?
More than half of the business is international--outside the U.S.--and, although we started out as an organization focused mainly on the small and medium-sized businesses, more than half of our revenue now comes from large enterprises.

Salesforce.com is very popular, so how do you compete with it?
We do and we have people who switch from Salesforce.com fairly routinely. But there are a number of companies that compete in this space and everyone wins some and everyone loses some.

What I like about our strategy is that Salesforce.com has a single operating model, which is that you rent it from them forever. Our software comes with a choice of either having an on-demand subscription offering or buying the software. If you talk to analysts today, they will tell you that, of all the CRM in use throughout the world, probably about 90 percent is deployed on the premises.

We want to give our customers the choice. Whether you want to go on-premises or to a cloud-based offering, the choice is yours.

Does this mean you don't find the cloud particularly valid?
It is not so much that--and this is a somewhat controversial view--but I don't really care. I have an agnostic model: if you want to buy the CRM software, great; if you want to go ahead with on-demand, that's also great.

For us, it is a single codebase. It's literally the same software running.

There is no difference between on-premise or on the network. The only difference is how long your network cable is. I think sometimes people get too caught up in it. The world is not really binary.

What new products are coming up?
About 11 months ago, we shipped our CRM 4.0, which is a fully multi-tenanted system that you can deploy from outside the cloud.

What we announced [earlier this month] is a new set of accelerators for CRM 4.0 [on sale Dec. 1]. These are extensions to the core system so that, when you add them to CRM 4.0, they offer capabilities such as e-service for handling Web cases. This lets customers submit cases online and check things such as status very easily.

What do you mean exactly by 'cases'?
Well, if you have a broken fridge and you want to contact the retailer or manufacturer, then you can go to the Web site and that will submit information that will flow into the CRM system and then trigger a workflow.

So e-service is really that Web interface to customer services, as opposed to the call center. What we provide is the data and the workflow to support e-service scenarios.

Another one is extended sales forecasting, which is a way to lock and manage forecasts that goes deeper than the usual sales-automation facilities.

There are new analytics in business intelligence; sales methodologies. And then there is enterprise search integration through SharePoint.

But are these new?
We talked about them at our partner conference back in July in Houston but we haven't had a customer launch before [this month].

So presumably these are intended to deal with the view that Microsoft has not been in the CRM market very long and that, to understand and execute sophisticated CRM, you need a specialist supplier?
The wildly less popular ones? The ones that have had staggering adoption problems over the past 10 years? I think there is really a philosophical difference [between Microsoft and those suppliers]. We are not going to give you a gigantic list of features. You know how you want to run your business.

So, for us, user adoption is key. If they [the users] are not going to use the system, you are pretty much guaranteed a failed deployment.

We give you enough flexibility so that you can run the system how you want to. So I find we will beat a classic offering from your CRM vendors on end-user adoption and platform flexibility. Those factors will far outweigh the fact that other people have more pre-built stuff.

We went into a sales opportunity against a classic CRM vendor and measured its software. Out of the box, [only 7 percent of the software could be run without modification].

When you think about it, it is very difficult to sit in Palo Alto or Redwood and design something that is going to fit any business--a system that will work with every business in the world, whether it is in Turkey or South Africa. So the key now is flexibility. How easy is it to add the stuff we need?

I think the old model of 10 years ago, where you built a system that had a big slab of stuff that you had to adopt, has gone.

At the same time, we will still bring out our accelerators with pre-packaged software, and more and more of them. But we release them as open source. The idea is that we just put this stuff out there and let people use it. And, if our partners use it, all the better.

So are these products free?
Yes.

But you are charging people for the software.
You have to buy the core license but, once you have bought it, we are not going to try and nickel and dime people for bits of process and functionality. We don't believe in that.

We are taking the approach of wanting to make CRM much more affordable. Affordable in terms of TCO (total cost of ownership).

Part of that is in the core. We think we've done a pretty good job there and we keep adding pieces of incremental value through the accelerators. Even in the on-demand space, we want to go in and make it more affordable. CRM in on-demand tends to be relatively overpriced. So we want to make that price come down.

How do you charge? It is on a license basis?
We have a server price and a user price--what we call a server license and a client-access license. The server price is nominal, relatively low and doesn't tend to go up. The primary driver of price is how many people use it.

Source:zdnetasia.com

Sunday, January 4, 2009

Nine BI Megatrends for 2009

Open source software, rich, dynamic interfaces and column-store databases: these are just a few of the trends that will reshape business intelligence and information management in the year ahead. Get ready for the next waves that will help you ride out a tough economy.


By David Stodder

The world may be flat, as author and New York Times columnist Thomas Friedman has observed: but these days, it feels more like a deflated soufflé. Great sums of real and imaginary wealth blew out of the economy like overheated gas, leaving behind a sticky, gooey mess. Financial crises have been swirling about the globe, sucking the life out of commerce, launching a global recession and exposing weakness in nearly every business model. Sophisticated use of information and the deployment of advanced algorithms and software processes clearly helped organizations uncover opportunities and develop information-based products when things were going up. But apparently too few employed their wealth of knowledge to avoiding the downsides of risk. Now, organizations will need many of the same sorts of tools to scrutinize processes and decisions that exposed them to their current troubles and discover how to survive and prosper in a remade economic world.

During such difficult and rapidly changing times, it would be hard to argue against the importance of business intelligence for providing executives, managers and front-line knowledge workers with the tools they need to use information effectively. While it has always been tough to measure the exact impact of BI and data warehousing on overall business performance, it is certain that in many organizations, implementation of BI and data warehouse tools and technologies has forever changed how users access, analyze, report and share data. Excellence in these activities is for many organizations as important as transaction or process efficiency, and essential to the cycle of improvement. In other words, no matter how bad things get, it is unlikely that organizations will turn away from BI and go back to more primitive tools and methods.

Yet, there is also little doubt that BI and data warehousing providers and professionals will have to adjust to a deflated world and refashioned expectations and objectives. There will be less patience with delays in the development and deployment of systems and promised returns on investment. The margin for error will be tighter, with fewer second chances. CIOs, CTOs and business analysts will be under pressure to manage costs associated with information analysis and reporting.

With the omnipresent economic challenges serving as backdrop, what are this year's BI megatrends? Which BI technology developments are most important to your organization? This article describes nine important trends that are shaping the future of business intelligence and information management, from open source software and embedded BI to analysis of data relationships and business models to data warehousing alternatives including MapReduce and column-store databases. But first, let's consider what happened with last year's top megatrend.

Consolidation Conundrums

Twelve months ago, the uber-megatrend was industry consolidation. IBM, Oracle and SAP had just gobbled up Cognos, Hyperion and Business Objects respectively; together with Microsoft, these vendors seemed poised to bring down the curtain on the old BI industry and begin a new phase dominated by enterprise application and information management technology heavyweights.

As it turned out, in 2008 these four giants made significant technology moves but spent a good deal of their time under the hood, retooling their newly integrated and sometimes overlapping product, marketing and sales strategies. As expected, they have positioned BI as part of something they offer that's bigger; in fact, the term "business intelligence" doesn't get much respect anymore. IBM, for example, positions BI as part of a larger thrust toward "business optimization," which it sees as different from mere application automation. Optimization is about using information as a strategic asset to improve business processes, performance management, models, forecasts and policies; it can involve a range of technologies, not just traditional BI tools for data access, reporting and analysis. Oracle and SAP take a less visionary approach, offering the tools as items in their middleware or solutions portfolios (although Business Objects, as an SAP company, still has high visibility). You have to do a bit of searching on Microsoft's site to find information about BI, but that could be because it takes a village; Microsoft's Business Division, Data and Storage Platform Division and the Office Business Platform Group all contribute technologies that BI developers, IT managers and business users are seeking.

The internal machinations of the big four have left room for independents, including Actuate, Information Builders, Jaspersoft, MicroStrategy, Pentaho, SAS, TIBCO Spotfire and others in analytics, performance management, search and vertical industry areas. On the data warehouse systems and middleware side, IBM, Informatica, Oracle and Teradata come into 2009 as heavyweights. Not much was heard in 2008 from Hewlett-Packard's NeoView, but HP made its presence felt through partnerships, particularly with Oracle to produce a data warehouse appliance. Innovation in information management is being driven by increasing customer demand for more timely and diverse data, faster query performance, better data quality, master data management (MDM) and more. Appliances, column-oriented databases and other new approaches are providing organizations with more options and are sending a message to the heavyweights that complacency equals vulnerability. Even Teradata has loosened up its enterprise data warehouse dogma to bring to market an appliance that fits the price and performance requirements for departmental data marts.

Megatrend 1: The Impact of Open Source

Some commercial vendors might dismiss it, but open source is a growing factor in BI, data integration and data warehousing, and is clearly worthy to be cited as the first megatrend. The potential for lower total cost of ownership (TCO) is certainly attractive, but there are other important reasons that the open source BI movement is gaining momentum. Many organizations and developers are ready to build up from generally positive, maturing experiences with systems constructed using the LAMP stack (Linux, Apache, MySQL and PHP, Perl or Python) and the Eclipse development platform. Others like having more control over the code and direct involvement in the process of updating the software; these factors are important for organizations that prefer incremental deployment where users test components, provide feedback and revise requirements along the way rather than wait for finished systems. This is critical because in many organizations, BI deployments are delayed when user requirements have not been met or have changed during the time frame of the development cycle.

New vendors are drawn to open source because it offers a lower-cost, contemporary marketing model. Working with communities, partners, influential developer forums, social networking sites and bloggers, these vendors are executing marketing plans that creatively " but not too aggressively " build awareness and try to increase download rates. Coming on too strong can have negative repercussions, so wiser vendors calibrate their marketing to build confidence so that downloaders of free software become buyers of feature-rich, enterprise versions. Vendors also have to alleviate concerns that open source products will hit a ceiling as organizations scale up. Buoyed in most cases by infusions of venture capital, open source BI vendors have been developing their products rapidly. Pentaho's BI Suite Enterprise Edition, for example, offers features that are comparable to commercial BI products. Actuate, which has its roots in the commercial software market, based its current major release (Actuate 10) on the Eclipse Business Intelligence Reporting Tool (BIRT) project.

The most prominent open source data integration vendor is Talend; several open source BI and data warehouse vendors resell the company's products. The ability to customize data integration middleware through access to the source code is attractive because like it or not, many organizations need to tailor such routines to their requirements. Out-of-the-box routines only go so far. Cost is also a critical factor, given that commercial software and services for data integration have traditionally been one of the more expensive parts of the BI and data warehouse stack.

Finally, thanks to open source, database management for BI, analytics and data warehousing are undergoing something of a renaissance. Kickfire, Infobright and others are focusing particularly on the needs of maturing MySQL shops and are using aspects of the open source model to gain traction. In sum, the open source style of marketing products and services is changing how nearly all vendors " not just open source ones " engage with customers and introduce new technology.

Megatrends 2, 3 & 4: BI Tool Innovation

With the economy in turmoil, the last thing users need are BI tools that lock them into older ways of accessing, analyzing, reporting and sharing information. Search engines, mashups and social networking have changed the way most people interact with information and have raised the bar for BI tools to provide a similarly rich experience in a business context. Of course, therein lies the rub; business requirements are tougher than typical searching and sharing. Nonetheless, vendors are making progress in modernizing the BI experience, certainly compared to its roots in batch querying, canned reporting and analytics that once required hand-holding by PhDs. Here are three BI tool megatrends to track in 2009:

BI becomes less isolated. As analysis, information access and reporting become integral to operational decisions and actions, BI tools and methods cannot live in a silo. A recent Ventana Research "Operational BI benchmark study found that most operational users currently employ reporting, access and analysis tools that come with functional applications rather than dedicated BI suites. While the dedicated suites can help break down silos, these deployments cannot simply become bigger BI silos; they have to be well integrated with, if not embedded in applications, services and processes that are important to operational users.

One frequently overlooked need is the integration of BI and performance management with project management. Managers need timely information and analytical capabilities to help them track multiple projects and evaluate with data whether ongoing projects are meeting strategic and budget objectives. KnowledgeRelay is an innovator in this area; its BI and performance management products and services have evolved out of experience in aerospace, defense and other industries known for having numerous interlocking and complex projects. From the project management side, Planview, a leading project portfolio management (PPM) vendor, beefed up the BI components of its Planview Enterprise suite in 2008.

Users demand a richer experience. Anyone looking at the latest BI user interfaces can see that the days of simple, canned reporting supplemented by ad hoc querying functions are slipping into the past. When BI first began migrating to Web browsers, functionality seemed to take a step backward. But now, BI dashboards and portals are on a path to become richer and more flexible. This is in large part due to their implementation of rich Internet application (RIA) tools and methods such as Adobe Flash. Actuate's Actuate 10 offers the most fully developed merger of RIA with BI, but other vendors are also delivering rich and dynamic BI interfaces.

On the R&D drawing boards of many vendors are tools to create mashups, allowing users to combine different data sources and visualizations in a self-service manner rather than having to rely on IT developers. Of course, mashups may require some users to jump through not just technological but also political, managerial and security hoops regarding what can appear on shared BI dashboards and portals. However, the richer interfaces have the potential to enable users to integrate "at the glass" a wider variety of data sources, such as from geographic information systems, external demographic data and more. Look for search, wikis and blogs to also play starring roles in BI dashboards and portals.

BI will focus on relationships. "Connect the dots" has become a mantra in government intelligence circles, but it is important for other objectives, such as customer experience management, risk and security management, Web data analysis, supply chain management and business performance management. For most of its history, BI has focused on reports that do not provide much flexibility for investigating data relationships, leaving this analysis to specialists using data mining or statistical analysis tools. But as BI becomes integral to operational decision making, users will need to examine real-time data, text and alerts in a context that improves understanding of relationships and dependencies. This requirement will increase the need for master data (or data relationship) management tools at the integration layer, but BI and business user analytics tools will also play a critical role. TIBCO Spotfire's DecisionSite and Analytics Server are pacesetters, with recent releases enabling organizations to visually analyze data relationships, including, for example, those created during drug tests or by real-time interactions between participants in social networks. And coming from a data-integration middleware direction, Composite Software introduced a new product, called Composite Discovery, which uses search, metadata and other techniques to discover and display data relationships.

Megatrend 5: Business Modeling Meets MDM

BI must support requirements for greater business agility at a time of rapid changes in the economy. Tools that offer self-service information access, analysis and reporting are critical, but users need more. They also need the ability to build and revise business models in ways that can be easily mapped to underlying data models. Business analysts and users have traditionally had a difficult time understanding data models, which can be complicated, hard to visualize and difficult to understand because they are often expressed in a specialized language or methodology. Important requirements can be lost in translation, leading to delays in the deployment of BI and data warehouse systems when functions must be added or significantly modified at later stages.

The megatrend here is about MDM and emerging semantic models, which could serve business modeling in the same way that data models, schema and metadata have served extract, transformation and loading (ETL) tools. However, MDM and business modeling must be integrated with underlying data models. Some vendors in the MDM market, looking for ways to appeal more directly to potential business user customers, are developing tools to improve business modeling and its relationship to data modeling. Kalido is at the front of the pack; in 2008, the company introduced Business Information Modeler, a graphical tool for business users and analysts to develop conceptual models that can more accurately represent information needs and business processes. Another vendor to watch is Expressor, led by Bob Potter, who was previously CEO at Kalido. Expressor's Semantic Data Integration System offers a set of component products that focus on the integration of business modeling with data transformation and business rules.

IBM Cognos is also addressing these needs with Business Viewpoint, which it introduced as part of IBM Cognos 8 version 4; integrated with IBM's InfoSphere MDM Server, the technology enables organizations to align business modeling with MDM. An important element of the Cognos offering is dimension management, which helps organizations improve their governance and maintenance of hierarchies and dimensions. These tasks prove difficult as the number of users, the size of the artifacts and requirements for refreshing the data increase. Dimension management is also featured in MDM products from Microsoft and Oracle, but IBM Cognos is ahead in enabling business users to have access to this management functionality in the BI platform.

Megatrends 6, 7 & 8: Breaking the BI/DW Mold

Pressures to provide more timely and comprehensive understanding of business performance, customer behavior, supply chain management, regulatory compliance and more are driving significant changes in BI and data warehouse tools, systems and architecture. The result has been one of the more creative periods in technology for analytics and information management in recent memory. Some innovations employ very large memory and near-line storage to provide faster querying and greater data availability; others achieve results through new programming and event processing techniques. Most of these innovations take advantage of hardware advances in parallelism, virtualization and blade servers to enable higher performance at a reasonable cost. Here are three megatrends to watch:

MapReduce meets large-scale data analysis. Implemented most famously by Google and Yahoo but also available in the open source Apache Hadoop framework for data-intensive distributed applications, the MapReduce programming technique is suddenly the rage. MapReduce is growing in importance as organizations begin to build parallel, virtualized architectures based on server farms using commodity hardware, in many cases to support anticipated growth in cloud computing, or software-as-a-service (SaaS). Not everyone is on board with MapReduce; database luminaries Michael Stonebraker and David DeWitt argue that it is a step backward (see "The Database Column" ). Nonetheless, data warehouse appliance vendor Greenplum and analytical database provider Aster Data Systems are the first out of the box with MapReduce technology. More are sure to follow.

Column-oriented databases take aim at performance woes. In the recent Ventana Research "Optimizing BI and Data Warehouse Performance" research study, more than half (58%) of participants said they are experiencing sometimes nightmarish performance problems when they have to scale to run more complex queries, and nearly half (48%) said they have the same problem when scaling to load more data. As a result, they are canceling important queries when they simply run too long. Thus, it's not surprising that the study found that organizations are evaluating appliances and column-oriented databases to remedy problems in these areas.

Sybase IQ is the established column-oriented database system; Infobright, ParAccel, and Vertica are leading contenders. In addition to column orientation, these products employ advanced compression technology and large memory to change the game for BI and data warehouse architecture. It's fair to expect that in 2009, database heavyweights IBM, Oracle and Microsoft will continue to make moves to increase their presence in the market for appliances and perhaps even column-oriented databases.

Event processing opens new analytical possibilities. Before the financial services industry cratered, that was where most of the work in event or stream processing could be found. Now, while algorithmic trading and other processes still consume this technology, the spotlight shines brighter on emergent applications in healthcare, telecommunications, government intelligence, IT management, gaming and Web analytics. Network events, sensor data from radio frequency identification (RFID) tags and surveillance data are among the new sources. Capturing events, correlating them and presenting the results of analytics in dashboards can potentially give organizations more actionable insight than traditional BI tools provide. However, to gain full business value, event processing must be deployed in an integrated fashion with not only BI and data warehouse systems but also process management and service-oriented architecture. IBM, which acquired AptSoft in 2008, also offers Cognos Now (formerly Celequest) and is flexing its muscles. Coral8, Progress Software (Apama), Streambase, TIBCO and Vitria are also important technology providers.

Megatrend 9: Too Big to Fail

Queries running against multi-billion row tables; data warehouses managing hundreds of terabytes and adding more every day; thousands of users dependent on information and analysis for daily operational decisions: Clearly, BI is something that organizations can't afford to take lightly. Yet, many organizations fly blind when it comes to managing workloads and setting priorities for improving performance. Research finds that while many IT database managers meet regularly with business users to solve performance problems, they often lack hard information to aid decisions about tuning existing systems and deploying new technology. Thus, a final megatrend is that we will see more implementation of BI to improve BI. Appfluent is a leading vendor of workload analysis and tuning tools that are being applied to BI performance management; Teleran is another.

Tighter budgets and a critical need for better alignment between business priorities and technology decisions mean that BI and data warehouse systems must be effectively and efficiently managed. New technology deployments and upgrades to existing systems must be carefully chosen to provide the greatest return on investment. Fortunately, thanks to a torrent of innovation in recent years, organizations have many technology options in front of them. While data security and availability concerns must be addressed, organizations are likely to see SaaS and cloud computing alternatives as part of the mix for BI, analytics and data warehousing as we progress into 2009.

The megatrends presented in this article show that BI is hardly a settled domain; there's too much at stake for organizations to stand pat when it comes to using information to improve decisions and take action. Failure is not an option when it comes to BI. It will take smarts to ride out a tough economy and identify where opportunities for growth lie. Fortunately, the BI market is responding to demand.

David Stodder is an analyst and writer focused on business intelligence, analytics and information management. He is a research fellow with Ventana Research, where he was previously vice president and research director with a concentration on information management, operational intelligence and IT performance management. He was the founding chief editor of Intelligent Enterprise.

Source:intelligententerprise.com