Technology continues to evolve as we move toward a future of modular digital experience technology architectures that are flexible and easy to maintain. As an industry body advocating for this future, one of our core objectives is supporting buyers to navigate the journey to composable architectures. Including when and where to start, and how to evaluate the right vendors. Our certification program is designed to help, with vendors, SIs and enablers evaluated against strict criteria.
Analyst evaluations like the Forrester Wave(™) and the Gartner® Magic Quadrant™ remain a hugely important resource for companies considering and evaluating solutions for their stack. However, there is some confusion in the market about how to read these reports through a MACH lens.
You are not “everyone”
It is important to remember that there is no such thing as a generic analyst evaluation. Every company has a different set of requirements for their business. Simply looking at the vendor positioning with the dots in the Wave or Quadrant, is not going to help you in selecting the solutions for you.
Analyst firms recommend you identify the solution that best serves your current technology gap and requirements. They specialize in defining and classifying criteria that vendors are benchmarked against and there is a lot of transparency underneath the ‘dot’ that can help buyers with the identification. Yet, few buyers seem to know about the additional layer that is so important for internal due diligence.
All Forrester analyst evaluations are designed for the respective target audiences of the analyst firms. This is typically upper mid-market and enterprise organizations, covering the full spectrum of technology adoption profiles - it’s for laggards as much as it is for early adopters. If you are a company looking to modernize your set-up, it is helpful to approach evaluations from your unique perspective. We believe this holds true for Gartner analyst evaluations as well.
How to customize a Forrester Wave
The Forrester Wave comprises three key components: A report which talks about trends and criteria in a summary version; the graphic with the dots that we all know and love; and a high level overview of the different criteria that make up the dots. It also provides vendor profiles with strengths and weaknesses of the solutions.
What is most interesting is how the dots come together. This is based on how weak or strong a vendor’s current offering is, and the functionality available today for you to use (ie. all the buttons you can click and APIs to work with etc.) on the y axis. Vendors that have been in the market the longest will typically have a more complete feature set. It also takes into account strategy on the x axis - usually how well does the vendor’s business model and roadmap set it up for the future. The size of the dot shows market presence usually based on number of customers and product revenue.
Analyst assessments the MACH way - build your own wave
Each Wave includes a list of criteria that companies are benchmarked against. Taking Forrester’s B2C Commerce Solutions Wave as an example, Forrester benchmarks vendors against criteria such as ‘commerce experience’, and ‘search and merchandising’. For Forrester clients, those criteria are explained in an underlying spreadsheet which also shows how the scoring is defined. Next to the Forrester scores and rankings, there is a ‘Custom Score’ tab. As a MACH buyer, if you are also a Forrester subscriber, this is where you can tailor the Wave to your business and architectural needs.
To briefly explain the scoring, every criterion, like commerce experience, is a direct or calculated score based on defined elements. A score of 1 is below par relative to others in this evaluation (important distinction). 3 is indicative of the current market standard as defined by Forrester and on par with others in the evaluation. Anything above 3 is above par compared to others in the evaluation and potentially ahead of what the market needs today.
To give a concrete MACH example, front end management for a company looking to go headless might not be as relevant as for one with a tightly coupled architectural mandate. A vendor may score low on a particular criterion because they have actively decided that certain functionality is not part of their native offering but to partner with adjacent vendors to retain focus and follow composable architectural principles. The custom score capability lets you weigh the features and criteria that are relevant and important to you, just as you would for your stack - the MACH way.
Based on your customized scoring, the vendor ranking on the first tab of the spreadsheet will change. As a MACH company, your priorities are different from the average buyer. It’s your responsibility to look at the report in the context of your business and technology strategy.
How to customize a Gartner® Magic Quadrant™
The Gartner Magic Quadrant according to Gartner, “is a culmination of research in a specific market, giving you a wide-angle view of the relative positions of the market’s competitors. By applying a graphical treatment and a uniform set of evaluation criteria, a Magic Quadrant helps you quickly ascertain how well technology providers are executing their stated visions and how well they are performing against Gartner’s market view.”
The interactive version provided with the report available to Gartner clients also shows how vendors have moved in the Quadrant over the past few years, which can be helpful to take vendor trajectory into account. Remember that, even if a vendor hasn’t moved, they may still be evolving as the criteria itself changes with the changing market.
Gartner doesn’t provide a spreadsheet. If you are a Gartner Client, you do have customization tools available on the Portal that allow you to change weighting of criteria. For instance, you may not care about a vendor’s sales strategy which goes into the Gartner positioning of the vendor on the Quadrant.
Gartner Critical Capabilities report explains the high level criteria and will help with your understanding of the assessment. It includes predefined Use Cases for every category to help you customize capabilities and weightings without the need to create them yourself.
From a MACH lens, Gartner has made some level of customization easier for us through its Composable Business Use Case in the Gartner Magic Quadrant for Digital Commerce and Critical Capabilities report. This use case is not yet part of all Magic Quadrant evaluations however and it remains the user’s responsibility to check, depending on the Quadrant and product category, if there is a MACH/composable equivalent. Though a detailed explanation of the criteria and scoring is not provided, the vendor write up articulates how they perform in different use cases.
Our final words of advice
Above and beyond all customization of analyst reports, the best way to read these assessments with your specific needs in mind is to call the evaluating analysts. As a Forrester or Gartner client, you may have Inquiries available that allow you to have a real-life two-way conversation and receive the analyst’s expert advice specific to your priorities.
As the monolith / composable divide in the enterprise technology space has grown, the analysts’ mandate has become more difficult because they have to serve both sides of the chasm equally - in one single evaluation. As a buyer, to make the right investment choice and avoid selecting technology that won’t fit your business strategy and priorities, you have to look beyond the dot - for now.
GARTNER and MAGIC QUADRANT are registered trademarks and service marks of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.