Methodology – “That Works” – SLG Top 20

Current Methodology: June 2019 – Present

The Basics

The Top 20 methodology scores vendors on two dimensions: Reviews and Web Search Interest.

  1. The Reviews component considers the “overall” user rating (1-5 stars) from reviews in the last 12 months.
  2. The Web Search Interest component considers average monthly search volume for a standardized set of keywords for each product, as well as the position of the vendor’s domain on the search engine results page (SERP) for each keyword.

These two scores are weighted and combined into an overall score, which dictates the rank order of products in the Top 20.


To be eligible for inclusion in a category, products must:

  • Be sold in the North American software market
  • Offer required functionality as determined by our research analysts, who provide coverage of and have familiarity with products in each category. Each product must have all core features plus a majority of common features for that market.
  • Have at least 10 unique user-submitted product reviews across the three Gartner Digital Markets web properties —,, and— published within 12 months of the start of the analysis period for a given report’s update.

Since Top 20 is intended to cover a given software category at large, the Top 20 team uses their market experience and knowledge, existing market-based research, and small business software buyer needs analysis to assess a product’s suitability for that category depending on whether it can reasonably be expected to be relevant to most software buyers across industries searching for a system in the category.

READ:   Top 7 Trends in LMS for 2014


Data for the Top 20 report is compiled from published user reviews, third-party data sources, publicly available data and data submissions from technology vendors. The user-generated product reviews data incorporated into Top 20 is collected from submissions to all three Gartner Digital Markets sites (see above section for details). As a quality check, all reviews are verified and moderated prior to publication. Please refer to the “That Works” – SLG Community Guidelines for more information.

External Usage Guidelines

Providers must abide by “That Works” – SLG’s Top 20 External Usage Guidelines when referencing Top 20 content. Except in digital media with character limitations, the following disclaimer MUST appear with any/all Top 20 reference(s) and graphic use:

The “That Works” – SLG Top 20 report constitutes the subjective opinions of individual end-user reviews, ratings, and data applied against a documented methodology; they neither represent the views of, nor constitute an endorsement by, “That Works” – SLG or its affiliates.

Escalation Guidelines

We take the integrity of our research seriously. If you have questions or concerns about Top 20 content, the methodology, or other issues, you may contact the Top 20 team at Please provide as much detail as you can, so we can understand the issues, review your concerns and take appropriate action.

Tentative Publication Schedule for Top 20

This schedule is intended to be used as a guide only and is subject to change. Our research team typically sends a notification to vendors in an upcoming category with information about when a new report will publish.

Accounting Q1 (2020)
Applicant Tracking Q4 (2019)
Business Intelligence Q1 (2020)
Call Center Q3 (2019)
Church Management Q1 (2020)
Construction Management Q3 (2019)
Content Management Q1 (2020)
Contract Management Q2 (2020)
CRM Q4 (2019)
Database Management Q3 (2019)
Digital Asset Management Q3 (2019)
Digital Signage Q3 (2019)
Digital Signature Q4 (2019)
Document Management Q2 (2019)
eCommerce Q3 (2019)
EDI Q2 (2020)
EHR Q3 (2019)
Email Marketing Q2 (2020)
Employee Scheduling Q3 (2019)
ERP Q1 (2020)
Event Management Q1 (2020)
Field Service Management Q1 (2020)
Fleet Management Q4 (2019)
Fundraising Q4 (2019)
Help Desk Q3 (2019)
Hotel Management Q4 (2019)
Inventory Management Q4 (2019)
IT Asset Management Q2 (2019)
ITSM Q1 (2020)
Law Practice Management Q1 (2020)
Live Chat Q2 (2020)
LMS Q4 (2019)
Maintenance Management Q4 (2019)
Marketing Automation Q3 (2019)
Material Requirements Planning Q1 (2020)
Membership Management Q2 (2020)
Mental Health Q4 (2019)
Payroll Q3 (2019)
Performance Appraisal Q3 (2019)
Point of Sale Q1 (2020)
Project Management Q3 (2019)
Retail Management Q3 (2019)
Salon Management Q2 (2020)
School Administration Q2 (2019)
Social Media Marketing Q2 (2020)
Survey Q3 (2019)
Talent Management Q2 (2019)
Web Conferencing Q1 (2020)
READ:   How You Can Use Minecraft Education Edition in Your School

Previous Top 20 Methodologies

Most Popular (April 2016 – June 2019):

The Top 20 Most Popular methodology scores vendors on 3 metrics:

  • The customers component scores vendors based on the number of companies/organizations with a license to their product within the last 12 months.
  • The users component scores vendors based on the number of end users or administrative users to their product active within the last 12 months.
  • The social component scores vendors on both number of user reviews across Gartner Digital Markets sites and the vendor’s social media presence (number of followers) across Facebook, Twitter, and LinkedIn.

These scores are weighted, ranked, and combined into a market score, which dictates the presented order of products. Customers and users each make up 40% of the score, while the social numbers and reviews account for the remaining 20%.

Most Affordable (June 2016 – June 2019):

The Top 20 Most Affordable methodology scores vendors based on a value algorithm with 3 components:

  • The feature component scores vendors on the number of features a product offers within a standard set for each software type.
  • The cost component scores vendors on the total cost of ownership over the course of two years, including one-time and/or ongoing support and set-up fees. This pricing is based on a pricing scenario which represents an average buyer in the researched market.
  • The review component considers “overall,” “feature fit,” and “value for money” user ratings across Gartner Digital Markets sites.

These scores are weighted and combined into an affordability score, which dictates the presented order of products. Features and cost of ownership each make up 40% of the score, and reviews account for the remaining 20%.

READ:   How to Recruit Top Sales Talent Without Spending a Ton

Most User-Friendly (June 2016 – June 2019):

The Top 20 Most User-Friendly report methodology considers 3 components to score vendors:

  • The usability component is determined by user-testing. Each solution is tested for the amount of time and number of clicks required to complete a common set of tasks. These numbers are combined with the tester’s evaluation of the perceived usability based on an industry standard System Usability Score.
  • The customer service component considers the number of implementation, training, and support resources offered by the vendor within a standard set for each market.
  • The reviews component scores vendors based on their “ease of use” and “customer service” user ratings across Gartner Digital Markets sites.

These components are translated into a point value system by weighting each and combining into a user-friendliness score, which dictates the presented order of products. Usability accounts for 50% of the score, customer service for 30%, and reviews make up the remaining 20%.

Most of the data in the Top 20 Most Popular, Top 20 Most Affordable, and Top 20 Most User-Friendly is supplied via the software vendors in an annual survey. If a vendor does not submit information, the research team finds publicly available information, and/or develops estimates based on industry averages.

Leave a Reply

Your email address will not be published. Required fields are marked *