The primary goal of the Audit and report is to help drive the adoption of best practices and provide prescriptive tools and resources to aid companies in enhancing their security, data protection and privacy practices. The secondary goal is to recognize companies who have demonstrated a commitment to online trust and consumer protection by designating them as recipients of the 2016 Online Trust Honor Roll. Last but not least, a third goal is to provide an incentive for consumer-facing brands to make security and privacy part of their brand promise.
Now in its 8th year, the 2016 Audit encompasses nearly 1,000 websites across multiple sectors, examining consumer protection, security and privacy protection practices, and has been embraced by organizations worldwide as an objective benchmark report.1 Changes in the 2016 sectors include expanding the News 50 to the News 100, which includes the top 100 news/media sites by unique monthly visitors. The Social 50 has been re-designated as the Consumer 100, including leading sites requiring account creation across multiple subsegments such as social networks, email, photo/file sharing sites, dating sites, travel, jobs, e-file sites, identity theft protection, ridesharing and other sites.2
Sectors examined and associated top-ranked organizations include:
- 2016 Internet Retailer Top 500 based on revenue3 (IR 100 & IR 500) – Gap Inc.
- FDIC top 100 banks based on net assets (FDIC 100) – IBERIABANK
- Top U.S. federal government sites (Fed 50) – Dept. of Health and Human Services (healthcare.gov)
- Top 100 consumer services sites (Consumer 100) – Twitter
- Top 100 news and media sites (News 100) – Google News
- OTA member companies (OTA) – Twitter
In recognition of the increased number of organizations qualifying for the Honor Roll, a new designation has been added this year – “Top of the Class” – recognizing sites with a total score of 95% or higher. These sites are highlighted in bold in the Honor Roll listing in Appendix A. These organizations represented 10% of the overall sites, and approximately 20% of those achieving Honor Roll status. Viewing “Top of the Class” recipients by sector, OTA members led with 48%, followed by the Consumer (23%) and the Federal 50 (18%). Approximately 3-6% of the online retailers, FDIC 100 and News 100 received this designation.
The top 10 overall scores represented a variety of sectors, led by consumer sites – 1) Twitter, 2) HealthCare.gov, 3) Pinterest, 4) the White House, 5) Dropbox, 6) FileYourTaxes, 7) LifeLock, 8) Instagram, 9) 1040.com, 10) Gap Inc.
As shown in Figure 1, of the organizations evaluated, 50% qualified for the Honor Roll (vs. 30% in 2014 and 44% last year). All sectors grew in achievement, but the sector having the largest impact on overall results was the Consumer 100, jumping from 58% to 71%. Other sectors having a major impact included the FDIC 100 (grew from 46% to 55%) and the News sites (which both doubled in sector size and rose in achievement from 8% to 23%).
As seen in previous years, a significant fraction of Honor Roll qualifiers are first time recipients – a total of 40% (183 sites, mostly from the Retail and Consumer sectors) were first-timers this year, down from 55% in 2015. A complete list of recipients is shown in Appendix A along with the number of consecutive years they have earned Honor Roll status. Approximately 12% of qualifiers (54) achieved Honor Roll status for the fifth year in a row, nearly 9% (40) qualified for the fourth year in a row, 8% (34) qualified for the third year in a row and 32% (143) qualified for the second year in a row.
The significant number of first-timers and the range of ranking of designees in the Internet Retailer Top 500 (from #1 to #493) shows that the Honor Roll is achievable by companies of all sizes and levels of technical resources and skills. By contrast, there were 78 sites (nearly 10%) that made the Honor Roll in 2015 but failed to repeat in 2016. This highlights that security and privacy practices are not a static process – sites need to continually monitor, update and evolve to keep pace with evolving threats.
As illustrated in Figure 2, Honor Roll achievement grew in all sectors despite more stringent criteria in this year’s Audit. For the third year in a row, the Consumer 100 (previously the Social 50) outscored all sectors with 72% achievement. Many of these sites benefit from homogeneous and integrated system architectures in contrast to other sectors which have a higher percentage of legacy systems. Online retailers, banks and federal government sites all achieved results in the 45-55% range, while the News 100 lagged with 23% achievement.
It should be noted that OTA Members, 96% of which qualified for the Honor Roll, have been omitted from the chart since their scores were found to distort and compress the axis. OTA acknowledges the results may be biased since member organizations by the nature of their membership are committed to data stewardship and responsible privacy practices. While the methodology is public, OTA members’ knowledge and awareness may be greater than others and the high achievement may somewhat skew the overall Honor Roll achievement shown in Figure 1. If OTA members were excluded, overall achievement this year would drop 4% to 46%.
As seen last year, improved Honor Roll achievement was noted in all sectors, with many (IR 100, FDIC, Consumer, News) jumping more than 5%. Despite more stringent criteria, improvement in email authentication had the biggest impact on the rising achievement level. Site security scores also had a significant impact, with most sectors increasing their scores 3-5%. Overall, privacy scores actually dipped modestly in 2016, directly due to the more rigorous scoring of privacy policies. Many companies hover near the Honor Roll threshold – in the Internet Retailer Top 500 alone, nearly 90 companies are within 5% of reaching the Honor Roll, though many of those sites have a failure that would have to be addressed.
It is also useful to examine the reasons why organizations did not achieve Honor Roll status. Of all sites analyzed, 42% had a failing grade (score of <55) in one or more categories (down from 46% in 2015), highlighting significant concerns regarding data security and privacy practices. Figure 3 shows the Honor Roll vs. Neither vs. Failure percentage breakdown for each sector. This chart clearly shows that results are nearly bi-modal, with only a small slice of sites in each sector that neither make the Honor Roll nor have a failure. Figure 4 breaks down the failures a step further to show which categories caused the failures.
Failures were most prominent in the News 100 sector, and least prominent in the Consumer 100 (the failure rate for OTA members, which is not shown, was 4%). For the fourth year in a row, the Internet Retailer Top 100 fared better than the Internet Retailer Top 500. As in 2015, results for the Federal 50 were bi-modal and slightly below the average of other sectors, with 46% achieving the Honor Roll and 54% failing in one or more categories.
Failure reasons and percentages varied widely by sector. Inadequate email authentication was the primary cause for failures in all but the News 100, led by Federal sites with a 50% failure rate. The main reason for failure in this category was a continued shift in scoring to place increased emphasis on email authentication at top-level domains and implementation of associated DMARC records. The absence of proper email authentication leaves consumers increasingly vulnerable to spearphishing and related exploits including ransomware, bank account takeovers and identity theft.
Inadequate privacy policies and practices were the second largest cause of failures, impacting more than onehalf of the News 50 and one-seventh of online retailers. Significant improvement was made by retailers, which reduced privacy failures by half, and the FDIC 100, which continued its streak of reducing failures. The News 100 sector lags significantly in privacy scores primarily due to heavy use of third-party data collection and tracking, reflecting their reliance on third-party advertisers to drive revenue. All other sectors reduced their privacy failure rates, though as noted, overall privacy scores went down. This is indicative of many sites adopting the bare minimum elements to avoid failure, but the bulk of sites not keeping pace with the more rigorous criteria in the 2016 Audit.
Site security was the lowest cause of failure for all sectors, showing that the vast majority of organizations are tracking with the minimum recommendations for site security, though the overall failure rate did rise from 6% in 2015 to nearly 10% this year. This increase can be attributed to more rigorous scoring which maps a failure of a major subcomponent (e.g., protocol support) to a failure of the category.
Additional insight can be gained by normalizing the 300 baseline points to a 100-point scale (called the “Online Trust Index”) and comparing the high, low and median scores across sectors, as shown in Figure 5. The Consumer 100 has the widest range, followed by the News 100 and FDIC 100. Note that some maximum scores exceeded 100 due to bonus points. This chart illustrates how the medians for several sectors (especially online retailers and the FDIC 100) sit at the 80% threshold, meaning many new sites could qualify for the Honor Roll through simple operational changes and support of best practices. Several sectors saw significant improvement in their median scores, with News sites rising 8 points, Federal sites rising 7 points and Consumer sites rising 5 points since last year.
Figure 6 shows the baseline scoring breakdown (out of 100) for all sectors by major category. This chart shows much more variability than the median scores, especially in the Consumer Protection and Privacy categories. Site security scores are more tightly clustered.
The primary driver of Consumer Protection scores is the implementation of email authentication on top-level and subdomains. Low scores are primarily due to lack of support at the top-level domains. This remains a concern since the majority of spoofing and malicious email purports to be sent from the recognizable corporate domains rather than marketing subdomains which are often delegated to email service providers. While marketers have embraced email authentication to drive inbox placement, increased focus and engagement is needed to maximize consumer protection for all domains.
Building on these email authentication protocols, DMARC provides ISPs and corporate networks direction on how to handle email that fails authentication. Since use of DMARC has no cost or impact to server performance, the low adoption is concerning and may be indicative of low awareness of the criticality and business value.
Overall privacy scores dipped modestly this year – in general, policy scores improved slightly while tracking scores dropped, yielding a net loss. Sector scores vary widely (a 34 point range). Several new elements were introduced into the baseline privacy policy scoring this year (e.g., layered notice, Do Not Track disclosure) which lowered scores in some sectors. Nearly one-third more “promiscuous” trackers were observed this year than in 2015, dragging down overall privacy scores. OTA encourages all sites to evaluate their privacy policies and practices and take steps to update them to reflect actual practices and respect for consumer privacy.
1 2016 Online Trust Audit – Virtual Press Room https://otalliance.org/2016-online-trust-honor-roll-virtual-press-room-vpr
2 While sector definitions and criteria for inclusion have remained constant, individual companies may be added or removed from sector lists due to reported revenues, site traffic ranking and the impact of market consolidation and acquisitions. This consistency allows year-over-year analysis within a sector. The analysis also assesses the top 100 retailers (“Internet Retailer Top 100”) in addition to the Internet Retailer Top 500, allowing comparison between larger and smaller companies. Note some sectors were increased to a sample of 100 or more reducing the impact of outliers and improving comparability from one sector to another.
3 Source list from Internet Retailer® https://www.internetretailer.com/top500/. In some charts and tables, for the sake of brevity, the Internet Retailer Top 100 and Top 500 are abbreviated “IR 100” and “IR 500”, respectively.
