Employing Advanced Natural Language Text Processing to Provide Guidance to Mid-Market Multiline Dealers and Product Distributors.
The present rage over such issues as ‘sentiment analysis’ and text mining of the public corpora is fertile ground for a fresh and focused analysis of the state of the CGM industry; such an analysis was recently undertaken by the author. The author’s privately commissioned report answered various questions, such as:
what is the state of the industry?
are early entrants employing a sustainable business model?
what is the traditional / entrenched competition?
what has been missed, where is the prime the opportunity?
Without betraying the verbatim text of the 90 day analysis compiled at the behest of the previous client, the author would indeed like to share the non-proprietary conclusions leading to strategies for future products and services.
The most abstract and brief statement of the analysis is that current CGM services being proffered to the marketplace by the early entrants (Cymfony, Buzzmetrics) are purely focused on Brand Owners, occupying positions chiefly within the fortune 1000. The services provided by these leaders are positioned against traditional brand equity services (provided by AC Nielsen, Arbitron, and Gallup). Cloning the established recurring campaign model, and offering such services to a limited cadre of potential clients (that already have access to repeatable data from entrenched leaders) is not a formula for success.
The author’s commissioned analysis for the previous client bears this out by verifying statements from some of the equity investors involved in early CGM ventures. Such bitter regrets led to the sale of Cymfony to TNS - the very nature of the acquiring entity1 is a repudiation of the brand equity campaign model being applied to Consumer Generated Media metrics.
Sentiment is the weakest of CGM metrics, and the notion that highly customized campaigns that are iteratively refined by the client and CGM agency, is simply unsustainable when compared to services provided by the multi-billion dollar brand equity leaders deriving solid, actionable data from surveys, focus groups, and other statistical sources.
Furthermore, the attribute of brand ‘equity’, or ownership, is truly limited to a very elite few, when viewed within the total business opportunity matrix. Therefore, the few high latency campaigns that have been sampled by the brand owners, as replacements or adjuncts to existing brand equity services, are simply not making a sustainable impression.
The author believes that the real, sustainable market for CGM analysis lies in the mid-market - those companies that are recipients of branding decisions and who make the daily decisions as to what lines to carry, and which products to drop; we are truly speaking of the multi-line retailers and product distributors that most of us deal with regularly in our personal and professional lives.
The mid-market makes up the lion share of commerce decisions; while the fortune 1000 CPG and durable goods markets toy with Sentiment Analysis, the mid-market struggles with allocation of limited dollars in actual cash and lines of finite credit. Which line shall we carry or drop, what action shall we take in light of incentives taking the form of cooperative advertising and floor-plan financing contributed by manufacturers and up-stream distributors? In short, what is the actionable intelligence we can gain from any service that can steer the ship of commerce?
The answer is to create this actionable market intelligence from multiple text corpora (for statistical accuracy), and to employ a greatly extended model of linguistic metrics based on phrasal Ontologies which detect consumer issues, outcomes, and declarations. Such issue detection methods result in real, actionable metrics. These metrics lend themselves to statistical scrutiny, and may be charted against offers from the brand owners - such as the aforementioned contributions of advertising cooperative program dollars, as well as seemingly advantageous adjustments to a multi-line dealer’s floor planning finance carrying charges.
With such a service in place, dealers and distributors who must make constant corrections to their product catalogs and inventory levels, etc., may avail-themselves of a more dispassionate decision support methodology. Such a hosted services solution can be monetized in many ways, i.e., by subscriptions, at varying service levels as a free or ad supported service, or as an up-sell generator for hosted CRM services.
By Linking the aforementioned perceptions and “statements of outcome” with variables that are derived from the decision matrix data important to the mid-market (such as cooperative advertising, floor-planning, and inventory financing incentive programs), such a subscriber based system can be created to offer guidance to multi-line dealers/distributors in regards to purchasing decisions, and weighing program incentives against overall market perceptions.
The component technologies that must be developed for this system are sufficiently novel, such that revenues from licensing, syndication style Web 2.0 widget embedding, and turnkey system via VAR are entirely possible as ancillary revenue streams in addition to offering a comprehensive, hosted solution.
The author is of the firm opinion that such a system is within reach, given the appropriate project management, sound architectural principles, and the application of creative thinking diligently applied to crystallizing superior solutions.
The market comprising text analytic products and services for brand monitoring is greatly in flux, and a mixed bag to boot. The most severe deficiencies, as covered by the author in previous reporting, is the sole focus on analysis of sentiment, with services targeted to serve only the very top of the market (as re-occurring campaigns) within the Fortune 1000 client constituency. It is wise to recall that these brand owners have had virtually unfettered access to classical consumer demographic services for brand management; such services offer repeatable, actionable brand and product intelligence through ongoing focus groups, surveys, etc. Nothing in the current state of text analytics for the social media sphere comes close to being as authoritative or actionable.
Sentiment is the weakest measurement of the “voice of the consumer” metrics palette, and the brand owners who are currently gingerly sampling the basic services offered by the field’s early entrants cannot be counted on to create a sustainable market segment for measurement and mining of ‘in-the-wild’ textual systems, such as the blogosphere. The era of the $200k sentiment analysis campaign is almost over before it started; the industry is virtually saturated, certainly somewhat disappointed, and more than ready for real change targeted at the thick and juicy part of the market occupied by tens of thousands of mid-market companies.
Companies comprising the mid-market are not always considered to be small-to-mid sized; the mid-market truly describes ‘brand intermediaries’ that interpose between brand owners and the consumers. Whereas it is true that the mid-market is mostly composed of the much vaunted SME, some are truly multinational distributors comparable in size to their Fortune 1000 brethren.
It is precisely this mid-market tier of distributors and multi-outlet regional and national retailers that require advanced systems that predict product and channel outcomes based on data mined from the public and private corpora. Such systems are much more sophisticated and provide truly actionable intelligence for the decision makers in these companies.
Merely providing the weakest metric of sentiment, measured as linguistic polarity, and pitching these services to the top tier of a handful of Fortune 1000 giants that mainly preoccupy themselves with traditional brand services from the likes of Gallup, Arbitron, and Neilsen, will net no long-term gains - the capital that has been poured into existing Sentiment startups is already grumbling,
The following describes a system to exceed the state of the art for providing directly actionable market intelligence for the influential mid-market companies that are the recipients and causal dependents of brand perceptions in the marketplace, by putting into action a data mining technique that measures the consumers perception of product and dealer outcomes.
The Proposed Product - Descriptive Targets / Functionality
The complete scope of the project comprises a broad range of market research used to determine not only the delivered functionality, but integral, fundamental investigations into modes of business and linguistic models used in various systems. Therefore, as current sentiment analysis services offered by the early entrants are patently abysmal at providing solid, understandable guidance to mid-market multi-line dealers, a new product or service should precisely provide the missing link, that is, provide guidance that aids decision makers in selecting, abandoning, or adjusting catalog and inventory levels, while taking popular dealer incentives (aka ‘spiffs’) into account.
Technology Challenges Overview
Please see the author’s previous monograph for an in-depth discussion of the advanced technical research issues relating to this product investigation. For the purpose of the present discussion regarding product applicability, we now turn to issues relating solely to the end user’s needs and the challenges arising thereof.
The chief challenge is to order and quantify the corpus into families of product perception and service performance outcomes. Consumers touch many facets of a brand’s ‘essence’, namely, the product itself, the dealer/retailer relationship, the service relationship, the product community, etc. Classifying the linguistic elements of ‘outcomes’ is therefore an important bedrock principle of this service. Beyond actual classification, a statistical/stochastic system is required to forecast and integrate these mined results with the previously mentioned external variables, and also for the creation of a predictor system.
Users of the system should be able to spot trend-lines regarding product perceptions and performance outcomes. In addition, by supplying retailer program variables, such as the aforementioned spiffs of all kinds, these trend-lines can be integrated with the issue detection and performance/perception text mining data, thereby deriving a sound system for recommended actions.
Statement of Work
The SOW is divided into product / market validation within the potential user/subscriber community, and the architectural challenges. Although early research performed at the previous contract provided positive anecdotal evidence of interest from the potential user community, a more formalized survey would be in order. The total SOW can be executed with authority, covering all issues within a six month research cycle; the total deliverable shall include UI mockups, functioning proof of concept code, and a detailed product monograph covering all of the strategic and architectural issues.
The following issues need to be clarified:
Market / Product
User cross section analysis - types of users, professional titles, budget authority.
Issues not addressed by current text mining leaders
Support of the mid-market decision makers
The daily business of multi-line product management at the retail dealer and regional distributor tiers.
Differential ‘at stake’ issues; what can be tangibly gained from such services?
Potential for leaders to clone services, other competitive issues.
Building out the fabric of the service beyond core offerings, i.e., what would a complete suite of services look like as applied to the mid-market multi-line b2b web-based offerings?
Interoperability issue as regarding features, partnerships, and OEM / affiliate opportunities; Once the web based core service is offered, are the up-sell opportunities to work with leading services / vendors (salesforce.com, CRM players) significant?
Technical / Architectural:
Basic validation of the text mining challenges for detection of conversational models via phrasal ontologies.
Use of Modular Ontologies defined in OWL, and instantiated in RDFS systems.
Establishing relationships with the academic labs engaged in the above as applied tools research
Algorithmic methods of reducing noise and refining the statistical and stochastic models of conversation dynamics detection.
Back-end systems issues, best storage models for persistence of state of RDF nodes. (author has backlog of vendor research).
User and Administrator UI.
Use of the internal Text extraction models to drive advertising supported revenue.
User driven query optimization across product lines and items of interest.
Automated optimization of queries - machine learning.
Creating opportunities for community building as a result of issue detection.
Connecting to cooperative, private corpora via PIRGS, government, and other.
Tender of Offer
The author is seeking a six month contract to conduct the above referenced research in concert with his technical counterpart, or a suitable research counterpart provided by the client. If there is interest in this proposal, the author would oblige with a 20 minute presentation further examining the potential product development opportunities.