Can you make it do this, can we give you sample text? Show me your code, you mutha....
I recently ran into this common response, as I do from time to time, in the course of meeting and writing to people about the work on advanced outcomes text mining and public corpus monitoring that grew out the work that me and my soon to be friend, Doc Martin, were performing at an EU telecom R&D lab.
As an analyst looking solely at the market dynamics and product strategies of the brand monitoring industry, I was a virgin, I knew nothing, I was coming in clean. Way over on the other side of the lab, was Charles Martin the outrageous, working on an unrelated machine learning project. It was pure chance that we started talking about the limitations of sentiment analysis, from both a market perspective (me) and machine processing and computational linguistic perspective (Charles). Two Jewish guys from good, East Coast and Mid-West stock, unrelated projects, working in an indifferent R&D environment that wouldn't know a hot topic if it fell in their pants.
Show me your code!
My job was to suss out the large industry issues from users perspectives - what would a good, sustainable, grow-able, web-based services look like in the brand monitoring world, 6 months, a year, five years?
I surveyed the users, the brand owners, several hundred who had partaken of the bounty that is today's state of the art in text mining and brand monitoring services. Talk about the corpus of the unimpressed! Still, these folks of the fortune 1000 brand owners, many of them visionaries, knew that this was a field of endeavor that needed to be cultivated.
It was an accidental left turn that led my research into the mid-market brand intermediaries that are the target of our product research. Very briefly, the mid-market is where decisions are made to add or drop product lines at the distributor or retail level. This is all covered in my other articles. The set of problems faced by the mid-market is completely different than the brand owners, who are the current target of sentiment scoring and brand monitoring; the market encompassed by these brand intermediaries is far larger and more fecund (from the POV of web based monitoring services) than the current brand monitoring sector could ever guess.
So, this complex web of issues regarding the scientific and strategic product issues that shall ultimately serve this mid-market, the true arbiters of commerce, is the starting point of our study - what do they want, what functional framework serves their needs to make decisions?
Show me your code.....does it do this?
In the world of what we might call, 'pre-product management', the analyst needs to be a bit more technical in order to understand the possible. We have here an entirely virgin decision support service territory that has been missed by the text mining industry - unless they are keeping their efforts secret.
The issues that are hinted at in my article are just the tip of the iceberg, and the merest crystallization of the larger body of work that has been done regarding the scientific underpinnings that need to be made, in a step by step way, practical, in......code!
But before that can happen, we need to move beyond the revelation that this very important market exists and is waiting, is not being served by the current industry leadership, formalize the product functional issues, re investigate the tools issues, ratify the academic issues and establish alliances with the academicians that are working on advanced linguistics.
A grand edifice of a product architecture needs to be built on the sound foundation of understanding the sector. That's the kind of research that I am proposing, which will lead to functional test in about six months....or we shall find that the issues are vexing and that new tools are required to perform these complex extractions...or.....
We found the market, we have engaged with it from a product research perspective, we have divined the opening issues, we have covered the potential technical challenges that need further study, now we seek comity with a sponsoring organization.
This issue is too grand for vencap, it needs to be nurtured, not harvested.