[IP] more on Heather MacDonald lashes out at "privacy fanatics" opposed to TIA, CAPPS II
Delivered-To: dfarber+@xxxxxxxxxxxxxxxxxx
Date: Sun, 04 Apr 2004 15:58:20 -0400
From: "David P. Reed" <dpreed@xxxxxxxx>
Subject: Re: [IP] more on Heather MacDonald lashes out at "privacy fanatics"
opposed to TIA, CAPPS II [priv]
X-Sender: mail.reed.com:dpreed@xxxxxxxxx
To: dave@xxxxxxxxxx, Hiawatha Bray <hbray@xxxxxxxxxxxxxx>
Dave - for IP.
Hiawatha misses the point (as does Ms. MacDonald). It wasn't just the
privacy wonks who objected to TIA. TIA was an idea that appealed to those
who think that rumors are definitive intelligence (like politicians of all
stripes who seek an intelligence service that tells them only what they
want to hear), and was being pursued with no scientific checks on its
implementation. Poindexter's flaw was that he was enamored of cool
technology, but was letting speculative ideas with no scientific validation
become operational programs.
Hiawatha is wrong in his presumption that TIA was a research project. TIA
was not a research project. It was a DARPA project, but plenty of DARPA
projects are not research - the bulk of DARPA work is advanced development,
and has always been.
I think it *would* be excellent scientific research to answer the question
of whether the kind of "garbage picking" data mining of random financial
transactions, etc. that TIA was interested in can provide scientifically
valid results. The kinds of inferences we should expect that were being
handwaved by its proponents were even less scientific in their methods than
"lie detector" testing, "astrology", or "life after death channeling" research.
But TIA was not doing that research. They were presuming that the answer
was "it will work" and embarking on engineering in the presumption that
such a system would provide reliable and valuable information. They had no
way of knowing it would, and worse, no way of knowing it would not provide
a cover for the kind of baseless intrepretations that has been shown to be
the output of analysts serving customers who make it clear they only want
to hear one kind of answer.
I'm sure TIA would discover a high degree of correlation between Muslim
religious charity contributions and anti-American sentiment, for
example. If I were a Muslim in America I would be pretty interested in
helping other Muslims who are suffering at the hand of our government
unnecessarily. So what would a large contribution to an Muslim charity
mean (scientifically)? It would contain little information (measured in
information theoretic terms). But in the hands of a skillful and
prejudiced analyst, it would be easy to use it as the basis of persecution
that might lead targeted contributors to more and more desperate measures.
The same kind of self-fulfilling science arises in the use of "intelligence
testing" in such tomes as "The Bell Curve" - ignoring the fact that IQ
tests were DESIGNED by Binet and colleagues to come up with a measure that
explained why the victorian upper classes were so much more successful than
the "poor" who were assumed to be that way because they were stupid, and
not because they were despised by their wealthier brethren because they
spoke with a Cockney accent, for example. When the correlation between
the test results and social class was optimized, they decided that the test
was "valid" and that it measured something real.
Those of us who have been working with computing for a long time have a
saying, Hiawatha - GIGO. But the TIA was basically a plan to do garbage
mining, expecting that there would be gold in the garbage, as if digital
processing can turn any dross into wonders.
If data mining of such sketchy data as financial transactions wants to
become respectable research, it first needs to be validated.
Credit card companies do such work, and their experience is
instructive. They use financial histories to discover patterns that can
increase by a few percent the likelihood that a person will be a profitable
customer. But looking at how they do this is constructive: they formulate
a hypothesis (based on some data) and then test that hypothesis by issuing
credit cards to a valid sample of that class of people. *After a year or
more*, they then determine whether they were right or not. If they were
right, they build on that learning by issuing more credit cards to people
who are similar. They use control groups, etc.
Whether they are right or not MATTERS to them, because its the difference
between making or losing a LOT of money. Even so, they have difficulty
using the data to make even a small amount of difference in their success
rate. It's useful to them because a high error rate is OK, as long as they
get a slim statistical edge that they wouldn't get otherwise.
Contrast this with the assumption that TIA makes. TIA assumes that you
will (among this blizzard of garbage) infer some "fact" with great
reliability, or even more speculatively, that AI wil "infer" activities
that you would never have dreamed were happening (the success so far of
broad domain A.I. inference would not lead one to that conclusion, but
perhaps *all* such research is classified...)
Then you in your infinite government wisdom decide, for example, to stop
all airplane flights on some day in June based on the intuition of the
intelligence officer who believes that computers are such good inference
engines that you don't have to validate the results. Or maybe you decide
instead to authorize a few hundred or a few thousand black bag operations
in people's houses, and some poor citizen accidentally blows the secret
police away thinking they are a rapist. (surely that would "prove" that
the killer was a bad guy, for having the effrontery to blow away a "good
guy" like, say, G. Gordon Liddy or Ollie North).
Somehow, I think that before we start assuming that TIA "could have been a
contenda" we ought to realize that it's probably just as good as all that
"powerful face recognition" that was gonna save us if we only bought enough
of it.
-------------------------------------
You are subscribed as roessler@xxxxxxxxxxxxxxxxxx
To manage your subscription, go to
http://v2.listbox.com/member/?listname=ip
Archives at: http://www.interesting-people.org/archives/interesting-people/