AI, human behaviors, bias, subtle unobserved data, & causality (long but worth it)

In my day job, I am now trying to measure and predict which software efforts from my teams will deliver how much incremental revenue for my company.  The inherent uncertainties (errors) of these estimations are larger than any forecast value.  70% (or more) of all software efforts fail across most industries.  The most frustrating part of my experience is that everyone lies and pretends their estimates are always perfect.  And there are no data or scholarly analysis for the justifications.  And, of course, the real "attribution" of any revenue to any specific effort in the complex ecosystem of a marketplace is very dodgy and is itself uncertain.

My colleagues in "analytics" data science claim to measure (perfectly, of course!) the exact percentage of people who "would have bought anyway" without my team's marketing effort / campaign / incentive.  The assumptions they make are (of course!) tested with biased assumptions in the test formulation and the predictive accuracy on unknown data are never tested.  But in general, they do great work and I agree with all of their reasoning if not all of their numerical methods.

But the main purpose of my rant here for why everyone in AI should pay more attention to the points Taleb raises in Incerto (uncertainty) is that many of the axioms and foundations upon which we are basing "AI" and "data science" are themselves very questionable:

• Judea Pearl has recently explained why our perceptions of "causality" are fundamentally wrong; current AI methods, and applications are exacerbating the consequences
• Megan Stevenson has recently shown why judges misperceptions and reliance on AI algorithms has not improved fairness in our legal proceeding

In this latter case there are very many reasons we need to be even more careful about AI in our judiciary and legal proceedings, not just those points raised by the author.   The origin data upon which judgments and the AI calculates are biased because of the humans who acted, recorded, selected, and encoded.  Judges, using ancient human unconscious perceptions of other humans' feelings, motivation, trustworthiness. 

Google is now monetizing those annoying "I'm not a robot" challenges

Google sells, and more frequently gives away the use of reCAPTCHA to protect your web site or web pages from bots.  When their algorithm suspects you are a bot or that you are abusing a site with a lot of traffic, they challenge you with a problem to solve that is difficult for a bot.

These challenges and the problems are usually image classification problems that help Google improve their image classification algorithms.  These human judgments the world provides to google are called "labels" in machine learning.

Now, Google is selling the use of those annoying reCAPTCHA challenges that billions of people perform for them for free. If you need your datasets labeled by people challenged by those annoying "I'm not a bot" prompts, you can sign up and pay Google for their free labor.

Brilliant!

The Day the Icicle Works Closed by Frederick Pohl (1960)

Holds up surprisingly well, great plot, 4/5 Stars.

Ruins of Empire by Jay Allan

Light, brainless military space opera with great melodrama, fun.  Terrible science 3/5 stars.

The Hive, Second Formic War, book 2 by Orson Scott Card and Aaron Johnston

The plot, dialogue, characters, and story-telling are all fantastic (as usual).  There were some celestial mechanics issues that irked me a little but then again the tech includes the "ansible" as it is set the Ender universe. The ending is a little abrupt and book 3 is not out yet.  So now we wait. 5/5 Stars.

Frost Kriminalhörspiel

Diese Geschichte war sehr gut. Das Schauspiel, Timing und die Postproduktion waren ebenfalls großartig.Die Zusammenfassung  war zu kurz. 4/5 Sterne.

Best SF Six by Edmund Crispin

All of these stories from 1964 - 1965 were "literary" and most of them are very dated with terrible science (even for their own era) and depressing themes / plot lines. 1/5 stars.

21 Lessons for the 21st Century by Yuval Noah Harari

I did not like this one as much as his other books.  The Buddhist proselytism and weak scholarship are more glaring and grating.  And his ridiculous use of perceptive certainty right after describing how consciousness is illusory and manufactured emphasizes his lazy metaphysics.  But Harari is a great writer and has some original, fun ideas.  3/5 Stars.

Recursion by Blake Crouch

Great plot and story line with terrible science and a little deus est machina in the magic system. The discovery of key "spells" (pseudo-science technique) in the magic system is a mystery that enhances the story.  3/5 stars.  I prefer hard science and space opera.

Am Ende Aller Zeiten von Adrian Walker

Die Geschichte ist sehr deprimierend und die Wissenschaft ist schlecht. Aber die Handlungsgeschichte und die Ur-Soziologie sind ziemlich gut. 2/5 Sterne.