Blog

A Short Lesson in Bad Decision-Making

Humans are simply bad at assessing risk
Friday, January 11, 2013, 5:02 PM

In business school I had to take an introductory class in statistics that we colloquially called "D&D." The official course name was Data & Decision-making.

In retrospect, it was a truly valuable class (one of very few I encountered in b-school). If you can figure out how to use statistics to determine the most probable outcome from a set of scenarios, or find predictive correlations from within a sea of data, that's real power. You can take a lot of the guesswork out of decision-making and consequently make the "right" call much more often.

I performed miserably in this class. But I had a lot of company; it was perennially voted the hardest in the school's core curriculum.

While I sometimes fantasize (masochistically) about taking the class again to master the black magic of statistics, I realize that I did learn a valuable lesson: Humans are innately poor at estimating probability.

This was proved to me time and again throughout that course, starting on the very first day.

A gifted young professor taught D&D. I was 27 at the time, and he was only a year or two older than I. Wicked smart guy.

Given his age, he had a few things to prove to us on that first day of class. He wanted to demonstrate he was a little more fun and hip than the stuffy older profs who taught our other courses. He also intended to show that even though he was the same age as us, his was the Alpha cerebrum in the room.

So as we took our seats for the first time, he asked: "Who wants to bet me $5 that two folks in this room have the same birthday?"

We all looked at each other. There were about 65 students, plus the prof. We were all thinking: "365 days in a year. Only 66 people. Those odds don't seem so bad..."

A hand went up, taking the bet. The prof asked the folks in the back row to start shouting out what their birthday was, one at a time. We got about 6 people in before someone in the middle of the room said, "That's my birthday, too." A $5 bill was passed up to the teacher.

"Anyone willing to bet me again?" he asked.

Another hand went up, figuring the odds just got much better as the "fluke" overlap had been removed.

The exercise repeated. It only took a few more shoutouts to hit another shared birthday. Another $5 was handed over.

"Anyone else?" said the professor.

Another brave student made the bet and lost his $5.

"Again?"

This time we students were much more tentative. But after a while I shot my hand up. "What the heck," I figured. "He's got to be running out of luck."

Wrong. I lost my $5 in about ten seconds.

What I later learned was that the professor was making an exceptionally safe bet that only appeared risky because we students were grossly misjudging his probability of being wrong.

In fact, as long as there were 57 students left, the prof's chances of winning the bet were over 99%(!). The odds would continue to be overwhelmingly in his favor all the way down to 23 people, at which point they would be 50/50.

It turns out this is a classic probability exercise known as The Birthday Problem. And it apparently has kept the beer funds of statistics professors well-capitalized for ages. 

The lesson I took from the experience is this: Probability estimation is non-intuitive. If you need to make an important decision about the odds of something occurring, don't go with your gut. It will be often wrong (sometimes wildly so). Get data, crunch the numbers, and consult a professional if you can't figure things out on your own.

This caution with regard to decision-making has served me well over my career. I've lost count of the number of times I began with a strongly-felt guesstimate that was torn to shreds by the time I did the math and learned how far reality was from my gut instinct.

And what worries me – scares me, is more accurate – is that I don't observe this same caution in the actions of the people making the truly big decisions. Like the Fed, Congress, and many of our state governors. Instead, I see people – many of whom don't have strong empirical skills or practical business experience – making rash decisions about debt, deficits, taxes, money supply, interest rates, pensions, etc. that will have implications on a staggering order of magnitude. I myself can't wrap my brain fully around some of these (classic example: The Crash Course Chapter 11: How Much Is a Trillion?). And even though I'm by no means the smartest guy in the room, I have little confidence that a career politician is able to comprehend these gargantuan repercussions at a materially higher level than I can. 

Sure, it's easy if you're an elected official to simply print more money. Or run trillion-dollar deficits. Or raise taxes instead of cutting spending. Or mint the coin. Or burn more oil. But these are not short-lived decisions. Their implications will manifest over generations.

And as the system becomes more unstable, the gut decisions will come faster and more furiously, made by those with the most hubris.

If only it were $5 of beer money that was on the line, instead of our global standard of living.

Related content

50 Comments

Arthur Robey's picture
Arthur Robey
Status: Diamond Member (Offline)
Joined: Feb 4 2010
Posts: 2502
Survival of the species

Thanks Adam. That does sound like a course that I need to take. 

My first thought was "Evolutionary pressure should have been in the opposite direction. It should have given us a keen sense of risk”. This is another anomaly along with the need for sleep.

Both these attributes should have been weeded out strongly. There is nothing subtle about their survival disadvantage.

Adam Taggart's picture
Adam Taggart
Status: Peak Prosperity Co-founder (Offline)
Joined: May 26 2009
Posts: 1876
Immediate vs long-term

Interesting point, Arthur.

My (unqualified) guess is that our species is probably well-wired to assess risks of the most immediate and mortal kind (e.g., the snarling saber-toothed tiger). It's the second-order, frequently longer-term, risks that we're blind to (why we'll avoid eating the rotten fruit so we don't keel over from Campylobacter enteritis,  but we'll pursue a steady diet of junk food that simply kills us more slowly)

It would be interesting to hear from any biologists or behaviorists reading this.

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
"Interesting point Arthur"

LOL!

I have no clue what the hell you Dudes are talking about. I 'll try again tomorrow. Maybe not. surprise

The brain power and commitment of so many here are most definitely a motivation to me.

BOB

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
OK, I have read your threads again and...

...have decided to slooowwwly move away from the lap top, close its cover as I move, as I am now frieghtened beyond belief!!! LOL

BOB

Travlin's picture
Travlin
Status: Diamond Member (Offline)
Joined: Apr 15 2010
Posts: 1322
Nice job

Adam

That was a nicely written essay.  You made an important point well.  I agree with your response to Arthur.

Travlin 

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
"Goooood Mooorrrning Vietnammmmm"...

Adam, the essay is sage advice, no question about that. Baseball is stats, stats, stats, and I get your message loud and clear.

Your bantering with Arthur was awe inspiring as you both are favorites and I'm glad you have each others in situations like this because honestly, I still have no clue how Arthur could garner his thoughts out of your essay, and THEN you responding. Education is knowledge, that's for sure.

OK, less scarred this morning and have wondered myself why we haven't evolved the ability to grow and retract our body hair as needed, over time, as on some of these cold days it sure would be nice! Am I in the ball park? Then again, we do have our razor! LOL!

Truly,... Respectfully Given

BOB

theisland's picture
theisland
Status: Bronze Member (Offline)
Joined: Mar 16 2012
Posts: 62
Ties to the "dismal" science

I think this is a very nice description of the problem with making assumptions based simply on what seems to be obvious. Some of the basic principles of economics 101 are just believed because they seem to make sense. Hence the "dismal" label - the scientific method does not seem to get applied in a transparent manner.

Everything needs to stand up to repeatable scientific measurement,  experimentation and review before it can be safely taken to be true - even the absolutely bleeding obvious. Eg it's obvious that heavy things fall faster than lighter things - until you measure their fall (Galileos famous experiment). As Adam says, intuition can be way off.

Its another angle on the fact versus belief concept that Chris has mentioned many times. 

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
Bowskill, I couldn't agree more and...

...the issue for me is separating the shaft from the corn. Let me explain: I can due the math, get the numbers right but then comes along a Princeton Professor who continuously baffles the process with Bullshit where North is now South and Up is in fact Down. Then doesn't share his findings. So, garbage in is garbage out and making sense of it all is just a gut feeling. If it's a gut feeling (that's bad but not always a losing proposition) then it's better to learn how to quantify bullshit, and exactly how does one write a formula for that? Yet, intuitively (again, statistically a bad formula) we KNOW something has to give and it will be monumental at least that's what my gut tells me as measuring these extraordinary times are just impossible (no math to definitively even guess, we just know it's unsustainable).

David and Goliath? My observations would have been wrong.

Respectfully Given

BOB

hooverdan's picture
hooverdan
Status: Member (Offline)
Joined: Apr 22 2012
Posts: 7
Evolution of Risk Assessment

All in due time... the risks we are facing (high-debt loads, investment failure, living beyond means) have not been with us long enough to invoke evolutionary change.  If we survive we will learn.

Wendy S. Delmater's picture
Wendy S. Delmater
Status: Diamond Member (Offline)
Joined: Dec 13 2009
Posts: 1582
what are the odds?

Back when my sons were in grade school, they did a science project abut the intuitive response to the laws of proability. It helped that their scoutmaster was a PhD mathematician who worked for Northrop Grumman, and he walked them through it; math is not my best skill, but I knew enough to know that this would be an interesting experiment.

The kids had eight flashlight batteries in a basket, and an empty flashlight. Two were fully charged, six of the batteries were dead: emptier than a politician's promises. The person viewing the exhibit was asked to guess the odds of getting the two charged batteries into the flashlight. Upon lifting a flap, the actual odds were revealed. Everone was shocked that the odds were 36 to 1. It was not intuitive, even for the science fair judges, who should have known better.

The exhibit was also covered with things like lottery tickets and pictures of casino games, with the odds of each game below a flap.

An interesting mathematical rant on the odds of winning a pick-six lottery is here. The short version?  A Pick-Six lotto gives you 1 chance in 13,983,816. Odds of 1/13,983,816. (That's a .000007% chance of winning and a 99.999992% chance of losing).

Humans are simply bad at assessing risk. That's why they gamble.

gillbilly's picture
gillbilly
Status: Gold Member (Offline)
Joined: Oct 22 2012
Posts: 382
Don't forget context!

I remember my econ statistic courses! I wasn't the best at them either (econometrics...uggh). One thing I think worth mentioning is context is everything. Scientific scrutiny is always good but intuition and context can be the balance to that scientific analysis if things in your gut are telling you that the data is somehow flawed. I don't see it as either or but rather the two working together. I think there is a difference between assumptions and intuition.

Adam, the example assumes that birthdays are random and that all students will be honest, and that's a pretty common sense assumption for a $5 bet. But if your prof had put more risk on the line, say $1000, how would that have changed the nature of the outcome? (probably no one would have taken the risk, I mean it is college, but hypothetically, would everyone have been honest?) I always found the assumptions on which much of the foundations of economic markets are based are flawed. We do not have instantaneous product information and even if we did, most people are not going to be "fully" informed, there's just not enough time or human will. I used to sit in my econ classes looking at those supply = demand graphs and say to myself...well yes that looks very mathematical and I can see it on the graph there, but is that really what the price 'should' be?  Then we run what we think are scientific models based on those assumptions and call them facts. I think this is where economics has real problems, it's impossible to scientifically reduce human behavior to fixed models when context is continually changing. The EEEs is a testament to this and why CM's (and many of us) gut is telling him there is something really wrong with the data.

Bob, baseball is all stats, but in the new context of the steroid use, now that we look back, what do those statistics mean? Who really deserves to be in the HOF? Do we have to go back and group the players that used and not used into different categories and run the stats over. Do we have a steroid HOF now?

A funny example...I received a call from a political pollster before the election (federal). He asked if I would take a a quick survey for a poll they were conducting. I was breathing pretty hard because I just stepped in the door after taking a run. He asked why I was breathing so hard, so I told him. He continued on and asked his first question..." Do you believe your town is moving in the right direction? What I mean is...is it getting better or worse for you locally?" I responded that some things were better and some were worse. There was a long pause as he looked at the available answers on his sheet and realized he didn't know what to ask next. I had not given him a better or worse answer. He responded "well it seems you're pretty tired after your run, I'll let you go, thanks for taking the call." I tried to say, no I'm fine, please go on, but he hung up.

As I watched the FOX news Carl Rove melt down on election night, I'm sure his data told him that Romney was supposed to win, and that's when I thought back to my phone call. How many people were cut off because they didn't answer the question within the confines of the script? Maybe it fit into the "margin of error," but then maybe not. I would guess human beings are predictable under certain circumstances, i.e. when the hockey stick is running on it's flat plane, but when curcumstances hit the steep curve, well, who knows?

Oh and Arthur I hate to think of sleep as an anomoly...I love my good night's sleep! lol

Thank You

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
Gillbilly, you asked:...

"Bob, baseball is all stats, but in the new context of the steroid use, now that we look back, what do those statistics mean? Who really deserves to be in the HOF? Do we have to go back and group the players that used and not used into different categories and run the stats over. Do we have a steroid HOF now?"

Geez I really could write quite the piece here Brother so I'll attempt to be brief. Steroids to purists (of which I am) is an unknown, known (think Rumsfield). What did speed or other enhancements do to the stats of the 60's and 70's ballplayer? Drink, cocaine and other drugs of the early games (pre-1930's)? Less games and media attention, and commercial contracts are all time spent not preparing for the next game, and how is this reflected in game day stats? What if the umpire called a ball instead of a strike, when clearly the ball was right up the middle waste high. Stats are basically subjective unless the numbers are quantifiable and I just don't think they are for the most part when everything is moving in each and every direction. Fixed objects fine but moving objects are entirely different and when you add human error then the gut and intuition is all you have as a supplement to the equation. It is TIME where stats prove there worth and current stats are not reflective really of the true wealth effect.

So, what do we know, the game plays out in a 162 game schedule and statistically the player will perform at 300 BA. 20 dingers and 100 RBI if the season has two rain outs in April and if the medium temperature is 76 degrees because this particular player needs a couple days off and warm weather because for every degree the temperature falls it can be shown that his batting average and power production drops.

Stats tell the tape, and we must adhere to the stats to even out the score long term. The same stats today are obsolete tomorrow even using the same information. For instance: Babe Ruth can't hit a slider down and away so you pound him down and away. Then Ruth thinks, OK, I'll go deep to left then, crowds the plate, pulls in his hands, shortens his stride and pops it out to left. Assumptions are great and numbers are great until their not.

Like everything in life, Risk/Reward. I say you get caught then banishment and asterisks. Without rules, CHAOS. It's why I sat for 3 days recently, and rightfully so, I broke the rules. Man up time then.

Pete Rose, every one's fall guy bet on the game while HE MANAGED, and for me, this is a banish-able offense. Why? He cheated the game. Steroids, same thing, no matter that the improbability of hitting a Nolan Ryan fastball is near 0. As I think about this everyone, every ball player has cheated the game in some form or another. All bad but most all in what is called the grey area. It has yet to be determined if steroids fall in the grey area but because such effort is being made to cover it up tells me that extreme guilt and cheating is the motivation. The writers will vote and the talk will banter about forever as is the case with Shoeless Joe Jackson. It's what makes the game so great.

Regards

BOB

baron1's picture
baron1
Status: Member (Offline)
Joined: Jan 12 2013
Posts: 1
Fascinating, can't wait to

Fascinating, can't wait to try it.  But, an econ prof has access to school records and would have the opportunity to check beforehand.  I know I would.  Just saying.

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
baron1...

...I suppose he could up his odds a bit but if the stats say 99.9 percent then you bet it all, and live with the consequences are my thoughts. Then again, checking would confirm the .01 percent and is why $5 dollars is way different than $1000 dollars, so forth and so on.

Welcome

BOB

OT Folks: My freezer alarm in the basement just blasted my home with a horrible sound that reminded me I had not shut the main freezer while stocking the refrigerator freezer with new supplies! I love that thing as sometimes we just forget to shut our freezer and that would have been a costly mistake. You can buy many different models for around $10 bucks! Do you have one?

tricky rick's picture
tricky rick
Status: Bronze Member (Offline)
Joined: Dec 9 2011
Posts: 78
Monty Hall syndrome

Nice article Adam:

Reminds me of the Monty Hall Problem  (Let's Make a Deal!).  Amazing that the odds of picking the car behind door number 2 or 3, AFTER showing it's not behind door #1 are 2 out of 3 - not just 50/50.   (a perennial wrong door picker I hope I have that right - google it!).

Gut feelings seem to me the obvious...  obviously not where the smart money is!! 

Mama told me not to ...

treebeard's picture
treebeard
Status: Gold Member (Offline)
Joined: Apr 18 2010
Posts: 390
Cultural Prejudices

Brings up an interesting point inadvertantly and displays our cultural prejudices.  We have a lot of guys who graduated from top ivy league schools with amazing cerebral credentials running things at the moment.  Some of them spend their time inventing new financial "instruments" and writing new "algos" to work the markets.  They are the guys and gals who came up with terminator genes in seeds for Monsanto.  They are inventing new drone technology to fly around third world countries.  We have a lot of "smart" guys in charge, but things aren't going to well.

The average joe says to himself, well I'm not too bright, glad we have these really "smart" peolpe running things for us.  We don't do well in a particular academic subjects and assume that we are not the "alpha male (female)" in the group and defer to other "smarter" people.  But I would like to propose a very different view of things.

We all have our nature and inclination of how we view reality.  There are those who lean towards knowledge, a fragmented, linear, scientific, reductionist and mathematical view of the world.  They tend to go into math and sciences.  Then there are those who lean towards wisdom, wholistic thinking, intuitive learning, and big picture thinking.  They tend to move into the arts and literature.   Both are equally intelegent or "smart", but we have a whole cultural paradigm that has become self selecting towards one point view.  If a person doesn't do well in a hard science subject, they then tend to believe they are not smart.

We all of course are a mixture of both, but we are now self our leaders of society from one extreme.  Knowledge is about gaining power and Wisdom is about love.  We need both, but the wisdom of whether or not we should do something at all is primary to the power of how to do something.

If we don't move out of a paradigm that worships power to one that is drive by love and wisdom, the ship will surely sink.  We need to believe in our own intelligence what ever form it takes and not doubt our view of reality.

gillbilly's picture
gillbilly
Status: Gold Member (Offline)
Joined: Oct 22 2012
Posts: 382
Good points Bob

Your point on "time" is a good one. Time is the great leveler in all areas of life isn't it? As you point out, with stats and scientific research over time, one small irritating new fact or revelation can render all of it obsolete. Godel's proof of incompleteness comes to mind...poor Whitehead and Russell...their enitre life's work negated by Godel's proof. Bummer!

Baseball is a great analogy for what's going on in the world. We are all to blame, well most of us at least, in regard to the steroid scandal. Who didn't get caught up in the Sosa/McGuire contest? The stakes are very high in the big leagues, and money is at the heart of it unfortunately. The pressure is extremely high for individual players to perform, so the incentive to enhance/cheat is strong. A player's biggest fear is being sent back to the minors. The sad part is that the stats on steroids are pretty well researched. In high doses they are extremely harmful to a person's health. If we say it's okay to allow them (as some have suggested, not me) then what does that say about us as a society? Should we say...well hey if the player wants to harm his body that way to reach his maximum income, who am I to tell him no? To me, that's the equivalent of the colluseum days of Rome.

The same is true for those playing in the big leagues of finance, the stakes are high. High frequency trading is in some ways the equivalent of steroids. As CM (and Treebeard circuitously above...nice post!) has pointed out, this type of trading really has nothing to do with supply and demand, and is a way of gaming the system.  Humans can't compete with computer algos when it comes to quantity of trades, but   our insatiable desire for performance, not just cheating, may be the reason we look the other way.

Your point on rules and risk/reward is good and I agree. I would add the rules have to be constantly evaluated and tweaked in light of the current context (which is what the league is doing).  Similarly, we need to reevaluate the global rules within the context of our insatiable harmful addiction to more speed/performance, efficiency, profit, and productivity.

Treebeard, I agree whole heartedly with your post!

Thank You

gillbilly's picture
gillbilly
Status: Gold Member (Offline)
Joined: Oct 22 2012
Posts: 382
Anecdotally

I was watching my daughter in gymnastics class today. She and another girl were working on walking across the balance beam. There are three balance beams each at different heights. When crossing the lowest one, they both shared the beam equally. They would start at the same time from either end and walk toward each other in the middle. When they reached each other, they would each jump off and walk back to the end. They did the same on the middle height beam. But when they went to the highest beam, their behavior changed. They took turns starting at one end, going across the beam (same amount of real estate), and then jumped off. I thought it was interesting how the perceived risk of falling off changed their behavior (there were mats on either side making the risk neglible). Implied in this is that there was greater reward to walking across the higher beam, but essentially they were doing the same thing as they were on the lower beams. I find the analogy interesting and coincidental. Are the risks real or just perceived? I guess it depends on the height of the beam and if there's a mat (safety net).

Peace

Grover's picture
Grover
Status: Platinum Member (Offline)
Joined: Feb 16 2011
Posts: 507
Optimism/Pessimism

Arthur Robey wrote:

My first thought was "Evolutionary pressure should have been in the opposite direction. It should have given us a keen sense of risk”. This is another anomaly along with the need for sleep.

Both these attributes should have been weeded out strongly. There is nothing subtle about their survival disadvantage.

Arthur,

I'm no biologist or behaviorist, but I have observed and noted both. Ironically, evolutionary pressure rewards optimists. Pessimists are less likely to take chances because they fear the possibility of negative outcomes. Optimists take chances because they hope for the reward coming. A pure optimist won't assess the severity of potential risks and will likely succumb before passing on genetic material. Likewise, a pure pessimist will never take a chance and will starve before procreating.

You don't need to be the fastest, strongest, smartest, etc. to survive. In some situations, being the second slowest is perfectly survivable. (My speculation) Humans' evolutionary success stemmed from our ability to work together. We can coordinate actions to behave like a much larger creature when needed. We can learn from others' mistakes (theoretically at least.) At the end of the day, we can separate and go our own ways. Some will die and most will live to see another day. The collective has to adjust in order to continue. It happened then and it happens now.

If you've ever seen a tweeker after a week of being awake, you'll see why sleep is rewarded by evolution. ;-)

Grover

bennfine's picture
bennfine
Status: Member (Offline)
Joined: Jan 12 2013
Posts: 2
Flashlight Shenanigans!

I'm calling shenanigans on Wendy Delmater...... to wit, she writes

The kids had eight flashlight batteries in a basket, and an empty flashlight. Two were fully charged, six of the batteries were dead: emptier than a politician's promises. The person viewing the exhibit was asked to guess the odds of getting the two charged batteries into the flashlight. Upon lifting a flap, the actual odds were revealed. Everone was shocked that the odds were 36 to 1. It was not intuitive, even for the science fair judges, who should have known better.

--------

First, she sats the odds were 36 to 1, which would imply the PROBABILITY to be 1/37 (see for example http://www.problemgambling.ca/en/resourcesforprofessionals/pages/probabi...).

Next-I don't even see the probability to be 1/36 or even 1/37.

8 choose 2 = 28. there is 1 combination that works, so the probability is 1/28

> combn(c("G","G","B","B","B","B","B","B"),2)
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
[1,] "G"  "G"  "G"  "G"  "G"  "G"  "G"  "G"  "G"  "G"   "G"   "G"   "G"   "B"  
[2,] "G"  "B"  "B"  "B"  "B"  "B"  "B"  "B"  "B"  "B"   "B"   "B"   "B"   "B"  
     [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25] [,26]
[1,] "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"  
[2,] "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"   "B"  
     [,27] [,28]
[1,] "B"   "B"  
[2,] "B"   "B"  
 

bennfine's picture
bennfine
Status: Member (Offline)
Joined: Jan 12 2013
Posts: 2
sorry didn't realize i could respond directly to a post

sorry-double posting-didn't realize i could respond directly.

I'm calling shenanigans:

The kids had eight flashlight batteries in a basket, and an empty flashlight. Two were fully charged, six of the batteries were dead: emptier than a politician's promises. The person viewing the exhibit was asked to guess the odds of getting the two charged batteries into the flashlight. Upon lifting a flap, the actual odds were revealed. Everone was shocked that the odds were 36 to 1. It was not intuitive, even for the science fair judges, who should have known better.

--------

First, you say the odds were 36 to 1, which would imply the PROBABILITY to be 1/37 (see for example http://www.problemgambling.ca/en/resourcesforprofessionals/pages/probabi...).

Next-I don't even see the probability to be 1/36 or even 1/37.

8 choose 2 = 28. there is 1 combination that works, so the probability is 1/28

bn(c("G1","G2","B1","B2","B3","B4","B5","B6"),2)
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
[1,] "G1" "G1" "G1" "G1" "G1" "G1" "G1" "G2" "G2" "G2"  "G2"  "G2"  "G2"  "B1"
[2,] "G2" "B1" "B2" "B3" "B4" "B5" "B6" "B1" "B2" "B3"  "B4"  "B5"  "B6"  "B2"
     [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25] [,26]
[1,] "B1"  "B1"  "B1"  "B1"  "B2"  "B2"  "B2"  "B2"  "B3"  "B3"  "B3"  "B4"
[2,] "B3"  "B4"  "B5"  "B6"  "B3"  "B4"  "B5"  "B6"  "B4"  "B5"  "B6"  "B5"
     [,27] [,28]
[1,] "B4"  "B5"
[2,] "B6"  "B6"

macro2682's picture
macro2682
Status: Gold Member (Offline)
Joined: Sep 3 2009
Posts: 423
Birthdays

I think that birthday trick has something to do with the popular birthdays of late September and mid November.  Which of course correspond to the popular conception days of late December and mid February.  It isn't just math that wins bets. 

macro2682's picture
macro2682
Status: Gold Member (Offline)
Joined: Sep 3 2009
Posts: 423
Batteries

I think the batteries answer is 1/28.  2/8 times 1/7.

treebeard's picture
treebeard
Status: Gold Member (Offline)
Joined: Apr 18 2010
Posts: 390
Optimism and pessimism

Paul Hawken goes after this optimism pessimism thing pretty well.  He articulates this better than I could.  The intiuitive mind can see the new dawn, while the rational mind drowns in darkness.  We are the ones we have been waiting for.

Arthur Robey's picture
Arthur Robey
Status: Diamond Member (Offline)
Joined: Feb 4 2010
Posts: 2502
Even Sharks do it.

Thanks Grover. Tweeker . You live and learn. Well, maybe some do.

I have yet to come up with a plausible reason why the need for sleep has not been at least minimised by evolution. The need for sleep is so obvious that it is neigh on impossible to ask the question, why do sentient beings all do this pathological behaviour? Even sharks do it. It seems to be a fundamental price for sentience. I would even go so far as to say that we will know that Robots are truly sentient when they need to sleep.

The true mystery seems to be hiding in plain sight.

I use this line of reasoning to conclude that I must be magnificently sentient.

macro2682's picture
macro2682
Status: Gold Member (Offline)
Joined: Sep 3 2009
Posts: 423
Sleeph

Arthur,

Robots do need sleep. Have you ever left your computer on for 2 weeks strait?  Things start getting weird, as they do for a person who's been awake too long.  Just because it makes evolutionary sense doesn't mean it makes practical sense. By your thinking, we should have all developed super speed and skin that can stop bullets. Entropy affects everything, we just need to sleep every once in a while. 

I guess if you really want to go into it, you could argue that if we never needed to sleep then we wouldn't have been driven to learn to create a safe shelter, which is necessary for a family and other complex social developments. If we didn't need to sleep we wouldn't need to stop moving around, and we wouldn't have built villages and towns.  

But I like to think that it isn't that complicated, and we just need to cool off the wiring. 

macro2682's picture
macro2682
Status: Gold Member (Offline)
Joined: Sep 3 2009
Posts: 423
More about sleep

It's also worth mentioning that yawning may be one of first forms of evolutionary communication. Many animals that need to hunt together in a pack or community have yawns that seem contagious.  Have you ever made someone else yawn just by yawning yourself?  Some scientists say that yawning was a crucial tool used by communities to keep the group on the same schedule, sleeping at the same time in order to hunt together.

SteveW's picture
SteveW
Status: Gold Member (Offline)
Joined: Jan 21 2010
Posts: 490
Body hair, batteries and bias (mathematical).

RJE: The most fascinating thing I find about my sparse body hair is that I get goose bumps when cold, an evolutionary adaptive response to increase the protective layer of warm air adjacent to my skin that is clearly redundant.

I also get 1/28 for the chance of getting both live batteries but my algebraic method is conditioned by my genetics background. There is a 2/8 chance of getting the first battery and a 1/7 chance of the second. 2/8 multiplied by 1/7 is 1/28. Maybe also influenced by playing poker, which fundamentally is an exploitation of one's probability and bluffing skills. 

Interesting comment about the sabre tooth tiger Adam. As a geneticist I'd say we are wired for those traits that have a selective advantage, i.e. allow reproduction. Immediate death from food precludes offspring whereas a slow death from a bad diet occurs in the post-reproductive years.

Wendy S. Delmater's picture
Wendy S. Delmater
Status: Diamond Member (Offline)
Joined: Dec 13 2009
Posts: 1582
shennanigans?

1/28 or 1/36 - still worse odds than people intuitively think.

And I had a math PhD help the kids 'cause I ain't a math wiz. Feel free to corrrect any mistakes and no hard feelings.

nickbert's picture
nickbert
Status: Diamond Member (Offline)
Joined: Jan 14 2009
Posts: 1125
Reminds me of my favorite bumper sticker...

....

"Lottery: A Tax on People Who Are Bad at Math"

Amusingly enough, I found that bumper sticker in a Las Vegas convenience store laugh

OITW's picture
OITW
Status: Bronze Member (Offline)
Joined: Mar 17 2012
Posts: 28
Shenanigans agins

Okay, I decided to test my BS meter against Adam's article. The mental trick is that one starts by thinking of the likelihood of someone else having one's own birthday. That's the defect that sets us up for the kill, all the prof has to do is find a match, not a particular match. Perhaps more importantly, the prof does not have to find the match, he gets the pool to self-declare its match. I find the How and Why of our self-deceptions more interesting than the actual statistics--and that itself is a gateway to self-deception.

That said, I tried to grok the wikipedia link, though I dare not try to work it with pencil and paper. In defense of Wendy, I think 1 in 37 is grossly low, but I won't try to defend that with calculations. Adam's problem was the probability of a match in a given pool, and as shown, the probability of a match was very high. But in the case of the batteries, we know there is a match. The challenge is to pick the two good batteries out of a group of eight. Can you imagine Adam's professor picking a birthday pair out of his class (without looking at the class roster)? The probability of that is exceedingly slim,a fraction of a percent. What if he had to pick out the four pairs WITHOUT KNOWING that with any of the pairs he picked he had reduced the overall number to choose from? The probability of there being a birthday match in a given group, and the probability of your being able to select it at random are two very different things (it seems to me).

Now, Wendy's brief statement of the problem didn't say how we know we've picked a good battery. Without a battery tester, in the real world we can't know until we put them in the flashlight--and by then it's too late. Two dead batteries, no problem. One good and one dead, you've just killed your chances of ever picking two good batteries, although you now have two half-good batteries. But with every bad battery you test with one of them, you've cut that remaining battery power in half.

So in Wendy's problem, you must guess correctly the first time, and the first time only. Finding a pair of bad batteries with your first guess actually helps, because at least with a complete failure you can eliminate 2/8 of the choices. But at the same time, your odds of choosing a mismatched battery pair increase have also increased proportionally.

In fact, numbers force us into bad decisions more often than not. I work in the realm of risk and threat, neither of which are mathematically calculable (risk is to an extent, but not threat), but which my overlords demand I squeeze into a mathematical likelihood (the threat is 60% likely). Hogwash, there are so many variables in what that threat can choose (the enemy always has a choice) and more introduced by our countermeasures, that there is no way of actually reducing a real life threat to a probability.

So, yes, knowing some statistics and probability is worth while, but it can't solve a lot of problems, and it doesn't even always provide good guidance on how to solve a lot of them.

OITW's picture
OITW
Status: Bronze Member (Offline)
Joined: Mar 17 2012
Posts: 28
Clarification

In fact, I would say that in the professor's wager there was no threat, just a gamble; and the risk was only $5.

But if one needed the flashlight to fix the generator as the house sunk into a deep freeze, the risk is substantial--one could lose everything, even one's life. At which point the threat of picking anything but the two good batteries becomes incredibly important, even lethal. Wisdom would have been to toss the dead batteries ages ago, or at least not to have mixed them with good batteries. I don't see statistics providing much guidance to help if one is stupid enough to get stuck having to guess at good batteries.

On the other hand, if someone offered you a bargain on a bag of batteries, say 20, with the caveat that maybe one or two of the batteries were dead, these statistical exercises today should be enough to warn you off of that "bargain."

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
Arthur you said:...

"The true mystery seems to be hiding in plain sight."

I have given this some thought and all I could come up with is TIME. Why? We have just come out of our fear as a species with regards to the night during the age of light. Before then Fear ruled the night and in response we slept, we still sleep, as not enough evolutionary time has passed to have changed the need for sleep. That and a host of quick answers but I think "evolutionary time" is the reason we still require sleep from sun down to sun up.

My habits today are different than when I was younger (less sleep needed) but is probably more choice than anything.

Happy Sailing and I'm going back to bed!

BOB

gillbilly's picture
gillbilly
Status: Gold Member (Offline)
Joined: Oct 22 2012
Posts: 382
Even more about sleep.

Don't you guys dream? lol C.G. Jung and Joseph Campbell say it best...

Peace! (ful) dreams

cowpoke's picture
cowpoke
Status: Member (Offline)
Joined: Nov 30 2008
Posts: 9
A Culture of Personality

And what worries me – scares me, is more accurate – is that I don't observe this same caution in the actions of the people making the truly big decisions.

-----------------------------------------------------------------------------------------------------------------------------------------------

In her book "Quiet", Susan Cain talks about how the west has gone from a Culture of Character to a Culture of Personality.  I think she is right, our culture rewards people for making snap decesions and moving on.  What is valued is salesmanship, being right about something is not as important as being willing and able to promote your own view.  Caution is seen as a weekness and the trick is to not  be around when the bill comes due for poor decisions.

What worries me is the amount of pain a nation must feel before considering that there is something wrong with the status quo.

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
Gillbilly...

you said:

"Baseball is a great analogy for what's going on in the world."

You have in this statement said the truth.

Baseball is seen through a prism of spoiled Rich dudes who have the world by the tail. The reality is these are our youths playing a game that frankly they would do for far less but will gladly take whatever they are offered.

Baseball is NO different than the world society at large, These kids are for the most part underprivileged and in many cases just third world boys who play a game well. They have their temptations as we all do, are competitive as we all are, and frankly are unprepared for what comes their way as most are.

The drugs or performance enhancing ones are pushed on them well before they get to the Bigs. Imagine if you would the temptation for a boy, 18, 19 years old who can hit the cover off the ball but needs to only hit that little white thing 10 more feet to hit the ball out or just hard enough to get it through the infield to make it to the Bigs, Taking something that is pressure pushed on them by agents or subtly through the organization they play for is powerful when winning and making it to the show is all encompassing. We are talking millions here as that is the average these kids, again, I say kids, will earn. 

Anyways, like I said and you referenced in your quote above, and that is baseball is very much a window pane into everything we see in the world today. I actually believe it's the influence of society that influence the game and, not the other way around. These kids are learning from the intellectuals and the lawless what is expected of them to succeed. As with everything blame must be assessed and ballplayers are an easier mark than having Congress go before the public and be paraded out in front of the world, and I dare say that every Congressman would plead the 5Th with every question asked of them. It's a double standard. Plus we are all guilty of playing the game, would do just about anything to feed and cloth our kids. Taking steroids so that we can get back to work sooner or taking alcohol and drugs for the lift it gives us as we sing and dance at times don't seem so bad. Moderation right, so these kids learn all this shit from us. Yet, they are to be perfect role models when they have learned, and have had reinforced on them all that we show as acceptable behavior. It really comes down to their mental and physical toughness, and that will change with every injury and responsability accumulated on the field and at home.

I just don't judge (severily) personally anyone myself. I am not walking in their shoes, and I know what I would do to take care of my family. Everything and anything short of taking another persons life or stealing to get what I wanted. I will work however and hard but I will work. 

These kids are doing what their families, coaches and owners of their contracts are asking of them, and then thrown under the bus when they get caught. That's just the truth, and it's on all of us for expecting more from them and not from ourselves.

The ones that actually get involved with enhancing are the older ball players that have lost a step, been hurt and don't hit it quite as far or hard, and are having to do something. Their life styles and families are more entrenched in the dream and finances as never before, and staying in the Bigs is the whole deal going bad or staying as is. That's a tough, tough decision, and one that every single one of us would have to make to stay in our positions in life. Risk/Reward.

Regards

BOB

Travlin's picture
Travlin
Status: Diamond Member (Offline)
Joined: Apr 15 2010
Posts: 1322
Thanks Cowpoke

cowpoke wrote:

In her book "Quiet", Susan Cain talks about how the west has gone from a Culture of Character to a Culture of Personality

Cowpoke

Thanks for sharing this insight.  It helps clarify a feeling I’ve had for a long time, but couldn’t express well.  The rest of your post shows how dangerous this trend is.

Travlin 

Brent1023's picture
Brent1023
Status: Member (Offline)
Joined: Oct 3 2010
Posts: 4
Selection time frames

Selection takes place over long time frames. People have only been able to conceive of numbers large than a few dozen for a relatively short time - certainly not long enough for selection to make a difference, even if ability with sets large enough to be statistically significant made a difference in passing on your genes.

There is a good chance that the species will never evolve along those lines. That is, that those skills will never be a factor in selection.

That does not mean that the cultural trait of using statistics could not become a factor in the survival of the culture - that a culture that values statistical inference (in particular, and science in general) may not come to dominate those that do not.

Do we have time before those who go with their gut drive us over the edge?

Not looking so good from here.

Brent1023's picture
Brent1023
Status: Member (Offline)
Joined: Oct 3 2010
Posts: 4
Bayesian Statistics

The article discusses statistics then asks a question.

I think it is important to know there are two kinds of statistical analysis. The first kind treats problems like the birthday problem. Or talks about taking colored balls out of urns. Here there is no history, just the present - the urn full of balls - and the future, possible selections.

The second kind talks about what happens when you know a bunch of stuff has already happened. The results here are even more startling. It is called Bayesian statistics.

The relevance?

If we ask - should the government simply print money to pay its bills? - the answer is No.

If we state

- the government has created programs for which it must pay and

- has set tax rates that are supposed to pay for those programs but cannot

then say

- should the government print money to pay its bills?

we come to a different conclusion.

What am I saying? I am saying that the analogy drawn in the article can lead to false conclusions, so is a bad analogy.

A better analogy takes history into account, does not treat the problem in isolation.

An interesting example is the Price is Right example. When offered three doors, you pick one. The host then opens a different door and shows you it is not the best door. Should you change your pick?

Going back to the questions at the end of the article - should the government print money to pay its bills?

These bills are a problem because the Congress put programs in place for which there was no revenue - wars, other programs. Revenue was never there to pay for these programs. Taxes were not raised. Revenue now is insufficient to pay for those programs.

Revenus is also down because the economy has shrunk over the last 5 years.

What to do?

In the government case, it can print the money. In this case, I believe it should.

My point? Decisions in real life need to be made in the context in which the implications of those decisions will happen. Not in a statistical distribution in which context is not a factor.

RJE's picture
RJE
Status: Diamond Member (Offline)
Joined: Aug 31 2008
Posts: 1369
Stats and Analysis I can believe in...

http://www.hussmanfunds.com/wmc/wmc130114.htm

BOB

Brent1023's picture
Brent1023
Status: Member (Offline)
Joined: Oct 3 2010
Posts: 4
Er ...

Probably 1 in 28.

On the first pick, 1 in 4 (well, 2 in 8) chances. If you get that right, 1 in 7 on the second, for 1 in 28.

Or, you could just put 8 choose 2 into google.

Brent1023's picture
Brent1023
Status: Member (Offline)
Joined: Oct 3 2010
Posts: 4
Optimism, Pessimism, Nomadic gene

Call it optimism and pessimism, or call it a nomadic gene, but it was shown some time ago that a species that does not have a few wanderers will die out. IIRC, the optimum percentage of wanderers was around 5%.

We don't usually associate human traits with non-humans, so using terms like optimism and pessimism is risky.

However, if no members of a species wander off see what is on the other side of the hill, then the species remains confined to one area. If problems hit that area - heat, cold, drought, floods - the species can be wiped out.

If too many wander, numbers could be too low to sustain the species anywhere.

Having said that, almost all of the wanderers simply die when they reach a place that cannot sustain them. In birds, first year birds that go north instead of south almost always die come winter.

So, wandering is essential but most wanderers die almost immediately.

Not a great role, but someone has to do it.

KugsCheese's picture
KugsCheese
Status: Platinum Member (Offline)
Joined: Jan 2 2010
Posts: 708
Most logical errors are due

Most logical errors are due to false assumptions.  For example, there is not equal chance a person could be born on any day of the year.   Think about cold weather.

The Monty Hall problem is a classic that most get wrong because most do not consider the complement probabilty, i.e. if an event has X chance of happening then the chance of it not happening is 1 - X.  Once you think of the complement many problems in probability become much easier.

I don't know how one would compute a probabilty for an an event within X time period for a complex system like the US economy since it is non-linear meaning at some value 'x' and 'y' it holds that: 1) xF(y) <> F(x * y) and 2) F(x + y) <> xF(x) + F(y).  Some call this an inflection point but in math it is called a non-continuity meaning the domain is not smooth.

Quercus bicolor's picture
Quercus bicolor
Status: Silver Member (Online)
Joined: Mar 19 2008
Posts: 243
re: sleep

Let's not forget: until recently, it was difficult to do anything useful at night because of the lack of practical, portable lighting.  The most valuable thing to do at night was to sleep so as to be at one's best during daylight hours.  Maybe after a few thousand more years of convenient, practical lighting, we'll need a lot less sleep!

cowpoke's picture
cowpoke
Status: Member (Offline)
Joined: Nov 30 2008
Posts: 9
Assessing Risk

Travlin,

I have a link you may be interested in, this is an RSA Animate video titled "Smile or Die".  I don't know if these have been widely posted here or not but it is quite well done and worth the time. 

In my mind I keep asking, how has it come to this, how did we get here, what is going on.  The only explanation that works for me is generational theory, such as Strauss and Howe and their ideas about a Fourth Turning.  Some dismiss their work as non-falsifable and all that but for me it works.

Travlin's picture
Travlin
Status: Diamond Member (Offline)
Joined: Apr 15 2010
Posts: 1322
Yes

Cowpoke

Yes, there were some interesting observations there.

Travlin 

silvervarg's picture
silvervarg
Status: Bronze Member (Offline)
Joined: Mar 28 2010
Posts: 44
Re: Brent1023

Great post Brent1023.

It does not just paint things in black and white, there are lots of shades in between.
Even so we now have the option of "Print money" or "Not print money".

Or, is that really the case? Do we have more options?

As you pointed out if we just got the question:
"should the goverment print money to pay its bills?"
We know that lots of moneyprinting will lead to inflation etc and we just makes things worse for the future, so it is easy to simple answer "No" quickly.

Then we look deeper into the problem and reallize that we are in deep shit, so the simple "No" answer will cause all sorts of problems, so it is simpler to just answer "Yes, print more money".

Thinking yat again we remember that it will ruin the life of our kids etc by constantly just print money to push the problem further on, as it makes things worse.

One logical conclusion would be that we need to make sure that we will not run long term masive money printing (more than GDP growth). Once we have setup a budget that all can agree on that is considerably better than the allowed long term money printing we can allow ourselves the lyxury to solve the immediate problems with money printing or more loans or MintTheCoin or whatever trick we choose to use.
 

rhare's picture
rhare
Status: Diamond Member (Offline)
Joined: Mar 30 2009
Posts: 1271
Crappy software abounds....

macro2682 wrote:

Have you ever left your computer on for 2 weeks strait?  Things start getting weird, as they do for a person who's been awake too long.

That's just crappy software at work. We have all had shifting baselines thanks to Microsoft on what is acceptable.  Why is it that rebooting is considered an acceptable repair technique?  If you write software for truly important things, ever having to do an unplanned reboot is not acceptable.  We used to measure up-time in months to years (3.5 years for one) for some of our critical systems.

It also shows why additional complexity leads to fragility.  More code - more bugs....

KugsCheese's picture
KugsCheese
Status: Platinum Member (Offline)
Joined: Jan 2 2010
Posts: 708
rhare wrote: macro2682

rhare wrote:

macro2682 wrote:

Have you ever left your computer on for 2 weeks strait?  Things start getting weird, as they do for a person who's been awake too long.

That's just crappy software at work. We have all had shifting baselines thanks to Microsoft on what is acceptable.  Why is it that rebooting is considered an acceptable repair technique?  If you write software for truly important things, ever having to do an unplanned reboot is not acceptable.  We used to measure up-time in months to years (3.5 years for one) for some of our critical systems.

It also shows why additional complexity leads to fragility.  More code - more bugs....

I only reboot my Linux-based music system when I apply a firmware update when I decide I want the new features (it is embedeed Linux and uses average of 20W with mirrored drives).  It has many times ran longer than 1 year and streams mostly FLAC files.

phecksel's picture
phecksel
Status: Silver Member (Offline)
Joined: May 24 2010
Posts: 156
Even worse then flawed

Even worse then flawed analysis is the predetermined outcome.  I see that way too often.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Login or Register to post comments