A question of table manners & human relations

[LarryHoward]

OK, there's "gambling," very broadly defined, and there's habitually taking astronomical sucker bets because it feels good.

Of course you're right. Sometimes I wonder if for the habitual gambler it's simply that the joy of winning more than makes up for the anguish of losing. Classic Tversky/Kahneman loss aversion says the opposite is usually true.

If I remember correctly, intermittent reward at the right frequency, increases dopamine release during the expectation phase and that's what keeps the gambler coming back. There is also some understanding of the physiology behind "chasing" losses, which I was never quite able to follow. The strange thing for me is that all of this stuff evolved to maximize survival in a partially predictable world, but it's so easy to fool.

I get the dopamine release. Mother Nature had to deliver a reward for risk taking to force the species to undertake low probability exercises with associates risks such as hunting.

The chasing losses comes from believing the "odds will catch up" and failure to understand independent experiments. Yes, a coin toss result is a 50-50 event but after 3 heads in a row, the probability of heads on the 4th trial is still exactly 50%. Or, to put it in lottery terms, each ticket is in 30 million or so. Buying a ticket a week doesn't change that. Buying 10 tickets in one week increases the odds all the way to 1 in 3 million, still tiny odds but with a 1 in 20 or so chance of a $3 payout, the seller can trigger a dopamine release for a losing proposition (spend 20. Win 3, ensuring the stooge will play again.

Then again, dropping $2 on a $100m jackpot once in a while is harmless fun. It's the poor folks dropping their paycheck every week on that sucker bet that is both dangerous and a tax on the poor.

[kdh]

Lotteries are a tax on the uninformed at least, I suppose, if not the poor.

Learning probability I was surprised, disturbed even, at the number of times I saw a proof of something that contradicted intuition.

A tip for lotteries. Choose numbers bigger than 31. You won't be sharing a jackpot with someone because of a birthday, anniversary, etc.

[Orestes Munn]

OK, there's "gambling," very broadly defined, and there's habitually taking astronomical sucker bets because it feels good.

Of course you're right. Sometimes I wonder if for the habitual gambler it's simply that the joy of winning more than makes up for the anguish of losing. Classic Tversky/Kahneman loss aversion says the opposite is usually true.

If I remember correctly, intermittent reward at the right frequency, increases dopamine release during the expectation phase and that's what keeps the gambler coming back. There is also some understanding of the physiology behind "chasing" losses, which I was never quite able to follow. The strange thing for me is that all of this stuff evolved to maximize survival in a partially predictable world, but it's so easy to fool.

I get the dopamine release. Mother Nature had to deliver a reward for risk taking to force the species to undertake low probability exercises with associates risks such as hunting.

The chasing losses comes from believing the "odds will catch up" and failure to understand independent experiments.

Yeah, the psychological theory is clear, but there's some science on the brain areas activated when human gamblers chase losses. Interestingly, animals don't seem to subscribe to the Gambler's Fallacy. Their behavior quickly conforms to the trial by trial probability.

[LarryHoward]

Lotteries are a tax on the uninformed at least, I suppose, if not the poor.

Learning probability I was surprised, disturbed even, at the number of times I saw a proof of something that contradicted intuition.

A tip for lotteries. Choose numbers bigger than 31. You won't be sharing a jackpot with someone because of a birthday, anniversary, etc.

At a $50M plus cash payout, I'd be willing to share . When I'm silly enough to drop $2.00 at the checkout, I figure any random combination is as good as any other of the 259M combinations. Lightening will strike one of them but I can't change the odds by agonizing over which numbers to bet on. The expected value of my payout is still infinitesimally small.

[Orestes Munn]

If I could teach the US population one thing beyond basic literacy it would be elementary probability and statistics. Think of the terrible errors we could avoid.

[kimbottles]

If I could teach the US population one thing beyond basic literacy it would be elementary probability and statistics. Think of the terrible errors we could avoid.

I would teach them basic personal finance. I mentored some of my former employees in that subject, amazing what many people believe!

[SloopJonB]

Lotteries are a tax on the uninformed at least, I suppose, if not the poor.

Learning probability I was surprised, disturbed even, at the number of times I saw a proof of something that contradicted intuition.

A tip for lotteries. Choose numbers bigger than 31. You won't be sharing a jackpot with someone because of a birthday, anniversary, etc.

I read some info on the odds of lotteries a long time ago. Apparently the Mob payed out more on their old "Numbers" racket than governments do on the legal lotteries.

Pretty bad when the Mob is more fair than your own government.

[LarryHoward]

If I could teach the US population one thing beyond basic literacy it would be elementary probability and statistics. Think of the terrible errors we could avoid.

I would teach them basic personal finance. I mentored some of my former employees in that subject, amazing what many people believe!

I agree. I was most disappointed that my daughter's university (with 34K students)offered one class section annually on personal finance, including investment basics, cost of credit, etc. It was fully subscribed within minutes of registration opening but they couldn't answer the question of "Why not offer more sections?"

I would add basic "project management" to the mandatory curriculum as well. Too many well schooled and intelligent grads can't understand getting from point a to point on a straightforward task.

[BeauV]

It seems to me that the purchase of one lotto ticket is the best improvement in odds available. The difference between zero tickets and one ticket is a gigantic improvement in the odds

That said, the purchase of the second ticket is only a tiny improvement so it's to be ignored.

I have to say, I could never bring myself to buy a ticket and cringe each time I see my sailing friends (who can't afford it) buying tickets.

[kdh]

If I could teach the US population one thing beyond basic literacy it would be elementary probability and statistics. Think of the terrible errors we could avoid.

It's even more ok in our culture not to be proficient in statistics ("sadistics") than it is not to be good at math. Imagine if someone said, "I'm terrible at writing, I'm practically illiterate." Compare that to "I'm awful at math, I can't even balance my checkbook" (not that balancing a checkbook has anything to do with math).

In the medical profession, for example, biostatisticians play second fiddle to doctors. Statistics is a support profession.

That's changing in many fields, however. "Data scientists" rule the day.

https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century/

[BeauV]

We have a disgusting anti-intellectual bend in our society. I've had people pronounce proudly that they've never bothered with Algebra or Geometry, let alone statistics.

If you really want to know why a small part of the population outstrips the vast majority in their earning power, this is a good place to start looking for a cause.

[LarryHoward]

We have a disgusting anti-intellectual bend in our society. I've had people pronounce proudly that they've never bothered with Algebra or Geometry, let alone statistics.

If you really want to know why a small part of the population outstrips the vast majority in their earning power, this is a good place to start looking for a cause.

I guess I like to think folks who major in the humanities and arts add a lot to our civilization. But I will pay a lot more for an engineer than I will for an English lit major.

[SloopJonB]

We have a disgusting anti-intellectual bend in our society. I've had people pronounce proudly that they've never bothered with Algebra or Geometry, let alone statistics.

If you really want to know why a small part of the population outstrips the vast majority in their earning power, this is a good place to start looking for a cause.

Chris Rock had a great bit about that re: ghetto dwellers - "Who's the President?" - "Ah don' know that sheeit man".

Ignorant and proud of it.

Or as David Niven claimed to have received on a report card "Ignorant and unashamed".

[Orestes Munn]

If I could teach the US population one thing beyond basic literacy it would be elementary probability and statistics. Think of the terrible errors we could avoid.

It's even more ok in our culture not to be proficient in statistics ("sadistics") than it is not to be good at math. Imagine if someone said, "I'm terrible at writing, I'm practically illiterate." Compare that to "I'm awful at math, I can't even balance my checkbook" (not that balancing a checkbook has anything to do with math).

In the medical profession, for example, biostatisticians play second fiddle to doctors. Statistics is a support profession.

That's changing in many fields, however. "Data scientists" rule the day.

https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century/

The medical profession per se is just a bunch of docs treating patients with no involvement from statisticians. Physicians tend to have a better understanding of statistics than lay people, but they rarely let that get in the way of flying by the seat of their pants. It is said, and I strongly agree, that the three most dangerous words in medicine are "in my experience." Likewise, basic science is full of crummy designs, underpowered studies, and irreproducible results. It is estimated that about 2/3 of what's published in peer reviewed articles is other than true. In the world of clinical research, however, statisticians fill a key role and have great power because lives and big money depend on it. The funding and regulatory bodies require it and clinical trials, with full statistical plans, must be registered beforehand in order to be published in respectable journals.

I rarely do real clinical trials, but because I work with humans, any piece of research I want to do has to undergo a thorough review of the design, power analysis, and statistical plan, among many other things, before I can do it and there are now even people looking at the publications to make sure that I say so if I deviate from the planned analysis. One of the toughest things to explain to my PhD postdocs is how and why post hoc analyses are inherently weak and that it is critical to caveat them. They don't seem to teach that in grad school.

[Ish]

I buy a lottery ticket once in a while, knowing the odds. I have a regular set of numbers (that start with 31, strangely enough). Over the years I have won a couple of thousand, combined with my very irregular buying history I'm above water on it. Being one of those ignorant English lit majors, I have given up on making millions in the workplace.
However, that doesn't mean financial illiteracy. I designed and kept the books for my incorporated photolab for 12 years. It also doesn't mean mechanical illiteracy. I built a professional photolab from scratch, upgraded and maintained it. I also do virtually all my own maintenance on the boat, including engine rebuilds.
I prefer to think of myself as a well-rounded Renaissance type, as opposed to a gadfly.

[BeauV]

I buy a lottery ticket once in a while, knowing the odds. I have a regular set of numbers (that start with 31, strangely enough). Over the years I have won a couple of thousand, combined with my very irregular buying history I'm above water on it. Being one of those ignorant English lit majors, I have given up on making millions in the workplace.
However, that doesn't mean financial illiteracy. I designed and kept the books for my incorporated photolab for 12 years. It also doesn't mean mechanical illiteracy. I built a professional photolab from scratch, upgraded and maintained it. I also do virtually all my own maintenance on the boat, including engine rebuilds.
I prefer to think of myself as a well-rounded Renaissance type, as opposed to a gadfly.

I can completely relate to this - as someone with a degree in Philosophy. A lot of us are self-taught in dozens of topics ( including statistics ) and, to be frank, I actually enjoy learning the new field almost as much as whatever forced me to learn the topic. Sailors do tend to end up being sewn from this cloth.

[Bull City]

Pat came to the pub last night, but alas, no salad.

[kimbottles]

Pat came to the pub last night, but alas, no salad.

Maybe he lurks here?

[Slick470]

Wow... that was a very impressive thread drift.

For Bull's pub/salad friend... I had a friend from years ago who enlisted in the Army. When he was home once on leave from basic training, another buddy and I took him out to one of those all you can eat pasta places, spaghetti warehouse or something like that. When his first bowl of pasta arrived he promptly did just what you're describing and finished the bowl in less than a minute. After the second bowl of the same, my buddy and I just looked at him and asked why he's eating like that. He said that that's how he had to eat in basic. If you don't get it down in some short amount of time, you don't get to eat it. He was probably 4 bowls in before my other buddy and I finished our first.

Maybe your friend was in the Army and has never broken that habit?

[Bull City]

He's too short and fucked up.

[Slick470]

I dunno... they took my buddy

[kdh]

OK, there's "gambling," very broadly defined, and there's habitually taking astronomical sucker bets because it feels good.

Of course you're right. Sometimes I wonder if for the habitual gambler it's simply that the joy of winning more than makes up for the anguish of losing. Classic Tversky/Kahneman loss aversion says the opposite is usually true.

If I remember correctly, intermittent reward at the right frequency, increases dopamine release during the expectation phase and that's what keeps the gambler coming back. There is also some understanding of the physiology behind "chasing" losses, which I was never quite able to follow. The strange thing for me is that all of this stuff evolved to maximize survival in a partially predictable world, but it's so easy to fool.

I get the dopamine release. Mother Nature had to deliver a reward for risk taking to force the species to undertake low probability exercises with associates risks such as hunting.

The chasing losses comes from believing the "odds will catch up" and failure to understand independent experiments.

Yeah, the psychological theory is clear, but there's some science on the brain areas activated when human gamblers chase losses. Interestingly, animals don't seem to subscribe to the Gambler's Fallacy. Their behavior quickly conforms to the trial by trial probability.

I had never heard of the term "Gambler's Fallacy." I guess because statisticians are taught what's right and not what mistakes people made in figuring out what's right. Which may be a wrong way to teach, but I digress.

I wonder if the fallacy isn't exhibited in animals because it requires remembering early trial results and the benefit wasn't worth the energy for smaller brains.

Is the fallacy symmetric, in that gains are expected after a series of losses as strongly as losses are expected after a series of gains?

There are some subtleties with repeated trials that are easily not appreciated. For example, it's newsworthy when someone wins the lottery twice. Yes, the probability of me, or any individual, winning the lottery twice is infinitesimal. But the probability of someone winning the lottery twice is not. Think of it this way. The probability of my winning the lottery once is small, but the probability of someone winning the lottery once is 1.

[Orestes Munn]

I had never heard of the term "Gambler's Fallacy." I guess because statisticians are taught what's right and not what mistakes people made in figuring out what's right. Which may be a wrong way to teach, but I digress.

I wonder if the fallacy isn't exhibited in animals because it requires remembering early trial results and the benefit wasn't worth the energy for smaller brains.

Is the fallacy symmetric, in that gains are expected after a series of losses as strongly as losses are expected after a series of gains?

There are some subtleties with repeated trials that are easily not appreciated. For example, it's newsworthy when someone wins the lottery twice. Yes, the probability of me, or any individual, winning the lottery twice is infinitesimal. But the probability of someone winning the lottery twice is not. Think of it this way. The probability of my winning the lottery once is small, but the probability of someone winning the lottery once is 1.

You are right about the animals. Unlike perhaps all but the very smartest animals, humans have two memory systems an implicit/procedural one, which picks up skills and knowledge through repeated experience under the guidance of the reward system, and an explicit/episodic one, which stores knowledge of specific instances of facts and events. This is the system that gets knocked out in the famous amnestic patients, who can learn new skills and sort of function, but can't remember their autobiographies.

The implicit memory system is capable of learning and predicting the behavior of stochastic systems without consciousness of that knowledge, and there are several classic ways of teaching people stuff without their their awareness. In order to gamble effectively, all you need is this for this system to learn the probabilities assigned to the contingencies of the game. Animals and humans can do that. However, when humans start to store and count conscious, "explicit," representations of events their behavior becomes subject to all their stupid prejudices and beliefs about the world, including the Gambler's Fallacy.

I don't know if the fallacy is symmetrical. I think it is, but you'd have to ask a gambler.

The fact about the lottery is subtle and I'm sure my daily behavior doesn't conform to it. Repeated trials are very tricky in science, too. For instance, very few scientists appreciate the fact that the greater the number of studies of a phenomenon or, say, clinical trials of some novel treatment, the greater the likelihood of false positives and the less the positive predictive value of each individual study. I work in an area full of shitty little underpowered trials and it's very hard to convince even very smart people that what they're seeing is just noise shaped by our biases.

[LarryHoward]

I find it interesting that it's called a "Gambler's Fallacy" and not "Sucker's Dream."

A professional gambler, such as a poker player, knows the odds very well and bets to maximize his expected return. A loss or even folding a bad hand is a price to play until the odds turn in his favor and his expected value rises dramatically. HIs only real gamble is the bluffing of other professionals who also are carefully calculating odds.

Getting into trials and confidence intervals in a somewhat random scenario with little ability to control all of the possible contributing factors is an entirely different challenge. We used to have a real problem on weapons programs when folks would require high 90's reliability at a 90% or 95% confidence level. It was hard to explain to well meaning folks that that really translates into requiring perfection because a single failure drives the required number of trials up dramatically, far beyond even defense program budgets.

[SloopJonB]

Re: gamblers, I had an early and enlightening experience with their mentality. In the early 70's I crossed paths briefly with a pro poker player (it was illegal then). He went into a game with $9 in his pocket and left with a new V12 E-Type Jaguar. A couple of months later he went off the road in it into a rock pile and ripped the suspension off it, totaling it. He just hopped on a bus and went home and forgot about it.

I would have been near suicidal but to him he only lost $9.

Gave me a real understanding of "Easy come, easy go".

[kdh]

The fact about the lottery is subtle and I'm sure my daily behavior doesn't conform to it. Repeated trials are very tricky in science, too. For instance, very few scientists appreciate the fact that the greater the number of studies of a phenomenon or, say, clinical trials of some novel treatment, the greater the likelihood of false positives and the less the positive predictive value of each individual study. I work in an area full of shitty little underpowered trials and it's very hard to convince even very smart people that what they're seeing is just noise shaped by our biases.

Implicit in calculating the power of studies in the usual way is that one designs an experiment and tries once. The reality is we toil away to find something interesting and publish it or act on it only if it is. Big difference to a probabilist.

The simple example that comes to mind is the blood panel we get for a physical. That something in it is abnormal is quite likely (normal). We obsess over what that happens to be when we should really be thinking. "Oh well, practically no one's normal by all measures."

[Orestes Munn]

I find it interesting that it's called a "Gambler's Fallacy" and not "Sucker's Cream."

A professional gambler, such as a poker player, knows the odds very well and bets to maximize his expected return. A loss or even folding a bad hand is a price to play until the odds turn in his favor and his expected value rises dramatically. HIs only real gamble is the bluffing of other professionals who also are carefully calculating odds.

Getting into trials and confidence intervals in a somewhat random scenario with little ability to control all of the possible contributing factors is an entirely different challenge. We used to have a real problem on weapons programs when folks would require high 90's reliability at a 90% or 95% confidence level. It was hard to explain to well meaning folks that that really translates into requiring perfection because a single failure drives the required number of trials up dramatically, far beyond even defense program budgets.

Good point about the term. I'm sure there are better.

Failure is not an option, blah-blah, woof-woof.

[Orestes Munn]

The fact about the lottery is subtle and I'm sure my daily behavior doesn't conform to it. Repeated trials are very tricky in science, too. For instance, very few scientists appreciate the fact that the greater the number of studies of a phenomenon or, say, clinical trials of some novel treatment, the greater the likelihood of false positives and the less the positive predictive value of each individual study. I work in an area full of shitty little underpowered trials and it's very hard to convince even very smart people that what they're seeing is just noise shaped by our biases.

Implicit in calculating the power of studies in the usual way is that one designs an experiment and tries once. The reality is we toil away to find something interesting and publish it only if it is. Big difference to a probabilist.

The simple example that comes to mind is the blood panel we get for a physical. That something in it is abnormal is quite likely (normal). We obsess over what that happens to be when we should really be thinking. "Oh well, practically no one's normal by all measures."

This "we" pretty much has to stick to the pre-established plan or make it clear that we didn't, with the explicit acknowledgment that this lowers the statistical power or raises the threshold for significance. Things rarely go according completely according to plan, but you have to take that into account. I agree that most of science still doesn't do this out of an equal mixture of ignorance and dishonesty.

As for the interpretation of findings, the correct approach is Bayesian, i.e. based on the ratio of true to false potential relationships possible in the real world and their prior probability. The thinking about an abnormal test result should be, "why did I order this test?" Was I just idly wasting the insurance company's money, or did I do it because I had reason to suspect something in this patient or ? In latter case, the true relationship (between the test result and the condition it was designed to detect) becomes way more likely. In science, weak hypotheses require a higher standard of proof and significant p values mean much less when there are 100 or 1000 equally (un)likely explanations for the relationship.

One of my favorite methods papers of all time is titled, "Why Most Published Research Findings are False" and contains a clear proof that this must be so for a statistical significance threshold of p = 0.05.

[kdh]

I think the Bayesian idea is a good one. Even a strict frequentist can in principle formulate a model that encompasses all experiments, failures and successes, and make inferences and power calculations on that basis.

To me though there's no substitute for just generally making inferences from data responsibly and with integrity. It can be easy to hide in the apparent sophistication of statistical methods.

[BeauV]

Most of my experiences with statistics in really REALLY complicated computer systems falls prey to: Mistaking Precision for Accuracy.

I'm always getting highly precise answers that are obviously wrong.