About the courseThe goal of this course is to train students in the use of basic statistical techniques for exploratory data analysis and to present problems in which the statistical analysis of data supports decision-making.Recommended reading:BUSSAB, Wilton de O.; MORETTIN, Pedro A. Estatística básica. Saraiva, 2017 (a standard Brazilian introductory statistics textbook).Statistics and modelsAs a reminder: statistics deals with the collection, reduction, analysis and modeling of data. From now on, we will focus a lot on modeling. A statistical model is a simplified representation of reality.As the statistician George Box put it:“Essentially, all models are wrong, but some are useful”- George E. P. BoxWhen we say that a falling coin has a 50% chance of landing heads and a 50% chance of landing tails, we are creating a mathematical/statistical model that helps us understand this phenomenon and make decisions. If we stop to think, the coin might land on its edge (or never come down at all), but these possibilities are discarded in order to have a useful and understandable model.A model widely used in biology to estimate the size of a species' population is the Lincoln-Petersen estimator. It follows a few steps: capture some individuals, mark them, release them, wait a while and capture some individuals again. In the end, we have the following rule of three: First capture (marked) -------- Total population Marked animals found in the second capture -------- Second capture The idea is that the proportion of marked animals in the second capture should be similar to the proportion of marked animals in the whole population. Since we know how many animals we caught the first and the second time, and how many marked ones reappeared, the rule of three gives the total population:\[ N = \frac{n_1 \times n_2}{m_2} \]In the image below, 15 animals were captured and marked on the first occasion (\(n_1 = 15\)); on the second, 16 were captured (\(n_2 = 16\)), of which 4 were marked (\(m_2 = 4\)). The estimate is \(N = 15 \times 16 / 4 = 60\) animals. And there we have a model.The Lincoln-Petersen estimator has several caveats (thinking about what they are is left as an exercise), but it is indeed useful, and the final result is almost always close to the true value!The problem here lies in the use of almost always. Since scientific information is expected to be understood the same way by everyone, terms like this create doubt:What does almost always mean to you?Is the chance of getting close to the result the same in every situation?What counts as close to the result?For the scientific method to be uniform, models are not described with vague words, but with probabilities! The statement becomes something like:"In 99% of cases, the model misses the population size by less than 5%."In a pond, 40 fish were caught, marked and released. A week later, 50 fish were caught, 8 of which were marked. What is the Lincoln-Petersen estimate of the population?Answer: 250 — N = 40 × 50 / 8 = 250. Marked fish are 8/50 = 16% of the second capture; if the 40 marked fish are also 16% of the population, it has 40 / 0.16 = 250 fish. 2,000 is the mistake of multiplying without dividing by the recaptured fish.Probability: a bit of historyThe study of probability was born from the human desire to predict the future, from battle tactics to doubts about whether astrology had any basis. The Roman orator Cicero, in the 1st century BC, mocked astrology by recalling the battle of Cannae (216 BC), in which the Roman army was slaughtered by a smaller army led by Hannibal: "did all the Romans who fell at Cannae have the same horoscope?". According to Mlodinow, Cicero's greatest legacy to the field is the term probabilis, which gave rise to the word probability. [2]The Romans also left the Digest, part of the code of laws compiled by order of Emperor Justinian in the 6th century. According to Mlodinow, it is there that probability first appears as a legal concept: instead of treating an accusation as simply proven or not proven, the law began to weigh evidence and testimony, accepting that some evidence counts more than other evidence and that the judge decides for the more probable version. [2]But why were there no studies of probability before the Romans?“Why didn't the Greeks develop a theory of probability? One answer is that many Greeks believed the future unfolded according to the will of the gods. If the outcome of a game of astragali meant “marry the stocky Spartan girl who pinned you in that wrestling match behind the barracks”, a Greek boy would not see the toss as the lucky (or unlucky) result of a random process; he would see it as the will of the gods. With that view of the world, an understanding of randomness would be irrelevant. Therefore, a mathematical prediction of randomness would have seemed impossible.”- Leonard Mlodinow, The Drunkard's Walk [2]Why is probability so hard? Heuristics and biasesThe will of the gods is not the only explanation. The difficulty with probability also comes from the biases of the heuristics — the mental shortcuts — we use every day.Availability heuristicIt is the tendency to judge how frequent something is by how easily examples come to mind: recent or more vivid memories get more weight than they deserve. An example from Tversky and Kahneman: are there more seven-letter English words ending in -ing, or more seven-letter words whose sixth letter is n? Many people pick the first option, because words like "running" or "nothing" come to mind easily. But every seven-letter word ending in -ing has n as its sixth letter, so the second group contains the first and cannot be smaller. If you thought there were more words ending in -ing, you fell for the availability heuristic. [1]Representativeness heuristicIt is the use of mental reference models (stereotypes) as the basis for decisions: we judge the probability of something by how much it "looks like" the image we have in mind.An example, adapted from a classic experiment by Tversky and Kahneman: "Joana studied law, has always defended animals and opposed nuclear programs. She has always stood her ground and believes the world can get better." Which is more likely? (1) Joana is a bank teller; (2) Joana is a bank teller and a feminist. If you answered 2, you are part of the majority. But the set of feminist bank tellers is inside the set of bank tellers, so option 2 can never be more likely than option 1. This mistake is called the conjunction fallacy.Another example: "Diego is a shy young man who loves stories and books." Which is more likely? (1) Diego is a librarian; (2) Diego is a farmer. If you answered 1, you forgot that there are many more farmers than librarians; even if the profile fits a librarian better, the number of shy, book-loving farmers tends to be larger. Ignoring the size of each group is called insensitivity to base rates.Cases of prejudice, racism or "uncomfortable situations" also show how this heuristic affects our decisions — such as that of the Brazilian Jean Charles de Menezes, killed by London police in 2005 after being mistaken for a terrorist. [1]Anchoring and adjustment heuristicIt is the human tendency to "anchor" on a number or piece of information received and adjust too little from it, even when the anchor has nothing to do with the question. In a classic experiment (Ariely, Loewenstein and Prelec, 2003), students wrote down the last two digits of their Social Security number and then how much they would pay for products such as wine, chocolates and computer accessories. Those with higher digits offered much higher amounts: for several products, more than three times what those with the lowest digits offered. The explanation is that people had no price reference for those products and, without realizing it, the brain used the only number available as an anchor. [3] [5]The table below summarizes the heuristics we use daily and the biases linked to each one.HeuristicBiasAvailabilityEase of recall Presumed associations RetrievabilityRepresentativenessInsensitivity to base rates Insensitivity to sample size Misconceptions of chance Regression to the mean Conjunction fallacyAnchoring and adjustmentInsufficient anchor adjustment Overconfidence Confirmation trap Curse of knowledgeAna is 30 years old and runs marathons. Which is more likely?Answer: Ana is a teacher — every vegetarian teacher is a teacher, so adding a condition can never increase the probability: P(A and B) is at most P(A). Choosing one of the more detailed options because it 'fits' the description is the conjunction fallacy.The Monty Hall problemThe problem is inspired by the American TV show Let's Make a Deal, which premiered in 1963 and was hosted by Monty Hall. The host shows the contestant three doors: behind one of them there is a car (the good prize), and behind each of the others, a goat. The host knows where the car is. [4]In step 1, the contestant chooses one of the three doors (which is not opened yet);In step 2, Monty opens one of the two doors the contestant did not choose, always one with a goat, revealing that the car is not there;In step 3, Monty asks the contestant whether he wants to keep the door he chose at the start or switch to the other door that is still closed. With only two closed doors — since the one opened in step 2 has already shown it had no prize — and knowing the car is behind one of them, the contestant has to decide.Switch doors or keep the choice? The problem was published by the statistician Steve Selvin in 1975, but it became famous in 1990, when a reader sent it to Marilyn vos Savant's column in Parade magazine. Marilyn was known for having appeared in the Guinness Book of Records in the 1980s as the person with the highest recorded IQ (228). She answered that switching doors is advantageous, and this triggered an avalanche of letters: about 10,000 readers, nearly a thousand of them with PhDs, wrote to say she was wrong — 92% of the letters from the general public disagreed with her.Marilyn was right. On the first choice, the contestant picks the car with probability 1/3 and misses with probability 2/3. If he picked the car, switching makes him lose. But if he missed (2/3 of the time), the host is forced to open the other goat door, and the door left closed is precisely the one with the car. So whoever switches wins with probability 2/3, and whoever stays wins with probability 1/3. The case gave rise to the phrase:“Our brain was not very well designed to solve probability problems”- On the Monty Hall problemIn the Monty Hall problem, what is the probability of winning the car for someone who ALWAYS switches doors?Answer: 2/3 — whoever switches only loses when the first choice was already the car, which happens 1/3 of the time. The tempting answer, 1/2, treats the two closed doors as equal, but the host chose which one to open knowing where the car was, and that changes the odds.Something to think aboutHas any bias ever influenced your opinions?Think of models we could build for situations in our daily lives.ConclusionThis course aims to present models and probability situations from real life. Pay close attention to the flaws of models and to the flaws people have simply for being people, which can show up in data collection, analysis and inference. Remember that from now on we are in the field of science, and that is why the language changes: do not use vague terms, use terms that mean the same thing to everyone — such as probabilities!References[1] JUNIOR, Willian Gatti; NASCIMENTO, Paulo Tromboni de Souza. Heurísticas e vieses aplicados à gestão do relacionamento com fornecedores: Uma contribuição à teoria de operações comportamentais. XVIII SIMPEP–Simpósio de Engenharia de Produção. Bauru–SP: Nov, 2011.[2] MLODINOW, Leonard. The Drunkard's Walk: How Randomness Rules Our Lives. Pantheon, 2008. (Brazilian edition: O andar do bêbado. Zahar, 2009.)[3] Aronson, E. et al. (2003). Sozialpsychologie. Pearson Studium. ISBN 3827370841[4] https://pt.wikipedia.org/wiki/Problema_de_Monty_Hall[5] ARIELY, Dan; LOEWENSTEIN, George; PRELEC, Drazen. "Coherent arbitrariness": stable demand curves without stable preferences. The Quarterly Journal of Economics, v. 118, n. 1, p. 73–105, 2003.
ردود الفعل
0
0 تعليقات
كن أول من يعلق
لا شيء هنا بعد.